Deep learning on mobile devices: a review

This study proposes a compact deep learning (DL) architecture and a highly parallelized computing hardware platform to reconstruct the blood flow index (BFi) in diffuse correlation spectroscopy (DCS). We leveraged a rigorous analytical model to generate autocorrelation functions (ACFs) to train the DL network. We assessed the accuracy of the proposed DL using simulated and milk phantom data. Compared to convolutional neural networks (CNN), our lightweight DL architecture achieves 66.7% and 18.5% improvement in MSE for BFi and the coherence factor β, using synthetic data evaluation. The accuracy of rBFi over different algorithms was also investigated. We further simplified the DL computing primitives using subtraction for feature extraction, considering further hardware implementation. We extensively explored computing parallelism and fixed-point quantization within the DL architecture. With the DL model's compact size, we employed unrolling and pipelining optimizations for computation-intensive for-loops in the DL model while storing all learned parameters in on-chip BRAMs. We also achieved pixel-wise parallelism, enabling simultaneous, real-time processing of 10 and 15 autocorrelation functions on Zynq-7000 and Zynq-UltraScale+ field programmable gate array (FPGA), respectively. Unlike existing FPGA accelerators that produce BFi and the β from autocorrelation functions on standalone hardware, our approach is an encapsulated, end-to-end on-chip conversion process from intensity photon data to the temporal intensity ACF and subsequently reconstructing BFi and β. This hardware platform achieves an on-chip solution to replace post-processing and miniaturize modern DCS systems that use single-photon cameras. We also comprehensively compared the computational efficiency of our FPGA accelerator to CPU and GPU solutions.

Building the machines that advance scientific knowledge is one of the most significant challenges of our time. So-called Big Science instruments are unique; the expertise required to build them is multidisciplinary, and few countries can build and maintain them. Since resources and budgets are not unlimited, these require efficient solutions. Big Science machines take a decade to plan, a decade to build, and at least a decade to operate. One such machine is the largest fusion reactor project, ITER, which promises to be the next step in fusion research and has a big challenge in system integration. To mitigate the integration effort, ITER has invested in standardization, both in software and hardware, introducing standard instrumentation software named Nominal Device Support (NDSv3); Micro Telecommunication Computing Architecture (MTCA) as the long-term hardware standard choice, and EPICS as a general-purpose distributed control system. These control systems are essential in the management of the facility. They provide functions such as communication protocols, process management, monitoring, alarms, and human-machine interfaces. But also, more advanced access to databases or data acquisition systems. The latter is typically managed by a Field Programmable Gate Array (FPGA), of which facilities can easily have thousands of these FPGAs managing millions of signals within the control system environment. Nevertheless, these FPGA data acquisition (DAQ) systems continue to be labor-intensive. Standardized, modular, and open-source frameworks have been proposed to solve this problem, however, the FPGA programming expertise requirement remained. This thesis proposes a complete technology stack that considers these DAQ systems from the control system to the hardware. It provides a methodology using the Open Computing Language (OpenCL) to reduce the FPGA programming effort. The FPGA devices are programmed using C/C++-like environments, which are much simpler than traditional Hardware Description Languages (HDLs). The software, named IRIO-OpenCL, is written in C++ following ITER's NDSv3 model, which abstracts and generalizes the development of software device drivers and simplifies the interface to EPICS. IRIO-OpenCL is specifically designed to simplify the integration of intelligent instrumentation systems, allowing the integration of algorithms into the FPGA without modifying large parts of the supporting library. The MTCA standard was chosen to maintain modularity. The chassis backplane connects to the devices via Ethernet and PCIe, enabling standard communications. The Advanced Mezzanine Card is the typical MTCA FPGA-based DAQ carrier board. It connects the FPGA to the chassis backplane and typically provides FPGA Mezzanine Connectors (FMCs) for plugging in analog-to-digital (ADC) carrier boards. An ADC connected via the FMC interface samples the analog signals passed to the OpenCL processing kernels. Using OpenCL, the FPGA kernels communicate with the host machine in a standardized way. To demonstrate the applicability of this methodology, reference problems were selected from the literature and implemented, both with traditional and deep learning (DL) solutions. The methodology elaborates on connecting the OpenCL kernels for any use case, and the results show examples of kernel sets. The neutron flux measurement problem is solved at 1GS/s in real time using pulse counting and Campbell techniques, using 40% of the target Arria10 FPGA. The DL application example is based on a neutron/gamma discrimination neural network (NN) that achieves a classification rate of up to 1.3 MEvents/s using less than 25% of the target resource. RESUMEN Construir máquinas que avancen el conocimiento científico es uno de los desafíos más significativos de nuestro tiempo. Los instrumentos de la llamada Big Science son únicos; las competencias requeridas para su construcción son multidisciplinares, y pocos países pueden construirlos y mantenerlos. Dado que los recursos y presupuestos no son ilimitados, estos requieren soluciones eficientes. Las máquinas de Gran Ciencia tardan una década en planificarse, una década en construirse y al menos una década en operarse. Una de estas máquinas es el proyecto del reactor de fusión más grande, ITER, que promete ser el siguiente paso en la investigación de fusión y presenta un gran desafío en la integración de sistemas. Para mitigar el esfuerzo de integración, ITER ha invertido en estandarización, tanto en software como en hardware, introduciendo un software de instrumentación estándar llamado Nominal Device Support v3 (NDSv3); Micro Telecommunication Computing Architecture (MTCA) como la elección estándar de hardware a largo plazo, y EPICS como un sistema de control distribuido de propósito general. Estos sistemas de control son esenciales en la gestión de la instalación ya que proporcionan funciones como protocolos de comunicación, gestión de procesos, monitorización, alarmas e interfaces de usuario, además de acceso avanzado a bases de datos o sistemas de adquisición de datos. Este último es típicamente gestionado por un Field Programmable Gate Array (FPGA), de los cuales las instalaciones pueden tener fácilmente miles de unidades gestionando millones de señales dentro del entorno del sistema de control. Estos sistemas de adquisición de datos (DAQ) con FPGA siguen requiriendo programación experta y laboriosa. Se han propuesto marcos estandarizados, modulares y de código abierto para resolver este problema, sin embargo, la necesidad de experiencia en programación FPGA no se ha podido eludir. Esta tesis propone una pila de tecnologías que considera estos sistemas DAQ desde el sistema de control hasta el hardware. Proporciona una metodología utilizando el Open Computing Language (OpenCL) para reducir el esfuerzo de programación FPGA. Los dispositivos FPGA se programan utilizando entornos similares a C/C++, que son mucho más simples que los Lenguajes de Descripción de Hardware (HDL) tradicionales. El software, llamado IRIO-OpenCL, está escrito en C++ siguiendo el modelo de NDSv3 de ITER, que abstrae y generaliza el desarrollo de controladores de dispositivos de software y simplifica la interfaz con EPICS. IRIO-OpenCL está específicamente diseñado para simplificar la integración de sistemas de instrumentación inteligentes, permitiendo la integración de algoritmos en el FPGA sin modificar grandes partes de la biblioteca de soporte. El estándar MTCA fue elegido para mantener la modularidad. El backplane del chasis se conecta a los dispositivos a través de Ethernet y PCIe, habilitando comunicaciones estándar. La Advanced Mezzanine Card es la placa portadora DAQ FPGA típica de MTCA, que conecta el FPGA al backplane del chasis y típicamente proporciona FPGA Mezzanine Connectors (FMC) para conectar placas con conversores analógico-digitales (ADC). Un ADC conectado a través de la interfaz FMC muestrea las señales analógicas pasadas a los núcleos de procesamiento OpenCL. Utilizando OpenCL, los núcleos FPGA se comunican con la máquina anfitriona de una manera estandarizada. Para demostrar la aplicabilidad de esta metodología, se seleccionaron e implementaron problemas de referencia de la literatura, tanto con soluciones tradicionales como de Deep Learning (DL). La metodología elabora ejemplos de conexión de núcleos OpenCL para cualquier caso de uso, y los resultados muestran ejemplos de conjuntos de núcleos. El problema de medición del flujo de neutrones se resuelve a 1GS/s en tiempo real utilizando técnicas de conteo de pulsos y Campbell, utilizando el 40% del objetivo FPGA Arria10. El ejemplo de aplicación DL se basa en una red neuronal para la discriminación neutrones/gammas que logra una tasa de clasificación de hasta 1.3 MEventos/s utilizando menos del 25% de la FPGA objetivo. LABURPENA Ezagutza zientifikoa aurreratzen duten makinak eraikitzea gure garaiko erronka garrantzitsuenetako bat da. Big Science deritzonaren makinak bakarrak dira; beharrezko esperientzia diziplina anitzekoa da, eta herrialde gutxik eraiki eta mantendu ditzakete. Baliabideak eta aurrekontuak mugagabeak ez direnez, konponbide eraginkorrak behar dira. Zientzia Handi makinak hamarkada bat behar dute diseinatuak izateko, hamarkada bat eraikitzeko eta gutxienez hamarkada bat jarduteko. Makina horietako bat fusio-erreaktore handienaren proiektua da, ITER, fusio-ikerketan hurrengo urratsa izango dena eta sistemak integratzeko erronka handia aurkezten duena. Integrazio-ahalegina arintzeko, ITERek estandarizazioan inbertitu du, bai softwarean bai - eta EPICS helburu orokorreko kontrol-sistema banatua. Kontrol-sistema horiek funtsezkoak dira instalazioa kudeatzeko. Komunikazio-protokoloak, prozesuen kudeaketa, monitorizazioa, alarmak eta pertsona-makina interfazeak bezalako funtzioak ematen dituzte, datubaseetarako edo datuak eskuratzeko sistemetarako sarbide aurreratua izateaz gain. Azken hori "Field Programmable Gate Array" (FPGA) batek kudeatzen du, milaka FPGA konektatuz makina bakoitzean eta kontrol-sistemaz milioika seinale kudeatuz. FPGA duten datuak eskuratzeko sistemek (DAQ) programazio aditua eta neketsua eskatzen jarraitzen dute. Problema hori ebazteko marko estandarizatuak, modularrak eta kode irekikoak proposatu dira, hala ere, FPGA hardware programazio espezializatuaren beharra ezin izan da ukatu. Tesi honek DAQ sistema hauek kontrol sistematik hardwareraino kontsideratuz teknologia bilduma bat programazio-ahalegina murrizten du. FPGA gailuak C/C++ antzeko inguruneak erabiliz programatzen dira, - OpenCL izenekoa, C++n idatzita dago, ITERren "Nominal Device Support v3" (NDSv3) eredua jarraituz. Honek software-gailuen garapena diseinu abstraktu eta orokorrez EPICSekiko interfazea sinplifikatzen du. IRIOOpenCL berariaz diseinatuta dago instrumentazio-sistema adimendunen integrazioa sinplifikatzeko, eta algoritmoak FPGAn integratzea ahalbidetzen du, software zati handiak aldatu gabe. Modularitateari eusteko, MTCA estandarra aukeratu da. Txasisaren backplanea Ethernet eta PCIe bidez konektatzen da gailuetara, komunikazio estandarrak gaituz. "Advanced Mezzanine Card" MTCAren ohiko DAQ FPGA plaka eramailea da. FPGA txasisaren backplanera konektatzen du eta, normalean, "FPGA Mezzanine Connectors"ak (FMC) ditu plaka bihurgailu analogiko-digitalak (ADC) konektatzeko. ADCak, FMC interfazearen bidez konektatuta, OpenCL prozesatze-nukleoetara igarotako seinale analogikoak bihurtzen ditu. OpenCL erabiliz, FPGA nukleoak ordenagailuarekin modu estandarizatuan komunikatzen dira. Metodologia horren aplikagarritasuna egiaztatzeko, erreferentziazko adibideak literaturatik hautatu eta nukleoen konexio-adibideak egiten ditu, edozein kasutan erabiltzeko, eta emaitzek nukleo multzoen adibideak erakusten dituzte. Neutroi-fluxuaren neurketak 1GS/s-ko ebazten dira real time errendimenduaz, pultsuak eta Campbell zenbatzeko teknikak erabiliz, Arria10 FPGAren % 40 erabiliz. Azkenik, DL aplikazioaren adibidea, neutroi/gamma sailkapena, sare neuronal batean oinarritzen da, eta FPGAren baliabideen % 25 baino gutxiago erabiliz 1.3 MSailkapen/s sailkapen-tasa lortzen du.

Tridiagonal diagonally dominant linear systems arise in many scientific and engineering applications. The standard Thomas algorithm for solving such systems is inherently serial forming a bottleneck in computation. Algorithms such as cyclic reduction and SPIKE reduce a single large tridiagonal system into multiple small independent systems which can be solved in parallel. We have developed portable cyclic reduction and SPIKE algorithm OpenCL implementations with the intent to target a range of co-processors in a heterogeneous computing environment including Field Programmable Gate Arrays (FPGAs), Graphics Processing Units (GPUs) and other multi-core processors. In this paper, we evaluate these designs in the context of solver performance, resource efficiency and numerical accuracy.

The staggering innovations and emergence of numerous deep learning (DL) applications have forced researchers to reconsider hardware architecture to accommodate fast and efficient application-specific computations. Applications, such as object detection, image recognition, speech translation, as well as music synthesis and image generation, can be performed with high accuracy at the expense of substantial computational resources using DL. Furthermore, the desire to adopt Industry 4.0 and smart technologies within the Internet of Things infrastructure has initiated several studies to enable on-chip DL capabilities for resource-constrained devices. Specialized DL processors reduce dependence on cloud servers, improve privacy, lessen latency, and mitigate bandwidth congestion. As we reach the limits of shrinking transistors, researchers are exploring various application-specific hardware architectures to meet the performance and efficiency requirements for DL tasks. Over the past few years, several software optimizations and hardware innovations have been proposed to efficiently perform these computations. In this article, we review several DL accelerators, as well as technologies with emerging devices, to highlight their architectural features in application-specific integrated circuit (IC) and field-programmable gate array (FPGA) platforms. Finally, the design considerations for DL hardware in portable applications have been discussed, along with some deductions about the future trends and potential research directions to innovate DL accelerator architectures further. By compiling this review, we expect to help aspiring researchers widen their knowledge in custom hardware architectures for DL.

Automated invasive ductal carcinoma detection based using deep transfer learning with whole-slide images

This article aims at implementing a Deep Neural Network (DNN) using Field Programmable Gate Arrays (FPGAs) for real time deep learning inference in embedded systems. In now days DNNs are widely used where high accuracy is required. However, due to the structural complexity, deep learning models are highly computationally intensive. To improve the system performance, optimization techniques such as weight quantization and pruning are commonly adopted. Another approach to improve the system performance is by applying heterogeneous architectures. Processor with Graphics Processing Unit (GPU) architectures are commonly used for deep learning training and inference acceleration. However, GPUs are expensive and consume much power that not a perfect solution for embedded systems. In this paper, we implemented a deep neural network on a Zynq SoC which is a heterogenous system integrated of ARM processor and FPGA. We trained the model with MNIST database, quantized the model’s 32-bit floating point weights and bias into integer and implemented model to inference in FPGA. As a result, we deployed a network on an embedded system while maintaining inference accuracy and accelerated the system performance with using less resources.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

The development of artificial intelligence (AI) technologies demands significant computing power and specialized talent, giving an advantage to large corporations with abundant resources. While knowledge produced by these firms can foster follow-on innovation across the industry through positive spillovers to startups, it may also prove less relevant or accessible to smaller firms, potentially creating barriers to their innovation. This paper leverages the sudden rise of deep learning in 2012 as an exogenous variation to examine its impact on the AI innovation landscape. The findings indicate that, following the breakthrough in deep learning, big tech companies significantly increased their participation in AI development. However, this increased involvement was associated with a reduced likelihood of deep learning research being used in AI startup patents. Notably, this shift did not affect the probability of these papers being cited in other startup patents or overall patent citations. These results suggest that the trajectory of technological change in the AI industry increasingly aligns with the priorities of large incumbents, potentially at the expense of fostering innovation within the startup ecosystem.

Background and Purpose: The COVID-19 pandemic has given a massive proliferation of technology, particularly in education, redefining language learning from face-to-face (F2F) and off-classroom known as blended learning (BL) to a new kind in online distance learning/education (ODL/E). BL is now a mix of synchronous and asynchronous learning. The shift, however impressive it may appear, does not guarantee the effectiveness of the learning process. The gap that leads to the current study is how bibliometrics has shown minimal focus on undergraduates’ acceptance of these changes, especially in English as a Second Language (ESL) learning classes to the newly embraced online BL (OBL) and how humanistic values are important in ESL lessons. Thus the study sets out to understand several issues pertaining to the use of mobile communication devices as a learning tool in ESL MoBL.
 
 Methodology: The present study is a mixed-method research approach using a sequential exploratory design. A set of Technology Acceptance Model (TAM) questionnaires was distributed to understand 264 teacher-trainers, degree and diploma students’ inclination towards ESL mobile learning platforms after a semester of mobile open blended learning (MoBL) integration, whilst side-lining the unnecessary information of Unified Theory of Acceptance and Use of Technology (UTAUT2). Interviews based on Analysis, Design, Assess, and Belief (ADAB) Model further enriched the data on the humanisation aspects of MoBL.
 
 Findings: Findings showed that learners’ inclination to the new MoBL can be affected by gender, age, and maturity. Thus to implement a more humanistic learning modification demands a thorough understanding of students’ needs.
 
 Contributions: The novelty of these findings is the sampling contrasting both degree and diploma students along with teacher-trainees’ perceptions of humanistic values in OBL highlights the effects of, age, maturity and gender on technology in education. This paper suggests a discussion on the humanistic MoBL model for ESL learners. The future implication of the findings suggests age, maturity, and gender affect students’ inclination towards the new MoBL, and pushes the need for more humanistic essences in MoBL.
 
 Keywords: Education, ESL and ELT, humanistic values, mobile learning model, online blended learning, and open distance learning.
 
 Cite as: Tengku Intan Suzila, T. S., Mohd Yusri, M. N., Omar, S. R., & Teo, K. S. (2022). Humanizing mobile online ESL blended learning model. Journal of Nusantara Studies, 7(2), 473-494. http://dx.doi.org/10.24200/jonus.vol7iss2pp473-494

The rise and exploitation of artificial intelligence (AI) in general and in particular the medical field is fuelled by breakthroughs in deep learning (DL), advances in computing hardware devices, as well as the exponential growth of clinical data used for decision-making. Modern oncological research is intensively adopting AI-based technologies, with most of the related elements such as machine learning (ML) and DL models being utilised to improve the accuracy and efficiency of cancer diagnosis, prevention and treatment, with studies showing that AI could have many additional applications in cancer care. In most countries, numerous AI technologies have received governmental approvals for use in oncology, most notably in radiology. In response to the growing interest in the use of AI technologies in clinical oncology, the number of AI start-ups that concentrate their efforts on combating cancer has surged. The present research study adopts a mixed methodological approach to perform a comparative analysis of the top five AI start-ups that are focused on the application of AI technologies in clinical oncology. The five chosen AI start-ups include: CancerIQ (USA); Panakeia (UK); MultiplAI Health (UK); MNM Bioscience (USA) and X-Zell in Singapore. The selection of the AI start-ups was based on the availability of statistical metrics of interest to the study and the impact of their projects and applications on clinical oncology. The study performs a quantitative and qualitative analysis of selected metrics associated with the use of AI in clinical oncology and the related financial performance metrics.

In the last decade, deep learning has seen exponential growth due to rise in computational power as a result of graphics processing units (GPUs) and a large amount of data due to the democratization of the internet and smartphones. This chapter aims to throw light on both the theoretical aspects of deep learning and its practical aspects using PyTorch. The chapter primarily discusses new technologies using deep learning and PyTorch in detail. The chapter discusses the advantages of using PyTorch compared to other deep learning libraries. The chapter discusses some of the practical applications like image classification and machine translation. The chapter also discusses the various frameworks built with the help of PyTorch. PyTorch consists of various models that increases its flexibility and accessibility to a greater extent. As a result, many frameworks built on top of PyTorch are discussed in this chapter. The authors believe that this chapter will help readers in getting a better understanding of deep learning making neural networks using PyTorch.

Mango cultivation faces significant challenges from leaf diseases such as Gall Midge and Sooty Mould, which can reduce crop yield by up to 40% if not detected early. Traditional disease identification methods rely heavily on expert knowledge and laboratory analysis, which are time- consuming, expensive, and inaccessible to small-scale farmers. This study presents a comprehensive mobile-based solution for automated mango leaf disease detection using deep learning techniques deployed on mobile devices. The system employs a Convolutional Neural Network (CNN) optimized through quantization-aware training and deployed via TensorFlow Lite within a Flutter-based cross-platform mobile application. The dataset comprises 4,962 high-resolution images across three classes: Healthy, Gall Midge, and Sooty Mould, collected from agricultural research stations and field conditions.. The Flutter application enables farmers to capture leaf images through their smartphone cameras and receive immediate disease classification results with confidence scores, detailed disease information, and treatment recommendations. The system operates entirely offline, making it accessible in remote agricultural areas with limited internet connectivity. Results demonstrate that mobile-based deep learning can provide accurate, fast, and accessible plant disease detection, contributing to early intervention, reduced crop losses, and improved agricultural productivity for mango farmers globally. Keywords: Mango Leaf Disease Detection, TensorFlow Lite, Flutter Framework, Mobile Deep Learning, CNN Quantization, Agricultural Technology, Plant Disease Classification, On- Device Inference

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

Is Deep Learning useful for decision making in pig production?

Recent breakthroughs in deep learning (DL) have led to the emergence of many intelligent mobile applications and services, but in the meanwhile also pose unprecedented computing challenges on resource-constrained mobile devices. This paper builds a collaborative deep inference system between a resource-constrained mobile device and a powerful edge server, aiming at joining the power of both on-device processing and computation offloading. The basic idea of this system is to partition a deep neural network (DNN) into a front-end part running on the mobile device and a back-end part running on the edge server, with the key challenge being how to locate the optimal partition point to minimize the end-to-end inference delay. Unlike existing efforts on DNN partitioning that rely heavily on a dedicated offline profiling stage to search for the optimal partition point, our system has a built-in online learning module, called Autodidactic Neurosurgeon (ANS), to automatically learn the optimal partition point on-the-fly. Therefore, ANS is able to closely follow the changes of the system environment by generating new knowledge for adaptive decision making. The core of ANS is a novel contextual bandit learning algorithm, called μLinUCB, which not only has provable theoretical learning performance guarantee but also is ultra-lightweight for easy real-world implementation. We implement our system on a video stream object detection testbed to validate the design of ANS and evaluate its performance. The experiments show that ANS significantly outperforms state-of-the-art benchmarks in terms of tracking system changes and reducing the end-to-end inference delay.