A Real-Time Object Detection Processor With xnor-Based Variable-Precision Computing Unit

Real-time object detection and tracking in video streams has become a fundamental component of modern intelligent systems, enabling applications in surveillance, autonomous driving, robotics, healthcare monitoring, and smart cities. The rapid advancement of deep learning techniques has significantly improved the accuracy and efficiency of detecting and tracking multiple objects under complex and dynamic environments. This study presents a deep learning–based framework for real-time object detection and tracking in continuous video streams, emphasizing both computational efficiency and detection robustness.The proposed system integrates a state-of-the-art convolutional neural network (CNN)-based object detector with a reliable multi-object tracking algorithm. The detection module identifies objects of interest in each frame, while the tracking component maintains consistent identities across consecutive frames by utilizing motion prediction and feature association strategies. To achieve real-time performance, the model is optimized through lightweight network architectures and parallel processing on GPU hardware. Data augmentation and transfer learning techniques are employed to enhance generalization and reduce training time. Performance evaluation is conducted on benchmark video datasets using standard metrics such as precision, recall, mean Average Precision (mAP), and tracking accuracy. Experimental results demonstrate that the proposed framework achieves high detection accuracy while maintaining low latency, making it suitable for real-time deployment. Furthermore, the system effectively handles challenges such as occlusion, scale variation, and illumination changes. This research presents a comprehensive study on real-time object detection and tracking in video streams using deep learning techniques. The proposed framework integrates YOLO-based object detection with Deep SORT multi-object tracking to achieve robust spatial localization and temporal identity preservation. The study evaluates performance using benchmark datasets including COCO, MOT17, and KITTI. Mathematical formulations of detection loss, Kalman filtering, and data association are presented. Experimental analysis using precision, recall, mAP, MOTA, ROC-AUC, and PR-AUC demonstrates that the system achieves high detection accuracy while maintaining real-time performance above 35 frames per second. The findings confirm that deep learning-based frameworks provide scalable and efficient solutions for surveillance, autonomous vehicles, and intelligent monitoring systems.

The real-time detection of objects and persons in their approach remains vital for applications that need safety measures security protocols and efficiency in autonomous systems and surveillance operations and industrial automation. The proposed research creates a durable real-time detection system through sensor combination using ultrasonic along with infrared and LiDAR and radar sensors and computer vision-based sensors to enhance detection accuracy and response times in changing environmental scenarios. The method starts by processing data gathered from arrays of multiple sensors through optimized filtering techniques and extraction methods before machines learn to classify objects while reducing false detection instances. The assessment of performance takes place in different environmental conditions to track both detection speed along with accuracy levels and system reliability. Sensor fusion enables real-time detection systems to perform better because it reduces the impact of environmental changes and sensor distortions. System capabilities increase in different operational areas when researchers demonstrate pre-processing optimization works alongside intelligent data processing systems. The research findings combine autonomous navigation systems with smart surveillance technology together with safety protocols for industrial security which enable real-time scalability and reduced risk elements. Research demonstrates a specific achievement through its analysis that machines learn better when combined with multi-sensor integration over single-sensor models which improves system reliability when used in real-world applications. Multi-sensor arrays and advanced processing techniques generate enhanced system responsiveness alongside increased adaptability and reliability thus improving detection systems within industrial security measures and automation operations.

In recent years, Convolutional Neural Networks (CNNs) have repeatedly shown state-of-the-art performance for their accuracy in the task of object detection, but their heavy computational costs impede their ability for real-time detection when the supporting system is moving, particulary when it is accelerating. At the same time, recent progress on visual inertial systems takes great advantage of movement information to robustly estimate the robot state and its surrounding. This paper proposes to exploit the advantages of inertial odometry research for the purpose of real-time object detection system on mobile robots. We combine a CNN detector with VINS-Mono, a moving visual odometry system, and show reliable improvement in the detection process, especially when the robot accelerates or decelerates. Our system is ready-to-use in that it has very low deployment cost and requires no calibration. The resulting system allows for simultaneous robot state estimation and object detection, as well as object tracking. Lastly, this architecture proves to be flexible because not restrained to a specific object type or detector.

Object detection is a major branch and fundamental task in computer vision, aiming to localize, identify and classify even the smallest objects of interest in images. Features can be extracted efficiently by deep convolutional neural networks (CNNs) as the backbone for real-time or near real-time object detection performance than the hand-crafted-based traditional methods. In the past few years, the advent of transformer-based models with robust self-attention mechanisms has not only raised object detection performance to a higher level but has also enabled it to produce excellent results. Many object detection tasks in the real world require that 3D information about the object be obtained, thus strengthening active research in 3D object detection. However, the algorithms for detecting 3D objects are not easy to propagate in real-world applications due to many factors, making reconstruction of 2D object detection algorithms to 3D object detection algorithms the suitable alternative. Therefore, we review the evolution of 2D object detection algorithms for digital imaging applications, focusing on their developments, models, applications, datasets, evaluation metrics, strengths and weaknesses, for better understanding of their landmarks and contributions to the advancement of the field.

This research proposes a novel hardware-optimized design to accelerate Convolutional Neural Networks (CNNs) using Verilog HDL. The design is specifically developed for the DARKNET-19 system model, which serves as the backbone of the YOLOv3-tiny algorithm, a widely used framework for real-time object detection in dynamic environments. The CNN architecture was implemented in Verilog HDL and synthesized using Synopsys Design Compiler, with a focus on improving both object detection accuracy and hardware resource efficiency. The proposed design efficiently performs key CNN operations, including convolution, pooling, and activation, enabling faster real-time object detection compared to many existing methods. To improve performance, the hardware design incorporates parallel processing techniques, allowing multiple computations to be executed simultaneously. This significantly reduces the system latency and power consumption. The convolutional layers of the DARKNET-19 architecture are efficiently mapped onto the hardware platform, ensuring optimized data storage and fast memory access, which further enhances processing speed and detection accuracy. An innovative feature of the design is a 2-dimensional image preprocessing module that prepares input images before they are fed into the CNN. This preprocessing stage includes image resizing, brightness normalization, and color adjustment, which helps the CNN process visual data more effectively. After preprocessing, the images pass through several CNN layers. The convolutional layers extract key features from the images, while the pooling and activation layers refine these features to improve detection performance. Finally, the processed data is analyzed by the YOLOv3-tiny algorithm, which identifies and locates objects in the images with high precision. Experimental results demonstrate that the proposed high-speed and resource-efficient hardware architecture is well-suited for real-time object detection applications, particularly in highly dynamic and unpredictable environments.

Computer vision (CV) and human-computer interaction (HCI) are essential in many technological fields. Researchers in CV are particularly interested in real-time object detection techniques, which have a wide range of applications, including inspection systems. In this study, we design and implement real-time object detection and recognition systems using the single-shoot detector (SSD) algorithm and deep learning techniques with pre-trained models. The system can detect static and moving objects in real-time and recognize the object's class. The primary goals of this research were to investigate and develop a real-time object detection system that employs deep learning and neural systems for real-time object detection and recognition. In addition, we evaluated the free available, pre-trained models with the SSD algorithm on various types of datasets to determine which models have high accuracy and speed when detecting an object. Moreover, the system is required to be operational on reasonable equipment. We tried and evaluated several deep learning structures and techniques during the coding procedure and developed and proposed a highly accurate and efficient object detection system. This system utilizes freely available datasets such as MS Common Objects in Context (COCO), PASCAL VOC, and Kitti. We evaluated our system's accuracy using various metrics such as precision and recall. The proposed system achieved a high accuracy of 97% while detecting and recognizing real-time objects.

Sandstorm weather significantly affects the quality of remote sensing images captured by drones, and collecting sandstorm datasets presents various challenges. This paper introduces a method for synthesizing a mild sandstorm dataset using Cycle-Consistent Adversarial Networks (CycleGAN), termed Sand-CycleGAN, which aimed to create a new sandstorm dataset, Sand-VisDrone, for the drone dataset VisDrone. This approach enhances the performance of remote sensing images in sandstorm conditions. To ensure that the synthesized dataset accurately reflects these conditions, we implement a Scatter Loss based on an atmospheric scatter model. This loss function improves the details and textures of the generated images, thereby increasing their realism. Additionally, we use a Super-Resolution Convolutional Neural Network (SRCNN) to enhance image resolution and restore detailed features that may be lost during the synthesis process. To validate the effectiveness of the Sand-VisDrone dataset, we conducted experiments with various dust removal networks and evaluated their performance using both reference-based and no-reference image quality metrics. The study also examines the influence of sandstorm image augmentation on the performance of object detection algorithms. Training results show that the mean Average Precision at IoU=0.5 (mAP@0.5) improved by 1.3% and 6.8% when using You Only Look Once version 10 narrow (YOLOv10n) and Real-Time Detection with Transformer for Object Tracking and Detection (RT-DETR) object detection algorithms, respectively.

Unmanned Aerial Vehicles (UAVs) have gained significant traction for real-time surveillance applications such as object tracking and search-and-rescue missions. A key enabler of these capabilities is the integration of a real-time, highly accurate object detection system. While available two-stage detectors like R-CNN, Fast R-CNN, Faster R-CNN, and Mask R-CNN achieve high detection accuracy and precise localization by dividing the image into regions and classifying each region, they are often slow, complex, and resource-intensive, making them unsuitable for real-time applications. To address this limitation, this research utilizes a Darknet-based YOLOv3 (You Only Look Once, Version 3), a state-of-the-art algorithm for real-time object detection in videos, live feeds, and images. Leveraging a deep Convolutional Neural Network (CNN), YOLOv3 efficiently learns features and predicts object locations and class probabilities in a single pass, ensuring high-speed and reliable detection. A real-time aerial surveillance system for quadcopter application using machine vision is proposed. The training dataset, obtained from the internet and self-taken images from a camera, was manually annotated into three critical categories: suspect, bandit, and weapon. The dataset was subsequently divided into training and testing subsets. Experimental results demonstrate that the proposed system achieves outstanding detection accuracy, with an overall mean average precision (mAP@0.5) of 93.82%, precision of 94%, and recall of 83%. Compared to a ResNet-50-based Faster R-CNN model, the YOLOv3-based approach outperformed, achieving success rates of 0.81 for the Bandit class and 1.00 for both Suspect and Weapon classes. This research improves drone-based AI for real-time object detection in security systems, enhancing efficiency and adaptability. It also creates specialized datasets, including a tailored bandit dataset, to support future UAV security research.

Real-time object detection and ranging of multiple objects on the road are the essential tasks in the field of autonomous driving. In this paper, we introduce a system for simultaneous detection and ranging of vehicles, people, non-motor vehicles and lanes based on RGB-D images. Among them, the detection of vehicles, people and non-motor vehicles belongs to general detection task and the lane detection belongs to segmentation task. In order to improve the accuracy and speed, we use two networks to complete these two tasks. We propose a real-time synchronization method with multi-GPU, which achieves separate training and simultaneous detection of lane detectior and vehicle, people and non-motor vehicle detector. We also propose a center-selective ranging module based on binocular ranging technology to distance the detected object. The system reaches nearly 15 FPS with four 1080Ti GPUs. We construct datasets about these problems including daytime and night in which the system achieves high accuracy. A real-time test of the system on the streets of Tianjin, China has been conducted by us, it has proved that the system can be applied to actual driving.

Cloud computing infrastructures have become the de-facto platform for data driven machine learning applications. However, these centralized models of computing are unqualified for dispersed high-volume real-time edge data intensive applications such as real time object detection, where video streams may be captured at multiple geographical locations. While many recent advancements in object detection have been made using Convolutional Neural Networks, these performance improvements only focus on a single, contiguous object detection model. In this paper, we propose a distributed Edge-Cloud R-CNN pipeline. By splitting the object detection pipeline into components and dynamically distributing these components in the cloud, we can achieve optimal performance to enable real time object detection. As a proof of concept, we evaluate the performance of the proposed system on a distributed computing platform including cloud servers and edge-embedded devices for real-time object detection on live video streams.

For real-time object detection and speech alarms, Research offers a sophisticated blind aid solution that smoothly combines the YOLO algorithm with Open CV's DNN (Deep Neural Network) module. The main objective is to improve the safety and freedom of people with visual impairments by offering quick object recognition and audio feedback. The YOLO method, which is tuned for real-time inference, is used by the system to precisely recognize objects after using a webcam to record live video input. Then, using audio description, it produces speech notifications that provide crucial details about the things it has spotted. Because of its exceptional versatility, the research can provide speech outputs in the user's preferred language, increasing its usability and accessibility. Its versatility is further demonstrated by its capacity to precisely handle a variety of object classes, making it a priceless tool for greatly enhancing the lives of those who are blind or visually impaired. Challenges in real-time object detection include occlusion, scale variations, and cluttered environments. Researchers must navigate the trade-offs between accuracy and speed. Real- time object detection is pivotal in computer vision, enabling intelligent systems across diverse applications. Real-time object detection systems designed for assistive technologies are becoming increasingly important in empowering individuals with visual impairments. The research integrates the YOLO algorithm with OpenCV’s DNN module to deliver a highly responsive blind-aid solution capable of capturing live video through a webcam and processing it instantly. YOLO’s architecture, optimized for speed and accuracy, enables the system to detect and classify multiple objects within each video frame, even in dynamic or unpredictable environments. Once an object is identified, the system generates immediate speech output, providing descriptive audio alerts in the user’s preferred language, thus enhancing accessibility and user experience. This multilingual capability makes the solution versatile across different regions and cultures. Additionally, the system supports a wide range of object categories such as vehicles, household items, and obstacles helping visually impaired users navigate daily surroundings more safely and independently. However, the implementation faces challenges, including handling occlusions, varying object scales, low-light conditions, and cluttered backgrounds. These factors require careful balancing between detection accuracy and processing speed to maintain real-time performance. Despite these challenges, the integration of advanced deep learning models with user-friendly audio interfaces demonstrates significant potential to improve mobility, safety, and overall quality of life for people with visual impairments.

In recent years, object detection became more and more important following the successful results from studies in deep learning. Two types of neural network architectures are used for object detection: one-stage and two-stage. In this paper, we analyze a widely used two-stage architecture called Faster R-CNN to improve the inference time and achieve real-time object detection without compromising on accuracy. To increase the computation efficiency, pruning is first adopted to reduce the weights in convolutional and fully connected (FC) layers. However, this reduces the accuracy of detection. To address this loss in accuracy, we propose a reduced region proposal network (RRPN) with dilated convolution and concatenation of multi-scale features. In the assisted multi-feature concatenation, we propose the intra-layer concatenation and proposal refinement to efficiently integrate the feature maps from different convolutional layers; this is then provided as an input to the RRPN. Using the proposed method, the network can find object bounding boxes more accurately, thus compensating for the loss arising from compression. Finally, we test the proposed architecture using ZF-Net and VGG16 as a backbone network on the image sets in PASCAL VOC 2007 or VOC 2012. The results show that we can compress the parameters of the ZF-Net-based network by 81.2% and save 66% of computation. The parameters of VGG16-based network are compressed by 73% and save 77% of computation. Consequently, the inference speed is improved from 27 to 40 frames/s for ZF-Net and 9 to 27 frames/s for VGG16. Despite significant compression rates, the accuracy of ZF-Net is increased from 2.2% to 60.2% mean average precision (mAP) and that of VGG16 is increased from 2.6% to 69.1% mAP.

Deep Convolutional Neural Network (CNN) algorithm has recently gained popularity in many applications such as image classification, video analytic and object detection. Being compute-intensive and memory expensive, CNN-based algorithms are hard to be implemented on the embedded device. Although recent studies have explored the hardware implementation of CNN-based object classification models such as AlexNet and VGG, there is still a rare implementation of CNN-based object detection model on Field Programmable Gate Array (FPGA). Consequently, this study proposes the fixed-point (16-bit) implementation of CNN-based object detection model: Tiny-Yolo-v2 on Cyclone V PCIe Development Kit FPGA board using High-Level-Synthesis (HLS) tool: OpenCL. Considering FPGA resource constraints in term of computational resources, memory bandwidth, and on-chip memory, a data pre-processing approach is proposed to merge the batch normalization into convolution layer. To the best of our knowledge, this is the first implementation of Tiny-Yolo-v2 object detection algorithm on FPGA using Intel FPGA Software Development Kit (SDK) for OpenCL. Finally, the proposed implementation achieves a peak performance of 21 GOPs under 100 MHz working frequency.

— In today's era, the challenge of object detection within laboratory spaces is paramount. The need for automated monitoring systems supporting object detection within laboratory settings, particularly for service robots, is crucial for scientific research and safety. Utilizing convolutional artificial neural networks has shown promising capabilities in object recognition. This paper proposes a novel method for detecting and localizing specific objects within laboratory environments using deep learning techniques. Employing a lightweight primary feature extractor allows for the identification of crucial object features without imposing excessive computational demands. Additionally, a depthwise separable convolution module is introduced, designed to capture essential components within multi-level convolutional patches specific to laboratory conditions. To address the challenges encountered within laboratory spaces, a new dataset is proposed, encompassing issues such as lighting variations, blurriness, occlusions, and complex backgrounds. Evaluation results indicate that the proposed model surpasses other lightweight object detection models in laboratory settings, offering high accuracy and efficiency. This model can be efficiently implemented on low-tier devices, ensuring real-time object detection and processing within laboratory environments. Keywords — object detection, service robot, convolutional neural networks, You Only Look Once version 5. Abstrak — Di era sekarang ini, tantangan deteksi objek di dalam ruang laboratorium menjadi sangat penting. Kebutuhan akan sistem pemantauan otomatis yang mendukung deteksi objek di dalam laboratorium, khususnya untuk robot servis, sangat penting untuk penelitian ilmiah dan keselamatan. Memanfaatkan jaringan saraf tiruan convolutional telah menunjukkan kemampuan yang menjanjikan dalam pengenalan objek. Makalah ini mengusulkan metode baru untuk mendeteksi dan melokalisasi objek tertentu di dalam lingkungan laboratorium menggunakan teknik pembelajaran mendalam. Menggunakan ekstraktor fitur utama yang ringan memungkinkan identifikasi fitur objek yang penting tanpa membebani komputasi yang berlebihan. Selain itu, modul konvolusi yang dapat dipisahkan secara mendalam juga diperkenalkan, yang dirancang untuk menangkap komponen penting dalam patch konvolusi multi-level yang spesifik untuk kondisi laboratorium. Untuk mengatasi tantangan yang dihadapi di dalam ruang laboratorium, dataset baru diusulkan, yang mencakup masalah seperti variasi pencahayaan, kekaburan, oklusi, dan latar belakang yang kompleks. Hasil evaluasi menunjukkan bahwa model yang diusulkan melampaui model pendeteksian objek ringan lainnya di lingkungan laboratorium, menawarkan akurasi dan efisiensi yang tinggi. Model ini dapat diimplementasikan secara efisien pada perangkat tingkat rendah, memastikan pendeteksian dan pemrosesan objek secara real-time di lingkungan laboratorium. Kata kunci — deteksi objek, robot layanan, jaringan saraf konvolusi, Anda Hanya Melihat Sekali versi 5.

Object detection is a fundamental task in computer vision, which is usually based on convolutional neural networks (CNNs). While it is difficult to be deployed in embedded devices due to the huge storage and computing consumptions, binary neural networks (BNNs) can execute object detection with limited resources. However, the extreme quantification in BNN causes diversity of feature representation loss, which eventually influences the object detection performance. In this paper, we propose a method balancing Information Retention and Deviation Control to achieve effective object detection, named IR-DC Net. On the one hand, we introduce the KL-Divergence to compose multiple entropy for maximizing the available information. On the other hand, we design a lightweight convolutional module to generate scale factors dynamically for minimizing the deviation between binary and real convolution. The experiments on PASCAL VOC, COCO2014, KITTI, and VisDrone datasets show that our method improved the accuracy in comparison with previous binary neural networks.