Research on Real-Time Object Detection and Tracking for UAV Surveillance Based on Deep Learning

Real-time object detection and tracking in video streams has become a fundamental component of modern intelligent systems, enabling applications in surveillance, autonomous driving, robotics, healthcare monitoring, and smart cities. The rapid advancement of deep learning techniques has significantly improved the accuracy and efficiency of detecting and tracking multiple objects under complex and dynamic environments. This study presents a deep learning–based framework for real-time object detection and tracking in continuous video streams, emphasizing both computational efficiency and detection robustness.The proposed system integrates a state-of-the-art convolutional neural network (CNN)-based object detector with a reliable multi-object tracking algorithm. The detection module identifies objects of interest in each frame, while the tracking component maintains consistent identities across consecutive frames by utilizing motion prediction and feature association strategies. To achieve real-time performance, the model is optimized through lightweight network architectures and parallel processing on GPU hardware. Data augmentation and transfer learning techniques are employed to enhance generalization and reduce training time. Performance evaluation is conducted on benchmark video datasets using standard metrics such as precision, recall, mean Average Precision (mAP), and tracking accuracy. Experimental results demonstrate that the proposed framework achieves high detection accuracy while maintaining low latency, making it suitable for real-time deployment. Furthermore, the system effectively handles challenges such as occlusion, scale variation, and illumination changes. This research presents a comprehensive study on real-time object detection and tracking in video streams using deep learning techniques. The proposed framework integrates YOLO-based object detection with Deep SORT multi-object tracking to achieve robust spatial localization and temporal identity preservation. The study evaluates performance using benchmark datasets including COCO, MOT17, and KITTI. Mathematical formulations of detection loss, Kalman filtering, and data association are presented. Experimental analysis using precision, recall, mAP, MOTA, ROC-AUC, and PR-AUC demonstrates that the system achieves high detection accuracy while maintaining real-time performance above 35 frames per second. The findings confirm that deep learning-based frameworks provide scalable and efficient solutions for surveillance, autonomous vehicles, and intelligent monitoring systems.

Object detection and tracking are two critical tasks in computer vision, widely applied in security surveillance, autonomous vehicles, and behavioral analysis. Strong performance in object detection has been demonstrated by recent Transformer-based models, such as RT-DETR (Real-Time Detection Transformer), due to their global context modeling and high accuracy. However, an inherent tracking mechanism is lacking in RT-DETR, which requires additional components to maintain identity consistency across frames. To address this limitation, an integration of RT-DETR with DeepSORT is proposed, leveraging the strengths of both models to enhance real-time object detection and tracking. A comparative evaluation with YOLOv8, a widely used real-time detector, is conducted to highlight the advantages of the proposed approach in tracking accuracy and robustness. Experiments show that effective performance is achieved in challenging scenarios such as object occlusion and intersection. Specifically, an IDF1 score of 60.0%, a MOTA of 42.4%, and a MOTP of 43.3% are obtained by RTl+DeepSORT on the MOT17-02-DPM dataset, outperforming YOLOv8x+DeepSORT. These results indicate that significant improvements in tracking accuracy are attained while real-time efficiency is maintained, making the proposed approach well-suited for applications such as intelligent surveillance and autonomous navigation.

Object detection and tracking is one of the key applications of wireless sensor networks (WSNs). The key issues associated with this application include network lifetime, object detection and localization accuracy. To ensure the high quality of the service, there should be a trade-off between energy efficiency and detection accuracy, which is challenging in a resource-constrained WSN. Most researchers have enhanced the application lifetime while achieving target detection accuracy at the cost of high node density. They neither considered the system cost nor the object localization accuracy. Some researchers focused on object detection accuracy while achieving energy efficiency by limiting the detection to a predefined target trajectory. In particular, some researchers only focused on node clustering and node scheduling for energy efficiency. In this study, we proposed a mobile object detection and tracking framework named the Energy Efficient Object Detection and Tracking Framework (EEODTF) for heterogeneous WSNs, which minimizes energy consumption during tracking while not affecting the object detection and localization accuracy. It focuses on achieving energy efficiency via node optimization, mobile node trajectory optimization, node clustering, data reporting optimization and detection optimization. We compared the performance of the EEODTF with the Energy Efficient Tracking and Localization of Object (EETLO) model and the Particle-Swarm-Optimization-based Energy Efficient Target Tracking Model (PSOEETTM). It was found that the EEODTF is more energy efficient than the EETLO and PSOEETTM models.

Artificial vision in robotics involves real time detection of objects for fast decision making. Such intelligent systems require efficient algorithms and big learning database of examples for producing robust classifiers. Several methods of objects detection and tracking have been proposed in the literature. However, even though the detection rates have been improved, the processing time and the complexity of the models still representing a key challenge. In this paper, we present a real time object detection and tracking framework based on Adaboost classification, where a strong classifier is generated using an iterative combination of weak learners. This method is based on the use of discriminative features by analyzing different regions of the input image. Instead of performing a full traversal in the entire search space of all possible visual features, we propose to use intelligent heuristics for accelerating time processing and extracting relevant features in the image that lead to a best detection rate. The meta-heuristics involve the use of genetic algorithms, particle swarm optimization, random walk and a novel hybrid combination of these methods. The obtained results, in a case of intelligent transportation system, have shown considerable improvements in term of computation time, efficiency and accuracy.

Computer vision (CV) and human-computer interaction (HCI) are essential in many technological fields. Researchers in CV are particularly interested in real-time object detection techniques, which have a wide range of applications, including inspection systems. In this study, we design and implement real-time object detection and recognition systems using the single-shoot detector (SSD) algorithm and deep learning techniques with pre-trained models. The system can detect static and moving objects in real-time and recognize the object's class. The primary goals of this research were to investigate and develop a real-time object detection system that employs deep learning and neural systems for real-time object detection and recognition. In addition, we evaluated the free available, pre-trained models with the SSD algorithm on various types of datasets to determine which models have high accuracy and speed when detecting an object. Moreover, the system is required to be operational on reasonable equipment. We tried and evaluated several deep learning structures and techniques during the coding procedure and developed and proposed a highly accurate and efficient object detection system. This system utilizes freely available datasets such as MS Common Objects in Context (COCO), PASCAL VOC, and Kitti. We evaluated our system's accuracy using various metrics such as precision and recall. The proposed system achieved a high accuracy of 97% while detecting and recognizing real-time objects.

Efficient and accurate object detection, classification and tracking have played a very important role in the advancement of computer vision systems. With the rise of deep learning techniques, the accuracy of visioning systems has increased a lot. This proposed work targets to incorporate state-of-the-art technology with the aim of achieving very high accuracy for real-time multiclass multiple object tracking with occlusion handling. A real-time multiclass multiple object tracker (MOT) is proposed using state-of-the-art object detection framework and deep convolutional neural networks. The proposed model is developed using YOLOv3 architecture for object detection and for real-time tracking, and a longer period of occlusion handling simple online and real-time tracking (SORT) is used as a base algorithm. The velocity and acceleration of objects along with their appearance information are used to predict the object location upon occlusion. Later detections are compared with MOT dataset detections and the YOLO bounding boxes, and based on their IOU matrix, normalized detections are provided.

The objective of this study is to examine two powerful components: the YOLO (You Only Look Once) framework for object recognition on edge devices and the MobileNet architecture. With the rising demand for real-time, efficient, and accurate object detection in constrained environments, these models offer distinct advantages suited to different applications. YOLO is renowned for its high accuracy and speed, making it ideal for complex detection tasks, whereas MobileNet is optimized for efficiency, enabling deployment on devices with limited computational resources. The paper discusses the evolution of these models, their architectural differences, and their respective strengths and limitations when implemented on edge devices. Furthermore, we explore the feasibility of integrating these frameworks into a hybrid model that leverages the lightweight feature extraction of MobileNet and the robust detection capabilities of YOLO. Such a hybrid approach could significantly enhance real-time object detection performance across a wide range of hardware platforms.  Researchers and practitioners seeking to deploy advanced object detection models in real-world applications, particularly where computational efficiency and accuracy are paramount may find this review as a valuable resource.

Conventional street cleaning methods include street sweepers going to various spots in the city and manually verifying if the street needs cleaning and taking action if required. However, this method is not optimized and demands a huge investment in terms of time and money. This paper introduces an automated framework which addresses street cleaning problem in a better way by making use of modern equipment with cameras and computational techniques to analyze, find and efficiently schedule clean-up crews for the areas requiring more attention. Deep learning-based neural network techniques can be used to achieve better accuracy and performance for object detection and classification than conventional machine learning algorithms for large volume of images. The proposed framework for street cleaning leverages the deep learning algorithm pipeline to analyze the street photographs and determines if the streets are dirty by detecting litter objects. The pipeline further determines the the degree to which the streets are littered by classifying the litter objects detected in earlier stages. The framework also provides information on the cleanliness status of the streets on a dashboard updated in real-time. Such framework can prove effective in reducing resource consumption and overall operational cost involved in street cleaning.

The rising demand for real-time perception in aerial platforms has intensified the need for lightweight, hardware-efficient object detectors capable of reliable onboard operation. This survey provides a focused examination of real-time aerial object detection, emphasizing algorithms designed for edge devices and UAV onboard processors, where computation, memory, and power resources are severely constrained. We first review the major aerial and remote-sensing datasets and analyze the unique challenges they introduce, such as small objects, fine-grained variation, multiscale variation, and complex backgrounds, which directly shape detector design. Recent studies addressing these challenges are then grouped, covering advances in lightweight backbones, fine-grained feature representation, multi-scale fusion, and optimized Transformer modules adapted for embedded environments. The review further highlights hardware-aware optimization techniques, including quantization, pruning, and TensorRT acceleration, as well as emerging trends in automated NAS tailored to UAV constraints. We discuss the adaptation of large pretrained models, such as CLIP-based embeddings and compressed Transformers, to meet onboard real-time requirements. By unifying architectural strategies, model compression, and deployment-level optimization, this survey offers a comprehensive perspective on designing next-generation detectors that achieve both high accuracy and true real-time performance in aerial applications.

The natural phenomenon of harmful algae bloom (HAB) has a bad impact on the quality of pure and freshwater. It increases the risk to human health, water bodies and overall aquatic ecosystem. It is necessary to continuously monitor and perform proper action against HAB. The inspection of algae blooms by using conventional methods, like algae detection under microscopes, is a difficult, expensive, and time-consuming task, however, computer vision-based deep learning models play a vital role in identifying and detecting harmful algae growth in aquatic ecosystems and water reservoirs. Many studies have been conducted to address harmful algae growth by using a CNN based model, however, the YOLO model is considered more accurate in identifying the algae. This advanced deep learning method is extensively used to detect algae and classify them according to their corresponding category. In this study, we used various versions of the convolution neural network (CNN) based on the You Only Look Once (YOLO) model. Recently YOLOv5 has been getting more attention due to its performance in real-time object detection. We performed a series of experiments on our custom microscopic images dataset by using YOLOv3, YOLOv4, and YOLOv5 to detect and classify the harmful algae bloom (HAB) of four classes. We used pre-processing techniques to enhance the quantity of data. The mean average precision (mAP) of YOLOv3, YOLOv4, and YOLO v5 is 75.3%, 83.0%, and 91.0% respectively. For the monitoring of algae bloom in freshwater, computer-aided based systems are very helpful and effective. To the best of our knowledge, this work is pioneering in the AI community for applying the YOLO models to detect algae and classify from microscopic images.

In several applications of computer vision and image processing, the inception of the processing starts with object detection and subsequently tracking, if the need arises. In recent years, there has been extensive research in the field of object detection and tracking. Many remarkable algorithms have been developed for object detection and tracking, including color segmentation, edge tracking and many more. However, all these algorithms faced the limited success in their implementation in the real world and were also bounded by the constraints such as white/plain background. This paper is the result of our research where our research team developed and implemented object detection and tracking system operational in an unknown background, using real-time video processing and a single camera. The proposed system has been extensively tested to operate in complex, real world, non-plain, light variant, changing background.

The remote sensing object detection technique has shown considerable applicability across various domains. Visible light remote sensing detection is easily affected by environmental factors such as lighting and weather conditions, resulting in poor detection performance. Infrared remote sensing detection is less affected by environmental factors, but its limited texture detail restricts its accuracy in object detection. Visible imagery and infrared imagery combined for multispectral object recognition effectively address complex remote sensing scenarios. Nonetheless, prior multispectral detection algorithms are plagued by challenges like inadequate modality information fusion, inferior performance in multi-scale object detection, and insufficient exploitation of semantic information. We propose a novel multispectral fusion architecture to tackle these difficulties. Initially, we developed the Cross-modal Adaptive Frequency Fusion (CAFF) module to independently decompose and integrate highfrequency and low-frequency information, thereby achieving fine fusion of images from different modalities in different frequency domains. Subsequently, we integrated the Multi-scale Receptive Field Aggregator (MRFA) into the SPPF module to ensure that the target obtains sufficient global contextual information while maintaining fine spatial details. We have developed a feature pyramid network for multispectral object detection, significantly improving feature interaction and fusion. This constitutes a thorough VIF-YOLO detection framework. On the VEDAI and LLVIP datasets, VIF-YOLO attains <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{mAP}_{50}$</tex> values of 75.1 % and 96.3 %, respectively, illustrating the efficacy of our approach in enhancing detection capabilities.

Real-time detection and tracking for fast moving object has important applications in various fields. However, available methods, especially low-cost ones, can hardly achieve real-time and long-duration object detection and tracking. Here we report an image-free and cost-effective method for detecting and tracking a fast moving object in real time and for long duration. The method employs a spatial light modulator and a single-pixel detector for data acquisition. It uses Fourier basis patterns to illuminate the target moving object and collects the resulting light signal with a single-pixel detector. The proposed method is able to detect and track the object with the single-pixel measurements directly without image reconstruction. The detection and tracking algorithm of the proposed method is computationally efficient. We experimentally demonstrate that the method can achieve a temporal resolution of 1,666 frames per second by using a 10,000 Hz digital micro-mirror device. The latency time of the method is on the order of microseconds. Additionally, the method acquires only 600 bytes of data for each frame. The method therefore allows fast moving object detection and tracking in real time and for long duration. This image-free approach might open up a new avenue for spatial information acquisition in a highly efficient manner.

The development of computer technology and computer vision has had a significant positive impact on the daily lives of blind people, especially in efforts to improve their navigation skills. This research aims to introduce a superior object detection method, especially to support the sustainability and effectiveness of blind navigation. The main focus of the research is the use of YOLOv8, the latest version of YOLO, as an object detection method and distance measurement technology from OpenCV. The main challenge to address involves improving object detection accuracy and performance, which is an important key to ensuring safe and effective navigation for blind people. In this context, blind people often face obstacles in their mobility, especially when walking in environments that may be full of obstacles or obstacles. Therefore, better object detection methods become essential to ensure the identification of nearby objects that may involve obstacles or potential threats, thus preventing possible accidents or difficulties in daily commuting. Involving YOLOv8 as an object detection method provides the advantage of a high level of accuracy, although with a slight increase in detection duration and GPU power consumption compared to previous versions. The research results show that YOLOv8 provides a low error rate, with an average error percentage of 3.15%, indicating very optimal results. Using a combined performance evaluation approach of YOLOv8 and OpenCV distance measurement metrics, this research not only seeks to improve accuracy but also efficiency in detection time and power consumption. This research makes an important contribution to the presentation of technological solutions that can help improve mobility and safety for blind people, bringing a real positive impact on the facilitation of their daily lives.

Deep Neural-Network (DNN) based Object Detection is one of the most important and time-consuming stages of Autonomous Driving software in cars. In non-critical domains, the performance and energy requirements of object detection can be reduced at the cost of accuracy in the detected objects. This is not the case in a critical domain like automotive, for which a delicate balance between performance/energy overheads and accuracy of object detection must be found. We propose IntPred to achieve such a balance by leveraging on the fact that, with high frame rates, objects do not move significantly across frames. IntPred tailors object interpolation for the case of object detection in autonomous driving frameworks, in line with approaches devised for other domains, thus heavily reducing the performance requirements of full-fledged DNN-based object prediction. IntPred results in comparable accuracy to the original object detection, while saving more than 70% of the computations. The latter allows using lower-performance and cheaper platforms resulting in saving energy and reducing heat dissipation: for instance, in an NVIDIA Jetson TX2 platform, specific for autonomous driving systems, our technique increases the frame processing rate by 4.6x. IntPred also allows consolidating additional applications onto the same platform.