A Real-Time Object Detection and Tracking Framework Based on RT-DETR and DeepSORT

Real-time object detection and tracking in video streams has become a fundamental component of modern intelligent systems, enabling applications in surveillance, autonomous driving, robotics, healthcare monitoring, and smart cities. The rapid advancement of deep learning techniques has significantly improved the accuracy and efficiency of detecting and tracking multiple objects under complex and dynamic environments. This study presents a deep learning–based framework for real-time object detection and tracking in continuous video streams, emphasizing both computational efficiency and detection robustness.The proposed system integrates a state-of-the-art convolutional neural network (CNN)-based object detector with a reliable multi-object tracking algorithm. The detection module identifies objects of interest in each frame, while the tracking component maintains consistent identities across consecutive frames by utilizing motion prediction and feature association strategies. To achieve real-time performance, the model is optimized through lightweight network architectures and parallel processing on GPU hardware. Data augmentation and transfer learning techniques are employed to enhance generalization and reduce training time. Performance evaluation is conducted on benchmark video datasets using standard metrics such as precision, recall, mean Average Precision (mAP), and tracking accuracy. Experimental results demonstrate that the proposed framework achieves high detection accuracy while maintaining low latency, making it suitable for real-time deployment. Furthermore, the system effectively handles challenges such as occlusion, scale variation, and illumination changes. This research presents a comprehensive study on real-time object detection and tracking in video streams using deep learning techniques. The proposed framework integrates YOLO-based object detection with Deep SORT multi-object tracking to achieve robust spatial localization and temporal identity preservation. The study evaluates performance using benchmark datasets including COCO, MOT17, and KITTI. Mathematical formulations of detection loss, Kalman filtering, and data association are presented. Experimental analysis using precision, recall, mAP, MOTA, ROC-AUC, and PR-AUC demonstrates that the system achieves high detection accuracy while maintaining real-time performance above 35 frames per second. The findings confirm that deep learning-based frameworks provide scalable and efficient solutions for surveillance, autonomous vehicles, and intelligent monitoring systems.

Artificial vision in robotics involves real time detection of objects for fast decision making. Such intelligent systems require efficient algorithms and big learning database of examples for producing robust classifiers. Several methods of objects detection and tracking have been proposed in the literature. However, even though the detection rates have been improved, the processing time and the complexity of the models still representing a key challenge. In this paper, we present a real time object detection and tracking framework based on Adaboost classification, where a strong classifier is generated using an iterative combination of weak learners. This method is based on the use of discriminative features by analyzing different regions of the input image. Instead of performing a full traversal in the entire search space of all possible visual features, we propose to use intelligent heuristics for accelerating time processing and extracting relevant features in the image that lead to a best detection rate. The meta-heuristics involve the use of genetic algorithms, particle swarm optimization, random walk and a novel hybrid combination of these methods. The obtained results, in a case of intelligent transportation system, have shown considerable improvements in term of computation time, efficiency and accuracy.

Real-time detection and tracking for fast moving object has important applications in various fields. However, available methods, especially low-cost ones, can hardly achieve real-time and long-duration object detection and tracking. Here we report an image-free and cost-effective method for detecting and tracking a fast moving object in real time and for long duration. The method employs a spatial light modulator and a single-pixel detector for data acquisition. It uses Fourier basis patterns to illuminate the target moving object and collects the resulting light signal with a single-pixel detector. The proposed method is able to detect and track the object with the single-pixel measurements directly without image reconstruction. The detection and tracking algorithm of the proposed method is computationally efficient. We experimentally demonstrate that the method can achieve a temporal resolution of 1,666 frames per second by using a 10,000 Hz digital micro-mirror device. The latency time of the method is on the order of microseconds. Additionally, the method acquires only 600 bytes of data for each frame. The method therefore allows fast moving object detection and tracking in real time and for long duration. This image-free approach might open up a new avenue for spatial information acquisition in a highly efficient manner.

To address the challenges of low accuracy and insufficient real-time performance in dynamic object detection for UAV surveillance, this paper proposes a novel tracking framework that integrates a lightweight improved YOLOv5s model with adaptive motion compensation. A UAV-view dynamic feature enhancement strategy is innovatively introduced, and a lightweight detection network combining attention mechanisms and multi-scale fusion is constructed. The robustness of tracking under motion blur scenarios is also optimized. Experimental results demonstrate that the proposed method achieves a mAP@0.5 of 68.2% on the VisDrone dataset and reaches an inference speed of 32 FPS on the NVIDIA Jetson TX2 platform. This significantly improves the balance between accuracy and efficiency in complex scenes, offering reliable technical support for real-time applications such as emergency response.

In several applications of computer vision and image processing, the inception of the processing starts with object detection and subsequently tracking, if the need arises. In recent years, there has been extensive research in the field of object detection and tracking. Many remarkable algorithms have been developed for object detection and tracking, including color segmentation, edge tracking and many more. However, all these algorithms faced the limited success in their implementation in the real world and were also bounded by the constraints such as white/plain background. This paper is the result of our research where our research team developed and implemented object detection and tracking system operational in an unknown background, using real-time video processing and a single camera. The proposed system has been extensively tested to operate in complex, real world, non-plain, light variant, changing background.

The use of unmanned aerial vehicles (UAVs) that can operate autonomously in dynamic and dangerous operational environments are becoming increasingly common. In such operations, object detection, classification and tracking can often be one of the main goals. In recent years there has been an increased focus on embedded hardware that is both small and powerful, making UAV on-board data processing more viable. Being able to process the video feed on-board the UAV calls for fast and robust real-time algorithms for object identification and tracking. This paper discusses the development and implementation of a machine vision system for a low-cost fixed-wing UAV with a total flying weight of less than 4kg. The machine vision system incorporates the use of a thermal imaging camera and on-board processing power to perform real-time object detection, classification and tracking of objects in the ocean surface. The system is tested on thermal video data from a test flight, and is found to be able to detect 99;6% of objects of interest located in the ocean surface. Of the detected objects, only 5% were false positives. Furthermore, it classifies 93; 3% of the object types it is trained to classify correctly. The classifier is highly agile, allowing the user to quickly define which object characteristics that should be considered during classification, and what types of objects to classify. Finally, the system is found to successfully track 85% of the object types it is actively searching for in a real-time simulation test.

One of the most important problems in Computer Vision is the computation of the 2D projective transformation (homography) that maps features of planar objects in different images and videos. This computation is required by many applications such as image mosaicking, image registration, and augmented reality. The real-time performance imposes constraints on the methods used. In this paper, we address the real-time detection and tracking of planar objects in a video sequence where the object of interest is given by a reference image template. Most existing approaches for homography estimation are based on two steps: feature extraction (first step) followed by a combinatorial optimization method (second step) to match features between the reference template and the scene frame. This paper has two main contributions. First, for the detection part, we propose a feature point classification which is applied prior to performing the matching step in the process of homography calculation. Second, for the tracking part, we propose a fast method for the computation of the homography that is based on the transferred object features and their associated local rawbrightness. The advantage of this proposed scheme is a fast and accurate estimation of the homography.

Object detection is a major branch and fundamental task in computer vision, aiming to localize, identify and classify even the smallest objects of interest in images. Features can be extracted efficiently by deep convolutional neural networks (CNNs) as the backbone for real-time or near real-time object detection performance than the hand-crafted-based traditional methods. In the past few years, the advent of transformer-based models with robust self-attention mechanisms has not only raised object detection performance to a higher level but has also enabled it to produce excellent results. Many object detection tasks in the real world require that 3D information about the object be obtained, thus strengthening active research in 3D object detection. However, the algorithms for detecting 3D objects are not easy to propagate in real-world applications due to many factors, making reconstruction of 2D object detection algorithms to 3D object detection algorithms the suitable alternative. Therefore, we review the evolution of 2D object detection algorithms for digital imaging applications, focusing on their developments, models, applications, datasets, evaluation metrics, strengths and weaknesses, for better understanding of their landmarks and contributions to the advancement of the field.

Low-cost real-time aerial object detection and GPS location tracking pipeline

This paper presents an investigation of the development of a real-time object tracking system that utilizes computer vision techniques and incorporates pan-tilt control mechanisms. The system architecture includes hardware components such as a camera and a pan-tilt mechanism, and software components such as object detection and tracking algorithms. To achieve efficient and accurate tracking, the Kalman filter, which uses a state-space model, was included in the object tracking process. The proposed system is implemented and evaluated through experiments to demonstrate its feasibility and effectiveness in real-world scenarios. The results indicate that the proposed system is capable of achieving real-time and accurate object tracking, demonstrating its potential for practical applications in various fields.

We present FUS3D, a fast and lightweight system for real-time 3D object detection and tracking on edge devices. Our approach seamlessly integrates stages for 3D object detection and multi-object-tracking into a single, end-to-end trainable model. FUS3D is specially tuned for indoor 3D human behavior analysis, with target applications in Ambient Assisted Living (AAL) or surveillance. The system is optimized for inference on the edge, thus enabling sensor-near processing of potentially sensitive data. In addition, our system relies exclusively on the less privacy-intrusive 3D depth imaging modality, thus further highlighting the potential of our method for application in sensitive areas. FUS3D achieves best results when utilized in a joint detection and tracking configuration. Nevertheless, the proposed detection stage can function as a fast standalone object detection model if required. We have evaluated FUS3D extensively on the MIPT dataset and demonstrated its superior performance over comparable existing state-of-the-art methods in terms of 3D object detection, multi-object tracking, and, most importantly, runtime.

Reliable detection and tracking of moving objects is essential for indoor robotics and smart environments. This paper presents a real-time 2D object detection and tracking system that uses a fixed planar LiDAR sensor combined with an inertial measurement unit (IMU), all implemented on a Raspberry Pi. The LiDAR is set up in a stationary position, which allows for continuous monitoring of a specific area and helps separate moving objects from static background structures based on motion. Unlike mobile sensing systems, this approach does not require estimating ego-motion. Instead, the IMU identifies and reduces measurement errors caused by mechanical vibrations and slight changes in orientation that can impact fixed LiDAR setups. Moving objects are detected by analysing consecutive range scans over time, grouped spatially using lightweight clustering, and tracked in the 2D plane with an efficient state model. While the system can cater to human-centric applications, it does not impose any object-specific shape or learning-based assumptions. This flexibility allows the framework to apply to a variety of moving objects. Experimental results in indoor settings show stable real-time performance even with limited computational resources. These findings confirm that low-cost LiDAR and IMU sensing is suitable for general object detection and tracking.

The real-time detection of objects and persons in their approach remains vital for applications that need safety measures security protocols and efficiency in autonomous systems and surveillance operations and industrial automation. The proposed research creates a durable real-time detection system through sensor combination using ultrasonic along with infrared and LiDAR and radar sensors and computer vision-based sensors to enhance detection accuracy and response times in changing environmental scenarios. The method starts by processing data gathered from arrays of multiple sensors through optimized filtering techniques and extraction methods before machines learn to classify objects while reducing false detection instances. The assessment of performance takes place in different environmental conditions to track both detection speed along with accuracy levels and system reliability. Sensor fusion enables real-time detection systems to perform better because it reduces the impact of environmental changes and sensor distortions. System capabilities increase in different operational areas when researchers demonstrate pre-processing optimization works alongside intelligent data processing systems. The research findings combine autonomous navigation systems with smart surveillance technology together with safety protocols for industrial security which enable real-time scalability and reduced risk elements. Research demonstrates a specific achievement through its analysis that machines learn better when combined with multi-sensor integration over single-sensor models which improves system reliability when used in real-world applications. Multi-sensor arrays and advanced processing techniques generate enhanced system responsiveness alongside increased adaptability and reliability thus improving detection systems within industrial security measures and automation operations.

In this paper, we have addressed the problem of real-time detection and tracking of dynamic objects using quadrotors. We have developed a novel object detection algorithm by analyzing and matching the color and spacial features of the target from monocular image sequences. The proposed object detection algorithm can track the objects with high Frame Per Second (FPS) which is suitable for low-end onboard computers that are used in quad-rotors. In addition, we also estimate the position of the target object in real world so that the drone can track the object accurately. A rigorous experimental analysis is provided to show the efficacy of the proposed approach in indoor as well as outdoor environments.

In recent years, Convolutional Neural Networks (CNNs) have repeatedly shown state-of-the-art performance for their accuracy in the task of object detection, but their heavy computational costs impede their ability for real-time detection when the supporting system is moving, particulary when it is accelerating. At the same time, recent progress on visual inertial systems takes great advantage of movement information to robustly estimate the robot state and its surrounding. This paper proposes to exploit the advantages of inertial odometry research for the purpose of real-time object detection system on mobile robots. We combine a CNN detector with VINS-Mono, a moving visual odometry system, and show reliable improvement in the detection process, especially when the robot accelerates or decelerates. Our system is ready-to-use in that it has very low deployment cost and requires no calibration. The resulting system allows for simultaneous robot state estimation and object detection, as well as object tracking. Lastly, this architecture proves to be flexible because not restrained to a specific object type or detector.