Real-Time Object Detection and Tracking in Video Streams Using Deep Learning

Object detection and tracking are two critical tasks in computer vision, widely applied in security surveillance, autonomous vehicles, and behavioral analysis. Strong performance in object detection has been demonstrated by recent Transformer-based models, such as RT-DETR (Real-Time Detection Transformer), due to their global context modeling and high accuracy. However, an inherent tracking mechanism is lacking in RT-DETR, which requires additional components to maintain identity consistency across frames. To address this limitation, an integration of RT-DETR with DeepSORT is proposed, leveraging the strengths of both models to enhance real-time object detection and tracking. A comparative evaluation with YOLOv8, a widely used real-time detector, is conducted to highlight the advantages of the proposed approach in tracking accuracy and robustness. Experiments show that effective performance is achieved in challenging scenarios such as object occlusion and intersection. Specifically, an IDF1 score of 60.0%, a MOTA of 42.4%, and a MOTP of 43.3% are obtained by RTl+DeepSORT on the MOT17-02-DPM dataset, outperforming YOLOv8x+DeepSORT. These results indicate that significant improvements in tracking accuracy are attained while real-time efficiency is maintained, making the proposed approach well-suited for applications such as intelligent surveillance and autonomous navigation.

Real-time detection and tracking for fast moving object has important applications in various fields. However, available methods, especially low-cost ones, can hardly achieve real-time and long-duration object detection and tracking. Here we report an image-free and cost-effective method for detecting and tracking a fast moving object in real time and for long duration. The method employs a spatial light modulator and a single-pixel detector for data acquisition. It uses Fourier basis patterns to illuminate the target moving object and collects the resulting light signal with a single-pixel detector. The proposed method is able to detect and track the object with the single-pixel measurements directly without image reconstruction. The detection and tracking algorithm of the proposed method is computationally efficient. We experimentally demonstrate that the method can achieve a temporal resolution of 1,666 frames per second by using a 10,000 Hz digital micro-mirror device. The latency time of the method is on the order of microseconds. Additionally, the method acquires only 600 bytes of data for each frame. The method therefore allows fast moving object detection and tracking in real time and for long duration. This image-free approach might open up a new avenue for spatial information acquisition in a highly efficient manner.

Artificial vision in robotics involves real time detection of objects for fast decision making. Such intelligent systems require efficient algorithms and big learning database of examples for producing robust classifiers. Several methods of objects detection and tracking have been proposed in the literature. However, even though the detection rates have been improved, the processing time and the complexity of the models still representing a key challenge. In this paper, we present a real time object detection and tracking framework based on Adaboost classification, where a strong classifier is generated using an iterative combination of weak learners. This method is based on the use of discriminative features by analyzing different regions of the input image. Instead of performing a full traversal in the entire search space of all possible visual features, we propose to use intelligent heuristics for accelerating time processing and extracting relevant features in the image that lead to a best detection rate. The meta-heuristics involve the use of genetic algorithms, particle swarm optimization, random walk and a novel hybrid combination of these methods. The obtained results, in a case of intelligent transportation system, have shown considerable improvements in term of computation time, efficiency and accuracy.

Low-cost real-time aerial object detection and GPS location tracking pipeline

In real time object tracking is an important task in various surveillance application. Nowadays, surveillance system very common in offices, collage campus, shopping malls etc. In this paper, an automated video surveillance system presented. Althrough tracking and counting system are commercially available today; there is need for further reaserach to address the challenges of real world scenarios. There is lot of surveillance cameras installed around us but there are no means to monitor all of them continuously. It is necessary to develop a computer vision based technologies that automatically process those images in order to detect situations or unusual behavior. In order to solve the problem of poor real time tracking performance using convolutional neural network. This paper proposes a fast and accurate real time video detection and tracking algorithm. Keyword: - Video surveillance system, Moving object detection, Tracking, Background subtraction. I. Introduction Real time human body detection and tracking in indoor environment deals with the concept of higher level application of image processing and computer vision. The following key components of identify to path of real time tracking are extracting a feature, background image subtraction. If two similar color objects come to close the tracker of one object can jump to associated object. The problem is called hijacking problem. Drifting problem occur when an object abruptly changes its direction to reverse. In that case it’s become very difficult to track the object because the motion model does not work. An object tracking can be understood as the problem of finding the path (i.e. Trajectory).

We present FUS3D, a fast and lightweight system for real-time 3D object detection and tracking on edge devices. Our approach seamlessly integrates stages for 3D object detection and multi-object-tracking into a single, end-to-end trainable model. FUS3D is specially tuned for indoor 3D human behavior analysis, with target applications in Ambient Assisted Living (AAL) or surveillance. The system is optimized for inference on the edge, thus enabling sensor-near processing of potentially sensitive data. In addition, our system relies exclusively on the less privacy-intrusive 3D depth imaging modality, thus further highlighting the potential of our method for application in sensitive areas. FUS3D achieves best results when utilized in a joint detection and tracking configuration. Nevertheless, the proposed detection stage can function as a fast standalone object detection model if required. We have evaluated FUS3D extensively on the MIPT dataset and demonstrated its superior performance over comparable existing state-of-the-art methods in terms of 3D object detection, multi-object tracking, and, most importantly, runtime.

One of the most important problems in Computer Vision is the computation of the 2D projective transformation (homography) that maps features of planar objects in different images and videos. This computation is required by many applications such as image mosaicking, image registration, and augmented reality. The real-time performance imposes constraints on the methods used. In this paper, we address the real-time detection and tracking of planar objects in a video sequence where the object of interest is given by a reference image template. Most existing approaches for homography estimation are based on two steps: feature extraction (first step) followed by a combinatorial optimization method (second step) to match features between the reference template and the scene frame. This paper has two main contributions. First, for the detection part, we propose a feature point classification which is applied prior to performing the matching step in the process of homography calculation. Second, for the tracking part, we propose a fast method for the computation of the homography that is based on the transferred object features and their associated local rawbrightness. The advantage of this proposed scheme is a fast and accurate estimation of the homography.

In several applications of computer vision and image processing, the inception of the processing starts with object detection and subsequently tracking, if the need arises. In recent years, there has been extensive research in the field of object detection and tracking. Many remarkable algorithms have been developed for object detection and tracking, including color segmentation, edge tracking and many more. However, all these algorithms faced the limited success in their implementation in the real world and were also bounded by the constraints such as white/plain background. This paper is the result of our research where our research team developed and implemented object detection and tracking system operational in an unknown background, using real-time video processing and a single camera. The proposed system has been extensively tested to operate in complex, real world, non-plain, light variant, changing background.

The use of unmanned aerial vehicles (UAVs) that can operate autonomously in dynamic and dangerous operational environments are becoming increasingly common. In such operations, object detection, classification and tracking can often be one of the main goals. In recent years there has been an increased focus on embedded hardware that is both small and powerful, making UAV on-board data processing more viable. Being able to process the video feed on-board the UAV calls for fast and robust real-time algorithms for object identification and tracking. This paper discusses the development and implementation of a machine vision system for a low-cost fixed-wing UAV with a total flying weight of less than 4kg. The machine vision system incorporates the use of a thermal imaging camera and on-board processing power to perform real-time object detection, classification and tracking of objects in the ocean surface. The system is tested on thermal video data from a test flight, and is found to be able to detect 99;6% of objects of interest located in the ocean surface. Of the detected objects, only 5% were false positives. Furthermore, it classifies 93; 3% of the object types it is trained to classify correctly. The classifier is highly agile, allowing the user to quickly define which object characteristics that should be considered during classification, and what types of objects to classify. Finally, the system is found to successfully track 85% of the object types it is actively searching for in a real-time simulation test.

Real time and accrual object tracking is ultimate aim of a visual tracker. Recently mean shift object tracking algorithm is studied in an all-round way and utilized to visual tracking in both civil and military fields. Compared to the traditional mean shift algorithm and its other versions distance weight mean shift algorithm can effectively improve the precision and robustness. It analyzes the dissimilarity of target feature color histogram with distance weight and candidate area feature color histogram with distance weight to calculate mean shift vector, and searches for target area via each iteration in candidate area. Though improved on basis of traditional mean shift algorithm, there are a large amount of calculations and the processing speed has already been a bottleneck in real time object tracking. Graphics processor unit (GPU) has special hardware parallel computing resources and has been already employed in various embedded systems. It provides us with an alternative implementation on GPU which can solve such bottleneck problem. The paper proposes a way of implementation of distance weight mean shift algorithm on GPU. The experiment results show that the execution performance of the implementation on GPU is over 56 times speedup of that completely on CPU at most, and can meet the requirement of real time and accuracy.

To address the challenges of low accuracy and insufficient real-time performance in dynamic object detection for UAV surveillance, this paper proposes a novel tracking framework that integrates a lightweight improved YOLOv5s model with adaptive motion compensation. A UAV-view dynamic feature enhancement strategy is innovatively introduced, and a lightweight detection network combining attention mechanisms and multi-scale fusion is constructed. The robustness of tracking under motion blur scenarios is also optimized. Experimental results demonstrate that the proposed method achieves a mAP@0.5 of 68.2% on the VisDrone dataset and reaches an inference speed of 32 FPS on the NVIDIA Jetson TX2 platform. This significantly improves the balance between accuracy and efficiency in complex scenes, offering reliable technical support for real-time applications such as emergency response.

Reliable detection and tracking of moving objects is essential for indoor robotics and smart environments. This paper presents a real-time 2D object detection and tracking system that uses a fixed planar LiDAR sensor combined with an inertial measurement unit (IMU), all implemented on a Raspberry Pi. The LiDAR is set up in a stationary position, which allows for continuous monitoring of a specific area and helps separate moving objects from static background structures based on motion. Unlike mobile sensing systems, this approach does not require estimating ego-motion. Instead, the IMU identifies and reduces measurement errors caused by mechanical vibrations and slight changes in orientation that can impact fixed LiDAR setups. Moving objects are detected by analysing consecutive range scans over time, grouped spatially using lightweight clustering, and tracked in the 2D plane with an efficient state model. While the system can cater to human-centric applications, it does not impose any object-specific shape or learning-based assumptions. This flexibility allows the framework to apply to a variety of moving objects. Experimental results in indoor settings show stable real-time performance even with limited computational resources. These findings confirm that low-cost LiDAR and IMU sensing is suitable for general object detection and tracking.

This paper presents an investigation of the development of a real-time object tracking system that utilizes computer vision techniques and incorporates pan-tilt control mechanisms. The system architecture includes hardware components such as a camera and a pan-tilt mechanism, and software components such as object detection and tracking algorithms. To achieve efficient and accurate tracking, the Kalman filter, which uses a state-space model, was included in the object tracking process. The proposed system is implemented and evaluated through experiments to demonstrate its feasibility and effectiveness in real-world scenarios. The results indicate that the proposed system is capable of achieving real-time and accurate object tracking, demonstrating its potential for practical applications in various fields.

Real time object tracking is an essential part of surveillance and robot applications. The performance of any object tracking system depends on its accuracy and its ability to deal with various sizes of objects, for better results the tracking should happen at a high speed. Object tracking can be defined as the process of segmenting the region of interest from the video sequence and keeping track of the motion in order to extract useful information. The basic fundamentals of video surveillance such as object detection, tracking and recognition using multiple cameras. Whenever an object enters a video frame, it is detected using background subtraction and the detected objects are tracked with a novel Bayesian Kalman filter with simplified Gaussian mixture (BKF-SGM). Object recognition algorithm is applied using Gait method which is employed to support multiple object tracking with its robust performance. Processing of image is done in MATLAB to get detection, tracking and recognized results.

There is a real need to develop a sensory system that can monitor and track the behavioral status of a person in real time while protecting his/her privacy. In addition, it is important to have the ability to detect stationary and moving objects in real time to support dynamic navigation needs for mobile robots. This paper describes the development of intelligent fiber-grating (FG)-based 3-D vision sensory system that enables real-time object detection, monitoring, and tracking. This system uses the richness and the strength of the vision while reducing the data load and computational cost by encoding the working space using a limited number of spatially interrelated 2-D laser spots. The sensor system generates the laser spots by projecting a coherent laser beam on two perpendicularly overlaid layers of FGs. The presence of a target within the visible operational view of the sensor disturbs the part of the projected laser spots' pattern where the target appeared. By calculating the displacement of the disturbed spots within the CCD image plane, the sensor system can detect, in a short time, an object/human and provide relevant information in terms of range, approximate shape, and position. To facilitate faster detection and real-time tracking, the author has developed two strategies that decompose the working space into zones and virtual planes