Advancements and challenges in multiobject tracking: a comprehensive survey on detection and association techniques

With the development of smart cities, video surveillance has become more prevalent in urban areas. The rapid growth of data brings challenges to video processing and analysis. Multi-object tracking (MOT), one of the most fundamental tasks in computer vision, has a wide range of applications and development prospects. MOT aims to locate multiple objects and maintain their unique identities by analyzing the video frame by frame. Most existing MOT frameworks are deployed in centralized systems, which are convenient for management but have problems such as weak algorithm adaptability, limited system scalability, and poor data security. In this paper, we propose a distributed MOT algorithm based on multi-agent reinforcement learning (DMARL-Tracker), which formulates MOT as a Markov decision process (MDP). Each object adjusts its tracking strategy during interactions with the environment. The benchmark results on MOT17 and MOT20 prove that our proposed algorithm achieves state-of-the-art (SOTA) performance. Based on this, we further integrate DMARL-Tracker into the blockchain and propose a blockchain-based collaborative MOT framework. All nodes collaborate and share information through the blockchain, achieving adaptation in different complex scenarios while ensuring data security. The simulation results show that our framework achieves good performance in terms of tracking and resource consumption.

The ability of predicting future frames in video sequences, known as video prediction, is an appealing yet challenging task in computer vision. This task requires an in-depth representation of video sequences and a deep understanding of real-word causal rules. Existing approaches for tackling the video prediction problem can be classified into two categories: deterministic and stochastic methods. Deterministic methods lack the ability of generating possible future frames and often yield blurry predictions. On the other hand, although current stochastic approaches can predict possible future frames, their models lack the ability of action control in the sense that they cannot generate the desired future frames conditioned on a specific action. In this paper, we propose new generative adversarial networks (GANs) for stochastic video prediction. Our framework, called VPGAN, employs an adversarial inference model and a cycle-consistency loss function to empower the framework to obtain more accurate predictions. In addition, we incorporate a conformal mapping network structure into VPGAN to enable action control for generating desirable future frames. In this way, VPGAN is able to produce fake videos of an object moving along a specific direction. Experimental results show that a combination of VPGAN and pre-trained image segmentation models outperforms existing stochastic video prediction methods.

The ability of predicting future frames in video sequences, known as video prediction, is an appealing yet challenging task in computer vision. This task requires an in-depth representation of video sequences and a deep understanding of real-word causal rules. Existing approaches for tackling the video prediction problem can be classified into two categories: deterministic and stochastic methods. Deterministic methods lack the ability of generating possible future frames and often yield blurry predictions. On the other hand, although current stochastic approaches can predict possible future frames, their models lack the ability of action control in the sense that they cannot generate the desired future frames conditioned on a specific action. In this paper, we propose new generative adversarial networks (GANs) for stochastic video prediction. Our framework, called VPGAN, employs an adversarial inference model and a cycle-consistency loss function to empower the framework to obtain more accurate predictions. In addition, we incorporate a conformal mapping network structure into VPGAN to enable action control for generating desirable future frames. In this way, VPGAN is able to produce fake videos of an object moving along a specific direction. Experimental results show that the combination of VPGAN with a pre-trained image segmentation model outperforms existing stochastic video prediction methods.

As the core technologies represented by Internet of Things (IoT) technology and computer vision technology booming, object detection algorithms based on deep learning have received high attention in various fields. Object detection is not only a core problem in the field of computer vision, but also a prerequisite and basis for many computer vision tasks, and has important applications in the fields of autonomous driving and video surveillance. The task of object detection is to find all the targets of interest in an image and determine their category and location. Object detection includes two tasks: object classification and object localization. However, object classification focuses on the most discriminative part of the feature map, and object localization requires a feature map focusing on the whole region of the object. Therefore, in order to improve the detection accuracy, we propose a CBAM-RetinaNet network model.CBAM (Convolutional Block Attention Module) is a lightweight convolutional attention module, which combines channel and spatial attention mechanism modules. The channel attention module focuses on the meaningful information in the input image, which would lead to an increase in the accuracy of object classification. The spatial attention module focuses on the location information of the object, which may improve the accuracy of object localization. The experiment results show that the proposed achieves better object detection performance on the PASCAL VOC2007 dataset, and the accuracy is 2% higher than that of the RetinaNet model, which has good experimental results.

Object detection and tracking are two critical tasks in computer vision, widely applied in security surveillance, autonomous vehicles, and behavioral analysis. Strong performance in object detection has been demonstrated by recent Transformer-based models, such as RT-DETR (Real-Time Detection Transformer), due to their global context modeling and high accuracy. However, an inherent tracking mechanism is lacking in RT-DETR, which requires additional components to maintain identity consistency across frames. To address this limitation, an integration of RT-DETR with DeepSORT is proposed, leveraging the strengths of both models to enhance real-time object detection and tracking. A comparative evaluation with YOLOv8, a widely used real-time detector, is conducted to highlight the advantages of the proposed approach in tracking accuracy and robustness. Experiments show that effective performance is achieved in challenging scenarios such as object occlusion and intersection. Specifically, an IDF1 score of 60.0%, a MOTA of 42.4%, and a MOTP of 43.3% are obtained by RTl+DeepSORT on the MOT17-02-DPM dataset, outperforming YOLOv8x+DeepSORT. These results indicate that significant improvements in tracking accuracy are attained while real-time efficiency is maintained, making the proposed approach well-suited for applications such as intelligent surveillance and autonomous navigation.

MOTT: A new model for multi-object tracking based on green learning paradigm

CFGVF: An improved correlation filters based visual tracking algorithm

Drones, or general UAVs, equipped with cameras have been fast deployed with a wide range of applications, including agriculture, aerial photography, and surveillance. Consequently, automatic understanding of visual data collected from drones becomes highly demanding, bringing computer vision and drones more and more closely. To promote and track the developments of object detection and tracking algorithms, we have organized three challenge workshops in conjunction with ECCV 2018, ICCV 2019 and ECCV 2020, attracting more than 100 teams around the world. We provide a large-scale drone captured dataset, VisDrone, which includes four tracks, i.e., (1) image object detection, (2) video object detection, (3) single object tracking, and (4) multi-object tracking. In this paper, we first present a thorough review of object detection and tracking datasets and benchmarks, and discuss the challenges of collecting large-scale drone-based object detection and tracking datasets with fully manual annotations. After that, we describe our VisDrone dataset, which is captured over various urban/suburban areas of 14 different cities across China from North to South. Being the largest such dataset ever published, VisDrone enables extensive evaluation and investigation of visual analysis algorithms for the drone platform. We provide a detailed analysis of the current state of the field of large-scale object detection and tracking on drones, and conclude the challenge as well as propose future directions. We expect the benchmark largely boost the research and development in video analysis on drone platforms. All the datasets and experimental results can be downloaded from https://github.com/VisDrone/VisDrone-Dataset.

Recently real-time detection, and recognition of an object of interest are becoming vital tasks in visual data processing and computer vision. Various models have been deployed to implement object detection and tracking in multiple fields. However, conventional classifiers are often faced with challenging tasks that visual frames come distorted due to overlapping, camera motion blur, changing subject appearances, and environmental variations. Models using OpenCV-based HAAR feature-based cascade classifiers, without integrating any error minimizing object detection algorithm, were unable to accurately detect an object and track it in a changing environment. Therefore, developing an embedded powerful framework for realtime object detection and recognition becomes more of a vital need for future implementation in various fields. This study presents a powerful technique for a real-time detector that utilizes integrated Deep Learning Neural Networks (DNN) for optimal computational accuracy. Deploying such a framework will ensure the flexibility and reliability of the detector by eliminating the sources of distortion previously mentioned. The model relies on integrating the ImageAI deep learning libraries and You Only Look Once (YOLO-v3) object detection method with a DarkNet53 architecture. The algorithm was trained using the TensorFlow framework to ensure accurate data processing. This paper targets one vital component of our long-term project of developing a multi-agent system, as the proposed model is to be implemented in autonomous agents for the detection of landmines, ocean debris, and wildlife beside environmental scanning missions. In this study the performance of the model has been assessed through detecting and collecting tennis balls as a preliminary test for real-world applications. The model was able to approach the desirable result of surpassing the accuracy of conventional detectors.

Algorithmic solutions for multi-object tracking (MOT) are a key enabler for applications in autonomous navigation and applied ocean sciences. State-of-the-art MOT methods fully rely on a statistical model and typically use preprocessed sensor data as measurements. In particular, measurements are produced by a detector that extracts potential object locations from the raw sensor data collected at discrete time steps. This preparatory processing step reduces data flow and computational complexity but may result in a loss of information. State-of-the-art Bayesian MOT methods that are based on belief propagation (BP) systematically exploit graph structures of the statistical model to reduce computational complexity and improve scalability. However, as a fully model-based approach, BP can provide highly suboptimal estimates when there is a mismatch between the statistical model and the true data-generating process. Existing BP-based MOT methods can further only make use of preprocessed measurements. In this paper, we introduce a variant of BP that combines model-based with data-driven MOT. The proposed neural enhanced belief propagation (NEBP) method complements the statistical model of BP by information learned from raw sensor data. This approach conjectures that the learned information can reduce model mismatch and thus improve data association and false alarm rejection. Our NEBP method improves tracking performance compared to model-based methods. At the same time, it inherits the advantages of BP-based MOT, i.e., it scales only quadratically in the number of objects, and it can thus generate and maintain a large number of object tracks. We evaluate the performance of our NEBP approach for MOT on the nuScenes autonomous driving dataset and demonstrate that it can achieve state-of-the-art performance. In particular, an average multi-object tracking accuracy (AMOTA) of 0.683 was obtained and, compared with non-BP-based methods, identity switches (IDS) and track fragments (Frag) were reduced by 23% and 19%, respectively.

Basic tasks in computer vision, object detection and localization have many applications in autonomous driving, robotics, surveillance, and healthcare [1]. Deep learning has enabled previously unheard-of levels of accuracy and robustness in object detection and localization tasks, revolutionizing the area in recent years. This paper provides a thorough overview of the most recent developments and approaches in the application of deep learning algorithms for localization and object detection. First, we go over the basic ideas behind object identification and localization, emphasizing the difficulties caused by different item sizes, occlusions, and cluttered backgrounds. We then explore the nuances of deep learning architectures designed specifically for these kinds of tasks: anchor-free approaches like Center Net, single-shot detectors like YOLO, and region-based detectors like Faster R-CNN. We investigate feature extraction networks, object detecting heads, and training procedures for best performance as we investigate the underlying mechanics of these architectures. We also go over important assessment measures and datasets that are necessary for benchmarking and training object detection algorithms. We stress the significance of using strong evaluation techniques to guarantee the dependability and capacity for generalization of the suggested algorithms. Apart from technical concerns, we investigate practical uses cases where object detection and localization are essential, like autonomous vehicle pedestrian identification, surveillance system object tracking, and medical imaging anomaly detection. Lastly, we examine new developments and directions for the discipline, such as integrating multimodal data, handling domain shifts, and improving object detection algorithms' interpretability and fairness. Our goal is to stimulate further developments in computer vision systems by offering scholars, professionals, and hobbyists insightful knowledge about the problems and cutting-edge methods of object recognition and localization through this thorough review. The content of the paper is succinctly summarized in this abstract, which also highlights the significance of deep learning, object detection and localization, important methodology, datasets, applications, and future developments in the field.

Object detection is a fundamental task in computer vision that involves identifying and localizing objects within images or video frames. This research focuses on implementing and evaluating the YOLOv5 (You Only Look Once version 5) model for real-time object detection. YOLOv5 is known for its efficiency, accuracy, and speed, making it a preferred choice for various applications such as autonomous driving, surveillance, and medical imaging. In this study, we explore the architecture, training process, and performance evaluation of YOLOv5 on benchmark datasets. The results demonstrate that YOLOv5 achieves high precision and recall, outperforming traditional object detection methods in terms of speed and accuracy. Keywords: Object Detection, YOLOv5, Deep Learning, Computer Vision, Real-Time Processing, Neural Networks.

It is essential for them to accurately and efficiently estimate both their own pose and the poses of surrounding dynamic objects in real time during autonomous exploration. Although most existing approaches achieve accurate self-localization or object detection and tracking individually, there is limited research on effectively integrating these two tasks, despite their inherently coupled and interactive nature. To address this challenge, this paper proposes a multi-object tracking and trajectory optimization framework based on enhanced data association and a two-stage optimization strategy, enabling robust localization of both the ego-agent and surrounding dynamic objects in highly dynamic urban environments. In the front-end, we incorporate an improved data association algorithm into a multi-object tracking pipeline and combine it with a GICP-based fast registration method to achieve efficient dynamic tracking. In the back-end, based on the front-end tracking results, we model the states of moving objects as dynamic tracking factors and integrate them into a global factor graph. This allows for joint optimization of the ego-pose and the poses of dynamic objects, resulting in high-precision multi-object tracking. Extensive comparative experiments on public high-dynamic datasets demonstrate the state-of-the-art performance and advantages of the proposed method in multi-object tracking and self-localization, where the accuracy of self-localization and tracking has increased by 19.7% and 3%, respectively, compared with the most advanced method.

Object detection and recognition is an essential task in computer vision with numerous real-world applications such as surveillance, self-driving cars, and robotics. In recent years, deep learning-based approaches have significantly improved the accuracy and speed of object detection and recognition. The You Only Look Once version 3 (YOLOv3) algorithm is a popular deep learning-based approach that can detect and recognize objects in real-time. The Common Objects in Context (COCO) dataset is a large-scale dataset with over 330,000 labeled images and more than 2.5 million object instances, making it a popular choice for object detection and recognition tasks. In this paper, we propose a deep learning-based approach for real-time object detection and recognition using the YOLOv3 architecture and COCO dataset. We evaluate our approach based on several performance metrics, including mean average precision (mAP), frames per second (FPS), total object detection time, object detection accuracy, false positive rate, number of detected objects, and mean intersection over union (mIoU). Our results show that our approach achieves a mean average precision of 0.76 on the COCO dataset and a real-time performance of 40 frames per second on a single GPU. Additionally, our approach achieves an object detection accuracy of 93.5%, a false positive rate of 6.5%, and a mean intersection over union of 0.65. Our proposed approach shows promising results for real-time object detection and recognition and can be applied to various real-world applications.

Object detection and tracking is one of the most important and challenging branches in computer vision, and have been widely applied in various fields, such as health-care monitoring, autonomous driving, anomaly detection, and so on. With the rapid development of deep learning (DL) networks and GPU’s computing power, the performance of object detectors and trackers has been greatly improved. To understand the main development status of object detection and tracking pipeline thoroughly, in this survey, we have critically analyzed the existing DL network-based methods of object detection and tracking and described various benchmark datasets. This includes the recent development in granulated DL models. Primarily, we have provided a comprehensive overview of a variety of both generic object detection and specific object detection models. We have enlisted various comparative results for obtaining the best detector, tracker, and their combination. Moreover, we have listed the traditional and new applications of object detection and tracking showing its developmental trends. Finally, challenging issues, including the relevance of granular computing, in the said domain are elaborated as a future scope of research, together with some concerns. An extensive bibliography is also provided.