SKT-MOT and DyTracker: A Multiobject Tracking Dataset and a Dynamic Tracker for Speed Skating Video

To adjust management practices and improve wine marketing strategies, accurate vineyard yield estimation early in the growing season is essential. Conventional methods for yield forecasting rely on phenotypic features' manual assessment, which is time- and labor-intensive and often destructive. We combined a deep object segmentation method, mask region-based convolutional neural network (Mask R-CNN), with two potential multi-object tracking algorithms, simple online and real-time tracking (SORT) and intersection-over-union (IOU) trackers to develop a complete visual system that can automatically detect and track individual inflorescences, enabling the assessment of the number of inflorescences per vineyard row from vineyard video footage. The performance of the two tracking algorithms was evaluated using our vineyard dataset, which is more challenging than conventional tracking benchmark datasets owing to environmental factors. Our evaluation dataset consists of videos of four vineyard rows, including 221 vines that were automatically acquired under unprepared field conditions. We tracked individual inflorescences across video image frames with a 92.1% multi-object tracking accuracy (MOTA) and an 89.6% identity F1 score (IDF1). This allowed us to estimate inflorescence count per vineyard row with a 0.91 coefficient of determination (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) between the estimated count and manual-annotated ground truth count. The impact of leaf occlusions on inflorescence visibility was lessened by processing multiple successive image frames with minimal displacements to construct multiple camera views. This study demonstrates the use of deep learning and multi-object tracking in creating a low-cost (requiring only an RGB camera), high-throughput phenotyping system for precision viticulture.

Traditional Multiple Object Tracking (MOT) methods in satellite videos mostly follow the Detection-Based Tracking (DBT) framework. However, the DBT framework assumes that all objects are correctly recognized and localized by the detector. In practice, the low resolution of satellite videos, small objects, and complex backgrounds inevitably leads to a decline in detector performance. To alleviate the impact of detector degradation on track, we propose Coarse-Fine Tracker, a framework that integrates the MOT framework with the Tracking Any Point (TAP) method CoTracker for the first time, leveraging TAP’s persistent point correspondence modeling to compensate for detector failures. In our Coarse-Fine Tracker, we divide the satellite video into sub-videos. For one sub-video, we first use ByteTrack to track the outputs of the detector, referred to as coarse tracking, which involves the Kalman filter and box-level motion features. Given the small size of objects in satellite videos, we treat each object as a point to be tracked. We then use CoTracker to track the center point of each object, referred to as fine tracking, by calculating the appearance feature similarity between each point and its neighboring points. Finally, the Consensus Fusion Strategy eliminates mismatched detections in coarse tracking results by checking their geometric consistency against fine tracking results and recovers missed objects via linear interpolation or linear fitting. This method is validated on the VISO and SAT-MTB datasets. Experimental results in VISO show that the tracker achieves a multi-object tracking accuracy (MOTA) of 66.9, a multi-object tracking precision (MOTP) of 64.1, and an IDF1 score of 77.8, surpassing the detector-only baseline by 11.1% in MOTA while reducing ID switches by 139. Comparative experiments with ByteTrack demonstrate the robustness of our tracking method when the performance of the detector deteriorates.

Multi-object tracking (MOT) in satellite videos is a challenging task due to the small size and blurry features of objects, which often lead to intermittent detection and tracking instability. Many existing object detection and tracking models often struggle with these issues, as they are not designed to effectively handle the unique characteristics of satellite videos. To address these challenges, we propose LocaLock, a joint detection and tracking framework for MOT that incorporates feature matching concepts from single object tracking (SOT) to enhance tracking stability and reduce intermittent tracking results. Specifically, LocaLock utilizes an anchor-free detection backbone for efficiency and employs a local cost volume (LCV) module to perform precise feature matching in the local area. This provides valuable object priors to the detection head, enabling the model to “lock” onto objects with greater accuracy and mitigate the instability associated with small object detection. Additionally, the local computation within the LCV module ensures low computational complexity and memory usage. Furthermore, LocaLock incorporates a novel motion flow (MoF) module to accumulate and exploit temporal information, further enhancing feature robustness and consistency across frames. Rigorous evaluations on the VISO dataset demonstrate the superior performance of LocaLock, surpassing existing methods in tracking accuracy and precision within the demanding satellite video analysis domain. Notably, LocaLock achieved state-of-the-art performance on the VISO benchmark, achieving a multi-object tracking accuracy (MOTA) of 62.6 while ensuring fast running speed.

Aiming at the problems of ID switching and tracking performance degradation caused by frequent occlusion and similar appearance of the tracked objects in dense scenes, a multi-object tracking method named TPFairMOT based on trajectory prediction and FairMOT is proposed in this paper. In the trajectory prediction branch, the object position of the future frame is predicted by using the object bounding box of the past frame and the velocity information learning network parameters, which overcomes the prediction failure caused by the uncertain motion state after the object is occluded in the tracking process. Secondly, the joint learning framework is used to combine the trajectory prediction branch with the detection and re-identification branch, and the tracking error caused by the high similarity between multiple objects in the tracking process is solved by integrating the appearance features and motion features of the tracked objects. Finally, MOTChallenge benchmarks (IDF1, IDs, MOTA, MT, and ML) are introduced to evaluate TPFairMOT, and different trajectory prediction strategies (FairMOT_KF and TPFairMOT_RNN) are used on FairMOT and TPFairMOT for comparative analysis. It is proved that the accuracy and ID switching times of trajectory prediction in this paper are better than other strategies. In addition, TPFairMOT, TPFairMOT_RNN, and FairMOT were compared on the public data sets MOT16, MOT17, and MOT20. The results show that TPFairMOT reduces the number of ID switching when the object is occluded, maintains the long-term validity of the identity information, and demonstrate good anti-occlusion performance.

Multi-object tracking (MOT) is crucial for edge-enabled industrial video surveillance. Many video streams are delivered to the edge via limited, dynamic communication channels to be processed efficiently for accurate, on-time surveillance responses. However, time-varying communication conditions and constrained edge computational resources challenge surveillance accuracy and latency. This paper proposes an edge-based industrial video surveillance framework with bandwidth adaptation and edge-end collaboration that operates under limited communication conditions. Besides, a communication-aware industrial video surveillance with edge-end collaboration method (CIVS) is devised to improve surveillance accuracy and latency performance across different bandwidth environments. Furthermore, an NP-hard integer non-linear problem is formulated to maximize the accuracy while minimizing the latency of MOT by configuring the matches between an industrial video network and an edge computing network, which is solved by a branch and bound algorithm. Simulations show CIVS achieves 0.5892s latency and 68.9% multi-object tracking accuracy, outperforming other state-of-the-art.

Individual pig tracking is key to stepping away from group-level treatment and towards individual pig care. By doing so we can monitor individual pig behaviour changes over time and use these as indicators of health and well-being, which, in turn, will assist in the early detection of disease allowing for earlier and more effective intervention. However, it is a much more computationally challenging than performing this task at group level; mistakes in identification and tracking accumulate and, over time, provide noise measures. We combine a deep CNN object localisation method, Faster Region-based convolutional neural network (R-CNN), with two potential real-time multi-object tracking methods in order to create a complete system that can autonomously localise and track individual pigs allowing for the extraction of metrics pertaining to individual pig behaviours from RGB cameras. We evaluate two different transfer learning strategies to adapt Faster R-CNN to our pig detection dataset that is more challenging than conventional tracking benchmark datasets. We are able to localise pigs in individual frames with 0.901 mean average precision (mAP), which then allows us to track individual pigs across video footage with 92% Multi-Object Tracking Accuracy (MOTA) and 73.4% Identity F1-Score (IDF1), and re-identify them after occlusions and dropped frames with 0.862 mAP (0.788 Rank 1 cumulative matching characteristic (CMC)). From these tracks we extract individual behavioural metrics for total distance travelled, time spent idle, and average speed with less than 0.015 mean squared error (MSE) for each. Changes in all these behavioural metrics have value in the detection of pig health and wellbeing.

Multi-target tracking is a difficult task in radar data processing field. To track multiple targets with the video data collected by a X-band surface surveillance radar, a two-level cascade detection algorithm is designed at first. We combine the motion model and appearance model of the targets together for multi-target tracking and propose a multi-target tracking algorithm based on scale adaptive kernelized correlation filter (SAKCF) and Kalman filter (KF) in this paper. Firstly, the extended targets are detected by the designed tow-level cascade detection algorithm and the centroids of the extended targets are extracted by centroid algorithm too. Then, a SAKCF tracker is assigned to each target. KF is utilized to predict the position of each target and each SAKCF tracker searches the region near the predicted position to determine the real position and scale of each target. The association between tracks and trace points is performed based on Hungary algorithm. Finally, the abnormal tracks are deleted. Experimental results based on the radar video data show that compared with the classical simple online real-time target tracking (SORT) algorithm and joint probabilistic data association (JPDA) algorithm, the proposed algorithm improves multi-object tracking precision (MOTP) and multi-object tracking accuracy (MOTA).

This paper describes a video surveillance system capable of recognizing human postures from video sequences. The system comprises of two key modules: human detection and posture classification. In the module of human detection, human blobs are extracted by the technique of background subtraction. An adaptive background model is employed to characterize the dynamics and complexity of outdoor scenes based on the mixture of Gaussians. In order to formulate the variations of human postures, pseudo 2D hidden Markov models (P2DHMM) is employed for representing and recognizing human postures based on its '2-D elastic matching' property. It is trained to differentiate human postures and tolerate the variations of the same human posture using embedded Viterbi and segmental K means algorithms. In the classification of human postures, observation sequence is extracted from current image frame. The probabilities of observation sequence corresponding to each P2DHMM model are computed by doubly embedded Viterbi optimization, and human blob is classified as the human posture with the highest likelihood.

Multi-object tracking (MOT) is a crucial task in computer vision, enabling the tracking of multiple objects across frames in video sequences while preserving their identities. This problem is fundamental in applications like autonomous driving, video surveillance, and robotic systems. This paper provides a concise overview of the current state of MOT, exploring key advancements and methods developed over recent years. We focus on the widely adopted tracking-by-detection paradigm, where object detection is followed by the association of detections across frames. Modern MOT systems, such as ByteTrack, BoT-SORT, and the integration of advanced deep learning models like YOLO, have significantly improved tracking performance by addressing challenges like occlusion, motion blur, and identity switching. Additionally, the integration of real-time systems, such as UAV-based MOT, utilizing big data technologies, highlights the practical scalability of these methods. We present a survey of these systems, examine their limitations, and suggest potential directions for future research.

Detector–tracker integration framework and attention mechanism for multi–object tracking

Algorithmic solutions for multi-object tracking (MOT) are a key enabler for applications in autonomous navigation and applied ocean sciences. State-of-the-art MOT methods fully rely on a statistical model and typically use preprocessed sensor data as measurements. In particular, measurements are produced by a detector that extracts potential object locations from the raw sensor data collected at discrete time steps. This preparatory processing step reduces data flow and computational complexity but may result in a loss of information. State-of-the-art Bayesian MOT methods that are based on belief propagation (BP) systematically exploit graph structures of the statistical model to reduce computational complexity and improve scalability. However, as a fully model-based approach, BP can provide highly suboptimal estimates when there is a mismatch between the statistical model and the true data-generating process. Existing BP-based MOT methods can further only make use of preprocessed measurements. In this paper, we introduce a variant of BP that combines model-based with data-driven MOT. The proposed neural enhanced belief propagation (NEBP) method complements the statistical model of BP by information learned from raw sensor data. This approach conjectures that the learned information can reduce model mismatch and thus improve data association and false alarm rejection. Our NEBP method improves tracking performance compared to model-based methods. At the same time, it inherits the advantages of BP-based MOT, i.e., it scales only quadratically in the number of objects, and it can thus generate and maintain a large number of object tracks. We evaluate the performance of our NEBP approach for MOT on the nuScenes autonomous driving dataset and demonstrate that it can achieve state-of-the-art performance. In particular, an average multi-object tracking accuracy (AMOTA) of 0.683 was obtained and, compared with non-BP-based methods, identity switches (IDS) and track fragments (Frag) were reduced by 23% and 19%, respectively.

DCTracker: Rethinking MOT in soccer events under dual views via cascade association

Appearance based multi-object tracking (MOT) is a challenging task, specially in complex scenes where objects have similar appearance or are occluded by background or other objects. Such factors motivate researchers to propose effective trackers which should satisfy real-time processing and object trajectory recovery criteria. In order to handle both mentioned requirements, we propose a robust online multi-object tracking method that extends the features and methods proposed for re-identification to MOT. The proposed tracker combines a local and a global tracker in a comprehensive two-step framework. In the local tracking step, we use the frame-to-frame association to generate online object trajectories. Each object trajectory is called tracklet and is represented by a set of multi-modal feature distributions modeled by GMMs. In the global tracking step, occlusions and mis-detections are recovered by tracklet bipartite association method based on learning Mahalanobis metric between GMM components using KISSME metric learning algorithm. Experiments on two public datasets show that our tracker performs well when compared to state-of-the-art tracking algorithms.

This paper presents a high-accuracy and robust multi-object tracking method for maritime vessel detection and tracking in complex marine environments, characterized by dense targets, large-scale variations, and frequent occlusions. The proposed approach adopts an enhanced YOLOv8-based detector with lightweight feature enhancement and attention mechanisms to improve its capability in detecting small-scale vessels and complex scenes. Furthermore, a tracking framework combining BOTSORT with an OSNet-based re-identification (ReID) model is employed to achieve stable and reliable vessel association. Experimental results on the Near-Infrared On-Shore (NIR) dataset demonstrate that the proposed method improves Precision, Recall, mAP@0.5, and mAP@0.5:0.95 by approximately 4.0%, 5.0%, 5.1%, and 5.4%, respectively, compared with the baseline YOLOv8, while reducing parameter count and model size by 11.6% and 6.5%. On the Visible On-Shore (VIS) dataset, the proposed method outperforms state-of-the-art approaches in detection accuracy and robustness, further validating its effectiveness and cross-modal generalization capability. In multi-object tracking tasks, the proposed CPM-YOLO and BOTSORT framework demonstrates clear advantages in trajectory continuity, occlusion handling, and identity preservation compared with mainstream tracking algorithms. On the NIR dataset, the proposed method achieves a competitive inference speed of 188 FPS, while running at 187 FPS on the VIS dataset, demonstrating that the accuracy improvements are achieved without sacrificing real-time performance. Overall, the proposed method achieves a favorable balance between detection accuracy, tracking robustness, and model efficiency, making it well-suited for practical maritime applications.

Multi-object tracking in unmanned aerial vehicle (UAV) scenes is a crucial task with numerous applications across various domains. The goal of this task is to track multiple objects such as people and vehicles over time as they move through the scene, often captured by cameras mounted on UAV. However, tracking objects in UAV scenes can be challenging due to several factors, including the scale variance of objects as they move through the scene and the frequent occlusions caused by complex scenes. To address these issues, we propose a Global-Local and Occlusion Awareness (GLOA) tracking network for unmanned aerial vehicles. It comprises two main components: global-local awareness detector (GLA-D) and the occlusion awareness data association (OADA). The GLA-D uses our specially designed global-local awareness block to extract scale variance feature information from the input frames. It then outputs more discriminative identity information by adding identity embedding branches to the prediction head. The GLA-D is designed to better handle scale variance issues and improve object tracking accuracy. The OADA method used different metrics for high and low scoring detection frames was to alleviate occlusion problems in tracking scenarios. By combining these two components, the GLOA provides a more robust and effective solution for multi-object tracking in UAV scenes. Experiments on two public datasets may indicate the effectiveness of the proposed method.