Exploiting Hierarchical Dense Structures on Hypergraphs for Multi-Object Tracking

The multi-object tracking (MOT) algorithms of the joint detection and embedding (JDE) approach estimate bounding boxes and re-identification (re-ID) features of objects with the single network, which balance the tracking accuracy and inference speed. However, when the appearance information between different objects is highly similar, these algorithms are usually easy to cause identity switches, and the comprehensive tracking performance is poor in crowded scenes. Aiming at the above problems, we propose a stronger multi-object tracking algorithm termed as ReIMOT, based on FairMOT. A joint loss function of combining normalized Softmax Loss and the center distance penalty term is designed to supervise the re-ID branch, which increases the intra-class similarity and makes the extracted appearance features more discriminative. To further improve the tracking performance, we introduce coordinate attention to make the encoder-decoder network focus more on features of interest. The experimental results show that the proposed ReIMOT is more effective than the other advanced multi-object tracking algorithms, and decreases the number of ID switches by 13.8% compared to FairMOT on the MOT17 dataset.

Pig husbandry constitutes a significant segment within the broader framework of livestock farming, with porcine well-being emerging as a paramount concern due to its direct implications on pig breeding and production. An easily observable proxy for assessing the health of pigs lies in their daily patterns of movement. The daily movement patterns of pigs can be used as an indicator of their health, in which more active pigs are usually healthier than those who are not active, providing farmers with knowledge of identifying pigs’ health state before they become sick or their condition becomes life-threatening. However, the conventional means of estimating pig mobility largely rely on manual observations by farmers, which is impractical in the context of contemporary centralized and extensive pig farming operations. In response to these challenges, multi-object tracking and pig behavior methods are adopted to monitor pig health and welfare closely. Regrettably, these existing methods frequently fall short of providing precise and quantified measurements of movement distance, thereby yielding a rudimentary metric for assessing pig health. This paper proposes a novel approach that integrates optical flow and a multi-object tracking algorithm to more accurately gauge pig movement based on both qualitative and quantitative analyses of the shortcomings of solely relying on tracking algorithms. The optical flow records accurate movement between two consecutive frames and the multi-object tracking algorithm offers individual tracks for each pig. By combining optical flow and the tracking algorithm, our approach can accurately estimate each pig’s movement. Moreover, the incorporation of optical flow affords the capacity to discern partial movements, such as instances where only the pig’s head is in motion while the remainder of its body remains stationary. The experimental results show that the proposed method has superiority over the method of solely using tracking results, i.e., bounding boxes. The reason is that the movement calculated based on bounding boxes is easily affected by the size fluctuation while the optical flow data can avoid these drawbacks and even provide more fine-grained motion information. The virtues inherent in the proposed method culminate in the provision of more accurate and comprehensive information, thus enhancing the efficacy of decision-making and management processes within the realm of pig farming.

Events cameras are ideal sensors for enabling robots to detect and track objects in highly dynamic environments due to their low latency output, high temporal resolution, and high dynamic range. In this paper, we present the Asynchronous Event Multi-Object Tracking (AEMOT) algorithm for detecting and tracking multiple objects by processing individual raw events asynchronously. AEMOT detects salient event blob features by identifying regions of consistent optical flow using a novel Field of Active Flow Directions built from the Surface of Active Events. Detected features are tracked as candidate objects using the recently proposed Asynchronous Event Blob (AEB) tracker in order to construct small intensity patches of each candidate object. A novel learnt validation stage promotes or discards candidate objects based on classification of their intensity patches, with promoted objects having their position, velocity, size, and orientation estimated at their event rate. We evaluate AEMOT on a new Bee Swarm Dataset, where it tracks dozens of small bees with precision and recall performance exceeding that of alternative event-based detection and tracking algorithms by over 37%. Source code and the labelled event Bee Swarm Dataset will be open sourced. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>https://github.com/angus-apps/AEMOT

Phytoplankton play a critical role in marine food webs and biogeochemical cycles, and their abundance must be monitored to prevent disasters and improve the marine environment. Although existing algorithms for automatic phytoplankton identification at the image level are available, there are currently no video-level algorithms. This lack of datasets is a significant obstacle to the development of video-level automatic identification algorithms for phytoplankton observations. Deep learning-based algorithms, in particular, require high-quality datasets to achieve optimal results. To address this issue, we propose the PMOT2023 (Phytoplankton Multi-Object Tracking), a multi-video tracking dataset based on 48,000 micrographs captured by in situ observation devices. The dataset comprises 21 classes of phytoplankton and can aid in the development of advanced video-level identification methods. Multi-object tracking algorithms can detect, classify, count, and estimate phytoplankton density. As a video-level automatic identification algorithm, multi-object tracking addresses trajectory tracking, concentration estimation, and other requirements in original phytoplankton observation, helping to prevent marine ecological disasters. Additionally, the PMOT2023 dataset will serve as a benchmark to evaluate the performance of future phytoplankton identification models and provide a foundation for further research on automatic phytoplankton identification algorithms.

Current multi-object tracking (MOT) algorithms are dominated by the tracking-by-detection paradigm, which divides MOT into three independent sub-tasks of target detection, appearance embedding, and data association. To improve the efficiency of this tracking paradigm, this paper presents an anchor-free one-stage learning framework to perform target detection and appearance embedding in a unified network, which learns for each point in the feature pyramid of the input image an object detection prediction and a feature representation. Two effective training strategies are proposed to reduce missed detections in dense pedestrian scenes. Moreover, an improved non-maximum suppression procedure is introduced to obtain more accurate box detections and appearance embeddings by taking the box spatial and appearance similarities into account simultaneously. Experiments show that our MOT algorithm achieves real-time tracking speed while obtaining comparable tracking performance to state-of-the-art MOT trackers. Code will be released to facilitate further studies of this problem.

Marine litter is a growing environmental threat, with severe ecological and socio-economic impacts. Most monitoring strategies rely on optical sensors to detect surface pollution, however these approaches fail to capture submerged plastics dispersed throughout the water column. Multibeam acoustic imaging offers a complementary solution, but the scarcity of annotated sonar datasets and the high noise levels of acoustic imagery make automated detection and tracking particularly challenging. This study presents a comparative evaluation of deep learning based multi-object tracking (MOT) algorithms applied to water column acoustic data. Pre-trained YOLOv8 detectors were integrated with tracking-by-detection frameworks including BoT-SORT, OC-SORT, ByteTrack, and DeepOC-SORT. Performance was assessed across acoustic frequencies and preprocessing strategies using standard MOT metrics. Results show that adaptive Gaussian thresholding and opening morphology improved robustness at lower frequencies (950 kHz and 1200 kHz), while unprocessed inputs proved more resilient to severe clutter at 1400 kHz. BoostTrack and ByteTrack achieved the most consistent tracking, effectively managing intermittent detections to maximise MOTA and IDF1. In contrast, OC-SORT underperformed, struggling with fragmented sonar trajectories. Furthermore, while efficient Nano models dominated at lower frequencies, Medium models were required under higher noise. These findings demonstrate the feasibility of applying MOT methods to sonar-based litter monitoring. Future work will explore unsupervised learning approaches to leverage intrinsic sonar data structure, reduce annotation needs, and enable scalable marine litter tracking.

Appearance based multi-object tracking (MOT) is a challenging task, specially in complex scenes where objects have similar appearance or are occluded by background or other objects. Such factors motivate researchers to propose effective trackers which should satisfy real-time processing and object trajectory recovery criteria. In order to handle both mentioned requirements, we propose a robust online multi-object tracking method that extends the features and methods proposed for re-identification to MOT. The proposed tracker combines a local and a global tracker in a comprehensive two-step framework. In the local tracking step, we use the frame-to-frame association to generate online object trajectories. Each object trajectory is called tracklet and is represented by a set of multi-modal feature distributions modeled by GMMs. In the global tracking step, occlusions and mis-detections are recovered by tracklet bipartite association method based on learning Mahalanobis metric between GMM components using KISSME metric learning algorithm. Experiments on two public datasets show that our tracker performs well when compared to state-of-the-art tracking algorithms.

Video security monitoring has always been an important mission for safety reason. In an entire surveillance system, there are usually several cameras distributed sparsely to cover a wide range of public areas (e.g., school, shopping mall or infrastructure). Tracking person through this cameras network is challenging due to different camera perspectives, illumination changes and pose variations. Several algorithms for Multi-Target Multi-Camera tracking (MTMCT) have been proposed in offline method which has delay in getting result. Addressing the need for real-time computation of people tracks through multi camera, the paper proposes an online tracking algorithm. The contributions include (1) online real-time framework which can be used in practical application, (2) extend a single camera multi object tracking (MOT) algorithm to be suitable for multi-camera tracking and (3) use spatial-temporal information to strengthen cross camera person recall performance. The proposed algorithm has been benchmarked against the literature review of MTMC algorithms.

Artificial reefs (ARS) are structures placed in the ocean, and their posture and position affect the placement effect of the reef. Quantitative data on the reef posture and position can provide references for the creation of a favorable environment for marine organisms. This paper focuses on improving the quality assessment methods for the deployment of ARS in marine habitats. Based on the images of ARS detected by forward-looking sonar, a new method is proposed to determine the three-dimensional position and deployment effectiveness of the reefs, with the aim of obtaining more detailed information about the ARS to assist in evaluating the quality of their deployment. By constructing a multi-object tracking (MOT) dataset based on the Oculus sonar, the YOLOv8n-pose series model was used for object detection. Subsequently, the three-dimensional position and subsidence of ARS were evaluated using trajectories identified by the MOT algorithm. This paper compares various MOT algorithms and improves the BOTSORT algorithm by introducing the Hungarian algorithm and the Joseph form of the Kalman filter, significantly enhancing the accuracy of target tracking. The experimental results demonstrate that, within the framework of the YOLOv8n-pose model, the detection and association accuracy of the BOTSORT algorithm, which has been refined to address the characteristics of sonar images, has been further enhanced. Additionally, this paper proposes a mathematical modeling method for the assessment of ARS and their subsidence, providing important technical support and solutions for future marine ranch construction.

We often execute multi-object tracking algorithms and pedestrian angle recognition algorithms independently when we solve complex problems that need to track multiple pedestrians and identify their angles simultaneously. Firstly, the multi-target tracking algorithm is used to locate the target and determine the target&#x2019;s identity. Then the pedestrian angle recognition model is used to recognize the pedestrian angle. This strategy is complicated and inefficient since the two models repeatedly extract target features. This paper realizes the combination of multi-object tracking algorithm and pedestrian angle recognition algorithm in response to this problem. Specifically, we added a new classification branch based on the JDE multi-object tracking algorithm. It enables the model to perform pedestrian angle recognition while detecting and embedding feature extraction. Finally, experiments show that the improved algorithm can perform pedestrian tracking and pedestrian angle recognition simultaneously. The algorithm&#x2019;s ability to track large and medium targets is the same as the JDE multi-object tracking algorithm. Compared with the Hydraplus model, the improved algorithm has a higher recognition accuracy. Besides, the algorithm can achieve real-time performance.

In previous research, most Multi-object tracking (MOT) algorithms focus on the optical image dataset, while the Synthetic Aperture Radar (SAR) image dataset faces the characteristics of few prior samples, high false alarm rate, and various defocusing interference. On the SAR image dataset, a robust MOT algorithm is proposed to fulfill multi-ship tracking in complex imaging conditions. First, the kernelized correlation filters (KCF) algorithm, a single-object tracking algorithm, is modified and applied to reduce the impact of false alarms on tracking performance. After that, different matching strategies are adaptively adapted to associate the targets based on the three intersection patterns between the predictions and the detections, which can reduce the impact of the deviated detections. Finally, the tracker’s time limit with Gaussian distribution is proposed to improve the re-association ability after the tracking interruption caused by the defocusing. The experiment results demonstrate the robust tracking ability of the proposed MOT algorithm.

To adjust management practices and improve wine marketing strategies, accurate vineyard yield estimation early in the growing season is essential. Conventional methods for yield forecasting rely on phenotypic features' manual assessment, which is time- and labor-intensive and often destructive. We combined a deep object segmentation method, mask region-based convolutional neural network (Mask R-CNN), with two potential multi-object tracking algorithms, simple online and real-time tracking (SORT) and intersection-over-union (IOU) trackers to develop a complete visual system that can automatically detect and track individual inflorescences, enabling the assessment of the number of inflorescences per vineyard row from vineyard video footage. The performance of the two tracking algorithms was evaluated using our vineyard dataset, which is more challenging than conventional tracking benchmark datasets owing to environmental factors. Our evaluation dataset consists of videos of four vineyard rows, including 221 vines that were automatically acquired under unprepared field conditions. We tracked individual inflorescences across video image frames with a 92.1% multi-object tracking accuracy (MOTA) and an 89.6% identity F1 score (IDF1). This allowed us to estimate inflorescence count per vineyard row with a 0.91 coefficient of determination (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) between the estimated count and manual-annotated ground truth count. The impact of leaf occlusions on inflorescence visibility was lessened by processing multiple successive image frames with minimal displacements to construct multiple camera views. This study demonstrates the use of deep learning and multi-object tracking in creating a low-cost (requiring only an RGB camera), high-throughput phenotyping system for precision viticulture.

3D multi-object tracking with boosting data association and improved trajectory management mechanism

A novel marine radar multi-object tracking (MOT) algorithm is proposed based on the Space-Time Memory Network (STMN), which is specifically designed to markedly enhance the accuracy and stability of vessel tracking in complex marine environments. Especially, compared to traditional point- or shape-based radar object tracking methods, the proposed algorithm can significantly enhance robustness with respect to the occluded and rapid target deformation. Firstly, we define the macro- and micro-velocities for each object, integrating them into the proposed algorithm for newborn targets recognition, thereby improving the accuracy of target identification in occluded or cluttered environments. Secondly, we propose a feature extraction module that enables STMN to handle multiple targets simultaneously, which can efficiently differentiate the newborn, survival, and suspicious targets in complex scenarios. Finally, to enhance the shape estimation accuracy of survival targets, we propose a Width-Height Difference Intersection Over Union (WH-IOU) algorithm, which can improve the consistency between detection boxes and object contours, thereby boosting overall tracking performance. Experimental results demonstrate that the proposed algorithm outperforms existing methods in X-band radar MOT tasks, achieving substantial improvements in both tracking accuracy and robustness.

Tracking multiple objects is important in many application domains. We propose a novel algorithm for multi-object tracking that is capable of working under very challenging conditions such as minimal hardware equipment, uncalibrated monocular camera, occlusions and severe background clutter. To address this problem we propose a new method that jointly estimates object tracks, estimates corresponding 2D/3D temporal trajectories in the camera reference system as well as estimates the model parameters (pose, focal length, etc) within a coherent probabilistic formulation. Since our goal is to estimate stable and robust tracks that can be univocally associated to the object IDs, we propose to include in our formulation an interaction (attraction and repulsion) model that is able to model multiple 2D/3D trajectories in space-time and handle situations where objects occlude each other. We use a MCMC particle filtering algorithm for parameter inference and propose a solution that enables accurate and efficient tracking and camera model estimation. Qualitative and quantitative experimental results obtained using our own dataset and the publicly available ETH dataset shows very promising tracking and camera estimation results.KeywordsCamera MotionCamera ParameterCamera ModelSingle CameraStructure From MotionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.