A robust method for multi object tracking in autonomous ship navigation systems

The ability to automatically detect and track people is a key component of intelligent surveillance system (ISS). ISS is able to automatically analyze surveillance camera data with limited human intervention. In this research, robust people detection and tracking methods are studied for public surveillance system. The proposed method composed of foreground-background segmentation and people detection using mixture of gaussians (MOG) and histogram of oriented gradients (HOG) and then followed by multi object tracking (MOT) method for people tracking. The proposed method is tested on a closed circuit television (CCTV) system at the engineering College of Sebelas Maret University (UNS) Surakarta. All of the people tracking methods has been tested, and the result is good enough with the success up to 100%.

Algorithmic solutions for multi-object tracking (MOT) are a key enabler for applications in autonomous navigation and applied ocean sciences. State-of-the-art MOT methods fully rely on a statistical model and typically use preprocessed sensor data as measurements. In particular, measurements are produced by a detector that extracts potential object locations from the raw sensor data collected at discrete time steps. This preparatory processing step reduces data flow and computational complexity but may result in a loss of information. State-of-the-art Bayesian MOT methods that are based on belief propagation (BP) systematically exploit graph structures of the statistical model to reduce computational complexity and improve scalability. However, as a fully model-based approach, BP can provide highly suboptimal estimates when there is a mismatch between the statistical model and the true data-generating process. Existing BP-based MOT methods can further only make use of preprocessed measurements. In this paper, we introduce a variant of BP that combines model-based with data-driven MOT. The proposed neural enhanced belief propagation (NEBP) method complements the statistical model of BP by information learned from raw sensor data. This approach conjectures that the learned information can reduce model mismatch and thus improve data association and false alarm rejection. Our NEBP method improves tracking performance compared to model-based methods. At the same time, it inherits the advantages of BP-based MOT, i.e., it scales only quadratically in the number of objects, and it can thus generate and maintain a large number of object tracks. We evaluate the performance of our NEBP approach for MOT on the nuScenes autonomous driving dataset and demonstrate that it can achieve state-of-the-art performance. In particular, an average multi-object tracking accuracy (AMOTA) of 0.683 was obtained and, compared with non-BP-based methods, identity switches (IDS) and track fragments (Frag) were reduced by 23% and 19%, respectively.

3D fluorescence microscopy of living organisms has increasingly become an essential and powerful tool in biomedical research and diagnosis. An exploding amount of imaging data has been collected, whereas efficient and effective computational tools to extract information from them are still lagging behind. This is largely due to the challenges in analyzing biological data. Interesting biological structures are not only small, but are often morphologically irregular and highly dynamic. Although tracking cells in live organisms has been studied for years, existing tracking methods for cells are not effective in tracking subcellular structures, such as protein complexes, which feature in continuous morphological changes including split and merge, in addition to fast migration and complex motion. In this paper, we first define the problem of multi-object portion tracking to model the protein object tracking process. A multi-object tracking method with portion matching is proposed based on 3D segmentation results. The proposed method distills deep feature maps from deep networks, then recognizes and matches objects' portions using an extended search. Experimental results confirm that the proposed method achieves 2.96% higher on consistent tracking accuracy and 35.48% higher on event identification accuracy than the state-of-art methods.

Multi-object tracking (MOT) in satellite videos is a new and challenging task. The difficulties stem from the extremely small objects and the low contrast between objects and background. To tackle the challenges of MOT in satellite videos, a multi-object tracking method is proposed in this paper to incorporate the local motion prior into the network. Specifically, we design a local cost volume construction module to obtain tracking offsets between adjacent frames. Based on the tracking offsets, features of previous frames can be propagated to the current frame to incorporate spatio-temporal information. We conduct extensive experiments on videos from Jilin-1 satellite, and the results demonstrate the effectiveness of the proposed method.

Most multi-object detection and tracking techniques suffer from the well-known "multi-object occlusion" problem. The abundant nodes of wireless video sensor networks (WVSNs) can be utilized to solve the problem, and the video nodes in WVSN have limited calculation capability and energy. In order to achieve effective multi-object tracking using WVSN, the main contributions of our proposed method are that: (1) the limits of field of view (FOV) of every video nodes are utilized to establish the consistent labeling of the objects in different views. (2) Mobile Agent is employed to communicate among network nodes, so the objects are assigned correct labels after multi-object occlusion. The performance of the approach has been demonstrated on real-world and the experimental results show that the proposed method is effective for resolving multi-object occlusions and meets the requirement of WVSN.

In this paper, we introduced NanoTrack, a novel multi-object tracking (MOT) method that leverages light-weight object detectors to enhance tracking performance in real-time applications where computational resources are scarce. While light-weight detectors are efficient, they often produce an imbalance in detection quality, generating a significant number of low-scoring detections that pose challenges for tracking algorithms. Our approach innovatively utilizes these low-scoring detections for track initialization and maintenance, addressing the shortcomings observed in existing tracking by two-stage tracking methods like ByteTrack, which struggle with the abundance of low-scoring detections. By integrating two new light-weight modules, Refind High Detection (RHD) and Duplicate Track Checking (DTC), NanoTrack effectively incorporates low-scoring detections into the tracking process. Additionally, we enhance the pseudo-depth estimation technique for improved handling in dense target environments, mitigating issues like ID Switching. Our comprehensive experiments demonstrate that NanoTrack surpasses state-of-the-art two-stage TBD methods, including ByteTrack and SparseTrack, on benchmark datasets such as MOT16, MOT17, and MOT20, thereby establishing a new standard for MOT performance using light-weight detectors. The code is open source in https://github.com/VjiaLi/NanoTrack

CAMSHIFT (Continuously Adaptive Mean-Shift) has been well accepted as one of the prominent methods in object tracking. CAMSHIFT is good for single hue object tracking and in the condition where object's color is different with background's color. In this paper, we enhance CAMSHIFT so it can be used for multi-object tracking and improve the robustness of CAMSHIFT for multi-hue object tracking especially in the situation where object's colors are similar with background's colors. We propose a more precise object localization by selecting each dominant color object part using a combination of Mean-Shift segmentation and region growing. Hue-distance, saturation and value color histogram are used to describe the object. We also track the dominant color object parts separately and combine them together to improve robustness of the tracking on multi-hue object. For multi-object tracking, we use a separate tracker for each object. Our experiments showed that those methods improved CAMSHIFT robustness significantly and enable CAMSHIFT for multi-object tracking.

Appearance based multi-object tracking (MOT) is a challenging task, specially in complex scenes where objects have similar appearance or are occluded by background or other objects. Such factors motivate researchers to propose effective trackers which should satisfy real-time processing and object trajectory recovery criteria. In order to handle both mentioned requirements, we propose a robust online multi-object tracking method that extends the features and methods proposed for re-identification to MOT. The proposed tracker combines a local and a global tracker in a comprehensive two-step framework. In the local tracking step, we use the frame-to-frame association to generate online object trajectories. Each object trajectory is called tracklet and is represented by a set of multi-modal feature distributions modeled by GMMs. In the global tracking step, occlusions and mis-detections are recovered by tracklet bipartite association method based on learning Mahalanobis metric between GMM components using KISSME metric learning algorithm. Experiments on two public datasets show that our tracker performs well when compared to state-of-the-art tracking algorithms.

Pig husbandry constitutes a significant segment within the broader framework of livestock farming, with porcine well-being emerging as a paramount concern due to its direct implications on pig breeding and production. An easily observable proxy for assessing the health of pigs lies in their daily patterns of movement. The daily movement patterns of pigs can be used as an indicator of their health, in which more active pigs are usually healthier than those who are not active, providing farmers with knowledge of identifying pigs’ health state before they become sick or their condition becomes life-threatening. However, the conventional means of estimating pig mobility largely rely on manual observations by farmers, which is impractical in the context of contemporary centralized and extensive pig farming operations. In response to these challenges, multi-object tracking and pig behavior methods are adopted to monitor pig health and welfare closely. Regrettably, these existing methods frequently fall short of providing precise and quantified measurements of movement distance, thereby yielding a rudimentary metric for assessing pig health. This paper proposes a novel approach that integrates optical flow and a multi-object tracking algorithm to more accurately gauge pig movement based on both qualitative and quantitative analyses of the shortcomings of solely relying on tracking algorithms. The optical flow records accurate movement between two consecutive frames and the multi-object tracking algorithm offers individual tracks for each pig. By combining optical flow and the tracking algorithm, our approach can accurately estimate each pig’s movement. Moreover, the incorporation of optical flow affords the capacity to discern partial movements, such as instances where only the pig’s head is in motion while the remainder of its body remains stationary. The experimental results show that the proposed method has superiority over the method of solely using tracking results, i.e., bounding boxes. The reason is that the movement calculated based on bounding boxes is easily affected by the size fluctuation while the optical flow data can avoid these drawbacks and even provide more fine-grained motion information. The virtues inherent in the proposed method culminate in the provision of more accurate and comprehensive information, thus enhancing the efficacy of decision-making and management processes within the realm of pig farming.

This paper presents a high-accuracy and robust multi-object tracking method for maritime vessel detection and tracking in complex marine environments, characterized by dense targets, large-scale variations, and frequent occlusions. The proposed approach adopts an enhanced YOLOv8-based detector with lightweight feature enhancement and attention mechanisms to improve its capability in detecting small-scale vessels and complex scenes. Furthermore, a tracking framework combining BOTSORT with an OSNet-based re-identification (ReID) model is employed to achieve stable and reliable vessel association. Experimental results on the Near-Infrared On-Shore (NIR) dataset demonstrate that the proposed method improves Precision, Recall, mAP@0.5, and mAP@0.5:0.95 by approximately 4.0%, 5.0%, 5.1%, and 5.4%, respectively, compared with the baseline YOLOv8, while reducing parameter count and model size by 11.6% and 6.5%. On the Visible On-Shore (VIS) dataset, the proposed method outperforms state-of-the-art approaches in detection accuracy and robustness, further validating its effectiveness and cross-modal generalization capability. In multi-object tracking tasks, the proposed CPM-YOLO and BOTSORT framework demonstrates clear advantages in trajectory continuity, occlusion handling, and identity preservation compared with mainstream tracking algorithms. On the NIR dataset, the proposed method achieves a competitive inference speed of 188 FPS, while running at 187 FPS on the VIS dataset, demonstrating that the accuracy improvements are achieved without sacrificing real-time performance. Overall, the proposed method achieves a favorable balance between detection accuracy, tracking robustness, and model efficiency, making it well-suited for practical maritime applications.

It is essential for them to accurately and efficiently estimate both their own pose and the poses of surrounding dynamic objects in real time during autonomous exploration. Although most existing approaches achieve accurate self-localization or object detection and tracking individually, there is limited research on effectively integrating these two tasks, despite their inherently coupled and interactive nature. To address this challenge, this paper proposes a multi-object tracking and trajectory optimization framework based on enhanced data association and a two-stage optimization strategy, enabling robust localization of both the ego-agent and surrounding dynamic objects in highly dynamic urban environments. In the front-end, we incorporate an improved data association algorithm into a multi-object tracking pipeline and combine it with a GICP-based fast registration method to achieve efficient dynamic tracking. In the back-end, based on the front-end tracking results, we model the states of moving objects as dynamic tracking factors and integrate them into a global factor graph. This allows for joint optimization of the ego-pose and the poses of dynamic objects, resulting in high-precision multi-object tracking. Extensive comparative experiments on public high-dynamic datasets demonstrate the state-of-the-art performance and advantages of the proposed method in multi-object tracking and self-localization, where the accuracy of self-localization and tracking has increased by 19.7% and 3%, respectively, compared with the most advanced method.

Improved foreign object tracking algorithm in coal for belt conveyor gangue selection robot with YOLOv7 and DeepSORT

The multi-object tracking method was proposed in this paper effectively using stereo vision. For the multi-object tracking, we present the method to combine the local gradient patterns (LGP) and the depth information for extracting the feature of region of interest (ROI) by the stereo camera. We used the LGP algorithm to get the feature of ROI, and used the difference of depth information between objects to solve the occlusion problem caused the multi-object tracking. Therefore in this paper, the algorithm suggested that the location and size of detected object using background modeling are assigned automatically. Through this experiment, our proposed method is implemented that it performs more precisely than existing methods.

The paper proposes a novel approach to the objects localization in the working area of a modular reconfigurable robot (MRR), which implies the installation of stationary monitoring points (SMP), consisting of detachable robot’s modules and in- stalled by robot itself. This approach is based on the architecture of the MRR control system previously proposed by the authors and a new method for comparing information about the speed and position obtained from various sensors. The key steps of the approach are following. Upon arriving in the target area, the MRR places SMPs, which consist of a power source, a computing device, a wireless transceiver and a sensor, detached from the robot. Then SMPs monitor the working area using different types of sensors (cameras, rangefinders, etc.), perform segmentation of the measured data and transfer this information to the robot. Further a sensor fusion is performed using a novel object tracking method, which makes it possible to localize target objects even in those cases when they are not visible by some of the SMPs. One of the key advantages of the new approach is a possibility of implementation in the distributed architecture of a MRR. The simulation results show that proposed method has Multiple Object Tracking Accuracy (MOTA) metric of 86 %, which is higher than the most of its analogues, while the estimated dynamic object localization error in a 8x7 m working area using 2 cameras and 1 rangefinder does not exceed 10 cm.

Traditional Multiple Object Tracking (MOT) methods in satellite videos mostly follow the Detection-Based Tracking (DBT) framework. However, the DBT framework assumes that all objects are correctly recognized and localized by the detector. In practice, the low resolution of satellite videos, small objects, and complex backgrounds inevitably leads to a decline in detector performance. To alleviate the impact of detector degradation on track, we propose Coarse-Fine Tracker, a framework that integrates the MOT framework with the Tracking Any Point (TAP) method CoTracker for the first time, leveraging TAP’s persistent point correspondence modeling to compensate for detector failures. In our Coarse-Fine Tracker, we divide the satellite video into sub-videos. For one sub-video, we first use ByteTrack to track the outputs of the detector, referred to as coarse tracking, which involves the Kalman filter and box-level motion features. Given the small size of objects in satellite videos, we treat each object as a point to be tracked. We then use CoTracker to track the center point of each object, referred to as fine tracking, by calculating the appearance feature similarity between each point and its neighboring points. Finally, the Consensus Fusion Strategy eliminates mismatched detections in coarse tracking results by checking their geometric consistency against fine tracking results and recovers missed objects via linear interpolation or linear fitting. This method is validated on the VISO and SAT-MTB datasets. Experimental results in VISO show that the tracker achieves a multi-object tracking accuracy (MOTA) of 66.9, a multi-object tracking precision (MOTP) of 64.1, and an IDF1 score of 77.8, surpassing the detector-only baseline by 11.1% in MOTA while reducing ID switches by 139. Comparative experiments with ByteTrack demonstrate the robustness of our tracking method when the performance of the detector deteriorates.