SimpleTrack: Understanding and Rethinking 3D Multi-object Tracking

LiDAR-based single object tracking plays a key role in intelligent vehicles. Current methods typically follow appearance matching or motion-centric frameworks. However, point clouds are usually sparse and incomplete, providing insufficient appearance information for matching. While the motion-centric framework predicts inter-frame motion of targets instead of performing appearance matching for tracking, it neglects contextual information matching of consecutive frames that is conducive to target motion modeling. In this paper, we propose an elegant and effective framework by leveraging Context Matching to guide motion modeling for accurate Tracking (CMTrack). The novel framework possesses two attractive properties: 1) It incorporates a context matching encoder-decoder network to match contextual information of consecutive frames, fully exploring informative cues relevant to target motion. 2) Benefiting from informative motion cues being modeling, CMTrack allows for accurate prediction of inter-frame motion of targets in a one-stage manner. Extensive experiments are conducted on several widely-adopted datasets, i.e., KITTI, NuScenes and Waymo Open Dataset. Without bells and whistles, our CMTrack demonstrates competitive tracking accuracy (e.g., 87.3% and 69.3% precision on KITTI and NuScenes, respectively) compared to state-of-the-art methods, while running at a high speed of 48 Fps on a single Titan Xp GPU.

A robust method for multi object tracking in autonomous ship navigation systems

Algorithmic solutions for multi-object tracking (MOT) are a key enabler for applications in autonomous navigation and applied ocean sciences. State-of-the-art MOT methods fully rely on a statistical model and typically use preprocessed sensor data as measurements. In particular, measurements are produced by a detector that extracts potential object locations from the raw sensor data collected at discrete time steps. This preparatory processing step reduces data flow and computational complexity but may result in a loss of information. State-of-the-art Bayesian MOT methods that are based on belief propagation (BP) systematically exploit graph structures of the statistical model to reduce computational complexity and improve scalability. However, as a fully model-based approach, BP can provide highly suboptimal estimates when there is a mismatch between the statistical model and the true data-generating process. Existing BP-based MOT methods can further only make use of preprocessed measurements. In this paper, we introduce a variant of BP that combines model-based with data-driven MOT. The proposed neural enhanced belief propagation (NEBP) method complements the statistical model of BP by information learned from raw sensor data. This approach conjectures that the learned information can reduce model mismatch and thus improve data association and false alarm rejection. Our NEBP method improves tracking performance compared to model-based methods. At the same time, it inherits the advantages of BP-based MOT, i.e., it scales only quadratically in the number of objects, and it can thus generate and maintain a large number of object tracks. We evaluate the performance of our NEBP approach for MOT on the nuScenes autonomous driving dataset and demonstrate that it can achieve state-of-the-art performance. In particular, an average multi-object tracking accuracy (AMOTA) of 0.683 was obtained and, compared with non-BP-based methods, identity switches (IDS) and track fragments (Frag) were reduced by 23% and 19%, respectively.

Multiobject tracking (MOT) is an important task in applications including autonomous driving, ocean sciences, and aerospace surveillance. Traditional MOT methods are model-based and combine sequential Bayesian estimation with data association and an object birth model. More recent methods are fully data-driven and rely on the training of neural networks. Both approaches offer distinct advantages in specific settings. In particular, model-based methods are generally applicable across a wide range of scenarios, whereas data-driven MOT achieves superior performance in scenarios where abundant labeled data for training is available. A natural thought is whether a general framework can integrate the two approaches. This paper introduces a hybrid method that utilizes neural networks to enhance specific aspects of the statistical model in Bayesian MOT that have been identified as overly simplistic. By doing so, the performance of the prediction and update steps of Bayesian MOT is improved. To ensure tractable computation, our framework uses belief propagation to avoid high-dimensional operations combined with sequential Monte Carlo methods to perform low-dimensional operations efficiently. The resulting method combines the flexibility and robustness of model-based approaches with the capability to learn complex information from data of neural networks. We evaluate the performance of the proposed method based on the nuScenes autonomous driving dataset and demonstrate that it has state-of-the-art performance.

Autonomous driving is conducted in complex scenarios, which requires to detect 3D objects in real time scenarios as well as accurately track these 3D objects in order to get such information as location, size, trajectory, velocity. MOT (Multi-Object Tracking) performance is heavily dependent on object detection. Once object detection gives false alarms or missing alarms, the multi-object tracking would be automatically influenced. In this paper, we propose a coupling system which combines 3D object detection and multi-object tracking into one framework. We use the tracked objects as a reference in 3D object detection, in order to locate objects, reduce false or missing alarms in a single frame, and weaken the impact of false and missing alarms on the tracking quality. Our method is evaluated on kitti dataset and is proved effective.

Multi-Object Tracking (MOT) in Unmanned Aerial Vehicle (UAV) scenarios is characterized by frequent and abrupt camera motion, which presents two unique challenges: nonlinear motion and appearance degradation. Traditional motion models, designed for smooth and consistent motion, struggle to capture the complex background motion patterns caused by UAV movement; while appearance-based methods are easily disrupted by occlusion and blur, leading to unreliable associations. Even though dense optical flow is widely utilized to model complex motion patterns, the entanglement of background and object motion often introduces interference, limiting its effectiveness. To this end, we propose MoDe-Track, a unified framework that explicitly decouples scene motion into background and object components, and serves as an elegant integration of three robust components. Specifically, the Scene Motion Decomposition (SMD) module decouples the motion into the background and object components based on robust principal component analysis, serving as the foundation for motion compensation and feature propagation. Afterwards, the Background Motion Compensation (BMC) uses the decomposed background flow to estimate and compensate for camera motion, mitigating the effects of nonlinear motion. Finally, the Foreground-guided Feature Propagation (FFP) module uses the decoupled object flow to guide feature propagation across frames, achieving temporal consistency and enhancing robustness against occlusion and motion blur. Extensive experimental results on two benchmarks, VisDrone2019 and UAVDT, demonstrate that MoDe-Track consistently outperforms current multi-object tracking methods. We achieve 56.0% MOTA on VisDrone2019 and 56.2% MOTA on UAVDT, reaching the state-of-the-art among existing methods.

Neural models for distantly supervised relation extraction (DS-RE) encode each sentence in an entity-pair bag separately. These are then aggregated for bag-level relation prediction. Since, at encoding time, these approaches do not allow information to flow from other sentences in the bag, we believe that they do not utilize the available bag data to the fullest. In response, we explore a simple baseline approach (PARE) in which all sentences of a bag are concatenated into a passage of sentences, and encoded jointly using BERT. The contextual embeddings of tokens are aggregated using attention with the candidate relation as query -- this summary of whole passage predicts the candidate relation. We find that our simple baseline solution outperforms existing state-of-the-art DS-RE models in both monolingual and multilingual DS-RE datasets.

Multi-object tracking through learning relational appearance features and motion patterns

Multi-object model-free tracking is challenging because the tracker is not aware of the objects' type (not allowed to use object detectors), and needs to distinguish one object from background as well as other similar objects. Most existing methods keep updating their appearance model individually for each target, and their performance is hampered by sudden appearance change and/or occlusion. We propose to use both appearance model and motion model to overcome this issue. We introduce an indicator variable to predict sudden appearance change and occlusion. When they happen, our model stops updating the appearance model to avoid parameter update based on background or incorrect object, and rely more on motion model to track. Moreover, we consider the correlation among all targets, and seek the joint optimal locations for all target simultaneously. We formulate the problem of finding the most likely locations jointly as a graphical model inference problem, and learn the joint parameters for both appearance model and motion model in an online fashion in the framework of LaRank. Experiment results show that our method outperforms the state-of-the-art.

Multi-object tracking (MOT) in cluttered and dynamic environments remains challenging, especially for maneuvering objects. While trajectory-based random finite set (RFS) filters provide principled solutions for trajectory estimation, they typically rely on a single motion model, limiting their adaptability. To address this gap, we propose the Multiple-Model Trajectory Poisson Multi-Bernoulli Mixture (MM-TPMBM) filter, which integrates jump Markov system (JMS) dynamics within the trajectory RFS framework. The filter enables closed-form Bayesian recursion for joint trajectory estimation and motion model switching. We derive its prediction and update equations, implement it using Gaussian mixtures for computational efficiency, and evaluate performance against benchmark filters (MM-PMBM, TPMBM, δ-GLMB) via Monte Carlo simulations. Results demonstrate that the MM-TPMBM filter achieves superior accuracy in trajectory estimation, localization, and cardinality, reducing the generalized optimal sub-pattern assignment (GOSPA) error by up to 24% and cardinality error by up to 52% compared to state-of-the-art methods, validating its robustness in complex tracking scenarios.

Multi-target tracking is a difficult task in radar data processing field. To track multiple targets with the video data collected by a X-band surface surveillance radar, a two-level cascade detection algorithm is designed at first. We combine the motion model and appearance model of the targets together for multi-target tracking and propose a multi-target tracking algorithm based on scale adaptive kernelized correlation filter (SAKCF) and Kalman filter (KF) in this paper. Firstly, the extended targets are detected by the designed tow-level cascade detection algorithm and the centroids of the extended targets are extracted by centroid algorithm too. Then, a SAKCF tracker is assigned to each target. KF is utilized to predict the position of each target and each SAKCF tracker searches the region near the predicted position to determine the real position and scale of each target. The association between tracks and trace points is performed based on Hungary algorithm. Finally, the abnormal tracks are deleted. Experimental results based on the radar video data show that compared with the classical simple online real-time target tracking (SORT) algorithm and joint probabilistic data association (JPDA) algorithm, the proposed algorithm improves multi-object tracking precision (MOTP) and multi-object tracking accuracy (MOTA).

Multi-object tracking (MOT) in satellite videos is a new and challenging task. The difficulties stem from the extremely small objects and the low contrast between objects and background. To tackle the challenges of MOT in satellite videos, a multi-object tracking method is proposed in this paper to incorporate the local motion prior into the network. Specifically, we design a local cost volume construction module to obtain tracking offsets between adjacent frames. Based on the tracking offsets, features of previous frames can be propagated to the current frame to incorporate spatio-temporal information. We conduct extensive experiments on videos from Jilin-1 satellite, and the results demonstrate the effectiveness of the proposed method.

3D fluorescence microscopy of living organisms has increasingly become an essential and powerful tool in biomedical research and diagnosis. An exploding amount of imaging data has been collected, whereas efficient and effective computational tools to extract information from them are still lagging behind. This is largely due to the challenges in analyzing biological data. Interesting biological structures are not only small, but are often morphologically irregular and highly dynamic. Although tracking cells in live organisms has been studied for years, existing tracking methods for cells are not effective in tracking subcellular structures, such as protein complexes, which feature in continuous morphological changes including split and merge, in addition to fast migration and complex motion. In this paper, we first define the problem of multi-object portion tracking to model the protein object tracking process. A multi-object tracking method with portion matching is proposed based on 3D segmentation results. The proposed method distills deep feature maps from deep networks, then recognizes and matches objects' portions using an extended search. Experimental results confirm that the proposed method achieves 2.96% higher on consistent tracking accuracy and 35.48% higher on event identification accuracy than the state-of-art methods.

Ultrasound strain imaging has demonstrated its ability to quantitatively assess myocardial viability and contractility altered by myocardial ischemia. However, current ultrasound strain imaging methods still use lower dimensional methods to monitor 3D heart motion. Some 3-D tracking algorithms have also been developed recently in different groups. Quantitative analysis using current methods depends on ultrasound probe orientation and selection of a centroid. To address this, 3D elasticity imaging derived using principal stretches independent of the centroid point is used to assess the contractility of myocardial fibers with 3D data from a commercial 3D scanner on an open-chest dog heart. In this study, an open-chest dog was performed according to a Yale institutional animal protocol and 3D radio frequency (RF) volume data were acquired using an commercial 2D phased array (iE33, Philips, Andover, MA) placed in front of the anterior wall of the left-ventricle with a small water stand-off. 3D speckle tracking was applied to estimate 3D displacement with tracking resolution of 1.2 mm in the axial direction and 4.5 mm in azimuthal and zenithal directions. Volume-to-volume tracking results were accumulated referenced to the heart geometry at the end of diastole until the end of systole. Three principal stretches were estimated using eigenvalue decomposition of the derived right Cauchy deformation tensor at each point. Initial results of strains based on principal stretch and the principal direction has successfully demonstrated that heart wall is thickening along the radial direction and shortening along the longitudinal direction during systole. Unlike ID or 2D methods, 3D speckle tracking can estimate myocardium's three-dimensional motion. Three strains based on the principal stretches were derived from 3D displacements to assess myocardial contractility independent of probe position and the selection of the centroid point.

In this paper, we introduced NanoTrack, a novel multi-object tracking (MOT) method that leverages light-weight object detectors to enhance tracking performance in real-time applications where computational resources are scarce. While light-weight detectors are efficient, they often produce an imbalance in detection quality, generating a significant number of low-scoring detections that pose challenges for tracking algorithms. Our approach innovatively utilizes these low-scoring detections for track initialization and maintenance, addressing the shortcomings observed in existing tracking by two-stage tracking methods like ByteTrack, which struggle with the abundance of low-scoring detections. By integrating two new light-weight modules, Refind High Detection (RHD) and Duplicate Track Checking (DTC), NanoTrack effectively incorporates low-scoring detections into the tracking process. Additionally, we enhance the pseudo-depth estimation technique for improved handling in dense target environments, mitigating issues like ID Switching. Our comprehensive experiments demonstrate that NanoTrack surpasses state-of-the-art two-stage TBD methods, including ByteTrack and SparseTrack, on benchmark datasets such as MOT16, MOT17, and MOT20, thereby establishing a new standard for MOT performance using light-weight detectors. The code is open source in https://github.com/VjiaLi/NanoTrack