Brain Strategy Algorithm for Multiple Object Tracking Based on Merging Semantic Attributes and Appearance Features.

Multiple object tracking is particularly challenging when many objects with similar appearances occlude one another. Most existing approaches concatenate the states of different objects, view the multi-object tracking as a joint motion estimation problem and search for the best state of the joint motion in a rather high dimensional space. However, this centralized framework suffers from a high computational load. We bring a new view to the tracking problem from a swarm intelligence perspective. In analogy with the foraging behavior of bird flocks, we propose a species-based particle swarm optimization algorithm for multiple object tracking, in which the global swarm is divided into many species according to the number of objects, and each species searches for its object and maintains track of it. The interaction between different objects is modeled as species competition and repulsion, and the occlusion relationship is implicitly deduced from the “power” of each species, which is a function of the image observations. Therefore, our approach decentralizes the joint tracker to a set of individual trackers, each of which tries to maximize its visual evidence. Experimental results demonstrate the efficiency and effectiveness of our method.

A review of object tracking methods: From general field to autonomous vehicles

Improvisation of Yolov4 Algorithm for Multiple Object Tracking Using Kalman Filtering Hungarian Method

Model-free object tracking is still challenging because of the limited prior knowledge and the unexpected variation of the target object. In this paper, we propose a feature learning algorithm for model-free multiple object tracking. First, we pre-learn generic features invariant to diverse motion transformations from auxiliary video data by using a deep network of anto-encoder. Then, we adapt the pre-learned features according to multiple target objects respectively in a multi-task learning manner. We treat the feature adaptation for each target object as one single task. We simultaneously learn the common feature shared by all target objects and the individual feature of each object. Experimental results demonstrate that our feature learning algorithm can significantly improve multiple object tracking performance.

The Constrained Admissible Region-Multiple Hypothesis Filter (CAR-MHF) has previously been successfully applied to the problem of determining initial state and parameter estimates for non-operational space objects in the near-geosynchronous Earth orbit (GEO) regime. The application of CAR-MHF to a dense population of synthetically-created uncorrelated tracks (UCT) has been primarily challenged by two characteristics of the current implementation. First, the rapid uncertainty growth due to the noise characteristics of processing short-arc optical tracklets leads to ambiguous uncertainty overlap and the prevalence of mis-associated observations, primarily with a large statistical association gate. Second, the reliance on measurement update decisions made on a frame-by-frame basis without a process that evaluates the track estimate based upon all available associated data has prevented additional data from resolving these ambiguous conditions (i.e. if data from another object is statistically valid at a given instant of large ambiguity, an update will be performed and usually drives the filter to diverge). The work addressed in this paper addresses these challenges by harmoniously applying joint probabilistic data association in concert with backward smoothing. This is coupled with the McReynolds Filter/Smoother consistency checks to further minimize mis-associations. These methodologies are implemented to reduce the impacts of early ambiguity by sharing information across track updates and delaying hard decisions on track formation until sufficient data have been processed. Through filter/smoother assessment against both sparse and dense synthetically-created GEO optical data, the utility of mutually applying these two distinct techniques within the construct of the CAR-MHF algorithm for robust multiple space object tracking is be demonstrated.

Multi-object tracking through learning relational appearance features and motion patterns

High tracking frame rates have been achieved based on traditional tracking methods which however would fail due to drifts of the object template or model, especially when the object disappears from the camera’s field of view. To deal with it, tracking-and-detection-combination has become more and more popular for long-term unknown object tracking, whose detector almost does not drift and can regain the disappeared object when it comes back. However, for online machine learning and multiscale object detection, expensive computing resources and time are required. So it is not a good idea to combine tracking and detection sequentially like Tracking-Learning-Detection algorithm. Inspired from parallel tracking and mapping, this article proposes a framework of parallel tracking and detection for unknown object tracking. The object tracking algorithm is split into two separate tasks—tracking and detection which can be processed in two different threads, respectively. One thread is used to deal with the tracking between consecutive frames with a high processing speed. The other thread runs online learning algorithms to construct a discriminative model for object detection. Using our proposed framework, high tracking frame rates and the ability of correcting and recovering the failed tracker can be combined effectively. Furthermore, our framework provides open interfaces to integrate state-of-the-art object tracking and detection algorithms. We carry out an evaluation of several popular tracking and detection algorithms using the proposed framework. The experimental results show that different tracking and detection algorithms can be integrated and compared effectively by our proposed framework, and robust and fast long-term object tracking can be realized.

The multi-object tracking (MOT) algorithms of the joint detection and embedding (JDE) approach estimate bounding boxes and re-identification (re-ID) features of objects with the single network, which balance the tracking accuracy and inference speed. However, when the appearance information between different objects is highly similar, these algorithms are usually easy to cause identity switches, and the comprehensive tracking performance is poor in crowded scenes. Aiming at the above problems, we propose a stronger multi-object tracking algorithm termed as ReIMOT, based on FairMOT. A joint loss function of combining normalized Softmax Loss and the center distance penalty term is designed to supervise the re-ID branch, which increases the intra-class similarity and makes the extracted appearance features more discriminative. To further improve the tracking performance, we introduce coordinate attention to make the encoder-decoder network focus more on features of interest. The experimental results show that the proposed ReIMOT is more effective than the other advanced multi-object tracking algorithms, and decreases the number of ID switches by 13.8% compared to FairMOT on the MOT17 dataset.

3D fluorescence microscopy of living organisms has increasingly become an essential and powerful tool in biomedical research and diagnosis. An exploding amount of imaging data has been collected, whereas efficient and effective computational tools to extract information from them are still lagging behind. This is largely due to the challenges in analyzing biological data. Interesting biological structures are not only small, but are often morphologically irregular and highly dynamic. Although tracking cells in live organisms has been studied for years, existing tracking methods for cells are not effective in tracking subcellular structures, such as protein complexes, which feature in continuous morphological changes including split and merge, in addition to fast migration and complex motion. In this paper, we first define the problem of multi-object portion tracking to model the protein object tracking process. A multi-object tracking method with portion matching is proposed based on 3D segmentation results. The proposed method distills deep feature maps from deep networks, then recognizes and matches objects' portions using an extended search. Experimental results confirm that the proposed method achieves 2.96% higher on consistent tracking accuracy and 35.48% higher on event identification accuracy than the state-of-art methods.

PHD filtering is a common and effective multiple object tracking (MOT) algorithm used in scenarios where the number of objects and their states are unknown. In scenarios where each object can generate multiple measurements per scan, some PHD filters can estimate the extent of the objects as well as their kinematic properties. Most of these approaches are, however, not able to inherently estimate trajectories and rely on ad-hoc methods, such as different labeling schemes, to build trajectories from the state estimates. This paper presents a Gamma Gaussian inverse Wishart mixture PHD filter that can directly estimate sets of trajectories of extended targets by expanding previous research on tracking sets of trajectories for point source objects to handle extended objects. The new filter is compared to an existing extended PHD filter that uses a labeling scheme to build trajectories, and it is shown that the new filter can estimate object trajectories more reliably.

We propose a novel multiple object tracking algorithm in a particle filter framework, where the input is a set of candidate regions obtained from Robust Principle Component Analysis (RPCA) in each frame, and the goals is to recover trajectories of objects over time. Our method adapts to the changing appearance of objects, due to occlusion, illumination changes and large pose variations, by incorporating a <i>l</i>1 minimization-based appearance model into the Maximize A Posterior (MAP) inference. Though L1 trackers have showed impressive tracking accuracy, they are computationally demanding for multiple object tracking. Conventional data association methods using simple nonparametric appearance model, such as histogram-based descriptor, may suffer from drastic changing object appearance. The robust tracking performance of our approach has been validated with a comprehensive evaluation involving several challenging sequences and state-of-the-art multiple object trackers.

Video security monitoring has always been an important mission for safety reason. In an entire surveillance system, there are usually several cameras distributed sparsely to cover a wide range of public areas (e.g., school, shopping mall or infrastructure). Tracking person through this cameras network is challenging due to different camera perspectives, illumination changes and pose variations. Several algorithms for Multi-Target Multi-Camera tracking (MTMCT) have been proposed in offline method which has delay in getting result. Addressing the need for real-time computation of people tracks through multi camera, the paper proposes an online tracking algorithm. The contributions include (1) online real-time framework which can be used in practical application, (2) extend a single camera multi object tracking (MOT) algorithm to be suitable for multi-camera tracking and (3) use spatial-temporal information to strengthen cross camera person recall performance. The proposed algorithm has been benchmarked against the literature review of MTMC algorithms.

A smart traffic analysis system could be used to reduce congestion, prevent accidents, as well as to control traffic flow. Such a system would need to make use of many technologies, such as computer networking, communication, image processing, object detection, and tracking. In this paper, we introduce an efficient vehicle tracking algorithm which could be used to help create a smart city traffic control system. Our algorithm is targeted to solve the multi-object tracking (MOT) problem. Our method follows a track-by-detection paradigm, i.e., it relies on vehicle detection results to decide whether a detected vehicle in sequential frames belong to the same track. Our vehicle tracker extends the intersection over union (IOU) tracker and improves upon it by fusing historical tracking information, taking into consideration the balance between tracking efficiency and effectiveness. We demonstrate the effectiveness and efficiency of our approach using the UA-DETRAC benchmark dataset. Our proposed method runs at an average speed of 1,264 frames per second. We conclude that our tracker could be useful for applications running in a real-time environment.

To adjust management practices and improve wine marketing strategies, accurate vineyard yield estimation early in the growing season is essential. Conventional methods for yield forecasting rely on phenotypic features' manual assessment, which is time- and labor-intensive and often destructive. We combined a deep object segmentation method, mask region-based convolutional neural network (Mask R-CNN), with two potential multi-object tracking algorithms, simple online and real-time tracking (SORT) and intersection-over-union (IOU) trackers to develop a complete visual system that can automatically detect and track individual inflorescences, enabling the assessment of the number of inflorescences per vineyard row from vineyard video footage. The performance of the two tracking algorithms was evaluated using our vineyard dataset, which is more challenging than conventional tracking benchmark datasets owing to environmental factors. Our evaluation dataset consists of videos of four vineyard rows, including 221 vines that were automatically acquired under unprepared field conditions. We tracked individual inflorescences across video image frames with a 92.1% multi-object tracking accuracy (MOTA) and an 89.6% identity F1 score (IDF1). This allowed us to estimate inflorescence count per vineyard row with a 0.91 coefficient of determination (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) between the estimated count and manual-annotated ground truth count. The impact of leaf occlusions on inflorescence visibility was lessened by processing multiple successive image frames with minimal displacements to construct multiple camera views. This study demonstrates the use of deep learning and multi-object tracking in creating a low-cost (requiring only an RGB camera), high-throughput phenotyping system for precision viticulture.

Event cameras utilize novel imaging sensors patterned after visual pathways in the brain that are responsive to low contrast, transient events. Specifically, the pixels of dynamic vision sensors (DVS) react independently and asynchronously to changes in light intensity, creating a stream of time-stamped events encoding the pixels’ (x, y) location in the sensor array and the sign of the brightness change. In contrast with conventional cameras that sample every pixel at a fixed rate, DVS pixels produce output only when the change in intensity has surpassed a set threshold, which leads to reduced power consumption in scenes with relatively little motion. Furthermore, compared to conventional CMOS imaging pixels, DVS pixels have extremely high dynamic range, low latency, and low motion blur. Taken together, these characteristics make event cameras uniquely qualified for persistent surveillance. In particular, we have been investigating their use in port surveillance applications. Such an application of DVS presents the need for automated pattern recognition and object tracking algorithms which can process event data. Due to the fundamentally different nature of the output relative to conventional frame-based cameras, traditional methods of machine learning for computer vision cannot be directly applied. Anticipating this need, this work details data collection and collation efforts to facilitate development of object detection and tracking algorithms in this modality. We have assembled a maritime dataset capturing several moving objects including sail boats, motor boats, large ships, etc.; as well as incidentally captured objects. The data was collected with lenses of various focal lengths and aperture settings to provide data variability and avoid unwanted bias to specific sensor parameters. In addition, the captured data was recorded with the camera in both static and dynamic states. These different states can be used to mimic potential behavior and help understand how this movement can affect the algorithms being developed for automated ship detection and tracking. We will describe the data captured, effects of hardware settings and lenses, as well as how lighting conditions and sensor movement contributed to the quality of the event data recorded. Finally, we will detail future data collection efforts.