Multiple object tracking: A literature review

Multiple Object Tracking (MOT) has attracted increasing attention due to its academic and commercial potential. However, it still remains challenging due to factors like data association ambiguity and abrupt appearance changes. To address these issues, we extended the multiple hypothesis tracking (MHT) by incorporating the appearance features which extracted from faster region convolutional neural network (Faster-RCNN). Additionally, a principal component analysis method was introduced to update discriminative appearance features. Thus, the data association ambiguity can be reduced surprisingly by exploiting Faster-RCNN, which means fewer hypothesis for MHT. Meanwhile, the issue of abrupt appearance changes can be addressed by modeling discriminative appearance feature based-on MHT. Many experiments on MOT benchmark show competitive performance of the proposed approach in comparison with other state-of-art tracking methods.

This thesis addresses the problem of tracking under severe appearance changes using a combination of receding horizon and Incremental Locally Linear Embeddings (ILLE) techniques that provides adaptation to both time varying dynamics and abrupt appearance changes. In addition to enabling sustained tracking under these conditions, the proposed technique allows for efficiently detecting appearance changes by simply computing the distance between successive embeddings. These results are illustrated with several examples showing the ability of the proposed technique to flag appearance changes while not breaking track.

This paper applies a pragmatic approach to study the real-time performance effect of software design methods for multiple object tracking (MOT) based on vision digital signal processing (vision DSP). The MOT system in the paper combines target detection, the Hungarian algorithm and the Kernel correlation filter (KCF) tracker. In addition, the MOT system needs to support multiway video streams, so higher speed and storage requirements are necessary for target tracking. Therefore, we carried out some studies on how to improve tracker speed performance and reduce system resource consumption under limited system resources. In the paper, we achieved the goal in two respects. Regarding the data processing, we studied how to efficiently process tracking data by utilizing the parallel characteristics of iDMA (integrated direct memory access) and a DSP core; and regarding the data storage, we proposed a time-sharing strategy to solve the DSP local memory (data RAM) usage issue for multiple tracking objects. In addition, regarding the software design, we propose a new strategy, which includes two levels of parallel computations: the frame-level parallel computations and the tracking object-level parallel computations. The experimental results show that the KCF tracking algorithm based on vision DSP achieves not only the desired real-time tracking speed but also the expected goal of system resource utilization. Our research methods also provide a reference for algorithm embedded applications in the field of computer vision.

The inverse synthetic aperture lidar (ISAL) have attracted increasing attention for its merits including small visual tracking which is considered as one of the important research topics in the field of computer vision due to its key role in versatile applications, such as precision guidance, intelligent video surveillance, human-computer interaction, robot navigation and public safety. The basic idea for implementing visual tracking is composed of finding the target object in a video or sequence of images, then determining its exact position in the next successive frames and finally generating the corresponding trajectory of this object. Visual tracking, however, is still a challenging problem in practice while taking into account the abrupt appearance changes of the target objects induced by their non-rigid transformation, the sophisticated lighting variation, the obstruction by the block or similar objects in the background and the camera jitter. Motivated by the successful applications in target detection and recognition in recent years, plenty of deep learning models have been integrated in the visual tracking and better performance over traditional methods was achieved in a series of data evaluations, which opens a new door in the field of visual tracking. In this paper, the overview and progress on visual tracking were summarized. The current challenges and corresponding solving approaches in this field are introduced firstly and in particular, several novel and mainstream visual tracking algorithms based on the deep learning are specially described and analyzed in details, including their basic ideas, advantages and disadvantages and future prospect.

This experiment was conducted to improve estrus detection using a thermal sensor based on multiple object tracking system. A total of six Hanwoo (Korean native beef cattle, BW= 404 ± 58 kg) were used in this study. They were housed in a 5 × 10 m pen by three animals per replication. The total amount of moving distance and feed intake were measured before and after estrus period. The infrared thermography camera (FLIR A615, FLIR SYSTEMS, USA) was placed approximately 9 m above ground to take thermal images. The video files were analyzed to calculate moving distance of experimental animals by the tracking system which was developed by our previous research. Each animal in a pen was tracked on thermal video files, and then moving distances were calculated by topographic surface-based object segmentation, velocity weight, and the river-bed refinement. Feed intake was calculated every day at 0900 by weighing residual feed. On the day before estrus, the total moving distance of experimental animals was 6,434 m (Pen 1= 6,873 m, Pen 2= 5,995m), whereas it was 15,121 m (Pen 1= 17,434 m and Pen 2= 12,808 m) on the estrus day, showing 2.35 times increase (P < 0.05) in moving activity. Feed intake was decreased (P < 0.05) by 11% on estrus day (29.4 vs. 26.1 kg). A decrease in feed intake and an increase in movement of beef cattle on the estrus day, were obvious signs that have been well documented not only in previous researches and but empirical knowledge. In this study, however, the exact moving distance of Korean native beef cattle was calculated by the thermal imaging video tracking system. Therefore, this multiple object tracking system could be applied in practice to detect estrus of beef cattle.

In agriculture, automating the accurate tracking of fruits, vegetables, and fiber is a very tough problem. The issue becomes extremely challenging in dynamic field environments. Yet, this information is critical for making day-to-day agricultural decisions, assisting breeding programs, and much more. To tackle this dilemma, we introduce <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NTrack</i> , a novel multiple object tracking framework based on the linear relationship between the locations of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">neighboring</i> tracks. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NTrack</i> computes dense optical flow and utilizes particle filtering to guide each tracker. Correspondences between detections and tracks are found through data association via direct observations and indirect cues, which are then combined to obtain an updated observation. Our modular multiple object tracking system is independent of the underlying detection method, thus allowing for the interchangeable use of any off-the-shelf object detector. We show the efficacy of our approach on the task of tracking and counting infield cotton bolls. Experimental results show that our system exceeds contemporary tracking and cotton boll-based counting methods by a large margin. Furthermore, we publicly release the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">first</i> annotated cotton boll video dataset to the research community. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This work is motivated by the need to provide highly-accurate estimates of the total number of cotton bolls across an entire farm. We provide a multiple object tracking framework for automating the counting process using a dynamic motion model that can reidentify severely occluded objects. This information is immensely beneficial to agronomists and breeders. For example, it can allow them to accelerate the selection of genotypes and identify cotton cultivars that exhibit tolerance to adverse environmental conditions (e.g., drought, poor soil quality, etc.) via yield prediction. Since the performance of our tracker is tied to the accuracy of the object detector, the ability to swap detectors is important for enhancing the usability of the system. Our dataset structure was modeled after similar multiple object tracking datasets for the purpose of increasing adoption among practitioners. To implement our system, it is assumed that the image/video capture device is of high resolution and data is acquired under ideal lighting conditions. The framework can be run on any commercial-off-the-shelf platform (e.g., ground-based robots, unmanned aerial vehicles, etc.), with sufficient computing and memory resources, using a vision-based sensor.

The task of reliable detection and tracking of multiple objects becomes highly complex for crowded scenarios. In this paper, a robust framework is presented for multi-Human tracking. The key contribution of the work is to use fast calculation for mean shift algorithm to perform tracking for the cases when Kalman filter fails due to measurement error. Local density maxima in the difference image - usually representing moving objects - are outlined by a fast non-parametric mean shift clustering procedure. The proposed approach has the robu st ability to track moving objects, both separately and in groups, in consecutive frames under some kinds of difficulties such as rapid appearance changes caused by image noise and occlusion.

Recently, much work has been devoted to multiple object tracking on the one hand and to appearance model adaptation for a single object tracker on the other side. In this paper, we do both tracking of multiple objects (faces of people) in a meeting scenario and on-line learning to incrementally update the models of the tracked objects to account for appearance changes during tracking. Additionally, we automatically initialize and terminate tracking of individual objects based on low-level features, i.e. face color, face size, and object movement. For tracking a particle filter is incorporated to propagate sample distributions over time. Numerous experiments on meeting data demonstrate the capabilities of our tracking approach. Additionally, we provide an empirical verification of appearance model learning during tracking of an outdoor scene which supports a more robust tracking.

An algorithm for identification of nonsimultaneously arriving measurements of unequal accuracy in multiband radar systems is considered. Its efficiency is estimated.

Multiple object tracking (MOT) techniques can help to build gate-free parking lot management systems purely under vision-based surveillance. However, conventional MOT methods tend to suffer long-term occlusion and cause ID switch problems, making applying them directly in crowded and complex parking lot scenes challenging. Hence, we present a novel disentanglement-based architecture for multi-object detection and tracking to relieve the ID switch issues. First, a background image is disentangled from the original input frame; then, MOT is applied separately to the background and original frames. Next, we design a fusion strategy that can solve the ID switch problem by keeping track of the occluded vehicles while considering complex interactions among vehicles. In addition, we provide a dataset with annotations in severe occlusions parking lot scenes that suits the application. The experiment results show our superiority over the state-of-the-art trackers quantitatively and qualitatively.

Transformer is widely used in Natural Language Processing (NLP), in which numerous papers have been proposed. Recently, the transformer has been borrowed for many computer vision tasks. However, there are few papers to give a comprehensive survey on the vision-based transformer. To this end, we give an in-depth review of the vision-based transformer. We conclude 15 articles covering transformers on image object detection, multiple object tracking, action classification, and visual segmentation. Furthermore, we summarize 6 related datasets for corresponding tasks as well as their metrics. We also provide a comprehensive experimental comparison to validate the strength of transformer-based methods. We provide a brief introduction to the transformer and its applications on computer vision tasks, which can help beginners to recognize it.

Multiple object tracking is one field of study in vision-based navigation. Basically, in many current-research of this study, the common method in object tracking consists two main parts such as object detection and motion prediction, Object detection is highly related to the image processing which has a high tendency in using biggest part of computation resources. But, for the certain purpose, the computation resources should be used in proper way. In this paper, the problem will be limited in the case of football player tracking which needs so many computation resources for the image processing such the motion prediction algorithm that should be significantly reduced this need without decrease the system performance. In this paper, there will be explained the implementation of modified Hungarian algorithm and Kalman Filter as the main core of the motion prediction system in this case. Kalman Filter is the simplest algorithm for motion prediction, but in the case of multiple objects, there should be an additional computing to identify detected object before it will be tracked. Besides that, the system should have an ability to maintain object's identity in many worst case like occlusion, and boundary condition. In this paper, there also be explained the performance and consideration to increase system performance. In the highlight, the implementation result state that multiple point tracking module successfully being implemented as a part of fast multiple object tracking systems for football. The main indicator is identity maintainability which reaches 10% in 800 sample frames. Besides that, the computation parameter could be quantified using statistics that include the threshold value for data association is 2 meters per frame and 6 meters per frame for occlusion threshold. In the computational resources needs analysis, using worst case approximation, the system will need approximately 26 KB memory for computing in each frame. By this result, the conclusion is that the tracking system is proper to be implemented in football player tracking system both from the computation resources need and the performance.

Multiple object tracking is one field of study in vision-based navigation. Basically, in many current-research of this study, the common method in object tracking consists two main parts such as object detection and motion prediction, Object detection is highly related to the image processing which has a high tendency in using biggest part of computation resources. But, for the certain purpose, the computation resources should be used in proper way. In this paper, the problem will be limited in the case of football player tracking which needs so many computation resources for the image processing such the motion prediction algorithm that should be significantly reduced this need without decrease the system performance. In this paper, there will be explained the implementation of modified Hungarian algorithm and Kalman Filter as the main core of the motion prediction system in this case. Kalman Filter is the simplest algorithm for motion prediction, but in the case of multiple objects, there should be an additional computing to identify detected object before it will be tracked. Besides that, the system should have an ability to maintain object's identity in many worst case like occlusion, and boundary condition. In this paper, there also be explained the performance and consideration to increase system performance. In the highlight, the implementation result state that multiple point tracking module successfully being implemented as a part of fast multiple object tracking systems for football. The main indicator is identity maintainability which reaches 10% in 800 sample frames. Besides that, the computation parameter could be quantified using statistics that include the threshold value for data association is 2 meters per frame and 6 meters per frame for occlusion threshold. In the computational resources needs analysis, using worst case approximation, the system will need approximately 26 KB memory for computing in each frame. By this result, the conclusion is that the tracking system is proper to be implemented in football player tracking system both from the computation resources need and the performance.

In multiple object tracking, accurate detection for each of objects that appear sequentially and effective tracking in complicated cases that they are overlapped with each other are very important. In this paper, we propose a multiple object tracking system that has a concrete detection and tracking characteristics by using multi-lateral histogram and SIFT feature extraction algorithm. Especially, by limiting the matching area to object's inside and by utilizing the location informations in the keypoint matching process of SIFT algorithm, we advanced the tracking performance for multiple objects. Based on the experimental results, we found that the proposed tracking system has a robust tracking operation in the complicated environments that multiple objects are frequently overlapped in various of directions.

Observers can visually track multiple objects that move independently even if the scene containing the moving objects is rotated in a smooth way. Abrupt scene rotations yield tracking more difficult but not impossible. For nonrotated, stable dynamic displays, the strategy of looking at the targets' centroid has been shown to be of importance for visual tracking. But which factors determine successful visual tracking in a nonstable dynamic display? We report two eye tracking experiments that present evidence for centroid looking. Across abrupt viewpoint changes, gaze on the centroid is more stable than gaze on targets indicating a process of realigning targets as a group. Further, we show that the relative importance of centroid looking increases with object speed.