Kalman filter and Iterative-Hungarian Algorithm implementation for low complexity point tracking as part of fast multiple object tracking system

Multiple object tracking is one field of study in vision-based navigation. Basically, in many current-research of this study, the common method in object tracking consists two main parts such as object detection and motion prediction, Object detection is highly related to the image processing which has a high tendency in using biggest part of computation resources. But, for the certain purpose, the computation resources should be used in proper way. In this paper, the problem will be limited in the case of football player tracking which needs so many computation resources for the image processing such the motion prediction algorithm that should be significantly reduced this need without decrease the system performance. In this paper, there will be explained the implementation of modified Hungarian algorithm and Kalman Filter as the main core of the motion prediction system in this case. Kalman Filter is the simplest algorithm for motion prediction, but in the case of multiple objects, there should be an additional computing to identify detected object before it will be tracked. Besides that, the system should have an ability to maintain object's identity in many worst case like occlusion, and boundary condition. In this paper, there also be explained the performance and consideration to increase system performance. In the highlight, the implementation result state that multiple point tracking module successfully being implemented as a part of fast multiple object tracking systems for football. The main indicator is identity maintainability which reaches 10% in 800 sample frames. Besides that, the computation parameter could be quantified using statistics that include the threshold value for data association is 2 meters per frame and 6 meters per frame for occlusion threshold. In the computational resources needs analysis, using worst case approximation, the system will need approximately 26 KB memory for computing in each frame. By this result, the conclusion is that the tracking system is proper to be implemented in football player tracking system both from the computation resources need and the performance.

In this paper a multiple object detection, recognition, and tracking system for unmanned aerial vehicles (UAVs) has been studied. The system can be implemented on any UAVs platform, with the main requirement being that the UAV has a suitable onboard computational unit and a camera. It is intended to be used in a maritime object tracking system framework for UAVs, which enables a UAV to perform multiobject tracking and situational awareness of the sea surface, in real time, during a UAV operation. Using machine vision to automatically detect objects in the camera's image stream combined with the UAV's navigation data, the onboard computer is able to georeference each object detection to measure the location of the detected objects in a local North‐East (NE) coordinate frame. A tracking algorithm which uses a Kalman filter and a constant velocity motion model utilizes an object's position measurements, automatically found using the object detection algorithm, to track and estimate an object's position and velocity. Furthermore, a global‐nearest‐neighbor algorithm is applied for data association. This is achieved using a measure of distance that is based not only on the physical distance between an object's estimated position and the measured position, but also how similar the objects appear in the camera image. Four field tests were conducted at sea to verify the object detection and tracking system. One of the flight tests was a two‐object tracking scenario, which is also used in three scenarios with an additional two simulated objects. The tracking results demonstrate the effectiveness of using visual recognition for data association to avoid interchanging the two estimated object trajectories. Furthermore, real‐time computations performed on the gathered data show that the system is able to automatically detect and track the position and velocity of a boat. Given that the system had at least 100 georeferenced measurements of the boat's position, the position was estimated and tracked with an accuracy of 5–15 m from 400 m altitude while the boat was in the camera's field of view (FOV). The estimated speed and course would also converge to the object's true trajectories (measured by Global Positioning System, GPS) for the tested scenarios. This enables the system to track boats while they are outside the FOV of the camera for extended periods of time, with tracking results showing a drift in the boat's position estimate down to 1–5 m/min outside of the FOV of the camera.

High tracking frame rates have been achieved based on traditional tracking methods which however would fail due to drifts of the object template or model, especially when the object disappears from the camera’s field of view. To deal with it, tracking-and-detection-combination has become more and more popular for long-term unknown object tracking, whose detector almost does not drift and can regain the disappeared object when it comes back. However, for online machine learning and multiscale object detection, expensive computing resources and time are required. So it is not a good idea to combine tracking and detection sequentially like Tracking-Learning-Detection algorithm. Inspired from parallel tracking and mapping, this article proposes a framework of parallel tracking and detection for unknown object tracking. The object tracking algorithm is split into two separate tasks—tracking and detection which can be processed in two different threads, respectively. One thread is used to deal with the tracking between consecutive frames with a high processing speed. The other thread runs online learning algorithms to construct a discriminative model for object detection. Using our proposed framework, high tracking frame rates and the ability of correcting and recovering the failed tracker can be combined effectively. Furthermore, our framework provides open interfaces to integrate state-of-the-art object tracking and detection algorithms. We carry out an evaluation of several popular tracking and detection algorithms using the proposed framework. The experimental results show that different tracking and detection algorithms can be integrated and compared effectively by our proposed framework, and robust and fast long-term object tracking can be realized.

Multiple Object Tracking(MOT) developed by Pylyshyn and Storm(1988) has been widely used in the study of capacity-limited and object-based attention. Many researchers interested in finding out whether or not the tracking processes in MOT occupy attentional resource and what type of resource is used. The typical paradigm used in this line of research is dual-task paradigm. Participants were asked to perform a visual or auditory task and the MOT task simultaneously. MOT performance was interfered by both the visual and the auditory task. However, the interference of visual task and auditory task with MOT occurred at different levels. The previous studies demonstrated that the MOT task and visual task occupy visual attention resources. Although the MOT task and auditory task don't occupy visual attention resources, they share more central attention resources(such as executive function). Multiple Identity Tracking(MIT) is a variant of MOT, in which each object carries a unique identity. The previous studies also included both visual or auditory task and MIT task, but those studies have not examined how the visual/auditory task affects the MIT task when the two tasks shared the same properties. The current study included 3 experiments and aimed to investigate the influence of a visual or auditory task on either MOT or MIT task. The first two experiments manipulated participants' eye movement to compare the different effects of visual task and auditory task on MOT performance. The result of experiment 1A showed that the auditory task interfered more with MOT than did the visual task when eyes were fixated at the center of the screen. However, the auditory task yielded less interference with MOT when there was no eye movement control in experiment 1B. The results indicated that the tracking processes in MOT not only occupy visual attention resources, but also occupy central attention resources(such as executive function). The experiment 2 applied visual and auditory digit judgment task and MIT task. When the object identity(marked by a digit) was identical to the number in the digit judgment, the MIT performance was facilitated by both the visual and auditory task. It is possible that the process of digit activated the target identities(the same digits) that was stored in visual working memory during tracking.

Motion prediction of an uncontrolled space target

This paper describes a method for faster and more accurate bronchoscope camera tracking by image registration and camera motion prediction using the Kalman filter. The position and orientation of the bronchoscope camera at a frame of a bronchoscopic video are predicted by the Kalman filter. Because the Kalman filter gives good prediction for image registration, estimation of the position and orientation of the bronchoscope tip converges fast and accurately. In spite of the usefulness of Kalman filters, there have been no reports on tracking bronchoscope camera motion using the Kalman filter. Experiments on eight pairs of real bronchoscopic video and chest CT images showed that the proposed method could track camera motion 2.5 times as fast as our previous method. Experimental results showed that the motion prediction increased the number of frames correctly and continuously tracked by about 4.5%, and the processing time was reduced by about 60% with the search space restriction also proposed in this paper.

Multiple objects tracking is a challenging task. This article presents an algorithm which can detect and track multiple objects, and update target model automatically. The contributions of this paper as follow: Firstly,we also use color histogram(CH) and histogram of orientated gradients(HOG) to represent the objects, model update is realized by kalman filter and gaussian model; secondly we use Gaussian Mixture Model(GMM) and Bhattacharyya distance to detect object appearance. Particle filter with combined features and model update mechanism can improve tracking results. Experiments on video sequences demonstrate that the method presented in this paper can realize multiple objects detection and tracking.

In this study, we propose a multiple vehicle tracking method using multiple hypotheses and the appearance model. The multiple hypotheses are associated with multiple tracks using track-to-multiple hypotheses association method. A target state is estimated using the maximum a posteriori probability estimation method. The posterior probability is proportional to the product of a priori probability and the likelihood that is calculated using similarities of multiple hypotheses and the appearance model. The posterior probability density function is estimated using the Markov chain Monte Carlo particle filter. An optimal posterior target state is determined using a sample with the maximum a posteriori probability. Our experimental results show that the proposed method can improve multiple objects tracking precision as well as multiple object tracking accuracy.

In dynamic environments, a central task of the attentional system is to keep track of objects changing their spatial location over time. In some instances, it is sufficient to track only the spatial locations of moving objects (i.e., multiple object tracking; MOT). In other instances, however, it is also important to maintain distinct identities of moving objects (i.e., multiple identity tracking; MIT). Despite previous research, it is not clear whether MOT and MIT performance emerge from the same tracking mechanism. In the present report, we study gaze coherence (i.e., the extent to which participants repeat their gaze behaviour when tracking the same object locations twice) across repeated MOT and MIT trials. We observed more substantial gaze coherence in repeated MOT trials compared to the repeated MIT trials or mixed MOT-MIT trial pairs. A subsequent simulation study suggests that MOT is based more on a grouping mechanism than MIT, whereas MIT is based more on a target-jumping mechanism than MOT. It thus appears unlikely that MOT and MIT emerge from the same basic tracking mechanism.

Kalman filter and its families have played an important role in information gathering, such as target tracking. Data association techniques have also been developed to allow the Kalman filter to track multiple targets simultaneously. This paper revisits the principle and applications of the Kalman filter for single target tracking and multiple hypothesis tracking (MHT) for multiple target tracking. We present the brief review of the Bayes filter family and introduce a brief derivation of the Kalman filter and MHT. We show examples for both single and multiple targets tracking in simulation to illustrate the efficacy of Kalman filter-based algorithms in target tracking scenarios.

The task of reliable detection and tracking of multiple objects is a challenging problem. In this paper, we introduce an online multi-object tracking approach based on Kalman filter framework. Moving objects are extracted relying on frame difference and labeled by an adaptive labeling method. Each object is assigned an independent Kalman filter to predict the location in consecutive frame. Data association is performed using Hungarian assignment algorithm. Experimental results show that the approach is able to automatically and robustly track a variable number of objects and maintain their identities in complex environment.

Multiple object tracking (MOT) is generally regarded as a pure attention-consuming task that draws heavily on attention resources. In the present study, we adopted a cross-channel visual-audio dual-task paradigm, i.e., the MOT task combined with the concurrent auditory N-back working memory task, to test whether working memory indeed plays a necessary role in the process of multiple tracking, as well as to further identify the specific types of working memory components involved in this process. Experiments 1a and 1b examined the relationship between the MOT task and nonspatial object working memory (OWM) processing by manipulatingthe tracking load and working memory load, respectively. Results in both experiments indicated that the concurrent nonspatial OWM task did not have a significant effect on the tracking capacity of the MOT task. In contrast, Experiments 2a and 2b examined the relationship between the MOT task and spatial working memory (SWM) processing by a similar approach. Results in both experiments indicated that the concurrent SWM task significantly impaired the tracking capacity of the MOT task, showing a gradual decrease with increasing SWM load. Overall, our study provides empirical evidence that multiple object tracking does involve working memory, primarily related to spatial working memory rather than nonspatial object working memory, which sheds more light on the mechanisms of multiple object tracking.

In this letter, we propose a fast mirror-drive pan-tilt target tracking system that can robustly track an object whose appearance varies in a complex background at 500 fps. By assuming a small image displacement between frames, which is a property of high-frame rate vision, we develop an fast object tracking algorithm by hybridizing the convolutional-neural-network (CNN) based object detection with template-matching (TM) based tracking operating at hundreds of frames per second (fps). For object tracking with high-speed visual feedback, the proposed tracking algorithm can remarkably reduce dozens-of-milliseconds-latency in the CNN-based object detection by simultaneously executing TM-based tracking for several images at consecutive frames within a few milliseconds. In the proposed pan-tilt tracking system, when the current tracked objects are occluded or out of the camera view, it can recognize objects to be newly tracked with CNN-based object detection at the rate of 33 fps with acceleration using graphic processing units (GPUs). Controlling the pan-tilt tracking system via visual feedback at 500 Hz, fast moving objects can be robustly tracked at the center of the camera view. The effectiveness of our method was experimentally demonstrated via several results when fast-moving pre-learned objects, such as toy cars were tracked in complex backgrounds.

In the tracking‐by‐detection approach of online multiple object tracking (MOT), a major challenge is how to associate object detections on the new video frame with previously tracked objects. Two important aspects that directly influence the performance of MOT are quality of detection and accuracy in data association. The authors propose an efficient and unified MOT framework for improved object detection, followed by enhanced object tracking. The object detection and tracking are considered as two independent functions in the tracking‐by‐detection paradigm. In this study, object detection accuracy has been increased by employing a faster region‐based convolutional neural network (Faster R‐CNN) modified with the feedback region proposals from the tracker. Target association is performed by the correlation filter‐based Siamese CNN model, which finds the similarity score between the input image patches. The Siamese CNN is trained using a supervised hard sample mining strategy. An optical flow‐based motion model is employed to predict the next probable location of the targets from the tracker and these region proposals are fed back to the classifier module of Faster R‐CNN. The authors’ extensive analysis of publicly available MOT benchmark datasets and comparison with the state‐of‐the‐art tracking methods demonstrate competitive tracking performance of the proposed MOT framework.

Motion prediction of a non-cooperative space target