English

Model-free tracking is a widely-accepted approach to track an arbitrary object in a video using a single frame annotation with no further prior knowledge about the object of interest. Extending this problem to track multiple objects is really challenging because: a) the tracker is not aware of the objects' type while trying to distinguish them from background (detection task), and b) The tracker needs to distinguish one object from other potentially similar objects (data association task) to generate stable trajectories. In order to track multiple arbitrary objects, most existing model-free tracking approaches rely on tracking each target individually by updating their appearance model independently. Therefore, in this scenario they often fail to perform well due to confusion between the appearance of similar objects, their sudden appearance changes and occlusion. To tackle this problem, we propose to use both appearance and motion models, and to learn them jointly using graphical models and deep neural networks features. We introduce an indicator variable to predict sudden appearance change and/or occlusion. When these happen, our model does not update the appearance model thus avoiding using the background and/or incorrect object to update the appearance of the object of interest mistakenly, and relies on our motion model to track. Moreover, we consider the correlation among all targets, and seek the joint optimal locations for all targets simultaneously as a graphical model inference problem. We learn the joint parameters for both appearance model and motion model in an online fashion under the framework of LaRank. Experiment results show that our method achieved superior performance compared to the competitive methods.

Multiple objects tracking is a challenging task. This article presents an algorithm which can detect and track multiple objects, and update target model automatically. The contributions of this paper as follow: Firstly,we also use color histogram(CH) and histogram of orientated gradients(HOG) to represent the objects, model update is realized by kalman filter and gaussian model; secondly we use Gaussian Mixture Model(GMM) and Bhattacharyya distance to detect object appearance. Particle filter with combined features and model update mechanism can improve tracking results. Experiments on video sequences demonstrate that the method presented in this paper can realize multiple objects detection and tracking.

We propose a real-time multiple object tracking method for smart environment applications. The proposed method combines identity features and tracking features for robust long-term object tracking. Face is a stable feature for identifying different subjects. However, face is not reliable for object tracking. Whole body color histogram is more resistant to object scale and pose changes, which has been widely applied in short-term object tracking. Unfortunately, color histograms are not stable over a relatively long period of time. In smart environments, same subjects may have different clothes over a long time span. We propose a method to combine these two features, the identity features and tracking features, to achieve reliable multiple object tracking in smart environments. A fast object labeling approach is proposed to track multiple objects in real time. The proposed method has clear advantage over traditional single feature methods. Experiments confirm that the proposed method can reliably track multiple human objects in real time through long video sequences.

This paper introduces a joint learning architecture (JLA) for multiple object tracking (MOT) and multiple object forecasting (MOF) in which the goal is to predict tracked objects’ current and future locations simultaneously. MOF is a recent formulation of trajectory forecasting where the full object bounding boxes are predicted rather than trajectories alone. Existing works separate multiple object tracking and multiple object forecasting. Such an approach can propagate errors in tracking to forecasting. We propose a joint learning architecture for multiple object tracking and forecasting (MOTF). Our approach reduces the chances of propagating tracking errors to the forecasting module. In addition, we show, through a new data association step, that forecasting predictions can be used for tracking objects during occlusion. We adapt an existing MOT method to simultaneously predict current and future object locations and confirm that JLA benefits both the MOT and MOF tasks.

In the tracking‐by‐detection approach of online multiple object tracking (MOT), a major challenge is how to associate object detections on the new video frame with previously tracked objects. Two important aspects that directly influence the performance of MOT are quality of detection and accuracy in data association. The authors propose an efficient and unified MOT framework for improved object detection, followed by enhanced object tracking. The object detection and tracking are considered as two independent functions in the tracking‐by‐detection paradigm. In this study, object detection accuracy has been increased by employing a faster region‐based convolutional neural network (Faster R‐CNN) modified with the feedback region proposals from the tracker. Target association is performed by the correlation filter‐based Siamese CNN model, which finds the similarity score between the input image patches. The Siamese CNN is trained using a supervised hard sample mining strategy. An optical flow‐based motion model is employed to predict the next probable location of the targets from the tracker and these region proposals are fed back to the classifier module of Faster R‐CNN. The authors’ extensive analysis of publicly available MOT benchmark datasets and comparison with the state‐of‐the‐art tracking methods demonstrate competitive tracking performance of the proposed MOT framework.

3D tracking of objects and hands in an object manipulation scenario is a very interesting computer vision problem with a wide variety of applications ranging from consumer electronics to robotics and medicine. Recent advances in this research topic allow for 3D tracking of complex scenarios involving bi-manual manipulation of several rigid objects using commodity hardware and with high accuracy. The problem with these approaches is that they treat tracking as a search problem whose dimensionality increases with the number of objects in the scene. This fact typically limits the number of the tracked objects and/or the processing framerate. In this paper we present a method that utilizes simple low level motion cues for dynamically assigning computational resources to parts of the scene where they are actually required. In a series of experiments, we show that this simple idea improves tracking performance dramatically at a cost of only a minor degradation of tracking accuracy. The works that are most related to ours are the approaches by Kyriazis and Argyros on top-down 3D tracking of multiple active objects from RGBD input [1, 2]. The methodological part of our contribution can be briefly described as an extra processing node in the pipeline of [2] which it extends. The tracking approach in [2], the Ensemble of Collaborative Trackers (ECT), regards a set of semi-independent trackers. Each tracker is associated with a distinct object in the scene. For an object to be tracked, a separate optimization problem is solved, one for each frame and each object. Each optimization problem is numerically solved using a black box optimizer, i.e. a variant of the Particle Swarm Optimization (PSO) algorithm. PSO treats the objective function as an oracle and queries it on purposefully evolved “guesses” in the search space, in order to find the optimum. The harder the problem is, the more guesses (budget, which is the product of PSO particles and PSO generations that need to be computed) are required for adequate accuracy to be achieved. For example, tracking the pose and the articulation of the hand amounts to solving a 27-parameter optimization problem. This is much harder than solving for the 3D pose of a rigid object (6 parameters). ECT [2] assigns a fixed amount of computational resources to each tracking sub-problem which depends on this notion of complexity and which is empirically estimated. In this work, we quantify at run time, how hard the tracking of an object should be, not only based on its intrinsic complexity, but also based on its observed dynamics. An object that appears static in the recent temporal window requires less resources compared to an object whose state is more dynamic. Thus, change detection is performed on the image space of color intensities and depth measurements of the RGBD input. In more detail, for the next tracking frame, and the ith tracker, a value mi is computed, which takes the value of 0 if the corresponding tracked object appears to be relatively static, and takes the value of 1 otherwise. For mi to be 1, any of the following need to be true: • the mean value of the pixel-wise differences between the observations of object i and the back-projection of the last estimated configuration of object i is high enough, • the kinetic energy of object i, as computed from so far tracked velocities, suggests a moving object, • the amount of missing depth measurements changes substantially, from one frame to the next, which might me attributed to change in the slant of object i, or • any of the above was satisfied in the recent past (damping). Budget has the trivial minimum value of 0. Moreover, as it has been shown in [3, 4], a PSO budget of 64 particles and 64 generations suffices to track a hand, even in interaction with another hand. No more budget should be required to track simpler structures such as rigid objects. The proposed dynamic budget allocation policy assigns a minimum budget Bmin of 64 particles running for 4 generations to the objects that are static and a maximum budget Bmax that never exceeds the aforementioned (a) Pouring pancake mix (b) Waiting for the mix to cook

We describe a novel probabilistic framework for real-time tracking of multiple objects from combined depth-colour imagery. Object shape is represented implicitly using 3D signed distance functions. Probabilistic generative models based on these functions are developed to account for the observed RGB-D imagery, and tracking is posed as a maximum a posteriori problem. We present first a method suited to tracking a single rigid 3D object, and then generalise this to multiple objects by combining distance functions into a shape union in the frame of the camera. This second model accounts for similarity and proximity between objects, and leads to robust real-time tracking without recourse to bolt-on or ad-hoc collision detection.

In dynamic environments, a central task of the attentional system is to keep track of objects changing their spatial location over time. In some instances, it is sufficient to track only the spatial locations of moving objects (i.e., multiple object tracking; MOT). In other instances, however, it is also important to maintain distinct identities of moving objects (i.e., multiple identity tracking; MIT). Despite previous research, it is not clear whether MOT and MIT performance emerge from the same tracking mechanism. In the present report, we study gaze coherence (i.e., the extent to which participants repeat their gaze behaviour when tracking the same object locations twice) across repeated MOT and MIT trials. We observed more substantial gaze coherence in repeated MOT trials compared to the repeated MIT trials or mixed MOT-MIT trial pairs. A subsequent simulation study suggests that MOT is based more on a grouping mechanism than MIT, whereas MIT is based more on a target-jumping mechanism than MOT. It thus appears unlikely that MOT and MIT emerge from the same basic tracking mechanism.

Multi-person tracking in videos is a promising but challenging visual task. Recent progress in this field is introducing deep convolutional features as appearance models, which achieves robust tracking results when coupled with proper motion models. However, model failures that often cause severe tracking problems, have not been well discussed and addressed in previous work. In this paper, we propose a solution by online detecting such failures and accordingly adjusting the coupling between appearance and motion models. The strategy is letting the functional models take over when certain model faces data association ambiguity, and at the same time suppressing the influence of inappropriate observations during model update. Experimental results prove the benefit of our proposed improvement. Multiple object tracking; deep neural network; online learning; tracking-by-detection; multiple hypothesis tracking (key words)

Adaptive tracking-by-detection methods use previous tracking results to generate a new training set for object appearance, and update the current model to predict the object location in subsequent frames. Such approaches are typically bootstarpped by manual or semi-automatic initialization in the first several frames. However, most adaptive tracking-bydetection methods focus on tracking of a single object or multiple unrelated objects. Although one can trivially engage several single object trackers to track multiple objects, such solution is frequently suboptimal because it does not utilize the inter-object constraints or the obejct layout information [2]. We propose in this paper an adaptive tracking-by-detection method for multiple objects, inspired by recent work in [1] and [2]. The constraints for structured Support Vector Machine (SVM) in [1] are modified to localize multiple objects simultaneously with both appearance and layout information. Moreover, additional binary constraints are introduced to detect the existences of respective objects and to prevent possible model drift. Thus the method can handle frequent occlusion in multiple object tracking, as well as objects entering or leaving the scene. Those binary constraints make the optimization problem significantly different from the original Structured SVM [3]. The inter-object constraints, embedded in a linear programming technique similar to [2] for optimal position assignment, are applied to diminish false detections. In single object tracking case, given a set of frames {x1,x2, . . . ,xn} indexed by time, and the corresponding set of labeling, i.e. bounding box, {y1,y2, . . . ,yn}, structured SVM tries to find a model f (x,y), such that the task of predicting object location in a testing frame x could be conquered by maximizing: f (x,y) = 〈w,Ψ(x|y)〉, (1)

This chapter presents a multi-object tracking system using scale space representation of objects, the method of linear assignment and Kalman filter. In this chapter basically two very prominent problems of multi object tracking have been resolved; the two prominent problems are (i) irrespective of the size of the objects, tracking all the moving objects simultaneously and (ii) tracking of objects under partial and/or complete occlusion. The primary task of tracking multiple objects is performed by the method of linear assignment for which few cost parameters are computed depending upon the extracted features of moving objects in video scene. In the feature extraction phase scale space representation of objects have been used. Tracking of occluded objects is performed by Kalman filter.

Real-time tracking of multiple objects in space-variant vision based on magnocellular visual pathway

With the increasing popularity of Unmanned Aerial Vehicles (UAVs) in computer vision-related applications, intelligent UAV video analysis has recently attracted the attention of an increasing number of researchers. To facilitate research in the UAV field, this paper presents a UAV dataset with 100 videos featuring approximately 2700 vehicles recorded under unconstrained conditions and 840k manually annotated bounding boxes. These UAV videos were recorded in complex real-world scenarios and pose significant new challenges, such as complex scenes, high density, small objects, and large camera motion, to the existing object detection and tracking methods. These challenges have encouraged us to define a benchmark for three fundamental computer vision tasks, namely, object detection, single object tracking (SOT) and multiple object tracking (MOT), on our UAV dataset. Specifically, our UAV benchmark facilitates evaluation and detailed analysis of state-of-the-art detection and tracking methods on the proposed UAV dataset. Furthermore, we propose a novel approach based on the so-called Context-aware Multi-task Siamese Network (CMSN) model that explores new cues in UAV videos by judging the consistency degree between objects and contexts and that can be used for SOT and MOT. The experimental results demonstrate that our model could make tracking results more robust in both SOT and MOT, showing that the current tracking and detection methods have limitations in dealing with the proposed UAV benchmark and that further research is indeed needed.

In this book chapter, we propose a fully automated approach for tracking of multiple objects across multiple cameras with overlapping and non-overlapping views in a unified framework without initial training or prior camera calibration. For tracking with a single camera, Kalman filter and adaptive particle-sampling techniques are integrated for multiple objects tracking. When extended to tracking over multiple cameras, the relations between adjacent cameras are learned systematically by using image registration techniques for consistent handoff of tracking-object labels across cameras. In addition, object appearance measurement is employed to validate the labeling results. Experimental results demonstrate the performance of our approach on real video sequences for cameras with overlapping and non-overlapping views.

In this study, we propose a multiple vehicle tracking method using multiple hypotheses and the appearance model. The multiple hypotheses are associated with multiple tracks using track-to-multiple hypotheses association method. A target state is estimated using the maximum a posteriori probability estimation method. The posterior probability is proportional to the product of a priori probability and the likelihood that is calculated using similarities of multiple hypotheses and the appearance model. The posterior probability density function is estimated using the Markov chain Monte Carlo particle filter. An optimal posterior target state is determined using a sample with the maximum a posteriori probability. Our experimental results show that the proposed method can improve multiple objects tracking precision as well as multiple object tracking accuracy.