Coupled data association andl1 minimization for multiple object tracking under occlusion

Model-free tracking is a widely-accepted approach to track an arbitrary object in a video using a single frame annotation with no further prior knowledge about the object of interest. Extending this problem to track multiple objects is really challenging because: a) the tracker is not aware of the objects' type while trying to distinguish them from background (detection task), and b) The tracker needs to distinguish one object from other potentially similar objects (data association task) to generate stable trajectories. In order to track multiple arbitrary objects, most existing model-free tracking approaches rely on tracking each target individually by updating their appearance model independently. Therefore, in this scenario they often fail to perform well due to confusion between the appearance of similar objects, their sudden appearance changes and occlusion. To tackle this problem, we propose to use both appearance and motion models, and to learn them jointly using graphical models and deep neural networks features. We introduce an indicator variable to predict sudden appearance change and/or occlusion. When these happen, our model does not update the appearance model thus avoiding using the background and/or incorrect object to update the appearance of the object of interest mistakenly, and relies on our motion model to track. Moreover, we consider the correlation among all targets, and seek the joint optimal locations for all targets simultaneously as a graphical model inference problem. We learn the joint parameters for both appearance model and motion model in an online fashion under the framework of LaRank. Experiment results show that our method achieved superior performance compared to the competitive methods.

Multiple objects tracking is a challenging task. This article presents an algorithm which can detect and track multiple objects, and update target model automatically. The contributions of this paper as follow: Firstly,we also use color histogram(CH) and histogram of orientated gradients(HOG) to represent the objects, model update is realized by kalman filter and gaussian model; secondly we use Gaussian Mixture Model(GMM) and Bhattacharyya distance to detect object appearance. Particle filter with combined features and model update mechanism can improve tracking results. Experiments on video sequences demonstrate that the method presented in this paper can realize multiple objects detection and tracking.

In this study, we propose a multiple vehicle tracking method using multiple hypotheses and the appearance model. The multiple hypotheses are associated with multiple tracks using track-to-multiple hypotheses association method. A target state is estimated using the maximum a posteriori probability estimation method. The posterior probability is proportional to the product of a priori probability and the likelihood that is calculated using similarities of multiple hypotheses and the appearance model. The posterior probability density function is estimated using the Markov chain Monte Carlo particle filter. An optimal posterior target state is determined using a sample with the maximum a posteriori probability. Our experimental results show that the proposed method can improve multiple objects tracking precision as well as multiple object tracking accuracy.

Model-free object tracking is still challenging because of the limited prior knowledge and the unexpected variation of the target object. In this paper, we propose a feature learning algorithm for model-free multiple object tracking. First, we pre-learn generic features invariant to diverse motion transformations from auxiliary video data by using a deep network of anto-encoder. Then, we adapt the pre-learned features according to multiple target objects respectively in a multi-task learning manner. We treat the feature adaptation for each target object as one single task. We simultaneously learn the common feature shared by all target objects and the individual feature of each object. Experimental results demonstrate that our feature learning algorithm can significantly improve multiple object tracking performance.

Adaptive tracking-by-detection methods use previous tracking results to generate a new training set for object appearance, and update the current model to predict the object location in subsequent frames. Such approaches are typically bootstarpped by manual or semi-automatic initialization in the first several frames. However, most adaptive tracking-bydetection methods focus on tracking of a single object or multiple unrelated objects. Although one can trivially engage several single object trackers to track multiple objects, such solution is frequently suboptimal because it does not utilize the inter-object constraints or the obejct layout information [2]. We propose in this paper an adaptive tracking-by-detection method for multiple objects, inspired by recent work in [1] and [2]. The constraints for structured Support Vector Machine (SVM) in [1] are modified to localize multiple objects simultaneously with both appearance and layout information. Moreover, additional binary constraints are introduced to detect the existences of respective objects and to prevent possible model drift. Thus the method can handle frequent occlusion in multiple object tracking, as well as objects entering or leaving the scene. Those binary constraints make the optimization problem significantly different from the original Structured SVM [3]. The inter-object constraints, embedded in a linear programming technique similar to [2] for optimal position assignment, are applied to diminish false detections. In single object tracking case, given a set of frames {x1,x2, . . . ,xn} indexed by time, and the corresponding set of labeling, i.e. bounding box, {y1,y2, . . . ,yn}, structured SVM tries to find a model f (x,y), such that the task of predicting object location in a testing frame x could be conquered by maximizing: f (x,y) = 〈w,Ψ(x|y)〉, (1)

Despite many years of research, object tracking remains a challenging problem, not only because of the variety of object appearances, but also because of the complexity of surrounding environments. In this research, we present an algorithm for single object tracking using a particle filter framework and color histograms. Particle filters are iterative algorithms that perform predictions in each iteration using particles, which are samples drawn from a statistical distribution. Color histograms are embedded in these particles, and the distances between histograms are used to measure likelihood between targets and observations. One downside of color histograms is that they ignore spatial information, which may produce tracking failure when objects appear that are similar in color. To overcome this disadvantage, we propose a saliency-based weighting scheme for histogram calculation. Given an image region, first its saliency map is generated. Next, its histogram is calculated based on the generated saliency map. Pixels located in salient regions have higher weights than those in others, which helps preserve the spatial information. Experimental results showed the efficiency of the proposed appearance model in object tracking under various conditions.

In dynamic environments, a central task of the attentional system is to keep track of objects changing their spatial location over time. In some instances, it is sufficient to track only the spatial locations of moving objects (i.e., multiple object tracking; MOT). In other instances, however, it is also important to maintain distinct identities of moving objects (i.e., multiple identity tracking; MIT). Despite previous research, it is not clear whether MOT and MIT performance emerge from the same tracking mechanism. In the present report, we study gaze coherence (i.e., the extent to which participants repeat their gaze behaviour when tracking the same object locations twice) across repeated MOT and MIT trials. We observed more substantial gaze coherence in repeated MOT trials compared to the repeated MIT trials or mixed MOT-MIT trial pairs. A subsequent simulation study suggests that MOT is based more on a grouping mechanism than MIT, whereas MIT is based more on a target-jumping mechanism than MOT. It thus appears unlikely that MOT and MIT emerge from the same basic tracking mechanism.

Principal component analysis (PCA) is widely used to extract features and reduce dimensionality in various computer vision and image/video processing tasks. Conventional approaches either lack robustness to outliers and corrupted data or are designed for one-dimensional signals. To address this problem, we propose a robust PCA model for two-dimensional images incorporating structured sparse priors, referred to as structured sparse 2D-PCA. This robust model considers the prior of structured and grouped pixel values in two dimensions. As the proposed formulation is jointly nonconvex and nonsmooth, which is difficult to tackle by joint optimization, we develop a two-stage alternating minimization approach to solve the problem. This approach iteratively learns the projection matrices by bidirectional decomposition and utilizes the proximal method to obtain the structured sparse outliers. By considering the structured sparsity prior, the proposed model becomes less sensitive to noisy data and outliers in two dimensions. Moreover, the computational cost indicates that the robust two-dimensional model is capable of processing quarter common intermediate format video in real time, as well as handling large-size images and videos, which is often intractable with other robust PCA approaches that involve image-to-vector conversion. Experimental results on robust face reconstruction, video background subtraction data set, and real-world videos show the effectiveness of the proposed model compared with conventional 2D-PCA and other robust PCA algorithms.

The Constrained Admissible Region-Multiple Hypothesis Filter (CAR-MHF) has previously been successfully applied to the problem of determining initial state and parameter estimates for non-operational space objects in the near-geosynchronous Earth orbit (GEO) regime. The application of CAR-MHF to a dense population of synthetically-created uncorrelated tracks (UCT) has been primarily challenged by two characteristics of the current implementation. First, the rapid uncertainty growth due to the noise characteristics of processing short-arc optical tracklets leads to ambiguous uncertainty overlap and the prevalence of mis-associated observations, primarily with a large statistical association gate. Second, the reliance on measurement update decisions made on a frame-by-frame basis without a process that evaluates the track estimate based upon all available associated data has prevented additional data from resolving these ambiguous conditions (i.e. if data from another object is statistically valid at a given instant of large ambiguity, an update will be performed and usually drives the filter to diverge). The work addressed in this paper addresses these challenges by harmoniously applying joint probabilistic data association in concert with backward smoothing. This is coupled with the McReynolds Filter/Smoother consistency checks to further minimize mis-associations. These methodologies are implemented to reduce the impacts of early ambiguity by sharing information across track updates and delaying hard decisions on track formation until sufficient data have been processed. Through filter/smoother assessment against both sparse and dense synthetically-created GEO optical data, the utility of mutually applying these two distinct techniques within the construct of the CAR-MHF algorithm for robust multiple space object tracking is be demonstrated.

Multi-person tracking in videos is a promising but challenging visual task. Recent progress in this field is introducing deep convolutional features as appearance models, which achieves robust tracking results when coupled with proper motion models. However, model failures that often cause severe tracking problems, have not been well discussed and addressed in previous work. In this paper, we propose a solution by online detecting such failures and accordingly adjusting the coupling between appearance and motion models. The strategy is letting the functional models take over when certain model faces data association ambiguity, and at the same time suppressing the influence of inappropriate observations during model update. Experimental results prove the benefit of our proposed improvement. Multiple object tracking; deep neural network; online learning; tracking-by-detection; multiple hypothesis tracking (key words)

Multiple Object Tracking(MOT) developed by Pylyshyn and Storm(1988) has been widely used in the study of capacity-limited and object-based attention. Many researchers interested in finding out whether or not the tracking processes in MOT occupy attentional resource and what type of resource is used. The typical paradigm used in this line of research is dual-task paradigm. Participants were asked to perform a visual or auditory task and the MOT task simultaneously. MOT performance was interfered by both the visual and the auditory task. However, the interference of visual task and auditory task with MOT occurred at different levels. The previous studies demonstrated that the MOT task and visual task occupy visual attention resources. Although the MOT task and auditory task don't occupy visual attention resources, they share more central attention resources(such as executive function). Multiple Identity Tracking(MIT) is a variant of MOT, in which each object carries a unique identity. The previous studies also included both visual or auditory task and MIT task, but those studies have not examined how the visual/auditory task affects the MIT task when the two tasks shared the same properties. The current study included 3 experiments and aimed to investigate the influence of a visual or auditory task on either MOT or MIT task. The first two experiments manipulated participants' eye movement to compare the different effects of visual task and auditory task on MOT performance. The result of experiment 1A showed that the auditory task interfered more with MOT than did the visual task when eyes were fixated at the center of the screen. However, the auditory task yielded less interference with MOT when there was no eye movement control in experiment 1B. The results indicated that the tracking processes in MOT not only occupy visual attention resources, but also occupy central attention resources(such as executive function). The experiment 2 applied visual and auditory digit judgment task and MIT task. When the object identity(marked by a digit) was identical to the number in the digit judgment, the MIT performance was facilitated by both the visual and auditory task. It is possible that the process of digit activated the target identities(the same digits) that was stored in visual working memory during tracking.

Multiple object tracking is particularly challenging when many objects with similar appearances occlude one another. Most existing approaches concatenate the states of different objects, view the multi-object tracking as a joint motion estimation problem and search for the best state of the joint motion in a rather high dimensional space. However, this centralized framework suffers from a high computational load. We bring a new view to the tracking problem from a swarm intelligence perspective. In analogy with the foraging behavior of bird flocks, we propose a species-based particle swarm optimization algorithm for multiple object tracking, in which the global swarm is divided into many species according to the number of objects, and each species searches for its object and maintains track of it. The interaction between different objects is modeled as species competition and repulsion, and the occlusion relationship is implicitly deduced from the “power” of each species, which is a function of the image observations. Therefore, our approach decentralizes the joint tracker to a set of individual trackers, each of which tries to maximize its visual evidence. Experimental results demonstrate the efficiency and effectiveness of our method.

In order to realize robust visual tracking in natural environments, a novel algorithm based on adaptive appearance model is proposed. The model can adapt to changes in object appearance over time. A mixture of three Gaussian distributions models the value of each pixel. An online Expectation Maximization (EM) algorithm is developed to update the parameters of the Gaussians. The observation model in the particle filter is designed based on the adaptive appearance model. Numerous experimental results demonstrate that our proposed algorithm can track objects well under illumination change, large pose variation, and partial or full occlusion.Keywordsvisual trackingadaptive appearance modelExpectation Maximization algorithmparticle filter

Chemical exchange saturation transfer (CEST) MRI requires the acquisition of multiple saturation-weighted images and can last several minutes. Misalignments among these images, which are often due to the inevitable motion of the subject, will corrupt CEST contrast maps and result in large quantification errors. Therefore, the registration of the CEST series is critical. However, registration is challenging since common intensity-based registration algorithms may fail to differentiate CEST signals from motion artifacts. Herein, we studied how different patterns of motion affect CEST quantification and proposed a cascaded two-step registration scheme by utilizing features extracted from the entire Z-spectral image series instead of direct registration to a single image. The proposed approach is conducted in two stages: during the first coarse registration, the Z-spectral image series is decomposed by robust principal component analysis (RPCA) to separate CEST contrast from motion. The recomposed image series using only the low-rank component, which contains minimized motion, are averaged to generate a reference for the alignment of the image series. To further remove residual misalignments, the coarse registration is followed by a refinement stage, which uses PCA iteratively to generate motionless synthetic reference series with the first few principal components (PCs) that correspond to CEST contrast. In the end, the quality check is performed to exclude the images with unsuccessful registration. The proposed registration scheme (RPCA + PCA_R) was assessed by both phantom experiments and in vivo data of tumor-bearing mouse brain, with simulated random rigid motion in different patterns applied to the acquired static Z-spectral image series. For comparison, previous correction schemes using an explicit image [either S0 or Ssat(∆ω)] as registration reference were also performed, named as S0_R and Ssat_R respectively. To illustrate the advantage of combination of RPCA and PCA, registration was also exploited using either only the RPCA-based method (RPCA_R) or only the PCA-based one (PCA_R). Compared with the above four methods, RPCA + PCA_R allowed for more accurate correction of the corrupted Z-spectral images, exhibiting smaller MTRasym(∆ω) error maps and lower residual Z-spectra referring to the static data. Among all the five correction methods, the corrected Z-spectral image series by RPCA + PCA_R and the resulting MTRasym(∆ω) maps achieved the highest correlation coefficients (CC) with respect to the static ones. The registration scheme of RPCA + PCA_R provides robust motion correction between two specific Z-spectral images and among an entire image series, through extraction of the static component from the entire Z-spectra set and inclusion of a PCA-based refinement step. Therefore, this method can help improve CEST acquisition and quantification.

This paper presents a novel algorithm - robust kernel principal component analysis (robust KPCA), on the basis of the research of kernel principal component analysis (KPCA) and robust principal component analysis (RPCA). First, this algorithm sets the radius of the images of the training samples in the feature space using kernel tricks, then determines whether the samples are outliers or not, and finally analyzes the training samples which have eliminated the outliers using KPCA algorithm. The improved KPCA algorithm not only retains the non-linearity property of KPCA algorithm but also gets better robustness. Because the effects of outliers are eliminated, robust KPCA algorithm gets higher accuracy than KPCA algorithm for data analysis. The simulation experiments show that the robust KPCA algorithm developed is better than the KPCA algorithm.