Multi-Channel Sparsity Histogram based Particle Filter for Hand Tracking

In visual tracking, developing a robust appearance model is a challenging task due to variations of object appearances such as background clutter, illumination variation and partial occlusion. In existing tracking algorithms, a target candidate is represented by linear combinations of target templates. However, the relationship between a target candidate and the corresponding target templates is nonlinear because of appearance variations. In this paper, we propose a kernelized convex hull based target representation for visual tracking. Namely, a target is represented by a nonlinear combination of target templates in a mapped higher dimensional feature space. The convex hull model can covers the target appearances that do not appear in the target templates. Experimental results demonstrate the robustness and effectiveness of the proposed tracking algorithm against several state-of-the-art tracking algorithms.

Various sparse-representation-based tracking meth-ods have been proposed to tackle visual tracking problems, and most of them use simple intensity feature as the observation of target object. Moreover, most of them only take into account either global or local image representation and only exploit the underlying relationship among target candidates in a single frame. All of these may make their appearance models less robust to deal with complex scenes. To overcome these problems, we propose a smooth manifold kernel sparse tracker under the framework of particle filter. The proposed method characterizes targets and candidates with region covariance matrix descriptors, and constructs object tracking as a kernel sparse learning model based on symmetric positive-definite (SPD) manifolds. The spatial-temporal interdependencies among candidates and global-local representations of candidates are jointly considered and unified via the kernel sparse learning model. Moreover, in order to make the model more robust, the detection of outlier tasks is also taken into account. To handle the variation of object appearance, we develop a robust and efficient online dictionary learning algorithm on SPD manifolds. Extensive experiments on multiple benchmark datasets demonstrate that our tracker performs favorably against state-of-the-art trackers.

Visual tracking is an open issue in computer vision. Designing an robust tracking algorithm is a challenging problem due to complicated appearance variations, such as partial occlusion, illumination variations, fast motion and out-of-plane rotation. Some tracking algorithms use a set of templates to represent a target candidate. Such representations are not robust to drastic appearance variations. In order to alleviate the influence of appearance variations and outliers, we propose a novel tracking algorithm based on a robust Bayesian matrix factorization, where sample data is decomposed into a basis matrix and a coefficient matrix. The sample data is consist of previous tracking results. A column vector is stacked by a tracking result in a frame. A target candidate is represented by a linear combinations of the columns in the basis matrix. The matrix factorization can capture the correlation between different features in sample data. Also the appearance variations in tracking processing can be learnt in matrix factorization. Experimental results demonstrate the appearance models is effective and the proposed tracker in a particle filter tracking framework is robust to outliers and complicated appearance variations.

Automatic prostate segmentation from MR images is an important task in various clinical applications such as prostate cancer staging and MR-guided radiotherapy planning. However, the large appearance and shape variations of the prostate in MR images make the segmentation problem difficult to solve. Traditional Active Shape/Appearance Model (ASM/AAM) has limited accuracy on this problem, since its basic assumption, i.e., both shape and appearance of the targeted organ follow Gaussian distributions, is invalid in prostate MR images. To this end, the authors propose a sparse dictionary learning method to model the image appearance in a nonparametric fashion and further integrate the appearance model into a deformable segmentation framework for prostate MR segmentation. To drive the deformable model for prostate segmentation, the authors propose nonparametric appearance and shape models. The nonparametric appearance model is based on a novel dictionary learning method, namely distributed discriminative dictionary (DDD) learning, which is able to capture fine distinctions in image appearance. To increase the differential power of traditional dictionary-based classification methods, the authors' DDD learning approach takes three strategies. First, two dictionaries for prostate and nonprostate tissues are built, respectively, using the discriminative features obtained from minimum redundancy maximum relevance feature selection. Second, linear discriminant analysis is employed as a linear classifier to boost the optimal separation between prostate and nonprostate tissues, based on the representation residuals from sparse representation. Third, to enhance the robustness of the authors' classification method, multiple local dictionaries are learned for local regions along the prostate boundary (each with small appearance variations), instead of learning one global classifier for the entire prostate. These discriminative dictionaries are located on different patches of the prostate surface and trained to adaptively capture the appearance in different prostate zones, thus achieving better local tissue differentiation. For each local region, multiple classifiers are trained based on the randomly selected samples and finally assembled by a specific fusion method. In addition to this nonparametric appearance model, a prostate shape model is learned from the shape statistics using a novel approach, sparse shape composition, which can model nonGaussian distributions of shape variation and regularize the 3D mesh deformation by constraining it within the observed shape subspace. The proposed method has been evaluated on two datasets consisting of T2-weighted MR prostate images. For the first (internal) dataset, the classification effectiveness of the authors' improved dictionary learning has been validated by comparing it with three other variants of traditional dictionary learning methods. The experimental results show that the authors' method yields a Dice Ratio of 89.1% compared to the manual segmentation, which is more accurate than the three state-of-the-art MR prostate segmentation methods under comparison. For the second dataset, the MICCAI 2012 challenge dataset, the authors' proposed method yields a Dice Ratio of 87.4%, which also achieves better segmentation accuracy than other methods under comparison. A new magnetic resonance image prostate segmentation method is proposed based on the combination of deformable model and dictionary learning methods, which achieves more accurate segmentation performance on prostate T2 MR images.

Real-time object tracking via self-adaptive appearance modeling

Face recognition has always been a challenging task in real-life surveillance videos, with partial occlusion being one of the key factors affecting the robustness of face recognition systems. Previous researches had approached the problem of face recognition with partial occlusions by dividing a face image into local patches, and training an independent classifier for each local patch. The final recognition result was then decided by integrating the results of all local patch classifiers. Such a local approach, however, ignored all the crucial distinguishing information presented in the global holistic faces. Instead of using only local patch classifiers, this paper presents a novel multi-level supporting scheme which incorporates patch classifiers at multiple levels, including both the global holistic face and local face patches at different levels. This supporting scheme employs a novel criteria-based class candidates selection process. This selection process preserves more class candidates for consideration as the final recognition results when there are conflicts between patch classifiers, while enables a fast decision making when most of the classifiers conclude to the same set of class candidates. All the patch classifiers will contribute their supports to each selected class candidate. The support of each classifier is defined as a simple distance-based likelihood ratio, which effectively enhances the effect of a “more-confident” classifier. The proposed supporting scheme is evaluated using the AR face database which contains faces with different facial expressions and face occlusions in real scenarios. Experimental results show that the proposed supporting scheme gives a high recognition rate, and outperforms other existing methods.

A novel compressive local appearance model‐based object tracking algorithm is presented to address challenging issues in object tracking. To efficiently preserve image patches of an object and reduce the dimensionality, a random projection‐based feature selection method is introduced. Modelling the object's appearance using a sparse representation over a set of templates leads to an ℓ 1 ‐regularisation problem. To solve this problem, both the reconstruction error and the residual matrix are considered which play a key role in tracking an object with severe appearance variations using the modified likelihood function. Experimental results demonstrate that the proposed method outperforms existing state‐of‐the‐art tracking methods in terms of dealing with long‐term partial occlusion, deformation and rotation.

Considering the problems of motion blur, partial occlusion and fast motion in target tracking, a target tracking method based on adaptive structured sparse representation with attention is proposed. Under the framework of particle filtering, the performance of high-quality templates is enhanced through an attention mechanism. Structure sparseness is used to build candidate target sets and sparse models between candidate samples and local patches of target templates. Combined with the sparse residual method, reconstruction error is reduced. After optimally solving the model, the particle with the highest similarity is selected as the prediction target. The most appropriate scale is selected according to the multiscale factor method. Experiments show that the proposed algorithm has a strong performance when dealing with motion blur, fast motion, partial occlusion.

Despite many years of research, object tracking remains a challenging problem, not only because of the variety of object appearances, but also because of the complexity of surrounding environments. In this research, we present an algorithm for single object tracking using a particle filter framework and color histograms. Particle filters are iterative algorithms that perform predictions in each iteration using particles, which are samples drawn from a statistical distribution. Color histograms are embedded in these particles, and the distances between histograms are used to measure likelihood between targets and observations. One downside of color histograms is that they ignore spatial information, which may produce tracking failure when objects appear that are similar in color. To overcome this disadvantage, we propose a saliency-based weighting scheme for histogram calculation. Given an image region, first its saliency map is generated. Next, its histogram is calculated based on the generated saliency map. Pixels located in salient regions have higher weights than those in others, which helps preserve the spatial information. Experimental results showed the efficiency of the proposed appearance model in object tracking under various conditions.

Model-free tracking is a widely-accepted approach to track an arbitrary object in a video using a single frame annotation with no further prior knowledge about the object of interest. Extending this problem to track multiple objects is really challenging because: a) the tracker is not aware of the objects' type while trying to distinguish them from background (detection task), and b) The tracker needs to distinguish one object from other potentially similar objects (data association task) to generate stable trajectories. In order to track multiple arbitrary objects, most existing model-free tracking approaches rely on tracking each target individually by updating their appearance model independently. Therefore, in this scenario they often fail to perform well due to confusion between the appearance of similar objects, their sudden appearance changes and occlusion. To tackle this problem, we propose to use both appearance and motion models, and to learn them jointly using graphical models and deep neural networks features. We introduce an indicator variable to predict sudden appearance change and/or occlusion. When these happen, our model does not update the appearance model thus avoiding using the background and/or incorrect object to update the appearance of the object of interest mistakenly, and relies on our motion model to track. Moreover, we consider the correlation among all targets, and seek the joint optimal locations for all targets simultaneously as a graphical model inference problem. We learn the joint parameters for both appearance model and motion model in an online fashion under the framework of LaRank. Experiment results show that our method achieved superior performance compared to the competitive methods.

This paper addresses issues on moving object tracking from videos. We propose a novel tracking scheme that jointly exploits local object features using consensus point correspondences, and global object appearance and shape models using adaptive particle filter-based eigen-tracking. The paper include the following main novelties: (a) employ consensus feature point correspondences to estimate the motion vector of shape model; (b) employ adaptive particle filters and motioncorrected state vector for joint appearance- and shape-based eigen-tracking. An adaptive number of particles is chosen automatically based on an updated estimation of covariancematrix. Further, online learning is made adaptive to avoid learning using partially-occluded objects. The proposed scheme is realized by integrating SURF and RANSAC [8, 9] for estimating consensus point correspondences, and modify an existing particle filter-based eigen-tracking [4]. Experiment results on tracking moving objects in videos have shown that the proposed scheme provides more accurate tracking, especially for objects with fast motion or long-term partial occlusions. The average number of particles is significantly reduced. Comparisons have been made with an existing method, results have shown that the proposed scheme has provided an improved tracking accuracy at the cost of more computations.

Robust infrared target tracking based on particle filter with embedded saliency detection

Deep auto-encoder (DAE) models have been successfully used in object tracking due to its strong capability of feature representation. However, single deep auto-encoder model would not be robust enough to represent the appearance model of outdoor vehicle for its harsh working environment, such as illumination variation, occlusion, cluttered background and so on. In this paper, a novel multiple-DAE-based tracking approach, that is, classifier adaptive fusion for robust outdoor vehicle visual tracking approach is proposed under particle filter framework. Firstly, two deep auto-encoders are offline trained by gray-scale image and gradient image of the raw training images, respectively to obtain the stronger feature representation of gray-scale image and gradient image. Secondly, two classifiers are constructed using the encoder of the two well-trained deep auto-encoders and the output of the each classifier is used to compute the confidence of the corresponding particles. Finally, the confidence output of the two classifiers is fused and applied in online tracking, where, the fusion weight of the each classifier is computed according to the distribution of particles represented by different classifier. Extensive tracking experiments conducted on visual tracking benchmark (VTB) show that the proposed tracking algorithm outperforms 9 popular tracking algorithms in the challenge scenes of outdoor vehicle tracking such as illumination variation, occlusion, cluttered background and scale variation.

This paper addresses the problem of very large-scale image retrieval, focusing on improving its accuracy and robustness. We target enhanced robustness of search to factors such as variations in illumination, object appearance and scale, partial occlusions, and cluttered backgrounds -particularly important when search is performed across very large datasets with significant variability. We propose a novel CNN-based global descriptor, called REMAP, which learns and aggregates a hierarchy of deep features from multiple CNN layers, and is trained end-to-end with a triplet loss. REMAP explicitly learns discriminative features which are mutually-supportive and complementary at various semantic levels of visual abstraction. These dense local features are max-pooled spatially at each layer, within multi-scale overlapping regions, before aggregation into a single image-level descriptor. To identify the semantically useful regions and layers for retrieval, we propose to measure the information gain of each region and layer using KL-divergence. Our system effectively learns during training how useful various regions and layers are and weights them accordingly. We show that such relative entropy-guided aggregation outperforms classical CNN-based aggregation controlled by SGD. The entire framework is trained in an end-to-end fashion, outperforming the latest state-of-the-art results. On image retrieval datasets Holidays, Oxford and MPEG, the REMAP descriptor achieves mAP of 95.5%, 91.5% and 80.1% respectively, outperforming any results published to date. REMAP also formed the core of the winning submission to the Google Landmark Retrieval Challenge on Kaggle.

Robust visual tracking for outdoor vehicle is still a challenging problem due to large appearance variations caused by illumination variation, occlusion and scale variation, etc. In this paper, a deep-learning-based approach for robust outdoor vehicle tracking is proposed. Firstly, a stacked denoising auto-encoder is pre-trained to learn the feature representation way of images. Then, a k-sparse constraint is added to the stacked denoising auto-encoder and the encoder of k-sparse stacked denoising auto-encoder (kSSDAE) is connected with a classification layer to construct a classification neural network. After fine-tuning, the classification neural network is applied to online tracking under particle filter framework. Extensive tracking experiments are conducted on a challenging single object online tracking evaluation platform benchmark to verify the effectiveness of our tracker. Experiments show that our tracker outperforms most state-of-the-art trackers.