Robust Visual Tracking Using Dynamic Classifier Selection with Sparse Representation of Label Noise

Appearance change of moving targets is a challenging problem in visual tracking. In this paper, we present a novel visual object tracking algorithm based on the observation dependent hidden Markov model (OD-HMM) framework. The observation dependency is computed by structure complexity coefficients (SCC) which is defined to predict the target appearance change. Unlike conventional methods addressing the appearance change problem by investigating different online appearance models, we handle this problem by addressing the fundamental reason of motion -related appearance change during visual tracking. Based on the analysis of motion-related appearance change, we investigate the relationship between the structure of the object surface and the appearance stability. The appearance of complex structural regions is easier to change compared with that of smooth structural regions with object moving. Based on this, we define SCC to predict the appearance stability of moving objects. Different from the standard HMM-based tracking algorithms where observations between different frames are assumed to be independent, we consider the observation dependency between consecutive frames with the information provided by SCC. Moreover , we present a novel outlier removing method in appearance model updating which helps to avoid error accumulation. Experimental results on challenging video sequences demonstrate that the proposed visual tracking algorithm with OD-HMM and SCC achieves better performance than existing related tracking algorithms.

As the spread of digital videos, digital cameras, and camera phones, lots of researches are reported about degraded character recognition. It is found that while the grayscale-based classifier is powerful for degraded character, the performance for clear character is not so good as binary-based classifier. In this paper, a dynamic classifier selection method is proposed to combine the two classifiers based on an estimation of the degradation level and the recognition reliability of the input character images. Experimental results show that the proposed method can achieve better recognition performance than the two individual ones.

Deep neural networks, albeit their great success on feature learning in various computer vision tasks, are usually considered as impractical for online visual tracking, because they require very long training time and a large number of training samples. In this paper, we present an efficient and very robust tracking algorithm using a single convolutional neural network (CNN) for learning effective feature representations of the target object in a purely online manner. Our contributions are multifold. First, we introduce a novel truncated structural loss function that maintains as many training samples as possible and reduces the risk of tracking error accumulation. Second, we enhance the ordinary stochastic gradient descent approach in CNN training with a robust sample selection mechanism. The sampling mechanism randomly generates positive and negative samples from different temporal distributions, which are generated by taking the temporal relations and label noise into account. Finally, a lazy yet effective updating scheme is designed for CNN training. Equipped with this novel updating algorithm, the CNN model is robust to some long-existing difficulties in visual tracking, such as occlusion or incorrect detections, without loss of the effective adaption for significant appearance changes. In the experiment, our CNN tracker outperforms all compared state-of-the-art methods on two recently proposed benchmarks, which in total involve over 60 video sequences. The remarkable performance improvement over the existing trackers illustrates the superiority of the feature representations, which are learned purely online via the proposed deep learning framework.

This paper presents a k-nearest oracle-based dynamic ensemble selection method in which overall local accuracy (OLA) and local class accuracy (LCA) are combined into a twostep selection scheme. The OLA and LCA are computed on the neighborhood of the test pattern in a validation set to filter out the classifiers selected by the k-nearest oracles. The complementary information of OLA and LCA has shown to be an interesting alternative to approximate the classification performance to that estimated for the oracle of the initial pool of classifiers. The results were compared with the recognition rates of the majority voting of all classifiers in the initial pool, and also with the recognition rates of related classifier and ensemble selection methods which have inspired the proposed method and its variants. The proposed method achieved the best results on 5 out of 8 experiments using small and large datasets of different applications.

Human observers can correctly attribute changes in the appearance of a scene either to changes in the incident light or to changes in the spectral-reflectance properties of the scene. This ability was assessed as a function of the time course of illuminant and spectral-reflectance changes. Observers were presented with computer simulations of Mondrian patterns of 49 randomly selected Munsell papers. On each trial a Mondrian pattern was presented for 1 s; the pattern then changed either instantaneously or gradually into another Mondrian pattern, also presented for 1 s, which was related to the first either by an illuminant change or by an illuminant change accompanied by additional changes in the spectral-reflectance functions of the individual papers. Illuminant and spectral-reflectance change were applied linearly in time (with respect to CIE coordinates) over intervals ranging from 0 to 7 s. Observers indicated whether there was a spectral-reflectance change. They were able to make reliable discriminations between illuminant and spectral-reflectance changes both when the changes were applied instantaneously and when they were applied gradually over time, but performance worsened progressively as the duration of the changes increased, that is, as their rate decreased. It is suggested that discrimination in this task depends on the extraction of a low-level transient signal which is generated in response to rapid changes in scene appearance and which is progressively attenuated as changes occur more and more gradually.

Sparse representation based methods have recently drawn much attention in visual tracking due to good performance against illumination variation and occlusion. They assume the errors caused by image variations can be modeled as pixel-wise sparse. However, in many practical scenarios these errors are not truly pixel-wise sparse but rather sparsely distributed in a structured way. In fact, pixels in error constitute contiguous regions within the object's track. This is the case when significant occlusion occurs. To accommodate for non-sparse occlusion in a given frame, we assume that occlusion detected in previous frames can be propagated to the current one. This propagated information determines which pixels will contribute to the sparse representation of the current track. In other words, pixels that were detected as part of an occlusion in the previous frame will be removed from the target representation process. As such, this paper proposes a novel tracking algorithm that models and detects occlusion through structured sparse learning. We test our tracker on challenging benchmark sequences, such as sports videos, which involve heavy occlusion, drastic illumination changes, and large pose variations. Experimental results show that our tracker consistently outperforms the state-of-the-art. © 2013 IEEE.

Multiple Instance Learning (MIL) recently provides an appealing way to alleviate the drifting problem in visual tracking. Following the tracking-by-detection framework, an online MILBoost approach is developed that sequentially chooses weak classifiers by maximizing the bag likelihood. In this paper, we extend this idea towards incorporating the instance significance estimation into the online MILBoost framework. First, instead of treating all instances equally, with each instance we associate a significance-coefficient that represents its contribution to the bag likelihood. The coefficients are estimated by a Bayesian formula that jointly considers the predictions from multiple randomized MILBoost classifiers. Next, we incorporate the estimates within a new boosting procedure for more effectively selection of weak classifiers. Experiments with challenging public datasets show that the proposed method outperforms both existing MIL based and boosting based trackers.

In order to realize robust visual tracking in natural environments, a novel algorithm based on adaptive appearance model is proposed. The model can adapt to changes in object appearance over time. A mixture of three Gaussian distributions models the value of each pixel. An online Expectation Maximization (EM) algorithm is developed to update the parameters of the Gaussians. The observation model in the particle filter is designed based on the adaptive appearance model. Numerous experimental results demonstrate that our proposed algorithm can track objects well under illumination change, large pose variation, and partial or full occlusion.Keywordsvisual trackingadaptive appearance modelExpectation Maximization algorithmparticle filter

In this paper we investigate the suitability of different types of Dynamic Classifier Selection approaches for the task of multimodal music mood classification. The dynamic selection methods evaluated were: KNORA-UNION, KNORAELIMINATE, Dynamic Ensemble Selection Performance, Overall Local Accuracy, Local Class Accuracy, Multiple Classifier Behaviour, A Priori and A Posteriori. The experiments were performed using the Brazilian Music Mood Database, which is a multimodal database, containing the audio signal itself, beyond their visual representation (i.e. spectrogram) and the lyrics. The obtained results have shown that the use of dynamic classifier selection methods can improve the classification results for the task at hand.

This paper describes an approach for lip tracking using dynamic deformable templates. The objective is to track lip parameters from an audio-visual corpus recording a voice talent who is reading text prompts in a natural and expressive way. The corpus presents challenges to the conventional method of lip tracking with deformable templates. This is because natural and expressive speech includes relatively large motions of the head and the lips. The head motions lead to changes in the illumination of the face region and changes in the observed lip shape. In addition, emphatic pronunciations lead to large changes in the lip shape. Video frames that are affected by face illumination changes present additional difficulty in locating the mouth region (i.e. region of interest, ROI). Video frames that are affected by changes in lip shapes present additional deviations from the lip templates and hence lower tracking accuracies. Our proposed method incorporates dynamicity in the deformable templates to render them adaptive to changes in head pose, face illumination and lip shapes. Experiments show that dynamic deformable templates consistently outperform the conventional deformable templates in lip tracking.

This paper presents a robust vision algorithm for tracking the boundary of an object with an arbitrary shape by using monocular image sequences. This method consists of a curve registration based optimization technique and a deformable contour model (snakes) for the global and the motion estimations, respectively. By combining techniques, we overcome, among other problems, inaccurate estimate of motion parameters in the curve registration method (which apparently only occur when a rigid or a flexible object is tracked), and the local position variation of the deformable contour model, variations, which are due to noisy images and/or complex backgrounds. The curve registration method uses an iterative algorithm to find the minimum normal distance between two curves, one before motion and the corresponding curve after it. Snakes overcome the limitation of the curve registration method, which suffers from the inaccuracy of motion models. We also propose an internal force, which increases robustness of the deformable contour to background noise. By using the refined snakes' control points, the global update of the previous curve is performed for the re-location of the registered curve. Additionally, we integrate the geometric invariant value of the boundary contour and the curve registration method to solve the occlusion problem in visual tracking. The proposed method is validated through experiments on real images.

Deep face recognition has achieved rapid development but still suffers from occlusions, illumination and pose variations, especially for face identification. The success of deep learning models in face recognition lies in large-scale high quality face data with accurate labels. However, in real-world applications, the collected data may be mixed with severe label noise, which significantly degrades the generalization ability of deep models. To alleviate the impact of label noise on face recognition, inspired by curriculum learning, we propose a self-paced deep learning model (SPDL) by introducing a negative \(l_1\)-norm regularizer for face recognition with label noise. During training, SPDL automatically evaluates the cleanness of samples in each batch and chooses cleaner samples for training while abandons the noisy samples. To demonstrate the effectiveness of SPDL, we use deep convolution neural network architectures for the task of robust face recognition. Experimental results show that our SPDL achieves superior performance on LFW, MegaFace and YTF when there are different levels of label noise.

Similarity mapping for robust face recognition via a single training sample per person

This paper describes an approach for lip tracking using dynamic deformable templates. The objective is to track lip parameters from an audio-visual corpus recording a voice talent who is reading text prompts in a natural and expressive way. The corpus presents challenges to the conventional method of lip tracking with deformable templates. This is because natural and expressive speech includes relatively large motions of the head and the lips. The head motions lead to changes in the illumination of the face region and changes in the observed lip shape. In addition, emphatic pronunciations lead to large changes in the lip shape. Video frames that are affected by face illumination changes present additional difficulty in locating the mouth region (i.e. region of interest, ROI). Video frames that are affected by changes in lip shapes present additional deviations from the lip templates and hence lower tracking accuracies. Our proposed method incorporates "dynamicity" in the deformable templates to render them adaptive to changes in head pose, face illumination and lip shapes. Experiments show that dynamic deformable templates consistently outperform the conventional deformable templates in lip tracking.

Visual object tracking for unmanned aerial vehicles (UAV) is widely used in many fields such as military reconnaissance, search and rescue work, film shooting, and so on. However, the performance of existing methods is still not very satisfactory due to some complex factors including viewpoint changing, background clutters and occlusion. The Siamese trackers, which offer a convenient way of formulating the visual tracking problem as a template matching process, have achieved success in recent visual tracking datasets. Unfortunately, these template match-based trackers cannot adapt well to frequent appearance change in UAV video datasets. To deal with this problem, this paper proposes a template-driven Siamese network (TDSiam), which consists of feature extraction subnetwork, feature fusion subnetwork and bounding box estimation subnetwork. Especially, a template library branch is proposed for the feature extraction subnetwork to adapt to the changeable appearance of the target. In addition, a feature aligned (FA) module is proposed as the core of feature fusion subnetwork, which can fuse information in the form of center alignment. More importantly, a method for occlusion detection is proposed to reduce the noise caused by occlusion. Experiments were conducted on two challenging benchmarks UAV123 and UAV20L, the results verified the more competitive performance of our proposed method compared to the existing algorithms.