Complementary Discriminative Correlation Filters Based on Collaborative Representation for Visual Object Tracking

One of the most important and challenging research topics in the area of computer vision is visual object tracking, which is relevant to many real-world applications. Recently, discriminative correlation filters (DCF) have been demonstrated to overcome the problems in visual object tracking efficiently. So far, only single-resolution feature maps have been utilized in DCF. Owing to this limitation, the potential of DCF has not been exploited. Moreover, convolutional features have demonstrated a better performance for visual tracking than histogram of oriented gradients (HOG) features and color features. Based on these facts, in this paper, we propose collaborative learning based on multi-resolution feature maps for DCF, employing convolutional features. Further, the confidence score, which represents the location of the target object, is selected from various candidates based on certain rules. In addition, the continuous filters are trained to handle the variations of appearance of the target. The extensive experimental results obtained using VOT2015 and OTB-100 benchmark datasets show that the proposed algorithm performs favorably against state-of-the-art tracking algorithms.

Appearance change of moving targets is a challenging problem in visual tracking. In this paper, we present a novel visual object tracking algorithm based on the observation dependent hidden Markov model (OD-HMM) framework. The observation dependency is computed by structure complexity coefficients (SCC) which is defined to predict the target appearance change. Unlike conventional methods addressing the appearance change problem by investigating different online appearance models, we handle this problem by addressing the fundamental reason of motion -related appearance change during visual tracking. Based on the analysis of motion-related appearance change, we investigate the relationship between the structure of the object surface and the appearance stability. The appearance of complex structural regions is easier to change compared with that of smooth structural regions with object moving. Based on this, we define SCC to predict the appearance stability of moving objects. Different from the standard HMM-based tracking algorithms where observations between different frames are assumed to be independent, we consider the observation dependency between consecutive frames with the information provided by SCC. Moreover , we present a novel outlier removing method in appearance model updating which helps to avoid error accumulation. Experimental results on challenging video sequences demonstrate that the proposed visual tracking algorithm with OD-HMM and SCC achieves better performance than existing related tracking algorithms.

Recently, Discriminative Correlation Filters (DCF) have shown excellent performance in visual object tracking. The correlation for a computing response map can be conducted efficiently in Fourier domain by Discrete Fourier Transform (DFT) of inputs, where the DFT of an image has symmetry on the Fourier domain. To enhance the robustness and discriminative ability of the filters, many efforts have been devoted to optimizing the learning process. Regularization methods, such as spatial regularization or temporal regularization, used in existing DCF trackers aim to enhance the capacity of the filters. Most existing methods still fail to deal with severe appearance variations—in particular, the large scale and aspect ratio changes. In this paper, we propose a novel framework that employs adaptive spatial regularization and temporal regularization to learn reliable filters in both spatial and temporal domains for tracking. To alleviate the influence of the background and distractors to the non-rigid target objects, two sub-models are combined, and multiple features are utilized for learning of robust correlation filters. In addition, most DCF trackers that applied 1-dimensional scale space search method suffered from appearance changes, such as non-rigid deformation. We proposed a 2-dimensional scale space search method to find appropriate scales to adapt to large scale and aspect ratio changes. We perform comprehensive experiments on four benchmarks: OTB-100, VOT-2016, VOT-2018, and LaSOT. The experimental results illustrate the effectiveness of our tracker, which achieved a competitive tracking performance. On OTB-100, our tracker achieved a gain of 0.8% in success, compared to the best existing DCF trackers. On VOT2018, our tracker outperformed the top DCF trackers with a gain of 1.1% in Expected Average Overlap (EAO). On LaSOT, we obtained a gain of 5.2% in success, compared to the best DCF trackers.

Recently, visual object tracking has become a hot topic in computer vision community owing to its extensive applications and fundamental research significance. Many researchers have paid attention to the discriminative correlation filter (DCF) due to its excellent tracking performance. The background-aware CF (BACF) is developed to handle the inevitable boundary effects of DCFs that shows superior performance compared with other CF-based trackers. However, BACF has a poor occlusion handling ability. To overcome this defect, we propose a saliency-aware CF tracking (SACF) model. In SACF, a saliency map of the target is introduced on CFs to strengthen the ability to extract the target from a complex background. Meanwhile, to adjust to the rapid variation of the target appearance, we adaptively modify the learning rate of the filter template using adaptive learning rate. By incorporating the saliency map into the BACF objective function and adaptive learning rate adjustment, the robustness and accuracy of the tracker are boosted significantly. Extensive experiments validate that our method can efficaciously solve the occlusion problem and achieves state-of-the-art performance in terms of accuracy and speed on standard benchmarks, i.e. OTB-2013, OTB-2015 and LaSOT. Compared with other state-of-the-art CF tracking methods, the proposed method is competitive considering accuracy and success rate.

Correlation filter (CF) theory has gained a sustainable attractiveness on the tracking field by virtue of its efficiency in training and decision stage. The seminal idea of cyclic shift operations on image patch makes the CF-based trackers accomplish the dense sampling scheme in an efficient manner. Due to all shifted samples merely provide the single appearance of the target in the previous frame, the filters cannot acquire sufficient information of the target appearance, which leads to the tracking failed. To tackle this problem, we designed a novel tracker using collaborative correlation filters, which collaboratively exploit the historical appearances of the target and the surrounding contexts of the previous frame for the filters learning. Therefore, the collaborative correlation filter simultaneously acquires the variations in target appearance and promotes discriminability when the target suffers from occlusion or motion blur. In addition, a unified optimization procedure is proposed to enable abundant information to be embedded in filter learning, which obtains a closed-form solution. Since the historical appearances of the target are collected from previous tracking results and the target appearance may change drastically or suffer from occlusion, using all of these templates for training is inappropriate. To eliminate the useless samples from the template set, we provide a template updating strategy based on collaborative representation. The experimental results on the standard $$\hbox {OTB}100$$ benchmark data sets illustrate the robustness of the proposed tracker.

The technology of visual object tracking based on correlation filter has good accuracy and efficiency. However, it is still necessary to be study further on the appearance model of the target, the scale variation of the target and so on. This paper proposes a tracking algorithm based on discriminative correlation filtering and a hybrid color feature. The hybrid color feature is composed of two parts, which are compressed color name features and Histogram of Oriented Gradient features based on opponent color space. These two parts features above are extracted from the target patch, respectively. For the first part, color-name features are extracted from a target patch firstly, and then block-based compressed color-name features are extracted according to these color-name features. For the second part, opponent color features are extracted from the target patch firstly, and then HOG features are extracted from these opponent color features. At the basis of the hybrid color feature, two different discriminative correlation filters are used to estimate the translation and the scale of the target, respectively. Finally, extensive experiments show that the tracking algorithm with the hybrid color features of this paper outperforming several state-of-the-art tracking algorithms.

Learning dynamic spatial-temporal regularized correlation filter tracking with response deviation suppression via multi-feature fusion

The Discriminative Correlation Filter (DCF) has been universally recognized in visual object tracking, thanks to its excellent accuracy and high speed. Nevertheless, these DCF-based trackers perform poorly in long-term tracking. The reasons include the following aspects—first, they have low adaptability to significant appearance changes in long-term tracking and are prone to tracking failure; second, these trackers lack a practical re-detection module to find the target again after tracking failure. In our work, we propose a new long-term tracking strategy to solve these issues. First, we make the best of the static and dynamic information of the target by introducing the motion features to our long-term tracker and obtain a more robust tracker. Second, we introduce a low-rank sparse dictionary learning method for re-detection. This re-detection module can exploit a correlation among these training samples and alleviate the impact of occlusion and noise. Third, we propose a new reliability evaluation method to model an adaptive update, which can switch expediently between the tracking module and the re-detection module. Massive experiments demonstrate that our proposed approach has an obvious improvement in precision and success rate over these state-of-the-art trackers.

Visual object tracking is a challenging problem due to appearance variation of target. Correlation filter (CF) -based trackers have shown competing results for visual object tracking. However, they perform poorly in the case of abrupt motion and heavy background clutter due to less use of contextual information. In this paper, we solve this problem by explicitly incorporating contextual information into a context-aware (CA) framework. Under this framework, deep features from higher convolutional layers encode more semantic information of target which are robust to appearance variations, and features from lower layers locate the target more precise. Compared with handcrafted features, DL-based representation learning require less human interventions and provide much better performance. Extensive experimental results on largescale benchmark datasets show that the proposed algorithm performs favorably against state-of-the-art methods.

Discriminative Correlation Filters (DCFs) have demonstrated excellent performance in visual object tracking. These methods utilize a periodic assumption of the training samples to efficiently learn a classifier on image patches; unfortunately, this also introduces unwanted boundary effects. Recently, Spatially Regularized Discriminative Correlation Filters (SRDCFs) were proposed to resolve this issue by introducing penalization weights to the filter coefficients, thereby efficiently reducing boundary effects by assigning higher weights to the background. However, due to the variable target scale, defining the penalization ratio is non trivial; thus, it is possible to penalize the image content while also penalizing the background. In this paper, we investigate SRDCFs and present a novel and efficient part-based tracking framework by exploiting multiple SRDCFs. Compared with existing trackers, the proposed method has several advantages. (1) We define multiple correlation filters to extract features within the range of the object, thereby alleviating the boundary effect problem and avoiding penalization of the target content. (2) Through the combination of cyclic object shifts with penalized filters to build part-based object trackers, there is no need to divide training samples into parts. (3) Comprehensive comparisons demonstrate that our approach achieves a performance equivalent to that of the baseline SRDCF tracker on a set of benchmark datasets, namely, OTB2013, OTB2015 and VOT2017. In addition, compared with other state-of-the-art trackers, our approach demonstrates superior performance.

Multi-level context-adaptive correlation tracking

Template-matching methods for visual tracking have gained popularity recently due to their comparable performance and fast speed. However, they lack effective ways to adapt to changes in the target object's appearance, making their tracking accuracy still far from state-of-the-art. In this paper, we propose a dynamic memory network to adapt the template to the target's appearance variations during tracking. An LSTM is used as a memory controller, where the input is the search feature map and the outputs are the control signals for the reading and writing process of the memory block. As the location of the target is at first unknown in the search feature map, an attention mechanism is applied to concentrate the LSTM input on the potential target. To prevent aggressive model adaptivity, we apply gated residual template learning to control the amount of retrieved memory that is used to combine with the initial template. Unlike tracking-by-detection methods where the object's information is maintained by the weight parameters of neural networks, which requires expensive online fine-tuning to be adaptable, our tracker runs completely feed-forward and adapts to the target's appearance changes by updating the external memory. Moreover, unlike other tracking methods where the model capacity is fixed after offline training --- the capacity of our tracker can be easily enlarged as the memory requirements of a task increase, which is favorable for memorizing long-term object information. Extensive experiments on OTB and VOT demonstrates that our tracker MemTrack performs favorably against state-of-the-art tracking methods while retaining real-time speed of 50 fps.

In this paper, we propose deep discriminative correlation matching within the Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter for online multiple human tracking. In this matching scheme, we mainly exploit the Convolutional Neural Network (CNN) based Discriminative Correlation Filter (DCF) as a target-specific classifier to discriminate the desired target from background and remaining targets. DCFs are learned through the extracted features obtained from the outputs of the last convolutional layers which are capable to encode the target appearances with better discriminativity and robustness to appearance changes. Moreover, we present a hybrid likelihood function that fuses the spatio-temporal relation and correlation matching score to collaboratively enhance the PHD association step. Experimental results on the MOT17 Challenge benchmark [1] confirm the improved performance of our proposed method as compared with other state-of-the-art techniques.

In visual tracking, developing an efficient appearance model is a challenging task due to the influence of various factors, such as illumination variation, occlusion, background clutter, and so on. Existing tracking algorithms use appearance samples from previous frames to form a template set upon which target appearance models are built. However, these appearance models are data-dependent, so they may be corrupted by significant appearance variation. It is difficult to update the templates in challenging environments. In this paper, we propose a robust visual tracking algorithm with an adaptive appearance model using a point-to-set metric learning technique. To do this, we first model a target representation using a set of target templates and a regularized affine hull (RAH) spanned by the target templates. Then, we learn a point-to-set distance metric, which is incorporated into the optimization process to obtain an adaptive target representation. The RAH model covers unseen target appearances by affine combinations of the target templates. Based on the proposed target appearance model, we design an effective template update scheme by adjusting the weights of the target templates. Experimental results on challenging video sequences with comparisons to several state-of-the-art tracking algorithms demonstrate the effectiveness and robustness of the proposed tracking algorithm.

In order to cope with the complex variation of target appearance during visual tracking, a robust tracking algorithm based on multi-scale kernelized least squares (KLS) is proposed. First, by showing that the dense sampling set of translated patches is circulant, using the well-established theory of circulant matrices, kernelized least squares is efficient computed with fast Fourier transform (FFT). Second, we propose a method to judge whether target and background changed significantly by the response variance of all candidate targets, and use it to improve the model updating. Meanwhile, robust scale estimation based on a scale pyramid representation is proposed to handle large scale variations. Extensive experimental results show that our approach outperforms state-of-the-art methods, while operating at real-time.