Environmental Perception with Spatial Regularization Correlation Filter for Visual Tracking

Discriminative correlation filters (DCF)-based trackers have been increasingly applied to visual tracking due to their high precision while running at high frame rates. However, most recent DCF-based methods solely concentrate on learning the correlation filter with spatial information and thus do not have sufficient descriptive power to discriminate the target from the background in the complex circumstances, such as full occlusion (OCC) and rapid target variation. In this article, we introduce a novel tracking framework that exploits the relationship between the target and its spatiotemporal context to improve tracking accuracy and robustness. Especially, we present our spatiotemporal context model in a hierarchical way, where each layer of the context pyramid is a spatial correlation filter learned from different temporal instances. For gaining an accurate spatiotemporal model, we propose an optimization fusion approach that can adaptively and efficiently learn the effect of each hierarchical layer and exploit these multiple temporal levels of correlation filters for visual tracking. Moreover, an adaptive model update strategy for correlation filters is introduced into the framework to dynamically select proper hierarchical layers, which boosts the temporal diversity of the target appearance, while radically reduces the number of model parameters and guarantees the real-time performance of the tracking method. The experimental results show that, with conventional handcrafted features, our tracker achieves the best success rates among available state-of-the-art trackers with handcrafted features, and provides state-of-the-art performance comparable to those of deep-learning-based trackers on OTB-2013, OTB-2015, VOT-2016, and UAV-20L benchmarks but runs significantly faster than deep trackers.

One of the most important and challenging research topics in the area of computer vision is visual object tracking, which is relevant to many real-world applications. Recently, discriminative correlation filters (DCF) have been demonstrated to overcome the problems in visual object tracking efficiently. So far, only single-resolution feature maps have been utilized in DCF. Owing to this limitation, the potential of DCF has not been exploited. Moreover, convolutional features have demonstrated a better performance for visual tracking than histogram of oriented gradients (HOG) features and color features. Based on these facts, in this paper, we propose collaborative learning based on multi-resolution feature maps for DCF, employing convolutional features. Further, the confidence score, which represents the location of the target object, is selected from various candidates based on certain rules. In addition, the continuous filters are trained to handle the variations of appearance of the target. The extensive experimental results obtained using VOT2015 and OTB-100 benchmark datasets show that the proposed algorithm performs favorably against state-of-the-art tracking algorithms.

Visual object tracking via enhanced structural correlation filter

Recent progress has witnessed continued attention in discriminative correlation filter (DCF) tracking algorithms due to its high-efficiency. However, the existing DCF inevitably introduces some cyclic repetitions in learning and detection, which might lead to the heavy drift problem encountered in significant appearance variants owing to occlusion, deformation and motion blur. In this paper, we propose a dilated-aware discriminative correlation filter framework for visual tracking, which fully exploits multi-scale receptive contextual information of correlation filter to mitigate the impact of unwanted boundary and model degradation. On the premise of nondestructive filtering structure, our method adopts a simple formulation based on Kronecker product over discriminative correlation filter. By hands of multiple dilated factors perceive the multi-level spatial receptive map on objects. The framework learns a reliable response map by the residual understanding of multiple factor-dilated correlations filters. Furthermore, experiment results in a recent comprehensive tracking benchmark demonstrate a promising performance of the proposed method subjectively and objectively compared with several state-of-the-art algorithms.

The correlation filter is suitable for tracking on account of its low computational complexity and promising performance. However, the number of available training samples is limited to the filter size, and the lack of samples leads to poor generalization. Moreover, spectral leakage degrades the filter quality. In this paper, we, therefore, propose a sampling operator for learning a scalable correlation filter in an enlarged window, whose size is larger than the object size. The scalable filter encodes the sparse frequency representation to reconstruct a larger filter with zeros outside of the object in the spatial domain. The sampling operator, which is composed of windowing and sampling operations, enables learning the scalable filter from a large window, and it suppresses spectral leakage. Our method was evaluated on the OTB-100, TC-128, and UAV-123 datasets and achieved the promising results in terms of precision and success rates.

Model drift is a challenging issue for visual object tracking. Most available approaches aim to address this issue by constructing of a stronger discriminative prediction model with the integration of multiple features. In this article, we also address the issue of model drift using multiple features, but we consider the fusion problem from a different point of view, namely, strengthening the features that are suitable for the current scenario while weakening the remaining ones. Therefore, an advanced tracker that adaptively redetermines the reliability for each feature during the tracking process is proposed. Furthermore, to correctly evaluate and redetermine these reliabilities, two different solutions, called model evaluation and numerical optimization, are proposed, and two independent trackers corresponding to these two solutions are implemented. Extensive experiments have been designed on five large datasets to validate the following: 1) The proposed tracking framework is superior for making the tracking model more robust, and 2) the two solutions proposed for redetermining the reliability for each feature are effective. As expected, the two implemented trackers do indeed improve the accuracy and robustness compared to state-of-the-art trackers. Especially on VOT2016, the proposed trackers based on model evaluation and numerical optimization achieve outstanding EAO scores (i.e., <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0.453</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0.428</b> , respectively), outperforming the recently developed top trackers by a large margin.

Correlation filter has been proven to be an effective tool for a number of approaches in visual tracking, particularly for seeking a good balance between tracking accuracy and speed. However, correlation filter-based models are susceptible to wrong updates stemming from inaccurate tracking results. To date, very little effort has been devoted towards handling the correlation filter update problem. In this paper, we propose a novel approach to address the correlation filter update problem. In our approach, we update and maintain multiple correlation filter models in parallel, and we use deep reinforcement learning for the selection of an optimal correlation filter model among them. To facilitate the decision process in an efficient manner, we propose a decision-net to deal with target appearance modeling, which is trained through hundreds of challenging videos using proximal policy optimization and a lightweight learning network. An exhaustive evaluation of the proposed approach on the OTB100 and OTB2013 benchmarks shows that the approach is effective enough to achieve the average success rate of 62.3% and the average precision score of 81.2%, both exceeding the performance of traditional correlation filter-based trackers.

Correlation filter is a simple yet efficient method to deal with the visual tracking task. However, the unwanted boundary effects hinder further performance improvement. Spatially Regularized DCF (SRDCF) has been proposed to address this problem with a pre-computed spatial penalty matrix, which improves the tracking performance greatly. In this paper, aiming to achieve more accurate spatial regularization, we present our spatial adaptive regularized correlation filter (SARCF). A coarse-to-fine scale estimation approach is proposed to change the spatial penalty area, which can efficiently deal with large scale variation. Moreover, temporal regularization is introduced for long-term tracking. Experimental results show that the proposed algorithm outperforms most advanced algorithms in tracking accuracy and success rate.

In this paper, we propose a novel tracking method by formulating tracking as a correlation filtering as well as a ridge regression problem. First, we develop a tight correlation filter-based tracking framework from the signal detection perspective. In this formulation, the correlation filter is set as the same size as the target, which can make full use of the relations of the adjacent image patches and effectively exclude the influence of the background. Specifically, we point out that the novel correlation filter model can be regarded as the ridge regression model which takes into account the different importance of the samples and has the consistent objective with tracking. Second, we focus on the scale variation problem in tracking. By making use of the spatial structure of the correlation filter, the multiscale filter banks can be generated via interpolation to handle the scale estimation problem easily. Third, we present a novel distance importance-based confidence calculation model to determine the final tracking result, which not only makes use of the fine discriminability of the correlation filter but also takes the distance importance of the candidate samples into account to alleviate the impact of similar distractors. Experimental results demonstrate that our method is superior to several state-of-the-art trackers and many other correlation filter-based methods in the benchmark datasets.

Recently, Discriminative Correlation Filters (DCF) have shown excellent performance in visual object tracking. The correlation for a computing response map can be conducted efficiently in Fourier domain by Discrete Fourier Transform (DFT) of inputs, where the DFT of an image has symmetry on the Fourier domain. To enhance the robustness and discriminative ability of the filters, many efforts have been devoted to optimizing the learning process. Regularization methods, such as spatial regularization or temporal regularization, used in existing DCF trackers aim to enhance the capacity of the filters. Most existing methods still fail to deal with severe appearance variations—in particular, the large scale and aspect ratio changes. In this paper, we propose a novel framework that employs adaptive spatial regularization and temporal regularization to learn reliable filters in both spatial and temporal domains for tracking. To alleviate the influence of the background and distractors to the non-rigid target objects, two sub-models are combined, and multiple features are utilized for learning of robust correlation filters. In addition, most DCF trackers that applied 1-dimensional scale space search method suffered from appearance changes, such as non-rigid deformation. We proposed a 2-dimensional scale space search method to find appropriate scales to adapt to large scale and aspect ratio changes. We perform comprehensive experiments on four benchmarks: OTB-100, VOT-2016, VOT-2018, and LaSOT. The experimental results illustrate the effectiveness of our tracker, which achieved a competitive tracking performance. On OTB-100, our tracker achieved a gain of 0.8% in success, compared to the best existing DCF trackers. On VOT2018, our tracker outperformed the top DCF trackers with a gain of 1.1% in Expected Average Overlap (EAO). On LaSOT, we obtained a gain of 5.2% in success, compared to the best DCF trackers.

Correlation filter (CF) based trackers have gained significant attention in the field of visual single-object tracking, owing to their favorable performance and high efficiency; however, existing trackers still suffer from model drift caused by boundary effects and filter degradation. In visual tracking, long-term occlusion and large appearance variations easily cause model degradation. To remedy these drawbacks, we propose a sparse adaptive spatial-temporal context-aware method that effectively avoids model drift. Specifically, a global context is explicitly incorporated into the correlation filter to mitigate boundary effects. Subsequently, an adaptive temporal regularization constraint is adopted in the filter training stage to avoid model degradation. Meanwhile, a sparse response constraint is introduced to reduce the risk of further model drift. Furthermore, we apply the alternating direction multiplier method (ADMM) to derive a closed-solution of the object function with a low computational cost. In addition, an updating scheme based on the APCE-pool and Peak-pool is proposed to reveal the tracking condition and ensure updates of the target’s appearance model with high-confidence. The Kalam filter is adopted to track the target when the appearance model is persistently unreliable and abnormality occurs. Finally, extensive experimental results on OTB-2013, OTB-2015 and VOT2018 datasets show that our proposed tracker performs favorably against several state-of-the-art trackers.

Visual object tracking is a significant technology for camera-based sensor networks applications. Multilayer convolutional features comprehensively used in correlation filter (CF)-based tracking algorithms have achieved excellent performance. However, there are tracking failures in some challenging situations because ordinary features are not able to well represent the object appearance variations and the correlation filters are updated irrationally. In this paper, we propose a local–global multiple correlation filters (LGCF) tracking algorithm for edge computing systems capturing moving targets, such as vehicles and pedestrians. First, we construct a global correlation filter model with deep convolutional features, and choose horizontal or vertical division according to the aspect ratio to build two local filters with hand-crafted features. Then, we propose a local–global collaborative strategy to exchange information between local and global correlation filters. This strategy can avoid the wrong learning of the object appearance model. Finally, we propose a time-space peak to sidelobe ratio (TSPSR) to evaluate the stability of the current CF. When the estimated results of the current CF are not reliable, the Kalman filter redetection (KFR) model would be enabled to recapture the object. The experimental results show that our presented algorithm achieves better performances on OTB-2013 and OTB-2015 compared with the other latest 12 tracking algorithms. Moreover, our algorithm handles various challenges in object tracking well.

Rotation-aware correlation filters for robust visual tracking

Discriminative correlation filter (DCF) has achieved promising performance in visual tracking for its high efficiency and high accuracy. However, DCF trackers usually suffer from some challenges, such as boundary effects and appearance changes. In this paper, we propose a novel correlation tracking method via spatial-temporal constraints and structured sparse regularization. Firstly, we introduce the background-aware selection strategy to extract real negative examples, and penalize the filter coefficients close to the boundary locations for spatial protection, both of which can alleviate the boundary effects. Secondly, we restrict the filters with structured sparse regularization to handle the local appearance changes, and exploit temporal consistent constraint on the filters to address the global appearance changes. Finally, we employ the alternative direction method of multipliers to optimize our correlation tracking model. In our optimization framework, we combine grayscale, color names, histogram of orientation gradient with deep features for appearance learning to improve the discrimination. Meanwhile, we penalize spatial constraint and structured sparse regularization alternatively based on occlusion detection to enhance processing efficiency. The qualitative and quantitative experiments are conducted on the OTB dataset. Experimental results demonstrate that the proposed tracker has better performance than other state-of-the-art trackers.

Region-filtering correlation tracking