DVTrack: Deformable Vision Transformer for Visual Object Tracking

Visual object tracking based on discriminant DCT features

Appearance change of moving targets is a challenging problem in visual tracking. In this paper, we present a novel visual object tracking algorithm based on the observation dependent hidden Markov model (OD-HMM) framework. The observation dependency is computed by structure complexity coefficients (SCC) which is defined to predict the target appearance change. Unlike conventional methods addressing the appearance change problem by investigating different online appearance models, we handle this problem by addressing the fundamental reason of motion -related appearance change during visual tracking. Based on the analysis of motion-related appearance change, we investigate the relationship between the structure of the object surface and the appearance stability. The appearance of complex structural regions is easier to change compared with that of smooth structural regions with object moving. Based on this, we define SCC to predict the appearance stability of moving objects. Different from the standard HMM-based tracking algorithms where observations between different frames are assumed to be independent, we consider the observation dependency between consecutive frames with the information provided by SCC. Moreover , we present a novel outlier removing method in appearance model updating which helps to avoid error accumulation. Experimental results on challenging video sequences demonstrate that the proposed visual tracking algorithm with OD-HMM and SCC achieves better performance than existing related tracking algorithms.

Visual object tracking is a key factor for unmanned aerial vehicles (UAVs). In this paper, we propose a robust and effective visual object tracking method with an appearance model based on the locally adaptive regression kernel (LARK). The proposed appearance model encodes the geometric structure of the target. The tracking problem is formulated as two binary classifiers via two support vector machines (SVMs) with online model update. The backward tracking which tracks the target in reverse of time is utilized to evaluate the accuracy and robustness of the two SVMs. The final locations are adaptively fused based on the results of the forward tracking and backward tracking validation. Several state-of-the-art tracking algorithms are evaluated on large scale benchmark datasets which include challenging factors such as heavy occlusion, pose variation, illumination variation and motion blur. Experimental results demonstrate that our method achieves appealing performance.

We propose a probabilistic filtering method which fuses joint measurements with depth images to yield a precise, real-time estimate of the end-effector pose in the camera frame. This avoids the need for frame transformations when using it in combination with visual object tracking methods. Precision is achieved by modeling and correcting biases in the joint measurements as well as inaccuracies in the robot model, such as poor extrinsic camera calibration. We make our method computationally efficient through a principled combination of Kalman filtering of the joint measurements and asynchronous depth-image updates based on the Coordinate Particle Filter. We quantitatively evaluate our approach on a dataset recorded from a real robotic platform, annotated with ground truth from a motion capture system. We show that our method is robust and accurate even under challenging conditions such as fast motion, significant and long-term occlusions, and time-varying biases. We release the dataset along with open-source code of our method to allow quantitative comparison with alternative approaches.

Recently, Gall & Lempitsky [6] and Okada [9] introduced Hough Forests (HF), which emerged as a powerful tool in object detection, tracking and several other vision applications. HFs are based on the generalized Hough transform [2] and are ensembles of randomized decision trees, consisting of both classification and regression nodes, which are trained recursively. Densly sampled patches of the target object {Pi = (Ai,yi,di)} represent the training data, where Ai is the appearance, yi the label, and di a vector pointing to the center of the object. Each node tries to find an optimal splitting function by either optimizing the information gain for classification nodes or the variance of offset vectors di for regression nodes. This yields quite clean leaf nodes according to both, appearance and offset. However, typically HFs are trained in off-line mode, which means that they assume having access to the entire training set at once. This limits their application in situations where the data arrives sequentially, e.g., in object tracking, in incremental, or large-scale learning. For all of these applications, on-line methods inherently can perform better. Thus, we propose in this paper an on-line learning scheme for Hough forests, which allows to extend their usage to further applications, such as the tracking of arbitrary target instances or large-scale learning of visual classifiers. Growing such a tree in an on-line fashion is a difficult task, as errors in the hard splitting rules cannot be corrected easily further down the tree. While Godec et al. [8] circumvent the recursive on-line update of classification trees by randomly growing the trees to their full size and just update the leaf node statistics, we integrate the ideas from [5, 10] that follow a tree-growing principle. The basic idea there is to start with a tree consisting of only one node, which is the root node and the only leaf at that time. Each node collects the data falling in it and decides on its own, based on a certain splitting criterion, whether to split this node or to further update the statistics. Although the splitting criteria in [5, 10] have strong theoretical support, we will show in the experiments that it even suffices to only count the number n of samples Pi that a node has already incorporated and split when n > γ , where γ is a predefined threshold. An overview of this procedure is given in Figure 1. This splitting criterion requires to find reasonable splitting functions with only a small subset of the data, which does not necessarily have to be a disadvantage when building random forests. As stated in Breiman [4], the upper bound for the generalization error of random forests can be optimized with a high strength of the individual trees but also a low correlation between them. To this end, we derive a new but simple splitting procedure for off-line HFs based on subsampling the input space on the node level, which can further decrease the correlation between the trees. That is, each node in a tree randomly samples a predefined number γ of data samples uniformly over all available data at the current node, which is then used for finding a good splitting function. In the first experiment, we demonstrate on three object detection data sets that both, our on-line formulation and subsample splitting scheme, can reach similar performance compared to the classical Hough forests and can even outperform them, see Figures 2(a)&(b). Additionally, during training both proposed methods are orders of magnitudes faster than the original approach (Figure 2(c)). In the second part of the experiments, we demonstrate the power of our method on visual object tracking. Especially, our focus lies on tracking objects of a priori unknown classes, as class-specific tracking with off-line forests has already been demonstrated before [7]. We present results on seven tracking data sets and show that our on-line HFs can outperform state-of-the-art tracking-by-detection methods. Figure 1: While labeled samples arrive on-line, each tree propagates the sample to the corresponding leaf node, which decides whether to split the current leaf or to update its statistics.

As a fundamental task in computer vision, visual object tracking has received much attention in recent years. Most studies focus on short-term visual tracking which addresses shorter videos and always-visible targets. However, long-term visual tracking is much closer to practical applications with more complicated challenges. There exists a longer duration such as minute-level or even hour-level in the long-term tracking task, and the task also needs to handle more frequent target disappearance and reappearance. In this paper, we provide a thorough review of long-term tracking, summarizing long-term tracking algorithms from two perspectives: framework architectures and utilization of intermediate tracking results. Then we provide a detailed description of existing benchmarks and corresponding evaluation protocols. Furthermore, we conduct extensive experiments and analyse the performance of trackers on six benchmarks: VOTLT2018, VOTLT2019 (2020/2021), OxUvA, LaSOT, TLP and the long-term subset of VTUAV-V. Finally, we discuss the future prospects from multiple perspectives, including algorithm design and benchmark construction. To our knowledge, this is the first comprehensive survey for long-term visual object tracking. The relevant content is available at https://github.com/wangdong-dut/Long-term-Visual-Tracking.

Visual tracking has attracted a significant attention in the last few decades. The recent surge in the number of publications on tracking-related problems have made it almost impossible to follow the developments in the field. One of the reasons is that there is a lack of commonly accepted annotated data-sets and standardized evaluation protocols that would allow objective comparison of different tracking methods. To address this issue, the Visual Object Tracking (VOT) workshop was organized in conjunction with ICCV2013. Researchers from academia as well as industry were invited to participate in the first VOT2013 challenge which aimed at single-object visual trackers that do not apply pre-learned models of object appearance (model-free). Presented here is the VOT2013 benchmark dataset for evaluation of single-object visual trackers as well as the results obtained by the trackers competing in the challenge. In contrast to related attempts in tracker benchmarking, the dataset is labeled per-frame by visual attributes that indicate occlusion, illumination change, motion change, size change and camera motion, offering a more systematic comparison of the trackers. Furthermore, we have designed an automated system for performing and evaluating the experiments. We present the evaluation protocol of the VOT2013 challenge and the results of a comparison of 27 trackers on the benchmark dataset. The dataset, the evaluation tools and the tracker rankings are publicly available from the challenge website (http://votchallenge.net).

Individual adaptive metric learning for visual tracking

Despite remarkable progress, visual object tracking is still a challenging task as objects usually suffer from significant appearance changes, fast motion, and serious occlusion. In this paper, we propose an anti-occlusion correlation filter-based tracking method (AO-CF) for robust visual tracking. We first propose an occlusion criterion based on continuous response values. Based on the criterion, objects are divided into four categories to adaptively identify the occlusion of objects. Then we propose a new detection condition for detecting proposals. When the occlusion criterion is triggered, the re-detection mechanism is executed and the tracker is commanded to stop, and then the re-detector selects the most reliable proposal to reinitialize the tracker. Experimental results show that our method outperforms other state-of-the-art trackers in terms of both precision rate and success rate on the widely used object tracking benchmark dataset. In addition, AO-CF is able to achieve real-time tracking speed.

While many visual tracking methods have been proposed, the challenges such as severe deformation, cluttered background and heavy occlusion are still under well solved. This paper proposed a new visual tracking method by combing the sparse representation and global measure. We flrst construct an appearance model by sparse representation with the patches from target rectangles and their surrounding backgrounds. With this appearance model, we successfully identify the patches of target undergoing heavy occlusion or severe deformation. Then, we deflne a global measure based on the cues of saliency and color contrast to detect the tracking object. This measure performs well in distinguishing object from background especially in cluttered background. Many experiments on difierent challenge image sequences demonstrate that our method is much more robust and stable in tracking.

This work deals with visual object tracking. The well known discriminant correlation filter (DCF) based approach is improved by multi-layer CNN features, spatial reliability (through a foreground mask) and conditionally model updating strategy. In the training stage, by calculating a foreground mask using the color histograms, for each chosen CNN layer, a correlation filter is trained under the foreground constraint to construct a weak tracker. In next frame, the tracking position is from the weighting of weak trackers, for which the weights are computed by Hedge method. The response peak and oscillation are both considered to estimate the confidence criteria. The model and weight of each weak tracker are updated only when the tracking is high-confident. We analyze and evaluate our system on OTB-13 dataset, and show that our approach performs superiorly against several state-of-the-art methods.

Visual object tracking is a classical, but still open research problem in computer vision, with many real world applications. The problem is challenging due to several factors, such as illumination variation, occlusions, camera motion and appearance changes. Such problems can be alleviated by constructing robust, discriminative and computationally efficient visual features. Recently, biologically-inspired channel representations [9] have shown to provide promising results in many applications ranging from autonomous driving to visual tracking.

Visual tracking has recently gained a great advance with the use of the convolutional neural network (CNN). Usually, existing CNN‐based trackers exploit the features from a single layer or a certain combination of multiple layers. However, these features only characterise an object from an invariable aspect and cannot adapt to scene variation, which limits the performance of such trackers. To overcome this limitation, the authors study the problem from a new perspective and propose a novel convolutional layer selection method. To obtain robust appearance representation, they investigate the advantages of features extracted from different convolutional layers. To determine the correctness of the tracking prediction and updated model, they design a verification mechanism based on historical retrospect, which can estimate the deviation for each layer by bidirectionally locating the target. Meanwhile, the deviation works as the layer‐wise selection criteria. Extensive evaluations on the OTB‐2013, visual object tracking (VOT)‐2016 and VOT‐2017 benchmarks demonstrate that the proposed tracker performs favourably against several state‐of‐the‐art trackers.

Due to the effects of deformation, scale changes and background interference, visual tracking has always been one of the challenging problems. In recent years, kernelized correlation filter (KCF) has attracted much attention. However, the tracking accuracy of KCF tracker is poor when the object fast motion, heavy occlusion and scale variation appear. Herein, this reason is analyzed that the KCF tracker generates candidate patches of object according to its location in the last frame. By establishing grey prediction model of object motion, we produce the candidate patches around the predicted location. Moreover, we apply an adaptive strategy to update model and employ a simple mechanism for scale estimation. We conduct our tracker on some tracking benchmarks, including OTB2013, VOT2016 and UAV123. Experimental results demonstrate the proposed tracking method has good and stable results.

Attention shake siamese network with auxiliary relocation branch for visual object tracking