Visual Object Tracking by Structure Complexity Coefficients

In visual tracking, developing an efficient appearance model is a challenging task due to the influence of various factors, such as illumination variation, occlusion, background clutter, and so on. Existing tracking algorithms use appearance samples from previous frames to form a template set upon which target appearance models are built. However, these appearance models are data-dependent, so they may be corrupted by significant appearance variation. It is difficult to update the templates in challenging environments. In this paper, we propose a robust visual tracking algorithm with an adaptive appearance model using a point-to-set metric learning technique. To do this, we first model a target representation using a set of target templates and a regularized affine hull (RAH) spanned by the target templates. Then, we learn a point-to-set distance metric, which is incorporated into the optimization process to obtain an adaptive target representation. The RAH model covers unseen target appearances by affine combinations of the target templates. Based on the proposed target appearance model, we design an effective template update scheme by adjusting the weights of the target templates. Experimental results on challenging video sequences with comparisons to several state-of-the-art tracking algorithms demonstrate the effectiveness and robustness of the proposed tracking algorithm.

Significant progress is made in the field of object tracking recently. Especially, trackers based on deep learning and correlation filters both have achieved excellent performance. However, object tracking still faces some challenging problems such as deformation and illumination. In such kinds of situations, the accuracy and precision of tracking algorithms plunge as a result. It is imminent to find a solution to this situation. In this paper, we propose a tracking algorithm based on features extracted by residual network called Resnet features and cascaded correlation filters to improve precision and accuracy. Firstly, features extracted by a deep residual network trained on other image processing datasets, are robust enough and retain higher resolution, therefore, we exploit Resnet-101 pretrained offline to obtain features extracted by middle and high layers for target appearance model representation. Resnet-101 is deeper compared with other deep neural networks which means it contains more semantic information. Then, the method we propose to combine our correlation filters is superior. We propose cascaded correlation filters generated by handcraft, middle-level and high-level features from residual network to gain better competence. Handcraft features localize target precisely because they contain more spatial details while Resnet features are robust to the target appearance change because they retain more semantic information. Finally, we conduct extensive experiments on OTB2013 and OTB2015 benchmark. The experimental results show that our tracker achieves high performance under all kinds of challenges and performs favorably against other state-of-the-art trackers.

In recent years, online building an adaptive target appearance model has been investigated for robust visual tracking in a dynamic environment. However one inherent problem of adaptive appearance trackers is drift, a gradual adaptation of the tracker to non-targets. To alleviate this problem, we consider visual tracking in a novel online manifold regularization (Online-MR) setting where labeled and unlabeled data arrive sequentially in large volume for tracker update. A discriminative target appearance model based on Haar wavelet is learned at each frame and its output score is used as the input sample feature for Online-MR tracker learning. Such a combination of Online-MR semi-supervised learning and online appearance model adaptation results in a robust tracking scheme that can prevent the tracker from drifting while retain its adaptation to appearance changing. Experimental results demonstrate the effectiveness of the proposed method from comparisons with adaptive appearance model based trackers on several challenging video sequences.

One of the main difficulties in visual tracking is to take into account appearance changes (not only of the target but also of or due to the scene, illumination for example). The use of a Bayesian framework is very flexible and has proven to be very efficient in visual tracking. Moreover, color or greylevel histograms allow to track an objet with a low computational cost. The recently proposed color-based trackers integrated in a probabilistic framework are efficient for a given application (face tracking for example) but can not be generalized easily, due to the initialization and the adjustment of the different tracker parameters that are dependent on the input sequence. This paper presents a method based on color integrated in a particle filter that allows to cope with some of the usual problems of visual tracking (occlusions, target appearance changes, changes in resolution or in illumination) and to adapt easily to different applications (tracking of structures in aerial imagery as well as football players). The novelty of the tracker is its ability to automatically regulate all the parameters needed for tracking, which makes it flexible and easily usable for different applications.

Correlation tracker is computation intensive (if the search space or the template is large), has template drift problem, and may fail in case of fast maneuvering target, rapid changes in its appearance, occlusion suffered by it and clutter in the scene. Kalman filter can predict the target coordinates in the next frame, if the measurement vector is supplied to it by a correlation tracker. Thus, a relatively small search space can be determined where the probability of finding the target in the next frame is high. This way, the tracker can become fast and reject the clutter, which is outside the search space in the scene. However, if the tracker provides wrong measurement vector due to the clutter or the occlusion inside the search region, the efficacy of the filter is significantly deteriorated. Mean-shift tracker is fast and has shown good tracking results in the literature, but it may fail when the histograms of the target and the candidate region in the scene are similar (even when their appearance is different). In order to make the overall visual tracking framework robust to the mentioned problems, we propose to combine the three approaches heuristically, so that they may support each other for better tracking results. Furthermore, we present novel method for (1) appearance model updating which adapts the template according to rate of appearance change of target, (2) adaptive threshold for similarity measure which uses the variable threshold for each forthcoming image frame based on current frame peak similarity value, and (3) adaptive kernel size for fast mean-shift algorithm based on varying size of the target. Comparison with nine state-of-the-art tracking algorithms on eleven publically available standard dataset shows that the proposed algorithm outperforms the other algorithms in most of the cases.

Video tracking uses the semantic information between video image sequences to process, analyze and study the target to achieve target tracking. In response to the problems of sharp changes in target position, large deformation, and occlusion due to similar background interference, changes in lighting conditions, and different shapes of targets in the target tracking process, the tracking algorithm is less accurate and less robust in target appearance. An improved target tracking algorithm based on multi-domain network (MDNet) is proposed to solve this problem. By adding the Image-Align layer to the video target tracking task, a more accurate target value is obtained; applying the Directed Acyclic Graph-Recurrent Neural Network (DAG-RNN) by combining it with a convolutional neural network and modeling image neighborhood context dependencies for the target region to be tracked, we improve the problem that conventional networks only perform multi-layer extraction for the appearance features of the target, thus resulting in poor robustness to changes in target appearance; ROI Align layer is added after the convolution layer to speed up target feature extraction.

In recent years, discriminative correlation filter (DCF) based algorithms have significantly advanced the state of the art in visual object tracking. The key to the success of DCF is an efficient discriminative regression model trained with powerful multi-cue features, including both hand-crafted and deep neural network features. However, the tracking performance is hindered by their inability to respond adequately to abrupt target appearance variations. This issue is posed by the limited representation capability of fixed image features. In this work, we set out to rectify this shortcoming by proposing a complementary representation of a visual content. Specifically, we propose the use of a collaborative representation between successive frames to extract the dynamic appearance information from a target with rapid appearance changes, which results in suppressing the undesirable impact of the background. The resulting collaborative representation coefficients are combined with the original feature maps using a spatially regularised DCF framework for performance boosting. The experimental results on several benchmarking datasets demonstrate the effectiveness and robustness of the proposed method, as compared with a number of state-of-the-art tracking algorithms.

One of the most important and challenging research topics in the area of computer vision is visual object tracking, which is relevant to many real-world applications. Recently, discriminative correlation filters (DCF) have been demonstrated to overcome the problems in visual object tracking efficiently. So far, only single-resolution feature maps have been utilized in DCF. Owing to this limitation, the potential of DCF has not been exploited. Moreover, convolutional features have demonstrated a better performance for visual tracking than histogram of oriented gradients (HOG) features and color features. Based on these facts, in this paper, we propose collaborative learning based on multi-resolution feature maps for DCF, employing convolutional features. Further, the confidence score, which represents the location of the target object, is selected from various candidates based on certain rules. In addition, the continuous filters are trained to handle the variations of appearance of the target. The extensive experimental results obtained using VOT2015 and OTB-100 benchmark datasets show that the proposed algorithm performs favorably against state-of-the-art tracking algorithms.

Tracking is regarded as one of the most fundamental tasks in computer vision. It is used in many computer vision applications in fields such as surveillance, robotic navigation and 3D reconstruction to name but a few. Despite decades of research, the goal of fully automatic tracking of arbitrary types of objects in real world conditions is still an open problem. In this paper, we take a step toward the goal of general real-world tracking, and demonstrate a unified generative model for Bayesian multifeature, adaptive target tracking, or AMFT for short (Adaptive Multiple Feature Tracker). We derive a unified generative model for multi-sensory adaptive tracking which cleanly integrates tracking and the modeling of appearance change across multiple features in the same framework. The unified multi-feature observation model ensures that if one feature is not confident, e.g., color after an object crosses into a region of shadow, it is automatically down-weighted in its contribution to the appearance model update. In this way, without pre-training of specific object models, we achieve an extensible tracker for general object types, robust to real-world problems of clutter, appearance/lighting change and target model drift. The standard modeling assumptions made by a non-adaptive generative model are illustrated by the probabilistic graphical model in Figure 1(a). The unknown target state (e.g., location, size, velocity) xt is assumed to change with time t according to some process parameterized by A. At every time t, we make some noisy observations zt of the target xt (e.g., raw image or color histograms). The target is then tracked online by computing the posterior, p(xt |z1:t) over the true target location recursively. In the case of the Kalman filter (KF), all the distributions involved are Gaussian. In the case of the particle filter (PF), all the distributions involved are represented non-parametrically by a set of samples [1]. The true target model, e.g., the appearance or color histogram to search for, is assumed to be part of the parameters H, i.e., it is known and fixed by an operator or initialized by some external process. In many cases however, the true appearance of the target H may change significantly in time, e.g., the appearance changes when a subject moves between shade and sunlight. This is the case for outdoor surveillance applications and is the motivation for this research. Adaptive trackers [2, 3, 4, 6] have been proposed to update the target appearance online in various heuristic ways. We can formalise this more general modeling assumption generatively, by the generalized dynamic Bayesian network illustrated in Figure 1(b). In contrast to Figure 1(a), the true target model which was previously included in the fixed parameters H, is now included as the the initial condition y0 of a dynamic latent variable yt , formalizing the modeling assumption that the target appearance can change over time. In addition to the target state xt , the target appearance yt will therefore be incrementally and recursively updated as part of the process of inferring the latent variables in this model p(xt ,yt |z1:t). The latent space is of course now greatly expanded, and poses a more challenging inference problem than that of Figure 1(a). In Section 2 of the paper, we detail the specific parametric form of the model and an efficient inference algorithm. We evaluate our method (AMFT) against three contemporary trackers: A standard single feature particle filter (PF), mean-shift (MS) [5] and incremental visual tracking (IVT) [4]. The PF and MS trackers are non-adaptive color-based trackers, while IVT aims for pose and illumination change robustness by performing online adaptation in a subspace appearance model. Note that the AMFT, PF and IVT trackers track object scale, but MS does not. We evaluated these methods on a series of challenging video clips exhibiting a wide variety of data and object types for tracking, including far-field indoor and outdoor pedestrians with and without carried objects, vehicle tracking, and near-field indoor face trackH 1 x2 x3

Effective object appearance model is one of the key issues for the success of visual tracking. Since the appearance of a target and the environment changes dynamically, the majority of existed visual tracking algorithms tend to drift away from targets. To address this issue, we propose a robust tracking algorithm by integrating the generative and discriminative model. The object appearance model is made up of generative target model and a discriminative classifier. For the generative target model, we adopt the weighted structural local sparse appearance model combining patch based gray value and Histogram of Oriented Gradients feature as the patch dictionary. By sampling positives and negatives, alignment-pooling features are obtained based on the patch dictionary through local sparse coding, then we use support vector machine to train the discriminative classifier. The proposed method is embedded into a Bayesian inference framework for visual tracking. A combined matching method is adopted to improve the proposal distribution of the particle filter. Moreover, in order to adapt the situation change, the patch dictionary and discriminative classifier are updated by incremental learning every five frames. Experimental results on some publicly available benchmarks of video sequences demonstrate the accuracy and effectiveness of our tracker.

The emerging technology revolution empowered by artificial intelligence related research has brought us closer to the implementation of systems only seen in science-faction movies before. Artificial vision systems built in complex modern artefacts are very important for supporting their execution and perception based on camera captured raw pixels is one of the keys to open the machine intelligence. Deep learning and visual object tracking are both hot topics for computer vision research in recent years. Many visual object tracking algorithms based on deep learning technique appeared. The effectiveness of deep features for visual tracking has been shown on benchmarks, these big guys showed more intelligent behaviour capturing target object in bounding box but the high computational cost and deep hungry for data contradicts our initial requirements for practical tracking algorithm. In this paper, we investigated several deep trackers to help discovering current trends in this research area and hope it could help researchers and practitioners to better understand and further enhance existing methods.

This paper presents a robust vision algorithm for tracking the boundary of an object with an arbitrary shape by using monocular image sequences. This method consists of a curve registration based optimization technique and a deformable contour model (snakes) for the global and the motion estimations, respectively. By combining techniques, we overcome, among other problems, inaccurate estimate of motion parameters in the curve registration method (which apparently only occur when a rigid or a flexible object is tracked), and the local position variation of the deformable contour model, variations, which are due to noisy images and/or complex backgrounds. The curve registration method uses an iterative algorithm to find the minimum normal distance between two curves, one before motion and the corresponding curve after it. Snakes overcome the limitation of the curve registration method, which suffers from the inaccuracy of motion models. We also propose an internal force, which increases robustness of the deformable contour to background noise. By using the refined snakes' control points, the global update of the previous curve is performed for the re-location of the registered curve. Additionally, we integrate the geometric invariant value of the boundary contour and the curve registration method to solve the occlusion problem in visual tracking. The proposed method is validated through experiments on real images.

This paper concerns on overcoming the challenges caused by drastic appearance change in visual tracking, especially the long-term appearance variation due to occlusion or large object deformation. We aim to build a long-term appearance model for robust tracking against large appearance change in two new respects: using historical and distinguishing cues to model target representation and extracting effective spatial objectness features from each frame to distinguish outliers. For the first purpose, an adaptive superpixel-based appearance model is formulated. Different from previous superpixel-based trackers, a complementary feature set is defined for the update model to preserve the features of those temporally disappeared object parts especially under occlusion and large deformation. For the second purpose, three new spatial objectness cues specially designed for tracking are defined, including surrounding comparison, edge density change and weighted superpixel straddling. With these spatial objectness cues, our method facilitates target object localization and ensures the target has similar edge distribution between adjacent frames. These cues greatly improve the ability of our method to distinguish the target from its surrounding background. The adaptive appearance model retains valuable features of historical results, and the spatial objectness cues are extracted from the current frame, and thus they are finally combined to complement with each other to solve large appearance changes. The extensive evaluations on the CVPR 2013 online object tracking benchmark and VOT 2014 datasets demonstrate the effectiveness of our method as compared with related trackers.

Visual tracking has been widely used in robot systems, and numerous approaches for visual tracking have been proposed. However, developing a robust and real-time visual tracking algorithm which can adaptively track the varying appearance of target under challenging conditions for mobile robot is still an open problem. This paper presents an adaptive probabilistic tracking algorithm with multiple cues integration. An effective evaluation function is proposed to evaluate each cue used for tracking based on their discriminating abilities between foreground and background. Then the likelihood functions of the cues are integrated in particle filter framework with different weights determined based on the evaluation scores. A novel target model updating strategy is proposed to adapt to the varying appearance of target resisting gradual drift which is still an unsolved problem in many adaptive tracking algorithms. Experimental results on a mobile robot demonstrate the robust performance of the proposed algorithm under challenging conditions.

Deep learning has been widely used in many visual recognition tasks owing to its powerful representation ability. However, online learning is a bottleneck to obstruct the application of deep learning in visual tracking. Although many algorithms have discarded the process of online learning during tracking, they demonstrate poor robustness to the online adaptation to appearance changes of the target. In this study, the authors design a tree structure specifically for online learning, which enables the appearance model to be updated smoothly. Once the target appearance has changed severely, a new branch is generated to avoid the fuzzy boundary of classification. In addition, active learning technique and artificial data are employed in the update to make the best of the limited knowledge about the interesting object during the tracking process. The proposed algorithm is evaluated on OTB2013 and VOT2017 benchmark and outperforms many state-of-the-art methods.