基于深度学习的目标跟踪方法研究现状与展望

When entering into the realm of Computer Vision, the first thing which comes in to mind is Visual tracking. Visual tracking by far comes into one of the most actively investigated research areas because of the fact that it has an extensive collection of applications in areas such as activity recognition, surveillance, motion analysis and as well as human computer interaction. Some serious challenges of this area which still create hindrance in achieving 100% accuracy are abrupt appearance and pose changes of an object along with its background blockage due to blockages called occlusion, illumination and lighting variances and changes in scale of target object in the frames. Moreover, diverse algorithms had been proposed for the resolution of said issue. Now in such cases, if we study the statistical analysis of correlation between two frames in a certain video, it can be efficiently utilized to get the most exact location of the targeted object. The algorithms in existence today do not completely exploit a strong spatio-temporal relationship that very often occurs between the two successive frames in a video sequence. Recent advances in correlation-based tracking systems have been proposed to address the problem in successive frames. In this thesis a very simple yet quite speedy and robust algorithm that in actual brings all the relevant information used for Visual Tracking. Two of the Models proposed are the “Locality Sensitive Histogram” and “Discriminative Scale Tracking Method”. These are robust enough to the variations which are based on appearance which are normally presented by blockage, pose, illumination and lighting variations alike. A scheme is proposed called scale adaptation which is very much clever to adapt variations of targeted scale in the most efficient manner. The Discriminative Scale Tracking Method is used for detection as well as scale change ultimately resulting in an effective tracking method in the end. Various different experiments with the best algorithms have demonstrated on challenging sequences that the suggested methodology attains promising results as far as robustness, accuracy, and speed is concerned.

Multiple Object Tracking (MOT) has attracted increasing attention due to its academic and commercial potential. However, it still remains challenging due to factors like data association ambiguity and abrupt appearance changes. To address these issues, we extended the multiple hypothesis tracking (MHT) by incorporating the appearance features which extracted from faster region convolutional neural network (Faster-RCNN). Additionally, a principal component analysis method was introduced to update discriminative appearance features. Thus, the data association ambiguity can be reduced surprisingly by exploiting Faster-RCNN, which means fewer hypothesis for MHT. Meanwhile, the issue of abrupt appearance changes can be addressed by modeling discriminative appearance feature based-on MHT. Many experiments on MOT benchmark show competitive performance of the proposed approach in comparison with other state-of-art tracking methods.

We present Siam R-CNN, a Siamese re-detection architecture which unleashes the full power of two-stage object detection approaches for visual object tracking. We combine this with a novel tracklet-based dynamic programming algorithm, which takes advantage of re-detections of both the first-frame template and previous-frame predictions, to model the full history of both the object to be tracked and potential distractor objects. This enables our approach to make better tracking decisions, as well as to re-detect tracked objects after long occlusion. Finally, we propose a novel hard example mining strategy to improve Siam R-CNN's robustness to similar looking objects. Siam R-CNN achieves the current best performance on ten tracking benchmarks, with especially strong results for long-term tracking. We make our code and models available at www.vision.rwth-aachen.de/page/siamrcnn.

In this paper, we propose a novel object tracking algorithm in video sequences. The formulation of the object tracking is based on variational calculus, where an adaptive parametric mixture model is used for object features representation. The tracking is based on matching the object mixture models between successive frames of the sequence by using active contours while adapting the mixture model to varying object appearance changes due to illumination conditions and camera geometry. The implementation of the method is based on level set active contours which allow for automatic topology changes and stable numerical schemes. We validate our approach on examples of object tracking performed on real video sequences.

Multiple object tracking: A literature review

This thesis addresses the problem of tracking under severe appearance changes using a combination of receding horizon and Incremental Locally Linear Embeddings (ILLE) techniques that provides adaptation to both time varying dynamics and abrupt appearance changes. In addition to enabling sustained tracking under these conditions, the proposed technique allows for efficiently detecting appearance changes by simply computing the distance between successive embeddings. These results are illustrated with several examples showing the ability of the proposed technique to flag appearance changes while not breaking track.

Intelligent video surveillance is currently one of the most active research topics in computer vision, especially when facing the explosion of video data captured by a large number of surveillance cameras. As a key step of an intelligent surveillance system, robust visual tracking is very challenging for computer vision. However, it is a basic functionality of the human visual system (HVS). Psychophysical findings have shown that the receptive fields of simple cells in the visual cortex can be characterized as being spatially localized, oriented, and bandpass, and it forms a sparse, distributed representation of natural images. In this article, motivated by these findings, we propose an effective appearance model based on sparse coding and apply it in visual tracking. Specifically, we consider the responses of general basis functions extracted by independent component analysis on a large set of natural image patches as features and model the appearance of the tracked target as the probability distribution of these features. In order to make the tracker more robust to partial occlusion, camouflage environments, pose changes, and illumination changes, we further select features that are related to the target based on an entropy-gain criterion and ignore those that are not. The target is finally represented by the probability distribution of those related features. The target search is performed by minimizing the Matusita distance between the distributions of the target model and a candidate using Newton-style iterations. The experimental results validate that the proposed method is more robust and effective than three state-of-the-art methods.

Making full use of temporal and spatial information is critical to cope with the appearance changes of objects in visual object tracking. However, existing methods in the tracking field, which employ a memory network at frame level to learn this information, bring redundancy and cannot build long-term relationships among historical frames due to the limited memory size. In this paper, we propose a novel memory network, Pixel-level Spatio-Temporal Memory (PSTM), which organizes object features in an efficient way to leverage temporal and spatial context information. Specifically, PSTM is constructed and updated by a memory writer, which includes a pixel-level updating strategy to maintain the temporal consistency and dynamically memorize the noteworthy variations. Furthermore, in order to exploit relationships between the object and search region and precisely estimate the state of the object, we propose a memory reader, Pixel-wise Matching and Refinement module (PMR), and model spatial context without a complex manual-designed mechanism. Comprehensive experiments and comparisons on challenging large-scale benchmarks, including GOT-10k, TrackingNet, LaSOT, OTB2015, VOT2020, and NfS, have demonstrated the effectiveness of our proposed method, which performs favorably against state-of-the-art trackers.

In recent years, discriminative correlation filter (DCF) based algorithms have significantly advanced the state of the art in visual object tracking. The key to the success of DCF is an efficient discriminative regression model trained with powerful multi-cue features, including both hand-crafted and deep neural network features. However, the tracking performance is hindered by their inability to respond adequately to abrupt target appearance variations. This issue is posed by the limited representation capability of fixed image features. In this work, we set out to rectify this shortcoming by proposing a complementary representation of a visual content. Specifically, we propose the use of a collaborative representation between successive frames to extract the dynamic appearance information from a target with rapid appearance changes, which results in suppressing the undesirable impact of the background. The resulting collaborative representation coefficients are combined with the original feature maps using a spatially regularised DCF framework for performance boosting. The experimental results on several benchmarking datasets demonstrate the effectiveness and robustness of the proposed method, as compared with a number of state-of-the-art tracking algorithms.

Visual tracking is a challenging issue in surveillance, human–computer Interaction, and intelligent robotics, among others. Managing appearance changes of the target object, illumination changes, rotations, non-rigid deformations, partial or full occlusions, background clutter, fast motion, and so forth is generally difficult. Among the numerous existing trackers, the correlation-filter-based tracker can achieve appealing performance with a fast speed for fast Fourier transform. Motivated by this property, the spatio-temporal context (STC) learning algorithm was proposed with the consideration of the information from the context around the target, and this algorithm achieved good results. However, STC only utilizes the overall intensity information. In this paper, we propose a multi-channel features STC learning algorithm with an improved scale-adaptive scheme. Our algorithm integrates powerful features, including Histogram of Oriented Gradients and color naming, using kernel methods on the basis of the STC algorithm to further enhance the overall tracking performance. Extensive experimental results obtained from various benchmark data sets demonstrate that the proposed tracker is promising for various challenging scenarios and maintains real-time performance at an average speed of 78 frames/s. According to the test results, our algorithm outperforms the STC algorithm and achieves performance that is competitive with the state-of-the-art algorithms.

Background compensation plays an important role in detecting and isolating object motion in visual tracking. Here, we propose a Genetic Hough Transform, which combines the Hough Transform and Genetic Algorithm, as a method for eliminating background motion. Our method can handle cases in which the background may contain only a few, if any, feature points. These points can be used to estimate the motion between two successive frames. In addition to dealing with featureless backgrounds, our method can successfully handle motion blur. Experimental comparisons of the results obtained using the proposed method with other methods show that the proposed approach yields a satisfactory estimate of background motion.

There is tradeoff between magnification and viewing width when recognizing features on a moving object, for example, the eyes or a mouth of a human who is moving around. To recognize an image feature precisely or correctly, we need to have enough resolution of the image. It is not easy, however, to keep the feature area steady and centred in the camera's image if the object with the feature is moving fast or far away. This paper presents a visual tracking method for feature recognition of moving human face. We use two cameras with different viewing-angles to catch information both with lower resolution for tracking a moving object and with higher resolution for object recognition. The statistical information of the difference image between successive frames is used to estimate the motion speed of a moving object. To demonstrate the idea of this visual tracking system, we show its application to our anthropomorphic agent system called VSA (visual software agent), which is an advanced human interface system with a realistic human facial image and a speech communication function.

ABSTRACTTo improve the quality of state-space exploration and the performance of object tracking, this paper proposes a new adaptive ball particle filter for robust visual tracking. The proposed algorithm guarantees the valid particles in propagation of particle filter using an innovative ball sampling mode. In contrast to conventional approaches, the proposed algorithm uses much fewer particles to ameliorate the diversity of distribution, and effectively overcomes the particle degeneration problem. By the iterative movement of the ball between successive frames, particles move towards the regions with higher values of the posterior density function. Furthermore, the number of particles is estimated adaptively by the tracked object and background. Theoretical analysis and simulated experiments show the superiority of the proposed method.

Existing Visual Object Tracking (VOT) only takes the target area in the first frame as a template. This causes tracking to inevitably fail in fast-changing and crowded scenes, as it cannot account for changes in object appearance between frames. To this end, we revamped the tracking framework with Progressive Context Encoding Transformer Tracker (ProContEXT), which coherently exploits spatial and temporal contexts to predict object motion trajectories. Specifically, ProContEXT leverages a context-aware self-attention module to encode the spatial and temporal context, refining and updating the multi-scale static and dynamic templates to progressively perform accurately tracking. It explores the complementary between spatial and temporal context, raising a new pathway to multi-context modeling for transformer-based trackers. In addition, ProContEXT revised the token pruning technique to reduce computational complexity. Extensive experiments on popular benchmark datasets such as GOT-10k and TrackingNet demonstrate that the proposed ProContEXT achieves state-of-the-art performance <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .

Adaptive pyramid mean shift for global real-time visual tracking