Enhancing discriminative appearance model for visual tracking

We present an approach for online learning of discriminative appearance models for robust multi-target tracking in a crowded scene from a single camera. Although much progress has been made in developing methods for optimal data association, there has been comparatively less work on the appearance models, which are key elements for good performance. Many previous methods either use simple features such as color histograms, or focus on the discriminability between a target and the background which does not resolve ambiguities between the different targets. We propose an algorithm for learning a discriminative appearance model for different targets. Training samples are collected online from tracklets within a time sliding window based on some spatial-temporal constraints; this allows the models to adapt to target instances. Learning uses an Ad-aBoost algorithm that combines effective image descriptors and their corresponding similarity measurements. We term the learned models as OLDAMs. Our evaluations indicate that OLDAMs have significantly higher discrimination between different targets than conventional holistic color histograms, and when integrated into a hierarchical association framework, they help improve the tracking accuracy, particularly reducing the false alarms and identity switches.

Sparse coding has been used for target appearance modeling and applied successfully in visual tracking. However, noise may be inevitably introduced into the representation due to background clutter. To cope with this problem, we propose a saliency weighted sparse coding appearance model for visual tracking. Firstly, a spectral filtering based visual attention computational model, which combines both bottom-up and top-down visual attention, is proposed to calculate saliency map. Secondly, pooling operation in sparse coding is weighted by calculated saliency map to help target representation focus on distinctive features and suppress background clutter. Extensive experiments on a recently proposed tracking benchmark demonstrate that the proposed algorithm outperforms state-of-the-art methods in tracking objects under background clutter.

In this paper, we present a soft biometrics based appearance model for multi-target tracking in a single camera. Track lets, the short-term tracking results, are generated by linking detections in consecutive frames based on conservative constraints. Our goal is to "re-stitching" the adjacent track lets that contain the same target so that robust long-term tracking results can be achieved. As the appearance of the same target may change greatly due to heavy occlusion, pose variations and changing lighting conditions, a discriminative appearance model is crucial for association-based tracking. Unlike most previous methods which simply use the similarity of color histograms or other low level features to construct the appearance model, we propose to use the fusion of soft biometrics generated from sub-track lets to learn a discriminative appearance model in an online manner. Compared to low level features, soft biometrics are robust against appearance variation. The experimental results demonstrate that our method is robust and greatly improves the tracking performance over the state-of-the-art method.

Co-trained generative and discriminative trackers with cascade particle filter

An appearance model is critical for most modern trackers. While numerous novel appearance models have been proposed with demonstrated success, challenges such as occlusion and drifting are still not well addressed. In this paper, we propose a novel contextual bag-of-words (CBOW) discriminative appearance model that appropriately handles drifting and occlusion. Specifically, a contextual region containing both the target and its surroundings is explored to construct a compact representation with two bags-of-words. Each word carries discriminative appearance information that is learned by Bayesian inference. An adaptive updating approach, where the background BOWs of the CBOW model acts as a "sentinel" to prevent the integration of the background appearance with the object model, is introduced to alleviate the drifting problem. Based on CBOW, visual tracking is posed within a Bayesian framework. Moreover, an explicit detection method is employed to handle severe occlusions, which further reduces drifting. Two trackers based on the same CBOW model are implemented using either handcrafted color/texture or deep convolutional features. Our trackers are evaluated based on the popular OTB50 and VOT2015 benchmarks and perform competitively against the current state of the art. In addition, they outperform two recent BOWs trackers by a large margin using the currently available figures of merit. To take into account a tracking breakdown, we propose a new figure of merit called the mean maximum-tracked-frame ratio (MTFR) that evaluates a tracker's temporal persistence without any interruption. Experiments with OTB50 demonstrate the superior robustness of our tracker compared with all other evaluated trackers on the basis of MTFR.

Tracking-by-detection methods have been widely studied and some promising results have been obtained. These methods use discriminative appearance models to train and update online classifiers. They also use a sliding window to detect samples which will then be classified. Then, the location of the sample with the maximum classifier response will be selected as the new location. Compressive tracking was recently proposed with an appearance model based on features extracted in the compressed domain. However, CT uses a fixed-size tracking box to detect samples, and this is unsuitable for practice applications. CT detects samples around the selected region of the previous frame within a fixed radius. Here, the classifier may become inaccurate if the selected region drifts. The fixed radius is also not suitable for tracking targets that experience abrupt acceleration changes. Furthermore, CT updates the classifier parameters with constant learning rate. If the target is fully occluded for an extended period, the classifier will instead learn the features of the cover object and the target will ultimately be lost. In this paper, we present a multi-scale compressive tracker. This tracker integrates an improved appearance model based on normalized rectangle features extracted in the adaptive compressive domain into the bootstrap filter. This type of feature extraction is efficient, and the computation complexity does not increase as the tracking regions become larger. The classifier response is utilized to generate particle importance weight and a re-sample procedure preserves samples according to weight. A 2-order transition model considers the target velocity to estimate the current position and scale status. In this way, the sampling is not limited to a fixed range. Here, feedback strategies are adopted to adjust learning rate for occlusion. Experimental results on various benchmark challenging sequences have demonstrated the superior performance of our tracker when compared with several state-of-the-art tracking algorithms.

In this paper, the consistent fish tracking strategies have been present for the underwater surveillance system with multiple static cameras in the overlapping field of views (FOV). The discriminative appearance model has been first introduced to distinguish the swimming fish and the background by the superpixel. The centroid coordinate homographic mapping and the Speeded Up Robust Features (SURF) technique have been taken to capture and match the same fish from the perspectives among multiple cameras by a set of strategies. Simulation experiments have shown that the proposed method is less dependent on the exact motion detection of the fish and could perform robustly and stably at a high speed in case of the occlusion and appearance variation under the sea.

We develop an algorithm for finding and kinematically tracking multiple people in long sequences. Our basic assumption is that people tend to take on certain canonical poses, even when performing unusual activities like throwing a baseball or figure skating. We build a person detector that quite accurately detects and localizes limbs of people in lateral walking poses. We use the estimated limbs from a detection to build a discriminative appearance model; we assume the features that discriminate a figure in one frame will discriminate the figure in other frames. We then use the models as limb detectors in a pictorial structure framework, detecting figures in unrestricted poses in both previous and successive frames. We have run our tracker on hundreds of thousands of frames, and present and apply a methodology for evaluating tracking on such a large scale. We test our tracker on real sequences including a feature-length film, an hour of footage from a public park, and various sports sequences. We find that we can quite accurately automatically find and track multiple people interacting with each other while performing fast and unusual motions.

Abnormal event detection and automated alarm are the important tasks in visual surveillance applications. In this paper, a novel automated alarm method based on intelligent visual analysis is proposed for alarm of abandoned objects and virtual cordon protection. Firstly the monitoring regions and cordons position are set artificially in the surveillance background scenes. The forground motion regions are segmented based on background subtraction model, and then are clustered by connected component analysis. After motion region segmentation and cluster, object tracking based on discriminative appearance model for monocular multi-target tracking is utilized. According to the motion segmentation and tracking results, alarm is triggered in comparison with the monitoring regions and cordons position. Experimental results show that the proposed automated alarm algorithms are sufficient to detect the abnormal events for alarm of abandoned objects and virtual cordon protection.

Video target tracking is a critical problem in the field of computer vision. Particle filters have been proven to be very useful in target tracking for nonlinear and non-Gaussian estimation problems. Although most existing algorithms are able to track targets well in controlled environments, it is often difficult to achieve automated and robust tracking of pedestrians in video sequences if there are various changes in target appearance or surrounding illumination. To surmount these difficulties, this paper presents multitarget tracking of pedestrians in video sequences based on particle filters. In order to improve the efficiency and accuracy of the detection, the algorithm firstly obtains target regions in training frames by combining the methods of background subtraction and Histogram of Oriented Gradient (HOG) and then establishes discriminative appearance model by generating patches and constructing codebooks using superpixel and Local Binary Pattern (LBP) features in those target regions. During the process of tracking, the algorithm uses the similarity between candidates and codebooks as observation likelihood function and processes severe occlusion condition to prevent drift and loss phenomenon caused by target occlusion. Experimental results demonstrate that our algorithm improves the tracking performance in complicated real scenarios.

Developing robust and discriminative appearance models has been a long-standing research challenge in visual object tracking. In the prevalent Siamese-based paradigm, the features extracted by the Siamese-like networks are often insufficient to model the tracked targets and distractor objects, thereby hindering them from being robust and discriminative simultaneously. While most Siamese trackers focus on designing robust correlation operations, we propose a novel single-branch tracking framework inspired by the transformer. Unlike the Siamese-like feature extraction, our tracker deeply embeds cross-image feature correlation in multiple layers of the feature network. By extensively matching the features of the two images through multiple layers, it can suppress non-target features, resulting in target-aware feature extraction. The output features can be directly used to predict target locations without additional correlation steps. Thus, we reformulate the two-branch Siamese tracking as a conceptually simple, fully transformer-based Single-Branch Tracking pipeline, dubbed SBT. After conducting an in-depth analysis of the SBT baseline, we summarize many effective design principles and propose an improved tracker dubbed SuperSBT. SuperSBT adopts a hierarchical architecture with a local modeling layer to enhance shallow-level features. A unified relation modeling is proposed to remove complex handcrafted layer pattern designs. SuperSBT is further improved by masked image modeling pre-training, integrating temporal modeling, and equipping with dedicated prediction heads. Thus, SuperSBT outperforms the SBT baseline by 4.7%,3.0%, and 4.5% AUC scores in LaSOT, TrackingNet, and GOT-10K. Notably, SuperSBT greatly raises the speed of SBT from 37 FPS to 81 FPS. Extensive experiments show that our method achieves superior results on eight VOT benchmarks.

In order to obtain the discriminative compact appearance model for tracking objects effectively, this paper proposes a new structural tracking strategy that includes multicue inverse sparse appearance model and optimal metric evaluation between online robust templates and a limited number of particle samples in the looping process. Multicue inverse sparse appearance model globally improves the efficient selection of informative particle samples that can avoid the cumbersome coding and decoding cost for the trivial random particle samples. Only the most potential crucial cases are involved in each tracking loop. This refrains from unreasonable, rough numerical reduction of particle samples and also keeps the unbiasedness and dynamic stochasticness of the sampling process. Meanwhile, low-rank self-representatives for positive and negative samples facilitate the formulation of a suitable code book that arranges the useful sparse coefficients for feature bags and facilitates optimal metric evaluation for online training. It also alleviates the accuracy degradation of tracking occluded objects and improves the robustness of the tracker. Both of them preserve the discriminative compactness of target which speeds up particle filtering localization to separate the target object from distractors. Moreover, the proposed method exploits online appearance representations to learn the sharing compact information that avoids massive calculation burdens for massive visual data.

Text extraction from scene images by character appearance and structure modeling

This paper proposes a distributed multi-camera tracking algorithm with interacting particle filters. A robust multi-view appearance model is obtained by sharing training samples between views. Motivated by incremental learning and [1], we create an intermediate data representation between two camera views with generative subspaces as points on a Grassmann manifold, and sample along the geodesic between training data from two views to uncover the meaningful description due to viewpoint changes. Finally, a Boosted appearance model is trained using the projected training samples on to these generative subspaces. For each object, a set of two particle filters i.e., local and global is used. The local particle filter models the object motion in the image plane. The global particle filter models the object motion in the ground plane. These particle filters are integrated into a unified Interacting Markov Chain Monte Carlo (IMCMC) framework. We show the manner in which we induce priors on scene specific information into the global particle filter to improve tracking accuracy. The proposed algorithm is validated with extensive experimentation in challenging camera network data, and compares favorably with state of the art object trackers.

Multi-object tracking via discriminative appearance modeling