Multi-object tracking via discriminative appearance modeling

We present an approach for online learning of discriminative appearance models for robust multi-target tracking in a crowded scene from a single camera. Although much progress has been made in developing methods for optimal data association, there has been comparatively less work on the appearance models, which are key elements for good performance. Many previous methods either use simple features such as color histograms, or focus on the discriminability between a target and the background which does not resolve ambiguities between the different targets. We propose an algorithm for learning a discriminative appearance model for different targets. Training samples are collected online from tracklets within a time sliding window based on some spatial-temporal constraints; this allows the models to adapt to target instances. Learning uses an Ad-aBoost algorithm that combines effective image descriptors and their corresponding similarity measurements. We term the learned models as OLDAMs. Our evaluations indicate that OLDAMs have significantly higher discrimination between different targets than conventional holistic color histograms, and when integrated into a hierarchical association framework, they help improve the tracking accuracy, particularly reducing the false alarms and identity switches.

In this paper, we present a soft biometrics based appearance model for multi-target tracking in a single camera. Track lets, the short-term tracking results, are generated by linking detections in consecutive frames based on conservative constraints. Our goal is to "re-stitching" the adjacent track lets that contain the same target so that robust long-term tracking results can be achieved. As the appearance of the same target may change greatly due to heavy occlusion, pose variations and changing lighting conditions, a discriminative appearance model is crucial for association-based tracking. Unlike most previous methods which simply use the similarity of color histograms or other low level features to construct the appearance model, we propose to use the fusion of soft biometrics generated from sub-track lets to learn a discriminative appearance model in an online manner. Compared to low level features, soft biometrics are robust against appearance variation. The experimental results demonstrate that our method is robust and greatly improves the tracking performance over the state-of-the-art method.

Multiple Object Tracking (MOT) has attracted increasing attention due to its academic and commercial potential. However, it still remains challenging due to factors like data association ambiguity and abrupt appearance changes. To address these issues, we extended the multiple hypothesis tracking (MHT) by incorporating the appearance features which extracted from faster region convolutional neural network (Faster-RCNN). Additionally, a principal component analysis method was introduced to update discriminative appearance features. Thus, the data association ambiguity can be reduced surprisingly by exploiting Faster-RCNN, which means fewer hypothesis for MHT. Meanwhile, the issue of abrupt appearance changes can be addressed by modeling discriminative appearance feature based-on MHT. Many experiments on MOT benchmark show competitive performance of the proposed approach in comparison with other state-of-art tracking methods.

The analysis of a multitemporal sequence of images of a given site makes it possible to exploit temporal information in addition to spectral and spatial information, and therefore represents a way to improve the accuracy with respect to the non-contextual single-time classification. The proposed contextual multitemporal classification scheme consists of two stages of multilayer perceptron (MLP) neural networks for each single-time image of the multitemporal sequence. The first stage is a one-hidden layer MLP whose ro/spl circ/le is to estimate, for each pixel, the single-time posterior probability of each class, given the feature vector. These probability estimates represent spectral information; in addition, they are utilized to generate a non-contextual classification map. The neighboring class labels of a given pixel in the non-contextual classification map are exploited to extract spatial information, while temporal information is deduced from the non-contextual maps produced by the remaining single-time images in the multitemporal sequence. Spatial and temporal contextual information together with spectral information serve as inputs for the second stage network of the classification scheme where the fusion takes place. As the network configuration can influence the classification performances, three MLP-based configurations are investigated. Experimental results on a multitemporal data set consisting of two multisensor (Landsat TM and ERS-1 SAR) images are presented. The performances of the proposed methods are discussed and compared with those obtained by a reference classifier based on the Markov random fields fusion approach in terms of classification accuracy. The results show that the proposed fusion approach based on neural networks may represent an interesting solution to the problem of multitemporal contextual fusion.

An appearance model is critical for most modern trackers. While numerous novel appearance models have been proposed with demonstrated success, challenges such as occlusion and drifting are still not well addressed. In this paper, we propose a novel contextual bag-of-words (CBOW) discriminative appearance model that appropriately handles drifting and occlusion. Specifically, a contextual region containing both the target and its surroundings is explored to construct a compact representation with two bags-of-words. Each word carries discriminative appearance information that is learned by Bayesian inference. An adaptive updating approach, where the background BOWs of the CBOW model acts as a "sentinel" to prevent the integration of the background appearance with the object model, is introduced to alleviate the drifting problem. Based on CBOW, visual tracking is posed within a Bayesian framework. Moreover, an explicit detection method is employed to handle severe occlusions, which further reduces drifting. Two trackers based on the same CBOW model are implemented using either handcrafted color/texture or deep convolutional features. Our trackers are evaluated based on the popular OTB50 and VOT2015 benchmarks and perform competitively against the current state of the art. In addition, they outperform two recent BOWs trackers by a large margin using the currently available figures of merit. To take into account a tracking breakdown, we propose a new figure of merit called the mean maximum-tracked-frame ratio (MTFR) that evaluates a tracker's temporal persistence without any interruption. Experiments with OTB50 demonstrate the superior robustness of our tracker compared with all other evaluated trackers on the basis of MTFR.

Co-trained generative and discriminative trackers with cascade particle filter

In addition to the strong correlation in the internal pixels of the image, the pixels in the two phases of image have a certain correlation, that is, temporal context information. Conditional Random Field (CRF) models not only model spatial context information, but also fuse temporal context information. A spatio-temporal model based on temporal context information and spatial context information is proposed to improve the classification accuracy of remote sensing images. Use of two-phase data, the temporal context information between two phases and the spatial context information between pixels within a single temporal phase are considered, and High-order CRF model the spatio-temporal context information. Make full use of spatio-temporal context information of two-phase image to improve the classification accuracy. The experimental results show that the proposed method has better classification accuracy than the classification accuracy without temporal context information.

This paper investigates the potential of accounting for temporal contextual information in order to improve the accuracy of land-cover classification in summer with Synthetic Aperture Radar (SAR) data. Bi-temporal multi-sensor datasets collected in the Nonsan area of Korea were used to illustrate this approach. Multi-sensor data, including Japanese Earth Resources Satellite (JERS)-1 Optical Sensor (OPS) data acquired in April, and three different SAR sensor datasets from European Resource Satellite (ERS)-2, JERS-1, and Radarsat-1 obtained in the following July, were used for supervised classification in July. By comparing the classification result in April with a training set in July, transition probabilities between land-cover classes in the April–July period were empirically estimated and regarded as the temporal contextual information. A tau model is applied as a main integration methodology to combine multiple SAR data and the temporal contextual information. From the evaluation of the classification results in terms of accuracy statistics, using multiple SAR sensor data showed an increase of about 29% in overall accuracy compared with the case of single SAR sensor data. The incorporation of temporal contextual information into scattering information greatly contributed to a significant improvement of about 25% in overall accuracy over multiple SAR sensor integration only, and showed the best discrimination capability.

In this paper, we propose an approach which uses compressive sensing features to improve Markov Decision Process (MDP) tracking framework. First, we design a single object tracker which integrates compressive tracking into Tracking-Learning-Detection (TLD) framework to complement each other. Then we apply this tracker into the MDP tracking framework to improve the multi-object tracking performance. A discriminative model is built for each object and updated online. With the built discriminative model, the features used for data association are also enhanced. In order to validate our method, we first test the designed single object tracker with a common dataset. Then we use the validation set from the multiple object tracking (MOT) training dataset to analyze each part of our method. Finally, we test our approach in the MOT benchmark. The results show our approach improves the original method and performs superiorly against several state-of-the-art online multi-object trackers.

Tracking-by-detection methods have been widely studied and some promising results have been obtained. These methods use discriminative appearance models to train and update online classifiers. They also use a sliding window to detect samples which will then be classified. Then, the location of the sample with the maximum classifier response will be selected as the new location. Compressive tracking was recently proposed with an appearance model based on features extracted in the compressed domain. However, CT uses a fixed-size tracking box to detect samples, and this is unsuitable for practice applications. CT detects samples around the selected region of the previous frame within a fixed radius. Here, the classifier may become inaccurate if the selected region drifts. The fixed radius is also not suitable for tracking targets that experience abrupt acceleration changes. Furthermore, CT updates the classifier parameters with constant learning rate. If the target is fully occluded for an extended period, the classifier will instead learn the features of the cover object and the target will ultimately be lost. In this paper, we present a multi-scale compressive tracker. This tracker integrates an improved appearance model based on normalized rectangle features extracted in the adaptive compressive domain into the bootstrap filter. This type of feature extraction is efficient, and the computation complexity does not increase as the tracking regions become larger. The classifier response is utilized to generate particle importance weight and a re-sample procedure preserves samples according to weight. A 2-order transition model considers the target velocity to estimate the current position and scale status. In this way, the sampling is not limited to a fixed range. Here, feedback strategies are adopted to adjust learning rate for occlusion. Experimental results on various benchmark challenging sequences have demonstrated the superior performance of our tracker when compared with several state-of-the-art tracking algorithms.

This paper proposes a distributed multi-camera tracking algorithm with interacting particle filters. A robust multi-view appearance model is obtained by sharing training samples between views. Motivated by incremental learning and [1], we create an intermediate data representation between two camera views with generative subspaces as points on a Grassmann manifold, and sample along the geodesic between training data from two views to uncover the meaningful description due to viewpoint changes. Finally, a Boosted appearance model is trained using the projected training samples on to these generative subspaces. For each object, a set of two particle filters i.e., local and global is used. The local particle filter models the object motion in the image plane. The global particle filter models the object motion in the ground plane. These particle filters are integrated into a unified Interacting Markov Chain Monte Carlo (IMCMC) framework. We show the manner in which we induce priors on scene specific information into the global particle filter to improve tracking accuracy. The proposed algorithm is validated with extensive experimentation in challenging camera network data, and compares favorably with state of the art object trackers.

The main challenge of Multiple Object Tracking (MOT) is that there is great uncertainty in data association when using the tracked predicted values and tracked trajectories. Meanwhile, the MOT is complex and time‐consuming. When equipment resources are limited or real‐time requirement is high, its application is very limited. Therefore, we propose a Lightweight Deep Appearance Embedding (LDAE) to assist the association of trajectories. Firstly, in addition to motion information in data association, we also introduce more discriminative appearance features to participate in the affinity measure to effectively distinguish similar targets. Secondly, according to the idea of feature mapping, we design a lightweight deep appearance embedding module. It can help extract appearance features with less computation. Finally, we propose a simulated occlusion strategy for the training of the LDAE, which helps improve the ability to recognise different targets in dense scenes. The LDAE dramatically reduces the computational cost and improves the accuracy of data association. Extensive experiments are conducted on the MOT datasets (MOT16, MOT17 and MOT20), which prove that the LDAE outperforms several state‐of‐the‐art trackers in the tracking accuracy and anti‐occlusion performance. Furthermore, we apply the LDAE to escalators, which can achieve fast and stable tracking effect.

Enhancing discriminative appearance model for visual tracking

Text extraction from scene images by character appearance and structure modeling

We propose a novel online multiple object tracker taking structure information into account. State-of-the-art multi-object tracking (MOT) approaches commonly focus on discriminative appearance features, while neglect in different levels structure information and the core of data association. Addressing this, we design a new tracker fully exploiting structure information and encoding such information into the cost function of the graph matching model. Firstly, a new measurement is proposed to compare the structure similarity of two graphs whose nodes are equal. With this measurement, we define a complete matching which performs association in high efficiency. Secondly, for incomplete matching scenarios, a structure keeper net (SKnet) is designed to adaptively establish the graph for matching. Finally, we conduct extensive experiments on benchmarks including MOT2015 and MOT17. The results demonstrate the competitiveness and practicability of our tracker.