Soft Biometrics Integrated Multi-target Tracking

We present an approach for online learning of discriminative appearance models for robust multi-target tracking in a crowded scene from a single camera. Although much progress has been made in developing methods for optimal data association, there has been comparatively less work on the appearance models, which are key elements for good performance. Many previous methods either use simple features such as color histograms, or focus on the discriminability between a target and the background which does not resolve ambiguities between the different targets. We propose an algorithm for learning a discriminative appearance model for different targets. Training samples are collected online from tracklets within a time sliding window based on some spatial-temporal constraints; this allows the models to adapt to target instances. Learning uses an Ad-aBoost algorithm that combines effective image descriptors and their corresponding similarity measurements. We term the learned models as OLDAMs. Our evaluations indicate that OLDAMs have significantly higher discrimination between different targets than conventional holistic color histograms, and when integrated into a hierarchical association framework, they help improve the tracking accuracy, particularly reducing the false alarms and identity switches.

Co-trained generative and discriminative trackers with cascade particle filter

Abnormal event detection and automated alarm are the important tasks in visual surveillance applications. In this paper, a novel automated alarm method based on intelligent visual analysis is proposed for alarm of abandoned objects and virtual cordon protection. Firstly the monitoring regions and cordons position are set artificially in the surveillance background scenes. The forground motion regions are segmented based on background subtraction model, and then are clustered by connected component analysis. After motion region segmentation and cluster, object tracking based on discriminative appearance model for monocular multi-target tracking is utilized. According to the motion segmentation and tracking results, alarm is triggered in comparison with the monitoring regions and cordons position. Experimental results show that the proposed automated alarm algorithms are sufficient to detect the abnormal events for alarm of abandoned objects and virtual cordon protection.

An appearance model is critical for most modern trackers. While numerous novel appearance models have been proposed with demonstrated success, challenges such as occlusion and drifting are still not well addressed. In this paper, we propose a novel contextual bag-of-words (CBOW) discriminative appearance model that appropriately handles drifting and occlusion. Specifically, a contextual region containing both the target and its surroundings is explored to construct a compact representation with two bags-of-words. Each word carries discriminative appearance information that is learned by Bayesian inference. An adaptive updating approach, where the background BOWs of the CBOW model acts as a "sentinel" to prevent the integration of the background appearance with the object model, is introduced to alleviate the drifting problem. Based on CBOW, visual tracking is posed within a Bayesian framework. Moreover, an explicit detection method is employed to handle severe occlusions, which further reduces drifting. Two trackers based on the same CBOW model are implemented using either handcrafted color/texture or deep convolutional features. Our trackers are evaluated based on the popular OTB50 and VOT2015 benchmarks and perform competitively against the current state of the art. In addition, they outperform two recent BOWs trackers by a large margin using the currently available figures of merit. To take into account a tracking breakdown, we propose a new figure of merit called the mean maximum-tracked-frame ratio (MTFR) that evaluates a tracker's temporal persistence without any interruption. Experiments with OTB50 demonstrate the superior robustness of our tracker compared with all other evaluated trackers on the basis of MTFR.

In this paper, the consistent fish tracking strategies have been present for the underwater surveillance system with multiple static cameras in the overlapping field of views (FOV). The discriminative appearance model has been first introduced to distinguish the swimming fish and the background by the superpixel. The centroid coordinate homographic mapping and the Speeded Up Robust Features (SURF) technique have been taken to capture and match the same fish from the perspectives among multiple cameras by a set of strategies. Simulation experiments have shown that the proposed method is less dependent on the exact motion detection of the fish and could perform robustly and stably at a high speed in case of the occlusion and appearance variation under the sea.

Tracking-by-detection methods have been widely studied and some promising results have been obtained. These methods use discriminative appearance models to train and update online classifiers. They also use a sliding window to detect samples which will then be classified. Then, the location of the sample with the maximum classifier response will be selected as the new location. Compressive tracking was recently proposed with an appearance model based on features extracted in the compressed domain. However, CT uses a fixed-size tracking box to detect samples, and this is unsuitable for practice applications. CT detects samples around the selected region of the previous frame within a fixed radius. Here, the classifier may become inaccurate if the selected region drifts. The fixed radius is also not suitable for tracking targets that experience abrupt acceleration changes. Furthermore, CT updates the classifier parameters with constant learning rate. If the target is fully occluded for an extended period, the classifier will instead learn the features of the cover object and the target will ultimately be lost. In this paper, we present a multi-scale compressive tracker. This tracker integrates an improved appearance model based on normalized rectangle features extracted in the adaptive compressive domain into the bootstrap filter. This type of feature extraction is efficient, and the computation complexity does not increase as the tracking regions become larger. The classifier response is utilized to generate particle importance weight and a re-sample procedure preserves samples according to weight. A 2-order transition model considers the target velocity to estimate the current position and scale status. In this way, the sampling is not limited to a fixed range. Here, feedback strategies are adopted to adjust learning rate for occlusion. Experimental results on various benchmark challenging sequences have demonstrated the superior performance of our tracker when compared with several state-of-the-art tracking algorithms.

Enhancing discriminative appearance model for visual tracking

Video target tracking is a critical problem in the field of computer vision. Particle filters have been proven to be very useful in target tracking for nonlinear and non-Gaussian estimation problems. Although most existing algorithms are able to track targets well in controlled environments, it is often difficult to achieve automated and robust tracking of pedestrians in video sequences if there are various changes in target appearance or surrounding illumination. To surmount these difficulties, this paper presents multitarget tracking of pedestrians in video sequences based on particle filters. In order to improve the efficiency and accuracy of the detection, the algorithm firstly obtains target regions in training frames by combining the methods of background subtraction and Histogram of Oriented Gradient (HOG) and then establishes discriminative appearance model by generating patches and constructing codebooks using superpixel and Local Binary Pattern (LBP) features in those target regions. During the process of tracking, the algorithm uses the similarity between candidates and codebooks as observation likelihood function and processes severe occlusion condition to prevent drift and loss phenomenon caused by target occlusion. Experimental results demonstrate that our algorithm improves the tracking performance in complicated real scenarios.

We develop an algorithm for finding and kinematically tracking multiple people in long sequences. Our basic assumption is that people tend to take on certain canonical poses, even when performing unusual activities like throwing a baseball or figure skating. We build a person detector that quite accurately detects and localizes limbs of people in lateral walking poses. We use the estimated limbs from a detection to build a discriminative appearance model; we assume the features that discriminate a figure in one frame will discriminate the figure in other frames. We then use the models as limb detectors in a pictorial structure framework, detecting figures in unrestricted poses in both previous and successive frames. We have run our tracker on hundreds of thousands of frames, and present and apply a methodology for evaluating tracking on such a large scale. We test our tracker on real sequences including a feature-length film, an hour of footage from a public park, and various sports sequences. We find that we can quite accurately automatically find and track multiple people interacting with each other while performing fast and unusual motions.

Developing robust and discriminative appearance models has been a long-standing research challenge in visual object tracking. In the prevalent Siamese-based paradigm, the features extracted by the Siamese-like networks are often insufficient to model the tracked targets and distractor objects, thereby hindering them from being robust and discriminative simultaneously. While most Siamese trackers focus on designing robust correlation operations, we propose a novel single-branch tracking framework inspired by the transformer. Unlike the Siamese-like feature extraction, our tracker deeply embeds cross-image feature correlation in multiple layers of the feature network. By extensively matching the features of the two images through multiple layers, it can suppress non-target features, resulting in target-aware feature extraction. The output features can be directly used to predict target locations without additional correlation steps. Thus, we reformulate the two-branch Siamese tracking as a conceptually simple, fully transformer-based Single-Branch Tracking pipeline, dubbed SBT. After conducting an in-depth analysis of the SBT baseline, we summarize many effective design principles and propose an improved tracker dubbed SuperSBT. SuperSBT adopts a hierarchical architecture with a local modeling layer to enhance shallow-level features. A unified relation modeling is proposed to remove complex handcrafted layer pattern designs. SuperSBT is further improved by masked image modeling pre-training, integrating temporal modeling, and equipping with dedicated prediction heads. Thus, SuperSBT outperforms the SBT baseline by 4.7%,3.0%, and 4.5% AUC scores in LaSOT, TrackingNet, and GOT-10K. Notably, SuperSBT greatly raises the speed of SBT from 37 FPS to 81 FPS. Extensive experiments show that our method achieves superior results on eight VOT benchmarks.

In order to obtain the discriminative compact appearance model for tracking objects effectively, this paper proposes a new structural tracking strategy that includes multicue inverse sparse appearance model and optimal metric evaluation between online robust templates and a limited number of particle samples in the looping process. Multicue inverse sparse appearance model globally improves the efficient selection of informative particle samples that can avoid the cumbersome coding and decoding cost for the trivial random particle samples. Only the most potential crucial cases are involved in each tracking loop. This refrains from unreasonable, rough numerical reduction of particle samples and also keeps the unbiasedness and dynamic stochasticness of the sampling process. Meanwhile, low-rank self-representatives for positive and negative samples facilitate the formulation of a suitable code book that arranges the useful sparse coefficients for feature bags and facilitates optimal metric evaluation for online training. It also alleviates the accuracy degradation of tracking occluded objects and improves the robustness of the tracker. Both of them preserve the discriminative compactness of target which speeds up particle filtering localization to separate the target object from distractors. Moreover, the proposed method exploits online appearance representations to learn the sharing compact information that avoids massive calculation burdens for massive visual data.

Automatic prostate segmentation from MR images is an important task in various clinical applications such as prostate cancer staging and MR-guided radiotherapy planning. However, the large appearance and shape variations of the prostate in MR images make the segmentation problem difficult to solve. Traditional Active Shape/Appearance Model (ASM/AAM) has limited accuracy on this problem, since its basic assumption, i.e., both shape and appearance of the targeted organ follow Gaussian distributions, is invalid in prostate MR images. To this end, the authors propose a sparse dictionary learning method to model the image appearance in a nonparametric fashion and further integrate the appearance model into a deformable segmentation framework for prostate MR segmentation. To drive the deformable model for prostate segmentation, the authors propose nonparametric appearance and shape models. The nonparametric appearance model is based on a novel dictionary learning method, namely distributed discriminative dictionary (DDD) learning, which is able to capture fine distinctions in image appearance. To increase the differential power of traditional dictionary-based classification methods, the authors' DDD learning approach takes three strategies. First, two dictionaries for prostate and nonprostate tissues are built, respectively, using the discriminative features obtained from minimum redundancy maximum relevance feature selection. Second, linear discriminant analysis is employed as a linear classifier to boost the optimal separation between prostate and nonprostate tissues, based on the representation residuals from sparse representation. Third, to enhance the robustness of the authors' classification method, multiple local dictionaries are learned for local regions along the prostate boundary (each with small appearance variations), instead of learning one global classifier for the entire prostate. These discriminative dictionaries are located on different patches of the prostate surface and trained to adaptively capture the appearance in different prostate zones, thus achieving better local tissue differentiation. For each local region, multiple classifiers are trained based on the randomly selected samples and finally assembled by a specific fusion method. In addition to this nonparametric appearance model, a prostate shape model is learned from the shape statistics using a novel approach, sparse shape composition, which can model nonGaussian distributions of shape variation and regularize the 3D mesh deformation by constraining it within the observed shape subspace. The proposed method has been evaluated on two datasets consisting of T2-weighted MR prostate images. For the first (internal) dataset, the classification effectiveness of the authors' improved dictionary learning has been validated by comparing it with three other variants of traditional dictionary learning methods. The experimental results show that the authors' method yields a Dice Ratio of 89.1% compared to the manual segmentation, which is more accurate than the three state-of-the-art MR prostate segmentation methods under comparison. For the second dataset, the MICCAI 2012 challenge dataset, the authors' proposed method yields a Dice Ratio of 87.4%, which also achieves better segmentation accuracy than other methods under comparison. A new magnetic resonance image prostate segmentation method is proposed based on the combination of deformable model and dictionary learning methods, which achieves more accurate segmentation performance on prostate T2 MR images.

A robust object tracking algorithm is proposed in this paper based on an on-line discriminative appearance modeling mechanism. In contrast with traditional trackers whose computations cover the whole target region and may easily be polluted by the similar background pixels, we divided the target into a number of patches and take the most discriminative one as the tracking basis. With the consideration of both the photometric and spatial information, we construct a discriminative target model on it. Then, a likelihood map can be got by comparing the target model with candidate regions, on which the mean shift procedure is employed for mode seeking. Finally, we update the target model to adapt to the appearance variation. Experiments confirm the robustness and reliability of our method.

This paper proposes a distributed multi-camera tracking algorithm with interacting particle filters. A robust multi-view appearance model is obtained by sharing training samples between views. Motivated by incremental learning and [1], we create an intermediate data representation between two camera views with generative subspaces as points on a Grassmann manifold, and sample along the geodesic between training data from two views to uncover the meaningful description due to viewpoint changes. Finally, a Boosted appearance model is trained using the projected training samples on to these generative subspaces. For each object, a set of two particle filters i.e., local and global is used. The local particle filter models the object motion in the image plane. The global particle filter models the object motion in the ground plane. These particle filters are integrated into a unified Interacting Markov Chain Monte Carlo (IMCMC) framework. We show the manner in which we induce priors on scene specific information into the global particle filter to improve tracking accuracy. The proposed algorithm is validated with extensive experimentation in challenging camera network data, and compares favorably with state of the art object trackers.

Multi-object tracking via discriminative appearance modeling