Adaptive particle filters for visual object tracking using joint PCA appearance model and consensus point correspondences

Rotoscoping, the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines. While pixel-wise segmentation techniques can help for this task, professional rotoscoping tools rely on parametric curves that offer the artists a much better interactive control on the definition, editing and manipulation of the segments of interest. Sticking to this prevalent rotoscoping paradigm, we propose a novel framework to capture and track the visual aspect of an arbitrary object in a scene, given a first closed outline of this object. This model combines a collection of local foreground/background appearance models spread along the outline, a global appearance model of the enclosed object and a set of distinctive foreground landmarks. The structure of this rich appearance model allows simple initialization, efficient iterative optimization with exact minimization at each step, and on-line adaptation in videos. We demonstrate qualitatively and quantitatively the merit of this framework through comparisons with tools based on either dynamic segmentation with a closed curve or pixel-wise binary labelling.

Rotoscoping, the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines. While pixel-wise segmentation techniques can help for this task, professional rotoscoping tools rely on parametric curves that offer the artists a much better interactive control on the definition, editing and manipulation of the segments of interest. Sticking to this prevalent rotoscoping paradigm, we propose a novel framework to capture and track the visual aspect of an arbitrary object in a scene, given an initial closed outline of this object. This model combines a collection of local foreground/background appearance models spread along the outline, a global appearance model of the enclosed object and a set of distinctive foreground landmarks. The structure of this rich appearance model allows simple initialization, efficient iterative optimization with exact minimization at each step, and on-line adaptation in videos. We further extend this model by so-called trimaps which serve as an input to alpha-matting algorithms to allow truly seamless compositing. To this end, we leverage local classifiers attached to the roto-curves to define a confidence measure that is well-suited to define trimaps with adaptive band-widths. The resulting trimaps are parametric, temporally consistent and remain fully editable by the artist. We demonstrate qualitatively and quantitatively the merit of this framework through comparisons with tools based on either dynamic segmentation with a closed curve or pixel-wise binary labelling.

Real-time object tracking via self-adaptive appearance modeling

Model-free tracking is a widely-accepted approach to track an arbitrary object in a video using a single frame annotation with no further prior knowledge about the object of interest. Extending this problem to track multiple objects is really challenging because: a) the tracker is not aware of the objects' type while trying to distinguish them from background (detection task), and b) The tracker needs to distinguish one object from other potentially similar objects (data association task) to generate stable trajectories. In order to track multiple arbitrary objects, most existing model-free tracking approaches rely on tracking each target individually by updating their appearance model independently. Therefore, in this scenario they often fail to perform well due to confusion between the appearance of similar objects, their sudden appearance changes and occlusion. To tackle this problem, we propose to use both appearance and motion models, and to learn them jointly using graphical models and deep neural networks features. We introduce an indicator variable to predict sudden appearance change and/or occlusion. When these happen, our model does not update the appearance model thus avoiding using the background and/or incorrect object to update the appearance of the object of interest mistakenly, and relies on our motion model to track. Moreover, we consider the correlation among all targets, and seek the joint optimal locations for all targets simultaneously as a graphical model inference problem. We learn the joint parameters for both appearance model and motion model in an online fashion under the framework of LaRank. Experiment results show that our method achieved superior performance compared to the competitive methods.

Effective object appearance model is one of the key issues for the success of visual tracking. Since the appearance of a target and the environment changes dynamically, the majority of existed visual tracking algorithms tend to drift away from targets. To address this issue, we propose a robust tracking algorithm by integrating the generative and discriminative model. The object appearance model is made up of generative target model and a discriminative classifier. For the generative target model, we adopt the weighted structural local sparse appearance model combining patch based gray value and Histogram of Oriented Gradients feature as the patch dictionary. By sampling positives and negatives, alignment-pooling features are obtained based on the patch dictionary through local sparse coding, then we use support vector machine to train the discriminative classifier. The proposed method is embedded into a Bayesian inference framework for visual tracking. A combined matching method is adopted to improve the proposal distribution of the particle filter. Moreover, in order to adapt the situation change, the patch dictionary and discriminative classifier are updated by incremental learning every five frames. Experimental results on some publicly available benchmarks of video sequences demonstrate the accuracy and effectiveness of our tracker.

A computer model was developed to establish the relationship between the two-dimensional (2D) and the three-dimensional (3D) parameters, specifically the number of particles and particle size distribution. The computer experiments were performed for both monodispersed and lognormally polydispersed systems. The model was based on a random distribution of a number of spherical grains in a cubic unit, with no intersection between the grains. The cubic unit was cut by a random plane and the number of particles which appeared in the section and the area fraction were measured. This procedure was repeated until the average number of particles and the average area fraction became constant. Finally, the 2D size distribution of the particles over all sections was obtained.It was concluded that the volume fraction is equal to the area fraction, irrespective of the particle size distribution, providing that the total number of measured 2D particles is large enough. As for the number of particles, an equation was found to calculate the number of 3D particles in a monodispersed system from the number of 2D particles and the volume fraction. However, in lognormally polydispersed systems the number of 3D particles could be calculated using the 3D mean and standard deviation, estimated from the 2D mean and standard deviation by a method specifically developed for the purpose.Finally, the method was applied to two ductile cast iron specimens and the applicability of the present model was verified.

Color-based tracking has proved efficient and robust recently. Trackers build the object appearance model with histogram statistics, search and evaluate hypothesis in a probabilistic framework. This method relies much on the discrimination between object and scene blobs. Color clutter in the scene, although not so many in quantity, may distract these trackers. We build explicitly object shape model and insert the head detector into the observation model to resist these clutters in the scene for improved tracker. The detector scans the image and output probability value as the possibility of current window being a candidate human head. Experiments demonstrate the method can work more accurately and robustly.

Visual object tracking is a significant technology for camera-based sensor networks applications. Multilayer convolutional features comprehensively used in correlation filter (CF)-based tracking algorithms have achieved excellent performance. However, there are tracking failures in some challenging situations because ordinary features are not able to well represent the object appearance variations and the correlation filters are updated irrationally. In this paper, we propose a local–global multiple correlation filters (LGCF) tracking algorithm for edge computing systems capturing moving targets, such as vehicles and pedestrians. First, we construct a global correlation filter model with deep convolutional features, and choose horizontal or vertical division according to the aspect ratio to build two local filters with hand-crafted features. Then, we propose a local–global collaborative strategy to exchange information between local and global correlation filters. This strategy can avoid the wrong learning of the object appearance model. Finally, we propose a time-space peak to sidelobe ratio (TSPSR) to evaluate the stability of the current CF. When the estimated results of the current CF are not reliable, the Kalman filter redetection (KFR) model would be enabled to recapture the object. The experimental results show that our presented algorithm achieves better performances on OTB-2013 and OTB-2015 compared with the other latest 12 tracking algorithms. Moreover, our algorithm handles various challenges in object tracking well.

This paper presents a probabilistic framework for discovering objects in video. The video can switch between different shots, the unknown objects can leave or enter the scene at multiple times, and the background can be cluttered. The framework consists of an appearance model and a motion model. The appearance model exploits the consistency of object parts in appearance across frames. We use maximally stable extremal regions as observations in the model and hence provide robustness to object variations in scale, lighting and viewpoint. The appearance model provides location and scale estimates of the unknown objects through a compact probabilistic representation. The compact representation contains knowledge of the scene at the object level, thus allowing us to augment it with motion information using a motion model. This framework can be applied to a wide range of different videos and object types, and provides a basis for higher level video content analysis tasks. We present applications of video object discovery to video content analysis problems such as video segmentation and threading, and demonstrate superior performance to methods that exploit global image statistics and frequent itemset data mining techniques.

Modeling Self-Occlusions/Disocclusions in Dynamic Shape and Appearance Tracking for Obtaining Precise Shape We present a method to determine the precise shape of a dynamic object from video. This problem is fundamental to computer vision, and has a number of applications, for example, 3D video/cinema post-production, activity recognition and augmented reality. Current tracking algorithms that determine precise shape can be roughly divided into two categories: 1) Global statistics partitioning methods, where the shape of the object is determined by discriminating global image statistics, and 2) Joint shape and appearance matching methods, where a template of the object from the previous frame is matched to the next image. The former is limited in cases of complex object appearance and cluttered background, where global statistics cannot distinguish between the object and background. The latter is able to cope with complex appearance and a cluttered background, but is limited in cases of camera viewpoint change and object articulation, which induce self-occlusions and self-disocclusions of the object of interest. The purpose of this thesis is to model self-occlusion/disocclusion phenomena in a joint shape and appearance tracking framework. We derive a non-linear dynamic model of the object shape and appearance taking into account occlusion phenomena, which is then used to infer self-occlusions/disocclusions, shape and appearance of the object in a variational optimization framework. To ensure robustness to other unmodeled phenomena 5 that are present in real-video sequences, the Kalman filter is used for appearance updating. Experiments show that our method, which incorporates the modeling of self-occlusion/disocclusion, increases the accuracy of shape estimation in situations of viewpoint change and articulation, and out-performs current state-of-the-art methods for shape tracking.

In this paper, we propose a novel fuzzy logic data association algorithm to resolve the problem of visual multi-object tracking. First of all, a fuzzy logic data association approach is proposed to calculate the association probabilities between the objects and observations (or detection responses) by use of a set of fuzzy if-then rules. Based on the similarities of object appearance, shape and motion models, several fuzzy inference systems are designed to confirm the fuzzy membership degrees between the objects and observations, the association probability is generated by weighted affinities, where the weight of the affinity is the output of fuzzy logic. Finally, the experiment results on several public sequences demonstrate that the proposed algorithm has superiorities over other comparatively leading tracking algorithms in terms of accuracy and robustness.

We present a novel local region approach for statistically characterizing appearance in the context of medical image segmentation via deformable models. Our appearance model reflects the inhomogeneity of tissue mixtures around the exterior of the object of interest by determining mixture-consistent local region types relative to the object boundary. The region types are formed by clustering local regional image descriptors. We partition the object boundary according to region type and apply principal component analysis on the cluster populations to acquire a statistical model of object appearance that accounts for local variability in the object exterior. We present results using this approach to segment bladders and prostates in CT in the context of day-to-day adaptive radiotherapy for prostate cancer. Results show improved fits versus those obtained with a previously developed method

Recent few-shot logical anomaly detection methods rely on external information for accurate detection. This is often done through handmade text prompts and category-specific procedures, making them infeasible to apply to new datasets. Full-shot methods do not utilise this additional information but extract meaningful representations of local and global structures. We hypothesise that a major drawback of few-shot logical anomaly detection methods is the over-reliance on external information and suboptimal image representation. However, matching the representations learned by full-shot methods is challenging due to the lack of data in a few-shot setting. We propose PyramidCore, a novel few-shot logical anomaly detection method that does not rely on external information but instead uses a robust appearance model that can be built from only a few examples. It builds a hierarchical model of object appearance, enabling the detection of complex logical anomalies at different scales. The proposed method achieves state-of-the-art results on the challenging MVTec LOCO Dataset.

We present an integrated two-level approach to computationally analyzing image sequences of static scenes by variational segmentation. At the top level, estimated models of object appearance and background are probabilistically fused to obtain an a-posteriori probability for the occupancy of each pixel. The data-association strategy handles object occlusions explicitly. At the lower level, object models are inferred by variational segmentation based on image data and statistical shape priors. The use of shape priors allows to distinguish between recognition of known objects and segmentation of unknown objects. The object models are sufficiently flexible to enable the integration of general cues like advanced shape distances. At the same time, they are highly constrained from the optimization viewpoint: the globally optimal parameters can be computed at each time instant by dynamic programming. The novelty of our approach is the integration of state-of-the-art variational segmentation into a probabilistic framework for static scene analysis that combines both on-line learning and prior knowledge of various aspects of object appearance.

Ortho-diffusion decompositions of graph-based representation of images