Learning local vessel appearance models using structured sparsity

Automatic and precise segmentation of hand bones is important for many medical imaging applications. Although several previous studies address bone segmentation, automatically segmenting articulated hand bones remains a challenging task. The highly articulated nature of hand bones limits the effectiveness of atlas-based segmentation methods. The use of low-level information derived from the image-of-interest alone is insufficient for detecting bones and distinguishing boundaries of different bones that are in close proximity to each other. In this study, we propose a method that combines an articulated statistical shape model and a local exemplar-based appearance model for automatically segmenting hand bones in CT. Our approach is to perform a hierarchical articulated shape deformation that is driven by a set of local exemplar-based appearance models. Specifically, for each point in the shape model, the local appearance model is described by a set of profiles of low-level image features along the normal of the shape. During segmentation, each point in the shape model is deformed to a new point whose image features are closest to the appearance model. The shape model is also constrained by an articulation model described by a set of pre-determined landmarks on the finger joints. In this way, the deformation is robust to sporadic false bony edges and is able to fit fingers with large articulations. We validated our method on 23 CT scans and we have a segmentation success rate of ~89.70 %. This result indicates that our method is viable for automatic segmentation of articulated hand bones in conventional CT.

We consider the problem of semantic segmentation, i.e. assigning each pixel in an image to a set of pre-defined semantic object categories. State-of-the-art semantic segmentation algorithms typically consist of three components: a local appearance model, a local consistency model and a global consistency model. These three components are generally integrated into a unified probabilistic framework. While it enables at training time a joint estimation of the model parameters and while it ensures at test time a globally consistent labeling of the pixels, it also comes at a high computational cost. We propose a simple approach to semantic segmentation where the three components are decoupled (this journal submission is an extended version of the following conference paper: G. Csurka and F. Perronnin, "A simple high performance approach to semantic segmentation", BMVC, 2008). For the local appearance model, we make use of the Fisher kernel. While this framework was shown to lead to high accuracy for image classification, to our best knowledge this is its first application to the segmentation problem. The semantic segmentation process is then guided by a low-level segmentation which enforces local consistency. Finally, to enforce image-level consistency we use global image classifiers: if an image as a whole is unlikely to contain an object class, then the corresponding class is not considered in the segmentation pipeline. The decoupling of the components makes our system very efficient both at training and test time. An efficient training enables to estimate the model parameters on large quantities of data. Especially, we explain how our system can leverage weakly labeled data, i.e. images for which we do not have pixel-level labels but either object bounding boxes or even only image-level labels. We believe that an important contribution of this paper is to show that even a simple decoupled system can provide state-of-the-art performance on the PASCAL VOC 2007, PASCAL VOC 2008 and MSRC 21 datasets.

Dual-scale structural local sparse appearance model for robust object tracking

A novel compressive local appearance model‐based object tracking algorithm is presented to address challenging issues in object tracking. To efficiently preserve image patches of an object and reduce the dimensionality, a random projection‐based feature selection method is introduced. Modelling the object's appearance using a sparse representation over a set of templates leads to an ℓ 1 ‐regularisation problem. To solve this problem, both the reconstruction error and the residual matrix are considered which play a key role in tracking an object with severe appearance variations using the modified likelihood function. Experimental results demonstrate that the proposed method outperforms existing state‐of‐the‐art tracking methods in terms of dealing with long‐term partial occlusion, deformation and rotation.

Accurate localization of representative points of a face is crucial to many face analysis and synthesis problems. Active shape model (ASM) is a powerful statistical tool for face alignment. However, it suffer from variations of pose, illumination and expressions. In this paper, we propose an improved ASM method, RG-ASM in which local appearance models of key points are modeled using statistical learning. RealBoost is proposed to build the likelihood model that ensures the ground truth position of each key point will more likely have a higher likelihood than its neighbors. Instead of using principle components analysis and one pixel Gabor coefficients. Gabor wavelet features of key point and its neighbors are used as the feature space in the learning procedure to model local structures of a face. Experimental results demonstrate that RG-ASM achieves more accurate results compared with original method used in ASM.

Effective object appearance model is one of the key issues for the success of visual tracking. Since the appearance of a target and the environment changes dynamically, the majority of existed visual tracking algorithms tend to drift away from targets. To address this issue, we propose a robust tracking algorithm by integrating the generative and discriminative model. The object appearance model is made up of generative target model and a discriminative classifier. For the generative target model, we adopt the weighted structural local sparse appearance model combining patch based gray value and Histogram of Oriented Gradients feature as the patch dictionary. By sampling positives and negatives, alignment-pooling features are obtained based on the patch dictionary through local sparse coding, then we use support vector machine to train the discriminative classifier. The proposed method is embedded into a Bayesian inference framework for visual tracking. A combined matching method is adopted to improve the proposal distribution of the particle filter. Moreover, in order to adapt the situation change, the patch dictionary and discriminative classifier are updated by incremental learning every five frames. Experimental results on some publicly available benchmarks of video sequences demonstrate the accuracy and effectiveness of our tracker.

Particle filters have been proven very successful for non-linear and non-Gaussian estimation problems and extensively used in object tracking. However, high computational costs and particle decadency problem limit its practical application. In this paper, we present a robust particle tracking approach based on spatio-temporal context learning and multi-task joint local sparse representation. The proposed tracker samples particles according to the confidence map constructed by the spatio-temporal context information of the target. This sampling strategy can ameliorate problems of sample impoverishment and particle degeneracy, target state distribution to obtain robust tracking performance. In order to locate the target more accurately and be less sensitive to occlusion, the local sparse appearance model is adopted to capture the local and structural information of the target. Finally, the multi-task learning where the representations of particles are learned jointly is employed to further improve tracking performance and reduce overall computational complexity. Both qualitative and quantitative evaluations on challenging benchmark image sequences have demonstrated that the proposed tracking algorithm performs favorably against several state-of-the-art methods.

We present an improved bundle adjustment method based on the online learned appearance subspaces of 3D points. Our method incorporates the additional information from the learned appearance models into bundle adjustment. Through the online learning of the appearance models, we are able to include more plausible observations of 2D features across diverse viewpoints. Bundle adjustment can benefit from such an increase in the number of observations. Our formulation uses the appearance information to impose additional constraints on the optimization. The detailed experiments with ground-truth data show that the proposed method is able to enhance the reliability of 2D correspondences, and more important, can improve the accuracy of camera motion estimation and the overall quality of 3D reconstruction.

Recently, much work has been devoted to multiple object tracking on the one hand and to appearance model adaptation for a single object tracker on the other side. In this paper, we do both tracking of multiple objects (faces of people) in a meeting scenario and on-line learning to incrementally update the models of the tracked objects to account for appearance changes during tracking. Additionally, we automatically initialize and terminate tracking of individual objects based on low-level features, i.e. face color, face size, and object movement. For tracking a particle filter is incorporated to propagate sample distributions over time. Numerous experiments on meeting data demonstrate the capabilities of our tracking approach. Additionally, we provide an empirical verification of appearance model learning during tracking of an outdoor scene which supports a more robust tracking.

We present a novel framework for learning a joint shape and appearance model from a large set of un-labelled training examples in arbitrary positions and orientations. The shape and intensity spaces are unified by implicitly representing shapes as “images” in the space of distance transforms. A stochastic chord-based matching algorithm is developed to align photo-realistic training examples under a common reference frame. Then dense local deformation fields, represented using the cubic B-spline based Free Form Deformations (FFD), are recovered to register the training examples in both shape and intensity spaces. Principal Component Analysis (PCA) is applied on the FFD control lattices to capture the variations in shape as well as on registered object interior textures. We show examples where we have built coupled shape and appearance prior models for the left ventricle and whole heart in short-axis cardiac tagged MR images, and used them to delineate the heart chambers in noisy, cluttered images. We also show quantitative validation on the automatic segmentation results by comparing to expert solutions.

View-based recognition of real-world textures

This study addresses the automatic multi‐person tracking problem in complex scenes from a single, static, uncalibrated camera. In contrast with offline tracking approaches, a novel online multi‐person tracking method is proposed based on a sequential tracking‐by‐detection framework, which can be applied to real‐time applications. A two‐stage data association is first developed to handle the drifting targets stemming from occlusions and people's abrupt motion changes. Subsequently, a novel online appearance learning is developed by using the incremental/decremental support vector machine with an adaptive training sample collection strategy to ensure reliable data association and rapid learning. Experimental results show the effectiveness and robustness of the proposed method while demonstrating its compatibility with real‐time applications.

This paper presents an appearance-based method for object pose recognition using single camera image. The basic idea of our method is to find the correlation between object image and its pose, and use it for object recognition and pose estimation. Canonical correlation analysis is introduced to derive such correlation and build a compact appearance model. In the approach, we first obtain a pair of training data set, i.e., object images and their pose parameters. The appearance model is given as the subspace spanned by the canonical vectors that maximize the correlation between images and poses. Pose parameters of currently observed image is predicted by finding the regression coefficient in this subspace. We also introduce the kernel methods to cope with the non-linearity lies in training data set. Experiments are conducted on object pose estimation and vehicle type classification problem. Performance of our appearance models is discussed through the comparison with conventional subspace method

This paper proposes a distributed multi-camera tracking algorithm with interacting particle filters. A robust multi-view appearance model is obtained by sharing training samples between views. Motivated by incremental learning and [1], we create an intermediate data representation between two camera views with generative subspaces as points on a Grassmann manifold, and sample along the geodesic between training data from two views to uncover the meaningful description due to viewpoint changes. Finally, a Boosted appearance model is trained using the projected training samples on to these generative subspaces. For each object, a set of two particle filters i.e., local and global is used. The local particle filter models the object motion in the image plane. The global particle filter models the object motion in the ground plane. These particle filters are integrated into a unified Interacting Markov Chain Monte Carlo (IMCMC) framework. We show the manner in which we induce priors on scene specific information into the global particle filter to improve tracking accuracy. The proposed algorithm is validated with extensive experimentation in challenging camera network data, and compares favorably with state of the art object trackers.

Aimed at tracking non-rigid objects with geometric appearance changes over time, we propose a novel patch-based appearance model to adapt to the changes of topology. Meanwhile, as an effective online updating scheme, superpixel learning is adopted to select and update the patches when a new frame arrives. We build a foreground-background vote map via superpixels to determine the confidence of the patches in case of drifting. Experimental results show the proposed approach enables tracking non-rigid targets robustly and accurately.