Chapter 13 - Robust and Scalable Shape Prior Modeling via Sparse Representation and Dictionary Learning

Shape information plays an important role in biomedical image analysis because of the strong shape characteristics of biological structures. It is often used as a prior to constrain or refine the intermediate shape information derived from low-level image features. In this paper, we give an overview of the sparse shape composition based prior modeling method and its various applications of biomedical image analysis. Instead of learning a generative shape model, it incorporates shape priors on-the-fly through the sparse shape composition. Particularly, a shape instance derived from low level image features is refined by a sparse linear combination of a sparse set of shapes in the repository. We also design three strategies to improve run-time efficiency. 1) When the shape repository contains a large number of instances, K-SVD can be used to learn a more compact but still informative shape dictionary. 2) If the derived shape instance has a large number of vertices, which often appears in 3D problems, affinity propagation method can be used to partition the surface into small subregions. 3) When there are multiple structures, hierarchical scheme is employed to model them simultaneously. These strategies decrease the scale of the sparse optimization problem and thus speed up the algorithm. Our method is applied on different biomedical image analysis problems, including localization, tracking and segmentation of anatomical structures. In all of these applications, this method achieves promising results.

Automatic prostate segmentation from MR images is an important task in various clinical applications such as prostate cancer staging and MR-guided radiotherapy planning. However, the large appearance and shape variations of the prostate in MR images make the segmentation problem difficult to solve. Traditional Active Shape/Appearance Model (ASM/AAM) has limited accuracy on this problem, since its basic assumption, i.e., both shape and appearance of the targeted organ follow Gaussian distributions, is invalid in prostate MR images. To this end, the authors propose a sparse dictionary learning method to model the image appearance in a nonparametric fashion and further integrate the appearance model into a deformable segmentation framework for prostate MR segmentation. To drive the deformable model for prostate segmentation, the authors propose nonparametric appearance and shape models. The nonparametric appearance model is based on a novel dictionary learning method, namely distributed discriminative dictionary (DDD) learning, which is able to capture fine distinctions in image appearance. To increase the differential power of traditional dictionary-based classification methods, the authors' DDD learning approach takes three strategies. First, two dictionaries for prostate and nonprostate tissues are built, respectively, using the discriminative features obtained from minimum redundancy maximum relevance feature selection. Second, linear discriminant analysis is employed as a linear classifier to boost the optimal separation between prostate and nonprostate tissues, based on the representation residuals from sparse representation. Third, to enhance the robustness of the authors' classification method, multiple local dictionaries are learned for local regions along the prostate boundary (each with small appearance variations), instead of learning one global classifier for the entire prostate. These discriminative dictionaries are located on different patches of the prostate surface and trained to adaptively capture the appearance in different prostate zones, thus achieving better local tissue differentiation. For each local region, multiple classifiers are trained based on the randomly selected samples and finally assembled by a specific fusion method. In addition to this nonparametric appearance model, a prostate shape model is learned from the shape statistics using a novel approach, sparse shape composition, which can model nonGaussian distributions of shape variation and regularize the 3D mesh deformation by constraining it within the observed shape subspace. The proposed method has been evaluated on two datasets consisting of T2-weighted MR prostate images. For the first (internal) dataset, the classification effectiveness of the authors' improved dictionary learning has been validated by comparing it with three other variants of traditional dictionary learning methods. The experimental results show that the authors' method yields a Dice Ratio of 89.1% compared to the manual segmentation, which is more accurate than the three state-of-the-art MR prostate segmentation methods under comparison. For the second dataset, the MICCAI 2012 challenge dataset, the authors' proposed method yields a Dice Ratio of 87.4%, which also achieves better segmentation accuracy than other methods under comparison. A new magnetic resonance image prostate segmentation method is proposed based on the combination of deformable model and dictionary learning methods, which achieves more accurate segmentation performance on prostate T2 MR images.

Shape information is essential in medical image analysis as the anatomical structures usually have strong shape characteristics. Shape priors can resolve ambiguities when the low level appearance is weak or misleading due to imaging artifacts and diseases. In this paper, we propose a shape prior model based on the Gaussian-Bernoulli Restricted Boltzmann Machine (GB-RBM). This powerful generative model is effective in capturing complex shape variations and handling nonlinear shape transformations. The model also shows great robustness, which is able to handle both outliers and Gaussian noise with large variance. We validate our model on synthetic data and a real clinical problem, i.e., lung segmentation in chest X-ray. Experiments show that our shape modeling method is qualitatively and quantitatively better than other widely-used shape prior methods.

Color and texture have been widely used in image segmentation; however, their performance is often hindered by scene ambiguities, overlapping objects, or missing parts. In this paper, we propose an interactive image segmentation approach with shape prior models within a Bayesian framework. Interactive features, through mouse strokes, reduce ambiguities, and the incorporation of shape priors enhances quality of the segmentation where color and/or texture are not solely adequate. The novelties of our approach are in (i) formulating the segmentation problem in a well-defined Bayesian framework with multiple shape priors, (ii) efficiently estimating parameters of the Bayesian model, and (iii) multi-object segmentation through user-specified priors. We demonstrate the effectiveness of our method on a set of natural and synthetic images.

Optimal multi-object segmentation with novel gradient vector flow based shape priors

In this paper, we propose a method to segment multiple rodent brain structures simultaneously. This method combines deformable models and hierarchical shape priors within one framework. The deformation module employs both gradient and appearance information to generate image forces to deform the shape. The shape prior module uses Principal Component Analysis to hierarchically model the multiple structures at both global and local levels. At the global level, the statistics of relative positions among different structures are modeled. At the local level, the shape statistics within each structure is learned from training samples. Our segmentation method adaptively employs both priors to constrain the intermediate deformation result. This prior constraint improves the robustness of the model and benefits the segmentation accuracy. Another merit of our prior module is that the size of the training data can be small, because the shape prior module models each structure individually and combines them using global statistics. This scheme can preserve shape details better than directly applying PCA on all structures. We use this method to segment rodent brain structures, such as the cerebellum, the left and right striatum, and the left and right hippocampus. The experiments show that our method works effectively and this hierarchical prior improves the segmentation performance.

In difficult object segmentation tasks, utilizing image information alone is not sufficient; incorporation of object shape prior models is necessary to obtain competitive segmentation performance. Most formulations that incorporate both shape and image information are in the form of energy functional optimization problems. This paper introduces a Bayesian latent marked Poisson process for segmenting multiple objects in an image. The model takes both shape and image feature/appearance into account—it generates object locations from a spatial Poisson process, then generates shape parameters from a shape prior model as the latent marks. Inferentially, this partitions the image: pixels inside objects are assumed to be generated from an object observation/appearance model and pixels outside objects come from a background model. The Poisson process provides (non-homogeneous) spatial priors for object locations and the marks allow the incorporation of shape priors. We develop a hybrid Gibbs sampler that addresses the variation in model order and nonconjugacy that arise in this setting and we present experimental results on synthetic images and two diverse domains in real images: cell segmentation in biological images and pedestrian and car detection in traffic images.

Difficult image segmentation problems, e.g., left atrium in MRI, can be addressed by incorporating shape priors to find solutions that are consistent with known objects. Nonetheless, a single multivariate Gaussian is not an adequate model in cases with significant nonlinear shape variation or where the prior distribution is multimodal. Nonparametric density estimation is more general, but has a ravenous appetite for training samples and poses serious challenges in optimization, especially in high dimensional spaces. Here, we propose a maximum-a-posteriori formulation that relies on a generative image model by incorporating both local intensity and global shape priors. We use deep autoencoders to capture the complex intensity distribution while avoiding the careful selection of hand-crafted features. We formulate the shape prior as a mixture of Gaussians and learn the corresponding parameters in a high-dimensional shape space rather than pre-projecting onto a low-dimensional subspace. In segmentation, we treat the identity of the mixture component as a latent variable and marginalize it within a generalized expectation-maximization framework. We present a conditional maximization-based scheme that alternates between a closed-form solution for component-specific shape parameters that provides a global update-based optimization strategy, and an intensity-based energy minimization that translates the global notion of a nonlinear shape prior into a set of local penalties. We demonstrate our approach on the left atrial segmentation from gadolinium-enhanced MRI, which is useful in quantifying the atrial geometry in patients with atrial fibrillation.

A new method incorporating deep learning with shape priors for left ventricular segmentation in myocardial perfusion SPECT images

Computer vision based train rolling stock examination

Recently support vector machines (SVM) using spatial pyramid matching (SPM) kernel have been highly successful in image classification applications. And linear spatial pyramid matching using sparse coding (ScSPM) scheme has been proposed to enhance the performance of SPM both in time and classification accuracy. In order to reduce the time complexity of dictionary construction process, sparse coding with affinity propagation method has been proposed in this paper. Because the dictionary used for sparse coding plays a key role in these methods, we also adopt supervised dictionary learning method to construct dictionary. The coding coefficients of each class have greater separability for SVM classification. Substantial experiments on Scene15 and CalTech101 image datasets have been conducted to investigate the performance of proposed approach in multi-class image classification; the results show that the approach can reach higher accuracy compared with ScSPM.

This paper investigates, using prior shape models and the concept of ball\nscale (b-scale), ways of automatically recognizing objects in 3D images without\nperforming elaborate searches or optimization. That is, the goal is to place\nthe model in a single shot close to the right pose (position, orientation, and\nscale) in a given image so that the model boundaries fall in the close vicinity\nof object boundaries in the image. This is achieved via the following set of\nkey ideas: (a) A semi-automatic way of constructing a multi-object shape model\nassembly. (b) A novel strategy of encoding, via b-scale, the pose relationship\nbetween objects in the training images and their intensity patterns captured in\nb-scale images. (c) A hierarchical mechanism of positioning the model, in a\none-shot way, in a given image from a knowledge of the learnt pose relationship\nand the b-scale image of the given image to be segmented. The evaluation\nresults on a set of 20 routine clinical abdominal female and male CT data sets\nindicate the following: (1) Incorporating a large number of objects improves\nthe recognition accuracy dramatically. (2) The recognition algorithm can be\nthought as a hierarchical framework such that quick replacement of the model\nassembly is defined as coarse recognition and delineation itself is known as\nfinest recognition. (3) Scale yields useful information about the relationship\nbetween the model assembly and any given image such that the recognition\nresults in a placement of the model close to the actual pose without doing any\nelaborate searches or optimization. (4) Effective object recognition can make\ndelineation most accurate.\n

Information about abdominal wall can be used for many applications from organ segmentation, registration, and surgical simulation. The challenges exist in abdominal wall extraction due to its varieties in shapes, connection to the internal organs and anterior layer edge formed between the muscle and fascia/fatty layer, which may distract the shape model. In this paper we present an approach to the posterior abdominal wall extraction using the shape model and other abdominal context, particularly with the rib-spine bone information and the wall image features. The shape model is constructed based on the training abdominal walls that are delineated manually. After bone information being extracted, the wall shape deforms from the prior shape model using the snake, which is constrained by the bone context and guided by the processed image energy map with the aim of removing distracted image features of anterior abdominal wall and the outer region from the original map. Meanwhile, an overall convex shape is maintained by limiting the angles of the contour points. The proposed approach is tested on abdominal CT data which provides encouraging results.

To present a model, which will reshape the raw 3D specific single-view images by learning the deformable object categories using unsupervised techniques. Using Convolutional autoencoders, each raw image will be taken as input, then this will be compressed by considering factors such as depth, albedo, viewpoint and illumination of a particular input. Autoencoders are basically used for efficient data codings in an unsupervised manner. These components can disentangled without any labels or supervisor based on the symmetric structure of the object. This model shows the reasoning about illumination which allows us to exploit the underlying object symmetry even if the appearance is not symmetric due to shading. Furthermore, this model objects that are probably, but not certainly, symmetric by predicting a symmetry probability map, learned end-to-end with the other components of the model. This model can recover very accurately the 3D shape of human faces, cat faces and cars from single-view images, without any supervision or a prior shape model. On benchmarks, model will be demonstrated with superior accuracy when compared to another method that uses supervision at the level of 2D image correspondences.

A new type of deformable model is presented that is able to combine some of the characteristics of both snakes and templates. It can be used to segment and recognize two dimensional objects when only vague prior knowledge about their shapes is available. A jump-diffusion process is used to fit the template to the image. The jumps allows the template to undergo abrupt discontinuous changes in shape and position and to decide among multiple target models. The diffusion process allows the template to perform continuous flowing deformations like a snake. A prior shape model is described that uses the local and global characteristics of each different target class. An efficient form for the image likelihood is given that extends to multiple attributes and multiple images. The jump transition kernel defines the probabilities of the template jumping to a new state. This is difficult to generate and sample in practice though. To allow for this a method is described where a marginal transition kernel is generated by integrating over the continuous internal parameters for subsets of jumps. This makes the sampling problem much easier while still providing effective inferencing. The relation of this approach to active contours and region competition is discussed. It is shown that with the appropriate choice of prior and likelihood that snakes can easily be modelled within the deterministic part of the diffusion process. The method is demonstrated with the detection of buildings and planes in infrared and optical images and a comparison with an active contour is also given.