Metamorphs: Deformable Shape and Appearance Models

Robust brain ROI segmentation by deformation regression and deformable shape model.

Deformable models are common in image modeling and analysis. Random objects provide major challenges as shapes and appearances are hard to quantify; hence, formulation of deformable models are much harder to construct and validate. In this paper, we examine the effect of randomness on building the shape and appearance models for small-size lung nodules (≤ 1cm) which appear in computed tomography (CT) of the human chest. We devise an approach for annotation, which lends a standard mechanism for building traditional active appearance (AAM), active shape (ASM) and active tensor models (ATM). We illustrate the effectiveness of AAM for nodule detection.

Automatic prostate segmentation from MR images is an important task in various clinical applications such as prostate cancer staging and MR-guided radiotherapy planning. However, the large appearance and shape variations of the prostate in MR images make the segmentation problem difficult to solve. Traditional Active Shape/Appearance Model (ASM/AAM) has limited accuracy on this problem, since its basic assumption, i.e., both shape and appearance of the targeted organ follow Gaussian distributions, is invalid in prostate MR images. To this end, the authors propose a sparse dictionary learning method to model the image appearance in a nonparametric fashion and further integrate the appearance model into a deformable segmentation framework for prostate MR segmentation. To drive the deformable model for prostate segmentation, the authors propose nonparametric appearance and shape models. The nonparametric appearance model is based on a novel dictionary learning method, namely distributed discriminative dictionary (DDD) learning, which is able to capture fine distinctions in image appearance. To increase the differential power of traditional dictionary-based classification methods, the authors' DDD learning approach takes three strategies. First, two dictionaries for prostate and nonprostate tissues are built, respectively, using the discriminative features obtained from minimum redundancy maximum relevance feature selection. Second, linear discriminant analysis is employed as a linear classifier to boost the optimal separation between prostate and nonprostate tissues, based on the representation residuals from sparse representation. Third, to enhance the robustness of the authors' classification method, multiple local dictionaries are learned for local regions along the prostate boundary (each with small appearance variations), instead of learning one global classifier for the entire prostate. These discriminative dictionaries are located on different patches of the prostate surface and trained to adaptively capture the appearance in different prostate zones, thus achieving better local tissue differentiation. For each local region, multiple classifiers are trained based on the randomly selected samples and finally assembled by a specific fusion method. In addition to this nonparametric appearance model, a prostate shape model is learned from the shape statistics using a novel approach, sparse shape composition, which can model nonGaussian distributions of shape variation and regularize the 3D mesh deformation by constraining it within the observed shape subspace. The proposed method has been evaluated on two datasets consisting of T2-weighted MR prostate images. For the first (internal) dataset, the classification effectiveness of the authors' improved dictionary learning has been validated by comparing it with three other variants of traditional dictionary learning methods. The experimental results show that the authors' method yields a Dice Ratio of 89.1% compared to the manual segmentation, which is more accurate than the three state-of-the-art MR prostate segmentation methods under comparison. For the second dataset, the MICCAI 2012 challenge dataset, the authors' proposed method yields a Dice Ratio of 87.4%, which also achieves better segmentation accuracy than other methods under comparison. A new magnetic resonance image prostate segmentation method is proposed based on the combination of deformable model and dictionary learning methods, which achieves more accurate segmentation performance on prostate T2 MR images.

We present a novel framework for learning a joint shape and appearance model from a large set of un-labelled training examples in arbitrary positions and orientations. The shape and intensity spaces are unified by implicitly representing shapes as “images” in the space of distance transforms. A stochastic chord-based matching algorithm is developed to align photo-realistic training examples under a common reference frame. Then dense local deformation fields, represented using the cubic B-spline based Free Form Deformations (FFD), are recovered to register the training examples in both shape and intensity spaces. Principal Component Analysis (PCA) is applied on the FFD control lattices to capture the variations in shape as well as on registered object interior textures. We show examples where we have built coupled shape and appearance prior models for the left ventricle and whole heart in short-axis cardiac tagged MR images, and used them to delineate the heart chambers in noisy, cluttered images. We also show quantitative validation on the automatic segmentation results by comparing to expert solutions.

The use of appearance and shape priors in image segmentation is known to improve accuracy; however, existing techniques have several drawbacks. For instance, most active shape and appearance models require landmark points and assume unimodal shape and appearance distributions, and the level set representation does not support construction of local priors. In this paper, we present novel appearance and shape models for image segmentation based on a differentiable implicit parametric shape representation called a disjunctive normal shape model (DNSM). The DNSM is formed by the disjunction of polytopes, which themselves are formed by the conjunctions of half-spaces. The DNSM's parametric nature allows the use of powerful local prior statistics, and its implicit nature removes the need to use landmarks and easily handles topological changes. In a Bayesian inference framework, we model arbitrary shape and appearance distributions using nonparametric density estimations, at any local scale. The proposed local shape prior results in accurate segmentation even when very few training shapes are available, because the method generates a rich set of shape variations by locally combining training samples. We demonstrate the performance of the framework by applying it to both 2-D and 3-D data sets with emphasis on biomedical image segmentation applications.

This paper presents an evaluation of the performance of a three-dimensional Active Shape Model (ASM) to segment the liver in 48 clinical CT scans. The employed shape model is built from 32 samples using an optimization approach based on the minimum description length (MDL). Three different gray-value appearance models (plain intensity, gradient and normalized gradient profiles) are created to guide the search. The employed segmentation techniques are ASM search with 10 and 30 modes of variation and a deformable model coupled to a shape model with 10 modes of variation. To assess the segmentation performance, the obtained results are compared to manual segmentations with four different measures (overlap, average distance, RMS distance and ratio of deviations larger 5mm). The only appearance model delivering usable results is the normalized gradient profile. The deformable model search achieves the best results, followed by the ASM search with 30 modes. Overall, statistical shape modeling delivers very promising results for a fully automated segmentation of the liver.

In this study, we propose a semantic grasping method that accounts for post-grasp stability without requiring a preexisting 3D shape of the object. Among multiple candidate grasp points, the method selects the one that minimizes the moment at the grasp point with respect to the object’s centroid. This paper proposes a semantic grasping method that explicitly considers post-grasp stability without requiring an instance-specific 3D shape model. The method estimates a grasp pose that minimizes moment at the grasp point with respect to the object’s centroid, thereby maintaining a stable orientation after lifting. In experiments, we evaluated post-grasp stability against conventional methods; the proposed method maintained a stable state in 18 out of 25 trials, outperforming the conventional method (4 out of 25).

Several studies report the mechanical properties of skin tissues but their values largely depend on the measurement method. Therefore, we investigated the feasibility of recognizing the cellular constituents mechanical properties of pigmented skin by Confocal Laser Scanner Microscopy (CLSM). With this purpose, an healthy volunteer was examined in three areas nearby a pigmented skin lesion in two configurations: deforming and non deforming the nevus. The tissue displacement of the nevus was then assessed by means of deformable registration of the images in these two configurations. There are several registration strategy able to overcome this task, among them, we proposed two methods with different deformation models: a Free Form Deformation (FFD) model based on b-spline and a second one based on Demons Registration Algorithm (DRA). These two strategies need the definition of several parameters in order to obtain optimal registration performances. Thus, we tuned these parameters by means of simulated data and evaluated their registration abilities on the real in vivo CLSM acquisitions in the two configurations. The results showed that the registration using DRA had a better performance in comparison to the FFD one, in particular in two out of the three areas the DRA performance was significantly better than the FFD one. The registration procedure highlighted deformation differences among the chosen areas.

Using the knowledge of information technology and physical mechanics to simulate the evolution of plant cell shapes is helpful to explore the mechanism of plant tissue and organ formation. However, the traditional numerical modeling based on curves and surfaces is difficult to accurately locate a geometric fragment of cell shape, resulting in the inability to analyze the stress and strain of the fragment freely. In this study, a plant cell shape is firstly decomposed into a body, faces, edges, and vertexes from top to bottom using the n-dimensional generalized mapping theory. Then, the 1-, 2-and 3-dimensional topologies of the cell shape are constructed by combining these topologies from bottom to top using the “half-edge” structure of 0-dimensional topology, and the corresponding retrieval algorithms are designed. Finally, the program is achieved by the C++ programming language and the OpenGL graphics library. The experimental results show that users could not only retrieve the geometric units of cell shape freely, but also highlight the query results in the program through the query interface of this program. The results of this study can provide an automatic topological retrieval of a cell shape for the elastic mechanical modeling of cell shape deformation.

Automatic and precise segmentation of hand bones is important for many medical imaging applications. Although several previous studies address bone segmentation, automatically segmenting articulated hand bones remains a challenging task. The highly articulated nature of hand bones limits the effectiveness of atlas-based segmentation methods. The use of low-level information derived from the image-of-interest alone is insufficient for detecting bones and distinguishing boundaries of different bones that are in close proximity to each other. In this study, we propose a method that combines an articulated statistical shape model and a local exemplar-based appearance model for automatically segmenting hand bones in CT. Our approach is to perform a hierarchical articulated shape deformation that is driven by a set of local exemplar-based appearance models. Specifically, for each point in the shape model, the local appearance model is described by a set of profiles of low-level image features along the normal of the shape. During segmentation, each point in the shape model is deformed to a new point whose image features are closest to the appearance model. The shape model is also constrained by an articulation model described by a set of pre-determined landmarks on the finger joints. In this way, the deformation is robust to sporadic false bony edges and is able to fit fingers with large articulations. We validated our method on 23 CT scans and we have a segmentation success rate of ~89.70 %. This result indicates that our method is viable for automatic segmentation of articulated hand bones in conventional CT.

The aim of facial reconstruction is to estimate the face of an unknown individual from the shape of his/her skull to aid in recognition and identification. In this paper a 3D computerized facial reconstruction method from 3D-MRI of human heads is presented. It based on a model deformable approach constrained by the knowledge of soft tissues thicknesses, at a certain number of characteristic landmarks. The facial reconstruction process is divided into two main stages. An automatic initialization step of the deformable model is based on a non-linear registration technique guided by B-Spline Free-Form Deformations (FFD) model. In the second step, the initial skin, resulting from the transformation computed previously, is refined by means of a 3D deform-able model based on simplexes meshes that is attracted by some characteristic landmarks previously computed by a calculation of Mean and Gaussian curvatures. The evolution of the deformable model is done according to different internal and external forces among which the Gradient Vectors Flow force (GVF) and a balloon force. This method has been tested on thirteen sets of skull/skin data, extracted from 3D-MRI of individual heads of children and adults.

Mobile devices are too small to operate freely using their input surfaces. To solve this problem, non-contact and natural gesture interfaces have been the focus of recent research. In this paper we propose a method of estimating multi-finger position and pose for operating such devices at high speed using a single camera. Our method achieves the finger tracking based on the appearance and shape deformation model by estimating the translational movements and the degree of bent finger. The experimental results show that our method can obtain the position of the hand and the pose of the each finger within 9.7 ms.

This paper describes the application of a new probabilistic shape and appearance model (PSAM) algorithm to the task of detecting polycystic kidney disease (PKD) in X-ray computed tomography images of laboratory mice. The genetically engineered PKD mouse is a valuable animal model that can be used to develop new treatments for kidney-related problems in humans. PSAM is a statistical-based deformable model that improves upon existing point distribution models for boundary-based object segmentation. This new deformable model algorithm finds the optimal boundary position using an objective function that has several unique characteristics. Most importantly, the objective function includes both global shape and local gray-level characteristics, so optimization occurs with respect to both pieces of information simultaneously. PSAM is employed to segment the mouse kidneys and then texture measurements are applied within kidney boundaries to detect PKD. The challenges associated with the segmentation non-rigid organs along with the availability of a priori information led to the choice of a trainable, deformable model for this application. In 103 kidney images that were analyzed as part of a preclinical animal study, the mouse kidneys and spine were segmented with an average error of 2.4 pixels per boundary point. In all 103 cases, the kidneys were successfully segmented at a level where PKD could be detected using mean-of-local-variance texture measurements within the located boundary.

In this work we automatically segment the left ventricle (LV) in cardiac MR images in the end-diastole (ED) and end-systole (ES) phases using a novel approach that combines statistical and deterministic deformable models. A 3D Active Appearance Model (AAM) is used to segment the ED phase. The AAM texture model is trained on radial samples from gradient magnitude images to make the fitting process faster and more discriminative. A trained ED-to-ES shape correspondence model is used to map a given ED shape to an ES shape. Once the AAM model converges to a shape in ED, the correspondence model is used to get an approximate ES shape. We segment the LV in the ES phase by first fitting a deformable superquadric to the AAM converged shape (in ED) using data range forces and then tracking the LV using image and data range forces (for the ES shape obtained from correspondence model). We test our approach by performing leave-one-out training on a 35 patient datasets. The data comprises 19 normal patients and 16 patients having heart abnormalities (cardiomyopathy and myocardial infarction). The composition makes it a challenging data collection with significant shape variation. The performance of our method is evaluated by measuring the mismatch between automatically segmented and expert delineated contours using the Mean Perpendicular Distance (MPD) and Dice metrics. The average MPD is 2.6mm for ED and 3.7mm for ES (error mostly towards the apex and base). The average Dice is 0.9 for ED and 0.8 for ES. These results show good potential for clinical use.

This paper addresses the problem of facial landmark localization and tracking from a single camera. We present a two-stage cascaded deformable shape model to effectively and efficiently localize facial landmarks with large head pose variations. For face detection, we propose a group sparse learning method to automatically select the most salient facial landmarks. By introducing 3D face shape model, we use procrustes analysis to achieve pose-free facial landmark initialization. For deformation, the first step uses mean-shift local search with constrained local model to rapidly approach the global optimum. The second step uses component-wise active contours to discriminatively refine the subtle shape variation. Our framework can simultaneously handle face detection, pose-free landmark localization and tracking in real time. Extensive experiments are conducted on both laboratory environmental face databases and face-in-the-wild databases. All results demonstrate that our approach has certain advantages over state-of-the-art methods in handling pose variations.