Handy: Towards a High Fidelity 3D Hand Shape and Appearance Model

The use of appearance and shape priors in image segmentation is known to improve accuracy; however, existing techniques have several drawbacks. For instance, most active shape and appearance models require landmark points and assume unimodal shape and appearance distributions, and the level set representation does not support construction of local priors. In this paper, we present novel appearance and shape models for image segmentation based on a differentiable implicit parametric shape representation called a disjunctive normal shape model (DNSM). The DNSM is formed by the disjunction of polytopes, which themselves are formed by the conjunctions of half-spaces. The DNSM's parametric nature allows the use of powerful local prior statistics, and its implicit nature removes the need to use landmarks and easily handles topological changes. In a Bayesian inference framework, we model arbitrary shape and appearance distributions using nonparametric density estimations, at any local scale. The proposed local shape prior results in accurate segmentation even when very few training shapes are available, because the method generates a rich set of shape variations by locally combining training samples. We demonstrate the performance of the framework by applying it to both 2-D and 3-D data sets with emphasis on biomedical image segmentation applications.

Automatic prostate segmentation from MR images is an important task in various clinical applications such as prostate cancer staging and MR-guided radiotherapy planning. However, the large appearance and shape variations of the prostate in MR images make the segmentation problem difficult to solve. Traditional Active Shape/Appearance Model (ASM/AAM) has limited accuracy on this problem, since its basic assumption, i.e., both shape and appearance of the targeted organ follow Gaussian distributions, is invalid in prostate MR images. To this end, the authors propose a sparse dictionary learning method to model the image appearance in a nonparametric fashion and further integrate the appearance model into a deformable segmentation framework for prostate MR segmentation. To drive the deformable model for prostate segmentation, the authors propose nonparametric appearance and shape models. The nonparametric appearance model is based on a novel dictionary learning method, namely distributed discriminative dictionary (DDD) learning, which is able to capture fine distinctions in image appearance. To increase the differential power of traditional dictionary-based classification methods, the authors' DDD learning approach takes three strategies. First, two dictionaries for prostate and nonprostate tissues are built, respectively, using the discriminative features obtained from minimum redundancy maximum relevance feature selection. Second, linear discriminant analysis is employed as a linear classifier to boost the optimal separation between prostate and nonprostate tissues, based on the representation residuals from sparse representation. Third, to enhance the robustness of the authors' classification method, multiple local dictionaries are learned for local regions along the prostate boundary (each with small appearance variations), instead of learning one global classifier for the entire prostate. These discriminative dictionaries are located on different patches of the prostate surface and trained to adaptively capture the appearance in different prostate zones, thus achieving better local tissue differentiation. For each local region, multiple classifiers are trained based on the randomly selected samples and finally assembled by a specific fusion method. In addition to this nonparametric appearance model, a prostate shape model is learned from the shape statistics using a novel approach, sparse shape composition, which can model nonGaussian distributions of shape variation and regularize the 3D mesh deformation by constraining it within the observed shape subspace. The proposed method has been evaluated on two datasets consisting of T2-weighted MR prostate images. For the first (internal) dataset, the classification effectiveness of the authors' improved dictionary learning has been validated by comparing it with three other variants of traditional dictionary learning methods. The experimental results show that the authors' method yields a Dice Ratio of 89.1% compared to the manual segmentation, which is more accurate than the three state-of-the-art MR prostate segmentation methods under comparison. For the second dataset, the MICCAI 2012 challenge dataset, the authors' proposed method yields a Dice Ratio of 87.4%, which also achieves better segmentation accuracy than other methods under comparison. A new magnetic resonance image prostate segmentation method is proposed based on the combination of deformable model and dictionary learning methods, which achieves more accurate segmentation performance on prostate T2 MR images.

Automatic and precise segmentation of hand bones is important for many medical imaging applications. Although several previous studies address bone segmentation, automatically segmenting articulated hand bones remains a challenging task. The highly articulated nature of hand bones limits the effectiveness of atlas-based segmentation methods. The use of low-level information derived from the image-of-interest alone is insufficient for detecting bones and distinguishing boundaries of different bones that are in close proximity to each other. In this study, we propose a method that combines an articulated statistical shape model and a local exemplar-based appearance model for automatically segmenting hand bones in CT. Our approach is to perform a hierarchical articulated shape deformation that is driven by a set of local exemplar-based appearance models. Specifically, for each point in the shape model, the local appearance model is described by a set of profiles of low-level image features along the normal of the shape. During segmentation, each point in the shape model is deformed to a new point whose image features are closest to the appearance model. The shape model is also constrained by an articulation model described by a set of pre-determined landmarks on the finger joints. In this way, the deformation is robust to sporadic false bony edges and is able to fit fingers with large articulations. We validated our method on 23 CT scans and we have a segmentation success rate of ~89.70 %. This result indicates that our method is viable for automatic segmentation of articulated hand bones in conventional CT.

Deformable models are common in image modeling and analysis. Random objects provide major challenges as shapes and appearances are hard to quantify; hence, formulation of deformable models are much harder to construct and validate. In this paper, we examine the effect of randomness on building the shape and appearance models for small-size lung nodules (≤ 1cm) which appear in computed tomography (CT) of the human chest. We devise an approach for annotation, which lends a standard mechanism for building traditional active appearance (AAM), active shape (ASM) and active tensor models (ATM). We illustrate the effectiveness of AAM for nodule detection.

This paper addresses issues on moving object tracking from videos. We propose a novel tracking scheme that jointly exploits local object features using consensus point correspondences, and global object appearance and shape models using adaptive particle filter-based eigen-tracking. The paper include the following main novelties: (a) employ consensus feature point correspondences to estimate the motion vector of shape model; (b) employ adaptive particle filters and motioncorrected state vector for joint appearance- and shape-based eigen-tracking. An adaptive number of particles is chosen automatically based on an updated estimation of covariancematrix. Further, online learning is made adaptive to avoid learning using partially-occluded objects. The proposed scheme is realized by integrating SURF and RANSAC [8, 9] for estimating consensus point correspondences, and modify an existing particle filter-based eigen-tracking [4]. Experiment results on tracking moving objects in videos have shown that the proposed scheme provides more accurate tracking, especially for objects with fast motion or long-term partial occlusions. The average number of particles is significantly reduced. Comparisons have been made with an existing method, results have shown that the proposed scheme has provided an improved tracking accuracy at the cost of more computations.

This paper presents an evaluation of the performance of a three-dimensional Active Shape Model (ASM) to segment the liver in 48 clinical CT scans. The employed shape model is built from 32 samples using an optimization approach based on the minimum description length (MDL). Three different gray-value appearance models (plain intensity, gradient and normalized gradient profiles) are created to guide the search. The employed segmentation techniques are ASM search with 10 and 30 modes of variation and a deformable model coupled to a shape model with 10 modes of variation. To assess the segmentation performance, the obtained results are compared to manual segmentations with four different measures (overlap, average distance, RMS distance and ratio of deviations larger 5mm). The only appearance model delivering usable results is the normalized gradient profile. The deformable model search achieves the best results, followed by the ASM search with 30 modes. Overall, statistical shape modeling delivers very promising results for a fully automated segmentation of the liver.

This paper provides methodology for fully automated model-based image segmentation. All information necessary to perform image segmentation is automatically derived from a training set that is presented in a form of segmentation examples. The training set is used to construct two models representing the objects--shape model and border appearance model. A two-step approach to image segmentation is reported. In the first step, an approximate location of the object of interest is determined. In the second step, accurate border segmentation is performed. The shape-variant Hough transform method was developed that provides robust object localization automatically. It finds objects of arbitrary shape, rotation, or scaling and can handle object variability. The border appearance model was developed to automatically design cost functions that can be used in the segmentation criteria of edge-based segmentation methods. Our method was tested in five different segmentation tasks that included 489 objects to be segmented. The final segmentation was compared to manually defined borders with good results [rms errors in pixels: 1.2 (cerebellum), 1.1 (corpus callosum), 1.5 (vertebrae), 1.4 (epicardial), and 1.6 (endocardial) borders]. Two major problems of the state-of-the-art edge-based image segmentation algorithms were addressed: strong dependency on a close-to-target initialization, and necessity for manual redesign of segmentation criteria whenever new segmentation problem is encountered.

This paper presents a new segmentation method for 3D ultrasound images of the pediatric kidney. Based on the popular active shape models, the algorithm is tailored to deal with the particular challenges raised by US images. First, a weighted statistical shape model allows to compensate the image variation with the propagation direction of the US wavefront. Second, an orientation correction approach is used to create a Gabor-based appearance model for each landmark at different scales. This multiscale characteristic is incorporated into the segmentation algorithm, creating a hierarchical approach where different appearance models are considered as the segmentation process evolves. The performance of the algorithm was evaluated on a dataset of 14 cases, both healthy and pathological, obtaining an average Dice's coefficient of 0.85, an average point-to-point distance of 4.07 mm, and 0.12 average relative volume difference.

vertebral fracture is a very common outcome of osteoporosis, which is one of the major public health concerns in the world. Early detection of vertebral fractures is important because timely pharmacologic intervention can reduce the risk of subsequent additional fractures. Our goal seeks to develop a computerized method for detection of vertebral fractures by measuring the shape and appearance of vertebrae on cervical xray radiographs in order to assist radiologist’s image interpretation and thus allow the early diagnosis of osteoporosis. The statistical models of shape and appearance are powerful tools for interpreting medical images. This work introduces the application of correlation filter classifiers for identification and verification of the osteoporosis presence in cervical vertebrae training/ testing set. Correlation filter classifiers have been previously applied to other biometric classification tasks, but not to classification of cervical vertebrae images. We describe how the extraction of an appropriate region of interest in the cervical vertebrae surface can be used to design correlation filters that accomplish 90 % recognition on a database of 50 cervical bone shapes.Keywordsx-ray radiographsSegmentationASM modelingCorrelation filtervertebral deformityOsteoporosis

The anatomical delineation of heart cavities in CT scans is a well-known problem with extended applications in cardiopulmonary diseases. A common approach to provide multi-cavity segmentations is the use of atlas-based statistical shape models. This methodology is of particular interest when just non-contrast and non-gated scans are available since the non-observable internal structures of the heart can be inferred from the fitting of the pericardium provided by the shape model. These inferred cavities have shown predictive power in clinical studies when compared to reference standards. However, although promising, the shape model is limited to the fitting of the pericardium and cannot take advantage of the geometrical inter-relations of internal cavities. This leads to inaccurate segmentations of the smaller structures such as the atria and wrong estimations of the internal cavities. In this work, we study the potential of CNNs to learn the inter-relations of a multicavity active shape model to provide accurate estimations of the internal cavities in non-gated and non-contrast CT scans. We propose an architecture that is able to learn the appearance model of the heart from an inaccurate training dataset. Results demonstrate that our segmentation method improves the correlation of ventricular volume estimations when compared against a semiautomatic active shape approach using cardiac MRI as the reference standard.

We present a computer vision-based approach to estimating the projected frontal surface area (pFSA) of cyclists from unconstrained images. Wind tunnel studies show a reduction in cyclists’ aerodynamic drag through manipulation of the cyclist’s pose. Whilst the mechanism by which reduction is achieved remains unknown, it is widely accepted in the literature that the drag is proportional to the cyclist’s pFSA. This paper describes a repeatable automatic method for pFSA estimation for the study of its relationship with aerodynamic drag in cyclists. The proposed approach is based on finding object boundaries in images. An initialised curve dynamically evolves in the image to minimise an energy function designed to force the curve to gravitate towards image features. To overcome occlusions and pose variation, we use a statistical cyclist shape and appearance models as priors to encourage the evolving curve to arrive at the desired solution. Contour initialisation is achieved using a discriminative object detection method based on offline supervised learning that yields a cyclist classifier. Once an instance of a cyclist is detected in an image and segmented, the pFSA is calculated from the area of the final curve. Applied to two challenging datasets of cyclist images, for cyclist detection our method achieves precision scores of 1.0 and 0.96 and recall scores of 0.68 and 0.83 on the wind tunnel and cyclists-in-natura datasets, respectively. For cyclist segmentation, it achieves 0.88 and 0.92 scores for the mean dice similarity coefficient metric on the two datasets, respectively. We discuss the performance of our method under occlusion, orientation, and pose conditions. Our method successfully estimates pFSA of cyclists and opens new vistas for exploration of the relationship between pFSA and aerodynamic drag.

This paper presents a new deformable modeling strategy aimed at integrating shape and appearance in a unified space. If we think traditional deformable models as active contours or evolving curve fronts, the new deformable shape and appearance models we propose are deforming disks or volumes. Each model has not only boundary shape but also interior appearance. The model shape is implicitly embedded in a higher dimensional space of distance transforms, thus represented by a distance map image. In this way, both shape and appearance of the model are defined in the pixel space. A common deformation scheme, the Free Form Deformations (FFD), parameterizes warping deformations of the volumetric space in which the model is embedded in, hence deforming both model boundary and interior simultaneously.

We present a novel framework for learning a joint shape and appearance model from a large set of un-labelled training examples in arbitrary positions and orientations. The shape and intensity spaces are unified by implicitly representing shapes as “images” in the space of distance transforms. A stochastic chord-based matching algorithm is developed to align photo-realistic training examples under a common reference frame. Then dense local deformation fields, represented using the cubic B-spline based Free Form Deformations (FFD), are recovered to register the training examples in both shape and intensity spaces. Principal Component Analysis (PCA) is applied on the FFD control lattices to capture the variations in shape as well as on registered object interior textures. We show examples where we have built coupled shape and appearance prior models for the left ventricle and whole heart in short-axis cardiac tagged MR images, and used them to delineate the heart chambers in noisy, cluttered images. We also show quantitative validation on the automatic segmentation results by comparing to expert solutions.

Active Appearance Models (AAMs) are widely used to fit shape models to new images. Recently it has been demonstrated that non-linear regression methods and sequences of AAMs can significantly improve performance over the original linear formulation. In this paper we focus on the ability of a model trained on one dataset to generalise to other sets with different conditions. In particular we compare two non-linear, discriminative regression strategies for predicting shape updates, a boosting approach and variants of Random Forest regression. We investigate the use of these regression methods within a sequential model fitting framework, where each stage in the sequence consists of a shape model and a corresponding regression model. The performance of the framework is assessed by both testing on unseen data taken from within the training databases, as well as by investigating the more difficult task of generalising to unrelated datasets. We present results that show that (a) the generalisation performance of the Random Forest is superior to that of the linear or boosted regression procedure and that (b) using a simple feature selection procedure, the Random Forest can be made to be as efficient as the boosting procedure without significant reduction in accuracy.

Model Based Biometric Identification for Improved Security