Robust active appearance models and their application to medical image analysis

A background robust active appearance model using active contour technique

The Active Appearance Model (AAM) is a widely used approach for model based vision showing excellent results. But one major drawback is that the method is not robust against occlusions. Thus, if parts of the image are occluded the method converges to local minima and the obtained results are unreliable. To overcome this problem we propose a robust AAM fitting strategy. The main idea is to apply a robust PCA model to reconstruct the missing feature information and to use the thus obtained image as input for the standard AAM fitting process. Since existing methods for robust PCA reconstruction are computationally too expensive for real-time processing we applied a more efficient method: Fast-Robust PCA (FR-PCA). In fact, by using our FR-PCA the computational effort is drastically reduced. Moreover, more accurate reconstructions are obtained. In the experiments, we evaluated both, the FR-PCA model on the publicly available ALOI database and the whole robust AAM fitting chain on facial images. The results clearly show the benefits of our approach in terms of accuracy and speed when processing disturbed data (i.e., images containing occlusions).

Active Appearance Models (AAMs) and Constrained Local Models (CLMs) are classical approaches for model-based image segmentation in medical image analysis. AAMs consist of global statistical models of shape and appearance, are known to be hard to fit, and often suffer from insufficient generalization capabilities in case of limited training data. CLMs model appearance only for local patches and relax or completely remove the global appearance constraint. They are, therefore, much easier to optimize but in certain cases they lack the robustness of AAMs. In this paper, we present a framework for patch-based active appearance modeling, which elegantly combines strengths of AAMs and CLMs. Our models provide global shape and appearance constraints and we make use of recent methodological advances from computer vision for efficient joint optimization of shape and appearance parameters during model fitting. Furthermore, the insufficient generalization abilities of those global models are tackled by incorporating and extending a recent approach for learning representative statistical shape models from small training populations. We evaluate our approach on publicly available chest radiographs and cardiac MRI data. The results show that the proposed framework leads to competitive results in terms of segmentation accuracy for challenging multi-object segmentation problems even when only few training samples are available.

In this work we automatically segment the left ventricle (LV) in cardiac MR images in the end-diastole (ED) and end-systole (ES) phases using a novel approach that combines statistical and deterministic deformable models. A 3D Active Appearance Model (AAM) is used to segment the ED phase. The AAM texture model is trained on radial samples from gradient magnitude images to make the fitting process faster and more discriminative. A trained ED-to-ES shape correspondence model is used to map a given ED shape to an ES shape. Once the AAM model converges to a shape in ED, the correspondence model is used to get an approximate ES shape. We segment the LV in the ES phase by first fitting a deformable superquadric to the AAM converged shape (in ED) using data range forces and then tracking the LV using image and data range forces (for the ES shape obtained from correspondence model). We test our approach by performing leave-one-out training on a 35 patient datasets. The data comprises 19 normal patients and 16 patients having heart abnormalities (cardiomyopathy and myocardial infarction). The composition makes it a challenging data collection with significant shape variation. The performance of our method is evaluated by measuring the mismatch between automatically segmented and expert delineated contours using the Mean Perpendicular Distance (MPD) and Dice metrics. The average MPD is 2.6mm for ED and 3.7mm for ES (error mostly towards the apex and base). The average Dice is 0.9 for ED and 0.8 for ES. These results show good potential for clinical use.

Active Appearance Models (AAMs) are generative parametric models that have been successfully used in the past to model faces. Anecdotal evidence, however, suggests that the performance of an AAM built to model the variation in appearance of a single person across pose, illumination, and expression (a Person Specific AAM) is substantially better than the performance of an AAM built to model the variation in appearance of many faces, including unseen subjects not in the training set (a Generic AAM). In this paper, we present an empirical evaluation that shows that Person Specific AAMs are, as expected, both easier to build and more robust to fit than Generic AAMs. Moreover, we show that: (1) building a generic shape model is far easier than building a generic appearance model, and (2) the shape component is the main cause of the reduced fitting robustness of Generic AAMs. We then proceed to describe two refinements to Generic AAMs to improve their performance: (1) a refitting procedure to improve the quality of the ground-truth data used to build the AAM and (2) a new fitting algorithm. For both refinements we demonstrate dramatically improved fitting performance. Finally, we evaluate the effect of these improvements on a combined model construction and fitting task.

Background. The prostate gland is delineated as the clinical target volume (CTV) in treatment planning of prostate cancer. Therefore, an accurate delineation is a prerequisite for efficient treatment. Accurate automated prostate segmentation methods facilitate the delineation of the CTV without inter-observer variation. The purpose of this study is to present an automated three-dimensional (3D) segmentation of the prostate using an active appearance model. Material and methods. Axial T2-weighted magnetic resonance (MR) scans were used to build the active appearance model. The model was based on a principal component analysis of shape and texture features with a level-set representation of the prostate shape instead of the selection of landmarks in the traditional active appearance model. To achieve a better fit of the model to the target image, prior knowledge to predict how to correct the model and pose parameters was incorporated. The segmentation was performed as an iterative algorithm to minimize the squared difference between the target and the model image. Results. The model was trained using manual delineations from 30 patients and was validated using leave-one-out cross validation where the automated segmentations were compared with the manual reference delineations. The mean and median dice similarity coefficient was 0.84 and 0.86, respectively. Conclusion. This study demonstrated the feasibility for an automated prostate segmentation using an active appearance with results comparable to other studies.

Face registration is a challenging problem due in part to the non-rigid nature of human faces. Active Appearance Models (AAMs) have been proposed as a useful technique for face registration in part because they can account for changes in shape. Fitting of AAMs to imagery is typically done using the Gauss-Newton method; however this approach is known to fail when either the initial shape estimate is to far off and/or the appearance model fails to direct search toward a good match. In this paper, we employ Evolution Strategies (ES) to search for a near optimal fit, i.e. set of model parameters, that relate an AAM to a novel face image. In addition, we dramatically reduce the dimensionality of the search problem by analytically determining the optimal texture parameters associated with any given shape. Experimental results reveal that the proposed approach outperforms the standard AAM parameter estimation technique and some of its variants. Further, the difference is statistically significant. Finally, some basic limitations of AAMs are identified using fitness landscape analysis.

Regression-based Active Appearance Model initialization for facial feature tracking with missing frames

Video-based face model fitting using Adaptive Active Appearance Model

PurposeRepresentation of anatomy appearance is one of the key problems in medical image analysis. An appearance model represents the anatomies with parametric forms, which are then vectorised for prior learning, segmentation and classification tasks.MethodsWe propose a part-based parametric appearance model we refer to as a deformable appearance pyramid (DAP). The parts are delineated by multi-scale local feature pyramids extracted from an image pyramid. Each anatomy is represented by an appearance pyramid, with the variability within a population approximated by local translations of the multi-scale parts and linear appearance variations in the assembly of the parts. We introduce DAPs built on two types of image pyramids, namely Gaussian and wavelet pyramids, and present two approaches to model the prior and fit the model, one explicitly using a subspace Lucas–Kanade algorithm and the other implicitly using the supervised descent method (SDM).ResultsWe validate the performance of the DAP instances with difference configurations on the problem of lumbar spinal stenosis for localising the landmarks and classifying the pathologies. We also compare them with classic methods such as active shape models, active appearance models and constrained local models. Experimental results show that the DAP built on wavelet pyramids and fitted with SDM gives the best results in both landmark localisation and classification.ConclusionA new appearance model is introduced with several configurations presented and evaluated. The DAPs can be readily applied for other clinical problems for the tasks of prior learning, landmark detection and pathology classification.

Visual tracking is one of the fundamental problems in computer vision. The pose of an object extracted through consecutive frames has a variety of applications ranging from robot navigation to camera based man-machine interfaces. In this paper we examine the use of Active Appearance Models(AAM) for the task of visual tracking. The original Active Appearance Model is limited to have all points of the model visible in all frames. We introduce a notion of visibility uncertainty for the points in the AAM, removing the above limitation and therefore allowing the object to contain self-occlusions. The visibility uncertainty is easily integrated into the existing AAM framework, keeping model initialization time to a minimum.We have experiments illustrating that the extension allow AAMs to track through self occlusions at near real-time.

We use the popular active appearance models (AAM) for extracting discriminative features from images of biological objects. The relevant discriminative features are combined principal component (PCA) vectors from the AAM and texture features from cooccurrence matrices. Texture features are extracted by extending the AAM's with a textural warp guided by the AAM shape. Based on this, texture cooccurrence features are calculated. We use the different features for classifying the biological objects to species using standard classifiers, and we show that even though the objects are highly variant, the AAM's are well suited for extracting relevant features, thus obtaining good classification results. Classification is conducted on two real data sets, one containing various vegetables and one containing different species of wood logs.

Although the conventional Active Appearance Model (AAM) has achieved some success for face alignment, it still suffers from the generalization problem when be applied to unseen subjects and images. In this paper, a novel Locality-constraint AAM (LC-AAM) algorithm is proposed to tackle the generalization problem of AAM. Theoretically, the proposed LC-AAM is a fast approximation for a sparsity-regularized AAM problem, where sparse representation is exploited for non-linear face modeling. Specifically, for an input image, its K-nearest neighbors are selected as the shape and appearance bases, which are adaptively fitted to the input image by solving a constrained AAM-like fitting problem. Essentially, the effectiveness of our LC-AAM algorithm comes from learning a strong localized shape and appearance prior for the input facial image through exploiting its K-similar patterns. To validate the effectiveness of our algorithm, comprehensive experiments are conducted on two publicly available face databases. Experimental results demonstrate that our method greatly outperforms the original AAM method and its variants. In addition, our method is better than the state-of-the-art face alignment methods and generalizes well to unseen subjects and images.

Pose robust face tracking by combining view-based AAMs and temporal filters

In this paper, a simulation of the human face evolution from youth to old age is presented. In this work, the goal is to maintain a person's appearance and accurately estimate the face at the target age; in such a way that the simulated face should be realistic and have the appearance and texture characteristics of the target age. To achieve this goal, the number of feature points and their arrangement on the face are very important. First, we suggest proper feature points. Then, using the active appearance model (AAM) method and the presented model, templates are obtained as representative of different age groups. To maintain the unique geometric characteristics of the input face and apply these characteristics to the target age template, the proposed feature point pattern and the MLS method have been used. Then, using the active appearance model method and having the target age template contain the geometric characteristics of the input face, and the steps of changing the input face age to reach the target age are presented. Finally, the results of the survey, using two methods of age recognition and real image recognition (separately for man and woman images), show an average of 80.77% (male images) and 81.36% (female images) correct answers of the participants in this Poll.