Accurate Regression Procedures for Active Appearance Models

Active Appearance Models (AAMs) are generative parametric models that have been successfully used in the past to model faces. Anecdotal evidence, however, suggests that the performance of an AAM built to model the variation in appearance of a single person across pose, illumination, and expression (a Person Specific AAM) is substantially better than the performance of an AAM built to model the variation in appearance of many faces, including unseen subjects not in the training set (a Generic AAM). In this paper, we present an empirical evaluation that shows that Person Specific AAMs are, as expected, both easier to build and more robust to fit than Generic AAMs. Moreover, we show that: (1) building a generic shape model is far easier than building a generic appearance model, and (2) the shape component is the main cause of the reduced fitting robustness of Generic AAMs. We then proceed to describe two refinements to Generic AAMs to improve their performance: (1) a refitting procedure to improve the quality of the ground-truth data used to build the AAM and (2) a new fitting algorithm. For both refinements we demonstrate dramatically improved fitting performance. Finally, we evaluate the effect of these improvements on a combined model construction and fitting task.

In this work we automatically segment the left ventricle (LV) in cardiac MR images in the end-diastole (ED) and end-systole (ES) phases using a novel approach that combines statistical and deterministic deformable models. A 3D Active Appearance Model (AAM) is used to segment the ED phase. The AAM texture model is trained on radial samples from gradient magnitude images to make the fitting process faster and more discriminative. A trained ED-to-ES shape correspondence model is used to map a given ED shape to an ES shape. Once the AAM model converges to a shape in ED, the correspondence model is used to get an approximate ES shape. We segment the LV in the ES phase by first fitting a deformable superquadric to the AAM converged shape (in ED) using data range forces and then tracking the LV using image and data range forces (for the ES shape obtained from correspondence model). We test our approach by performing leave-one-out training on a 35 patient datasets. The data comprises 19 normal patients and 16 patients having heart abnormalities (cardiomyopathy and myocardial infarction). The composition makes it a challenging data collection with significant shape variation. The performance of our method is evaluated by measuring the mismatch between automatically segmented and expert delineated contours using the Mean Perpendicular Distance (MPD) and Dice metrics. The average MPD is 2.6mm for ED and 3.7mm for ES (error mostly towards the apex and base). The average Dice is 0.9 for ED and 0.8 for ES. These results show good potential for clinical use.

The use of appearance and shape priors in image segmentation is known to improve accuracy; however, existing techniques have several drawbacks. For instance, most active shape and appearance models require landmark points and assume unimodal shape and appearance distributions, and the level set representation does not support construction of local priors. In this paper, we present novel appearance and shape models for image segmentation based on a differentiable implicit parametric shape representation called a disjunctive normal shape model (DNSM). The DNSM is formed by the disjunction of polytopes, which themselves are formed by the conjunctions of half-spaces. The DNSM's parametric nature allows the use of powerful local prior statistics, and its implicit nature removes the need to use landmarks and easily handles topological changes. In a Bayesian inference framework, we model arbitrary shape and appearance distributions using nonparametric density estimations, at any local scale. The proposed local shape prior results in accurate segmentation even when very few training shapes are available, because the method generates a rich set of shape variations by locally combining training samples. We demonstrate the performance of the framework by applying it to both 2-D and 3-D data sets with emphasis on biomedical image segmentation applications.

Pose robust face tracking by combining view-based AAMs and temporal filters

Background. The prostate gland is delineated as the clinical target volume (CTV) in treatment planning of prostate cancer. Therefore, an accurate delineation is a prerequisite for efficient treatment. Accurate automated prostate segmentation methods facilitate the delineation of the CTV without inter-observer variation. The purpose of this study is to present an automated three-dimensional (3D) segmentation of the prostate using an active appearance model. Material and methods. Axial T2-weighted magnetic resonance (MR) scans were used to build the active appearance model. The model was based on a principal component analysis of shape and texture features with a level-set representation of the prostate shape instead of the selection of landmarks in the traditional active appearance model. To achieve a better fit of the model to the target image, prior knowledge to predict how to correct the model and pose parameters was incorporated. The segmentation was performed as an iterative algorithm to minimize the squared difference between the target and the model image. Results. The model was trained using manual delineations from 30 patients and was validated using leave-one-out cross validation where the automated segmentations were compared with the manual reference delineations. The mean and median dice similarity coefficient was 0.84 and 0.86, respectively. Conclusion. This study demonstrated the feasibility for an automated prostate segmentation using an active appearance with results comparable to other studies.

Face registration is a challenging problem due in part to the non-rigid nature of human faces. Active Appearance Models (AAMs) have been proposed as a useful technique for face registration in part because they can account for changes in shape. Fitting of AAMs to imagery is typically done using the Gauss-Newton method; however this approach is known to fail when either the initial shape estimate is to far off and/or the appearance model fails to direct search toward a good match. In this paper, we employ Evolution Strategies (ES) to search for a near optimal fit, i.e. set of model parameters, that relate an AAM to a novel face image. In addition, we dramatically reduce the dimensionality of the search problem by analytically determining the optimal texture parameters associated with any given shape. Experimental results reveal that the proposed approach outperforms the standard AAM parameter estimation technique and some of its variants. Further, the difference is statistically significant. Finally, some basic limitations of AAMs are identified using fitness landscape analysis.

Regression-based Active Appearance Model initialization for facial feature tracking with missing frames

Video-based face model fitting using Adaptive Active Appearance Model

Active Appearance Models (AAMs) and Constrained Local Models (CLMs) are classical approaches for model-based image segmentation in medical image analysis. AAMs consist of global statistical models of shape and appearance, are known to be hard to fit, and often suffer from insufficient generalization capabilities in case of limited training data. CLMs model appearance only for local patches and relax or completely remove the global appearance constraint. They are, therefore, much easier to optimize but in certain cases they lack the robustness of AAMs. In this paper, we present a framework for patch-based active appearance modeling, which elegantly combines strengths of AAMs and CLMs. Our models provide global shape and appearance constraints and we make use of recent methodological advances from computer vision for efficient joint optimization of shape and appearance parameters during model fitting. Furthermore, the insufficient generalization abilities of those global models are tackled by incorporating and extending a recent approach for learning representative statistical shape models from small training populations. We evaluate our approach on publicly available chest radiographs and cardiac MRI data. The results show that the proposed framework leads to competitive results in terms of segmentation accuracy for challenging multi-object segmentation problems even when only few training samples are available.

A background robust active appearance model using active contour technique

Visual tracking is one of the fundamental problems in computer vision. The pose of an object extracted through consecutive frames has a variety of applications ranging from robot navigation to camera based man-machine interfaces. In this paper we examine the use of Active Appearance Models(AAM) for the task of visual tracking. The original Active Appearance Model is limited to have all points of the model visible in all frames. We introduce a notion of visibility uncertainty for the points in the AAM, removing the above limitation and therefore allowing the object to contain self-occlusions. The visibility uncertainty is easily integrated into the existing AAM framework, keeping model initialization time to a minimum.We have experiments illustrating that the extension allow AAMs to track through self occlusions at near real-time.

We use the popular active appearance models (AAM) for extracting discriminative features from images of biological objects. The relevant discriminative features are combined principal component (PCA) vectors from the AAM and texture features from cooccurrence matrices. Texture features are extracted by extending the AAM's with a textural warp guided by the AAM shape. Based on this, texture cooccurrence features are calculated. We use the different features for classifying the biological objects to species using standard classifiers, and we show that even though the objects are highly variant, the AAM's are well suited for extracting relevant features, thus obtaining good classification results. Classification is conducted on two real data sets, one containing various vegetables and one containing different species of wood logs.

Although the conventional Active Appearance Model (AAM) has achieved some success for face alignment, it still suffers from the generalization problem when be applied to unseen subjects and images. In this paper, a novel Locality-constraint AAM (LC-AAM) algorithm is proposed to tackle the generalization problem of AAM. Theoretically, the proposed LC-AAM is a fast approximation for a sparsity-regularized AAM problem, where sparse representation is exploited for non-linear face modeling. Specifically, for an input image, its K-nearest neighbors are selected as the shape and appearance bases, which are adaptively fitted to the input image by solving a constrained AAM-like fitting problem. Essentially, the effectiveness of our LC-AAM algorithm comes from learning a strong localized shape and appearance prior for the input facial image through exploiting its K-similar patterns. To validate the effectiveness of our algorithm, comprehensive experiments are conducted on two publicly available face databases. Experimental results demonstrate that our method greatly outperforms the original AAM method and its variants. In addition, our method is better than the state-of-the-art face alignment methods and generalizes well to unseen subjects and images.

Segmentation of diseased lungs in CT images is a nontrivial problem. As Active Appearance Model (AAM) has been applied effectively in this field, we propose a new approach for the construction of traditional AAM to segment the lung fields more accurately and efficiently: Matrixes based AAM (MatAAM). MatAAM is based on two-dimensional image matrixes rather one-dimensional vectors. Its appearance matrix does not need to be transformed into a vector prior to computing the appearance parameter. Instead, a covariance matrix is constructed directly using the normalized appearance matrixes and its eigenvectors are derived for the appearance parameter. The experiment results were compared to other landmark-based methods: Snake, Active Shape Model (ASM), AAM and several modified versions of them. For segmentation of lungs especially diseased lungs, MatAAM performed a superior result in both precision and efficiency.

Lip reading is a technique to recognize the spoken words base on lip movement. In this process, it is important to detect the correct features of the facial images. However, detection is not easy in the real situations because the facial images may be taken from various angles. To cope with this problem, lip reading from multi view facial images has been conducted in several research institutes. In this paper, we propose a lip reading approach using the 3D Active Appearance Models (AAM) features and the Hidden Markov Model (HMM)-based recognition model. The AAM is a parametric model constructed from both shape and appearance parameters. The parameters are compressed into the combination parameters in the AAM, and are used in lip reading or some other facial image processing applications. The 3D-AAM extends the traditional 2D shape model to 3D shape model built from three different view angles (frontal, left, and right profile). It provides an effective algorithm to align the model with the RGB and the 3D range images obtained by the RGBD-camera. The benefit of using 3D-AAM in lip reading is that it enables to recognize the spoken words from any angle of the facial images. In the experiment, we compared the accuracy of lip reading using 3D-AAM with that of the traditional 2D-AAM on various angles of facial images. Based on the result, we confirmed that 3D-AAM is effective in cross view lip reading despite using only the frontal images in the HMM training phase.