Feature extraction for classification problems and its application to face recognition

This research features a new method for automatic face recognition robust to variations in lighting, facial expression and eyewear. The new algorithm named SKKUfaces (Sungkyunkwan University faces) employs PCA (Principal Component Analysis) and FLD (Fisher's Linear Discriminant) in series similarly to Fisherfaces. The fundamental difference is that SKKUfaces effectively eliminates, in the reduced PCA subspace, portions of the subspace that are responsible for variations in lighting and facial expression and then applies FLD to the resulting subspace. This results in superb discriminating power for pattern classification and excellent recognition accuracy. We also propose an efficient method to compute the between-class scatter and within-class scatter matrices for the FLD analysis. We have evaluated the performance of SKKUfaces using YALE and SKKU facial databases. Experimental results show that the SKKUface method is computationally efficient and achieves much better recognition accuracy than the Fisherface method [1] especially for facial images with variations in lighting and eyewear.

With the improvement of security requirements of individuals, businesses and countries, the identity authentication system based on face recognition technology is applied widely. The identity methods based on single theory, due to a variety of factors, have their own inherent limitations. In order to improve the efficiency and accuracy of face recognition, based on fusion of wavelet packet sub-images, a face recognition algorithm named FW-FLD is proposed in this paper by combining PCA (Principal Component Analysis) and FLD(Fisher Linear Discriminant). Firstly, the training samples are decomposed using wavelet packet. The fusion weights of the decomposed sub-images are calculated according to their energy distribution and the weighted fusion images are obtained, which reserve the images characteristic in the frequency. Then the features of the fused images are extracted using PCA, and the optimized Fisher space is constructed using FLD. Finally, face images are classified by measuring the projection coefficients of optimized training samples and testing samples in Fisher space. Experimental results on CMU PIE, JAFFE and AR face databases show that the proposed algorithm is robust, and can adapt to face recognition with various illumination, facial expressions and gestures. Compared with other algorithms, it not only improves the face recognition rate, but also has a higher time efficiency.

Face recognition is one of the most fundamental functions for many kinds of intelligent robots. Among many methods proposed for face recognition, linear approaches such as Eigenface or principal component analysis (PCA), Fisher’s linear discriminant (FLD), Fisherface, and common vector (CV), have attracted great attention because of their simplicity. It is known that when the number of training examples is large enough, we should use FLD; otherwise, we should use CV. This paper proposes a rough common vector (RCV) approach. The basic idea of RCV is to divide the feature space into two subspaces. One is spanned by the eigenvectors corresponding to the largest eigenvalues of the within-class scatter matrix, and another is spanned by the eigenvectors corresponding to the smallest eigenvalues. The later plays the role of the null space of the within-class scatter matrix, and is important for extracting useful discriminative features for recognition. RCV can be used regardless the within-class scatter matrix is singular or not. Experimental results on four databases show that RCV outperforms nearest neighbor classifier, Eigenface, Fisherface, and CV in most cases.

The canonical correlation analysis (CCA) aims at measuring linear relationships between two sets of variables (views) that can be used for feature extraction in classification problems with multiview data. However, the correlated features extracted by the CCA may not be class discriminative, since CCA does not utilize the class labels in its traditional formulation. Although there is a method called discriminative CCA (DCCA) that aims to increase the discriminative ability of CCA inspired from the linear discriminant analysis (LDA), it has been shown that the extracted features with this method are identical to those by the LDA with respect to an orthogonal transformation. Therefore, DCCA is simply equivalent to applying single-view (regular) LDA to each one of the views separately. Besides, DCCA and the other similar DCCA approaches have generalization problems due to the sample covariance matrices used in their computation, which are sensitive to outliers and noisy samples. In this paper, we propose a method, called discriminative alternating regression (D-AR), to explore correlated and also discriminative features. D-AR utilizes two (alternating) multilayer perceptrons, each with a linear hidden layer, learning to predict both the class labels and the outputs of each other. We show that the features found by D-AR on training sets significantly accomplish higher classification accuracies on test sets of facial expression recognition, object recognition, and image retrieval experimental data sets.

The literature on independent component analysis (ICA)-based face recognition generally evaluates its performance using standard principal component analysis (PCA) within two architectures, ICA Architecture I and ICA Architecture II. In this correspondence, we analyze these two ICA architectures and find that ICA Architecture I involves a vertically centered PCA process (PCA I), while ICA Architecture II involves a whitened horizontally centered PCA process (PCA II). Thus, it makes sense to use these two PCA versions as baselines to reevaluate the performance of ICA-based face-recognition systems. Experiments on the FERET, AR, and AT&T face-image databases showed no significant differences between ICA Architecture I (II) and PCA I (II), although ICA Architecture I (or II) may, in some cases, significantly outperform standard PCA. It can be concluded that the performance of ICA strongly depends on the PCA process that it involves. Pure ICA projection has only a trivial effect on performance in face recognition.

Summary form only given. In the paper, the comparison of discriminative capabilities of conventional dimensionality reduction methods and the integration of a dimensionality reduction module and recognizer design with minimum classification error rate are discussed. Conventionally, principal component analysis (PCA) and Fisher's linear discriminant (FLD) are two most popular and widely used dimensionality reduction methods for pattern recognition. However, the objectives of these two methods are quite different. PCA basically tries to faithfully keep the original data representation but FLD tries to separate data distribution of different classes. It therefore seems that FLD can provide more discriminative characteristics to pattern recognition than PCA. However, a completely optimal feature extractor can never be anything but an optimal recognizer. It is only when constraints are placed on the classifier that one can formulate nontrivial problems. We apply a minimum error formulation (MEF) to integrate the design of dimensionality reduction module and pattern recognizer. The experimental results show that such an integration provides very good recognition performance on a data set of hand-written Chinese characters even when the feature number has been significantly reduced.

Real-world face recognition systems often have to face the single sample per person (SSPP) problem, that is, only a single training sample for each person is enrolled in the database. In this case, many of the popular face recognition methods fail to work well due to the inability to learn the discriminatory information specific to the persons to be identified. To address this problem, in this paper, we propose an Adaptive Generic Learning (AGL) method, which adapts a generic discriminant model to better distinguish the persons with single face sample. As a specific implementation of the AGL, a Coupled Linear Representation (CLR) algorithm is proposed to infer, based on the generic training set, the within-class scatter matrix and the class mean of each person given its single enrolled sample. Thus, the traditional Fisher's Linear Discriminant (FLD) can be applied to SSPP task. Experiments on the FERET and a challenging passport face database show that the proposed method can achieve better results compared with other common solutions to the SSPP problem.

Not alike to principal component analysis (PCA) based monitoring statistics (T2 and SPE), the control limits for independent component analysis (ICA) monitoring statistics (I2 , Ie 2 and SPE) cannot be determined directly from a particular approximation distribution due to latent variables do not follow Gaussian distribution. Lee et al. (2004) proposed to use kernel density estimation (KDE) to obtain the control limits. However, the KDE method is very sensitive to the choice of smoothing parameter. Therefore, this study utilizes the support vector machine (SVM) for process fault detection by taking information of ICA extracted statistics as inputs of SVM. The proposed method (named, ICA-SVM) will be implemented in the Tennessee Eastman Process. In which, several multivariate monitoring schemes such as PCA, ICA, modified ICA and ICA- PCA will be also compared to demonstrate the efficiency of proposed method.

Recently single trial classification of Event Related Potentials (ERPs) has received much attention to improve Brain Computer Interface (BCI) systems. BCI is one of the most recent fields in computer science which tries to help handicapped people; however, single trial EEG analysis is hard due to its low Signal to Noise Ratio (SNR). Many BCI systems are based on the analysis of ERPs such as P300. P300 is a positive peak that occurs approximately 300ms after a visual stimulus. But electrical potentials produced by blinks and eye movements cause serious problems in analyzing of EEG data. In this paper, a two stage algorithm is presented to detect P300 waves in single trial conditions in the presence of noise and artifact. In the first stage, the raw EEG data are denoised. For this reason, an ICA-wavelet based denoising method is proposed to automatically remove ocular artifact and it is also compared to other denoising methods. In the second stage, a detection algorithm is applied on denoised EEG data in order to discover P300 waves. In this method, a set of new features are extracted by applying ICA on each unknown incoming signal and finally they are classified using a neural network. The proposed method has been tested on more than 10 runs and an average accuracy of 71.5% in these runs is achieved in detecting P300 waves.

This paper address new face verification scheme based on Log-Gabor filter (texture based) and Gaussian Mixture Model. The proposed method consists of three parts. The first part is a Log-Gabor filtering on facial image. The second part is to model the Log-Gabor filter response using Gaussian Mixture Model to obtain more than one set of features. The third part is transforming the set of features using subspace methods such as Principal Component Analysis (PCA) and Independent Component Analysis (ICA). Thus, in this paper two methods namely Log-Gabor Mixture Model (LGMM) based on PCA and Log-Gabor Mixture Model based (LGMM) on ICA is proposed. Proposed methods are evaluated for its performance by conducting series of experiments on three image databases: ORL, AR, YALE face database. The nature of type \& size of databases chosen on the performance of these algorithms are also studied. The experimental results indicate the efficacy of the proposed methods and varied nature of results based on these algorithms.

We introduce two enhanced Fisher linear discriminant (FLD) models (EFM) in order to improve the generalization ability of the standard FLD based classifiers such as Fisherfaces. Similar to Fisherfaces, both EFM models apply first principal component analysis (PCA) for dimensionality reduction before proceeding with FLD type of analysis. EFM-1 implements the dimensionality reduction with the goal to balance between the need that the selected eigenvalues account for most of the spectral energy of the raw data and the requirement that the eigenvalues of the within-class scatter matrix in the reduced PCA subspace are not too small. EFM-2 implements the dimensionality reduction as Fisherfaces do. It proceeds with the whitening of the within-class scatter matrix in the reduced PCA subspace and then chooses a small set of features (corresponding to the eigenvectors of the within-class scatter matrix) so that the smaller trailing eigenvalues are not included in further computation of the between-class scatter matrix. Experimental data using a large set of faces-1,107 images drawn from 369 subjects and including duplicates acquired at a later time under different illumination-from the FERET database shows that the EFM models outperform the standard FLD based methods.

As an effective technique for feature extraction and pattern classification Fisher linear discriminant (FLD) has been successfully applied in many fields. However, for a task with very high-dimensional data such as face images, conventional FLD technique encounters a fundamental difficulty caused by singular within-class scatter matrix. To avoid the trouble, many improvements on the feature extraction aspect of FLD have been proposed. In contrast, studies on the pattern classification aspect of FLD are quiet few. In this paper, we will focus our attention on the possible improvement on the pattern classification aspect of FLD by presenting a novel linear discriminant criterion called maximum scatter difference (MSD). Theoretical analysis demonstrates that MSD criterion is a generalization of Fisher discriminant criterion, and is the asymptotic form of discriminant criterion: large margin linear projection. The performance of MSD classifier is tested in face recognition. Experiments performed on the ORL, Yale, FERET and AR databases show that MSD classifier can compete with top-performance linear classifiers such as linear support vector machines, and is better than or equivalent to combinations of well known facial feature extraction methods, such as eigenfaces, Fisherfaces, orthogonal complementary space, nullspace, direct linear discriminant analysis, and the nearest neighbor classifier.

A facial recognition system is a system for automatically recognizing a person from a digital image as the human eye recognizes. Here two algorithms principal component analysis (PCA) and Fisher discriminant analysis (FDA) of holistic approach of information theory have been analyzed. Recognition process comprises the two steps: training and testing. In the training phase a set of the eigenvectors of the covariance matrix of the images used for training. These eigenvectors are also called as eigenfaces. In testing phase when a new input image is given for recognition, this image will be projected into the eigenspace by using the already calculated eigenvectors. Test image will be compared with all the images in the eigenspace and measures the euclidean distance. The image with the lowest Euclidean distance is the matched image if the distance lies below some threshold value. Both algorithms works in the same manner, the difference lies in the calculation of face space. These two algorithms are evaluated experimentally on two databases each with the moderate subject size. Analysis and experimental results indicates that the PCA works well when the lightening variation is small. FDA works gives better accuracy in facial expression

One of the most successful process to accomplish human face recognition are the methods based on the principal component analysis (PCA), also known as eigenfaces. Recently, novel PCA approaches have been proposed: modular (MPCA) and two-dimensional (IMPCA). These approaches have achieved outstanding result in feature extraction and recognition. IMPCA is used for feature extraction based on 2D matrix representation and MPCA is based on image division to improve face recognition with variations like facial expressions, light and head pose. In this work we use some aspects of these methods to build a new technique called modular Image PCA (MIMPCA). The results achieved with the proposed method are superior in all experiments compared with the original techniques under different conditions of head pose angle, illumination and facial expression.

In this paper, a face recognition approach based on the hybrid of the approximate neural network with sparse RAM (SN-tuple) and Fisher's linear discriminant (FLD) is presented. Firstly, the data of original face images are imported into the SN-tuple to classify these faces roughly. Then FLD, a classical statistical pattern recognition method, are used to classify these faces precisely. Experimental results demonstrate that this proposed approach achieves better performance than the SN-tuple and FLD for face recognition respectively.