Fuzzy Maximum Scatter Discriminant Analysis and its Application to Face Recognition

This paper presents a new method for image feature extraction, namely, the fuzzy 2D discriminant locality preserving projections (F2DDLPP) based on the 2D discriminant locality preserving projections (2DDLPP) and fuzzy set theory. Firstly, we calculate the membership degree matrix by fuzzy k-nearest neighbor (FKNN), then we incorporate the membership degree matrix into the definition of the intra-class scatter matrix and inter-class scatter matrix, respectively. Secondly, we can get the fuzzy intra-class scatter matrix and fuzzy inter-class scatter matrix, respectively. The FKNN is implemented to achieve the distribution information of original samples, and this information is utilized to redefine corresponding scatter matrices. So, F2DDLPP can extract discriminative features from overlapping (outlier) samples which is different to the conventional 2DDLPP. Finally, Experiments on the Yale, ORL face databases, USPS database and PolyU palmprint database are demonstrated to verify the effectiveness of the proposed algorithm.

A new face recognition method is proposed in this paper. The proposed method is based on fuzzy regularized linear discriminant analysis (FR-LDA) and combines the regularized linear discriminant analysis (R-LDA) and the fuzzy set theory. R-LDA is based on a new regularized Fisher's discriminant criterion, which is particularly robust against the small sample size problem compared to the traditional one used in LDA. In the proposed method, we calculate the membership degree matrix by Fuzzy K-nearest neighbor (FKNN) and then incorporate the membership degree into the definition of the between-class and within-class scatter matrices and get the fuzzy between-class and within-class scatter matrices. Experimental results obtained on the FERET database demonstrate that the proposed method improves the classification rate performance.

Fuzzy MSD based feature extraction method for face recognition

Fault detection based on robust characteristic dimensionality reduction

The goal of this paper is to present a new fuzzy logic-based feature extraction method, called Fuzzy Generalized Two-Dimensional Fisher's Linear Discriminant (FG-2DFLD) method for face recognition. More specifically, the FG-2DFLD method is an extension of the G-2DFLD method. In FG-2DFLD method, we introduced fuzzy membership values to all the training samples and incorporated them into the within-class and between-class scatter matrices along the row and column directions. The class-wise mean images are also generated by considering the membership values. The fuzzy k-nearest neighbour (FKNN) algorithm is applied for generating the corresponding degrees of class membership. Finally, the feature matrix generated by solving the eigenvalue problem of these scatter matrices actually provide better discrimination information as supported by the simulation results on the AT&T (formally known as ORL) and UMIST face databases using a radial basis function (RBF) neural network.

In this paper, a novel image discriminant analysis method, coined two-dimensional discriminant transform based on scatter difference criterion (2DSDD), is developed for image representation. The proposed 2DSDD scheme adopts the difference of both between-class scatter and within-class scatter as discriminant criterion. In this way, the small sample size problem usually occurred in the traditional Fisher discriminant analysis (LDA) is essentially avoided. In addition, the developed method directly depends on image matrices. That is to say, it is not necessary to convert the image matrix into high-dimensional image vector like those conventional linear discriminant methods prior to feature extraction so that much computational time would be saved. Finally, the experimental results on the ORL face database indicate that the proposed method outperforms Fisherfaces, the standard scatter difference discriminant analysis, not only in the computation efficiency, but also in its recognition performance.

Tensor Rank One Discriminant Analysis—A convergent method for discriminative multilinear subspace selection

For multivariate data, tandem clustering is a well-known technique aiming to improve cluster identification through initial dimension reduction. Nevertheless, the usual approach using principal component analysis (PCA) has been criticized for focusing solely on inertia so that the first components do not necessarily retain the structure of interest for clustering. To address this limitation, a new tandem clustering approach based on invariant coordinate selection (ICS) is proposed. By jointly diagonalizing two scatter matrices, ICS is designed to find structure in the data while providing affine invariant components. Certain theoretical results have been previously derived and guarantee that under some elliptical mixture models, the group structure can be highlighted on a subset of the first and/or last components. However, ICS has garnered minimal attention within the context of clustering. Two challenges associated with ICS include choosing the pair of scatter matrices and selecting the components to retain. For effective clustering purposes, it is demonstrated that the best scatter pairs consist of one scatter matrix capturing the within-cluster structure and another capturing the global structure. For the former, local shape or pairwise scatters are of great interest, as is the minimum covariance determinant (MCD) estimator based on a carefully chosen subset size that is smaller than usual. The performance of ICS as a dimension reduction method is evaluated in terms of preserving the cluster structure in the data. In an extensive simulation study and empirical applications with benchmark data sets, various combinations of scatter matrices as well as component selection criteria are compared in situations with and without outliers. Overall, the new approach of tandem clustering with ICS shows promising results and clearly outperforms the PCA-based approach.

Feature extraction based on fuzzy 2DLDA

Local graph embedding based on maximum margin criterion via fuzzy set

A novel discriminant analysis criterion is derived in this paper under the theoretical framework of Bayes optimality. In contrast to the conventional Fisher's discriminant criterion, the major novelty of the proposed one is the use of L1 norm rather than L2 norm, which makes it less sensitive to the outliers. With the L1-norm discriminant criterion, we propose a new linear discriminant analysis (L1-LDA) method for linear feature extraction problem. To solve the L1-LDA optimization problem, we propose an efficient iterative algorithm, in which a novel surrogate convex function is introduced such that the optimization problem in each iteration is to simply solve a convex programming problem and a close-form solution is guaranteed to this problem. Moreover, we also generalize the L1-LDA method to deal with the nonlinear robust feature extraction problems via the use of kernel trick, and hereafter proposed the L1-norm kernel discriminant analysis (L1-KDA) method. Extensive experiments on simulated and real data sets are conducted to evaluate the effectiveness of the proposed method in comparing with the state-of-the-art methods.

This paper proposes a new method of face image feature extraction, namely, the fuzzy 2DLDA (F2DLDA) based on the 2D fisher discriminant criterion and fuzzy set theory. In the proposed method, we calculate membership degree matrix by FKNN, then we incorporate the membership degree into the definition of the between-class scatter matrix and within-class scatter matrix and get the fuzzy between-class scatter matrix and fuzzy within-class scatter matrix. Experiments on the ORL and FERET face databases show that the new method can work well.

In this paper, we present a novel approach to solving the supervised dimensionality reduction problem by encoding an image object as a general tensor of 2nd or higher order. First, we propose a discriminant tensor criterion (DTC), whereby multiple interrelated lower-dimensional discriminative subspaces are derived for feature selection. Then, a novel approach called k-mode cluster-based discriminant analysis is presented to iteratively learn these subspaces by unfolding the tensor along different tensor dimensions. We call this algorithm discriminant analysis with tensor representation (DATER), which has the following characteristics: 1) multiple interrelated subspaces can collaborate to discriminate different classes; 2) for classification problems involving higher-order tensors, the DATER algorithm can avoid the curse of dimensionality dilemma and overcome the small sample size problem; and 3) the computational cost in the learning stage is reduced to a large extent owing to the reduced data dimensions in generalized eigenvalue decomposition. We provide extensive experiments by encoding face images as 2nd or 3rd order tensors to demonstrate that the proposed DATER algorithm based on higher order tensors has the potential to outperform the traditional subspace learning algorithms, especially in the small sample size cases.

As an effective technique for feature extraction and pattern classification Fisher linear discriminant (FLD) has been successfully applied in many fields. However, for a task with very high-dimensional data such as face images, conventional FLD technique encounters a fundamental difficulty caused by singular within-class scatter matrix. To avoid the trouble, many improvements on the feature extraction aspect of FLD have been proposed. In contrast, studies on the pattern classification aspect of FLD are quiet few. In this paper, we will focus our attention on the possible improvement on the pattern classification aspect of FLD by presenting a novel linear discriminant criterion called maximum scatter difference (MSD). Theoretical analysis demonstrates that MSD criterion is a generalization of Fisher discriminant criterion, and is the asymptotic form of discriminant criterion: large margin linear projection. The performance of MSD classifier is tested in face recognition. Experiments performed on the ORL, Yale, FERET and AR databases show that MSD classifier can compete with top-performance linear classifiers such as linear support vector machines, and is better than or equivalent to combinations of well known facial feature extraction methods, such as eigenfaces, Fisherfaces, orthogonal complementary space, nullspace, direct linear discriminant analysis, and the nearest neighbor classifier.

Many multivariate statistical methods rely heavily on the sample covariance matrix. It is well known though that the sample covariance matrix is highly non-robust. One popular alternative approach for "robustifying" the multivariate method is to simply replace the role of the covariance matrix with some robust scatter matrix. The aim of this paper is to point out that in some situations certain properties of the covariance matrix are needed for the corresponding robust "plug-in" method to be a valid approach, and that not all scatter matrices necessarily possess these important properties. In particular, the following three multivariate methods are discussed in this paper: independent components analysis, observational regression and graphical modeling. For each case, it is shown that using a symmetrized robust scatter matrix in place of the covariance matrix results in a proper robust multivariate method.