Direct Orthogonal Marginal Fisher Analysis

In recent years, a variety of manifold-based learning dimensionality reduction techniques have been proposed, which attempt to project the original data into a lower dimensional feature space by preserving the local neighborhood structure. Among them, marginal fisher analysis (MFA) achieved high performance for face recognition. However, the optimal basis vectors obtained by MFA are non-orthogonal and MFA usually deteriorates when labeled information is insufficient. In order to resolve these problems, we present a new method called orthogonal semi-supervised marginal fisher analysis (OSMFA), which not only extracts all the orthogonal discriminant vectors but also preserves the global structure of labeled and unlabeled samples to learn a better subspace for classification. Experimental results on ORL database demonstrate the effectiveness of the proposed algorithm.

Linear Discriminant Analysis (LDA) has been one of the popular subspace methods for face recognition. But this method suffers from the small sample size (SSS) problem, also known as 'curse of dimensionality'. Various techniques have been proposed in literature to overcome this limitation. But it is still unclear which method provides the best solution to SSS problem. In this paper, we have investigated the performance of some popular subspace methods such as principal component analysis (PCA), PCA + LDA, LDA via QR decomposition, Null-space LDA, Exponential Discriminant Analysis (EDA), PCA+EDA etc. Extensive experiments have been performed on five publically available face datasets viz. AR, CMU-PIE, PIX, Yale and YaleB. The performance is measured in terms of average classification accuracy. Experimental results show that the performance increases with the increase in the number of images per person in training set irrespective of the datasets. There is no clear winner among the subspace methods under investigation. But, the performance of PCA+LDA and PCA+EDA is consistent in tackling SSS problem irrespective of the dataset and can also handle the illumination variation in face recognition.

We made two contributions in this paper. First, a new method called two-dimensional fisher discriminant analysis (2D-FDA) is proposed to deal with the small sample size (SSS) problem in LDA based face recognition. Second, appearance and depth information are combined to improve face recognition rate. Different from the conventional 1D-FDA (PCA plus LDA) approaches, 2D-FDA is based on 2D image matrices rather than column vectors so the image matrix does not need to be transformed into a long vector before feature extraction. The advantage arising in this way is that the SSS problem does not exist any more because the between-class and within-class scatter matrices constructed in 2D-FDA are both of full-rank. 2D FDA and 1D FDA (PCA plus LDA) are evaluated respectively with a problem that combines appearance and depth information for face recognition. We investigate the complete range of linear combinations to reveal the interplay between these two paradigms. The recognition rate by the combination is better than either appearance alone or depth alone. It is verified that 2D-FDA outperforms 1D FDA

A new image feature extraction method for face recognition called Tri-direction 2D-Fisher Discriminant Analysis (T2D-FDA) is proposed to deal with the Small Sample Size (SSS) problem in conventional 1D-Fisher Discriminant Analysis (1D-FDA). Moreover, the essence of T2D-FDA is investigated, and the equivalence of the left-multiplying 2D-FDA of the original image matrices and the left-multiplying D2D-FDA of diagonal image matrices is verified if each column is viewed as a computational unit. Different from the 1D-LDA based approaches, the T2D-FDA is based on 2D image matrices rather than column vectors, so the image matrix does not need to be transformed into a long vector before feature extraction. The advantage arising in this way is that the SSS problem does not exist anymore because within-class scatter matrices constructed is full-rank in T2D-FDA. The proposed method is applied to face recognition where only few training images exist for each subject. Experiment results show T2D-FDA outperforms the current linear subspace methods in SSS problem on four public databases: ORL, Yale, AR and FERET face databases.

Orthogonal discriminant analysis algorithms have recently been proposed. However, these methods don't address the singularity problem in the high dimensional feature space. In this paper, we present a new method called Direct Orthogonal Discriminant Analysis (DODA), which is able to extract all the orthogonal discriminant vectors simultaneously in the high-dimensional feature space and does not suffer the singularity problem. This method is very simple and easy to be implemented. Experimental results show that the proposed method is very competitive in comparison with some existing dimensionality reduction algorithms.

In this paper, a novel image discriminant analysis method, coined two-dimensional discriminant transform based on scatter difference criterion (2DSDD), is developed for image representation. The proposed 2DSDD scheme adopts the difference of both between-class scatter and within-class scatter as discriminant criterion. In this way, the small sample size problem usually occurred in the traditional Fisher discriminant analysis (LDA) is essentially avoided. In addition, the developed method directly depends on image matrices. That is to say, it is not necessary to convert the image matrix into high-dimensional image vector like those conventional linear discriminant methods prior to feature extraction so that much computational time would be saved. Finally, the experimental results on the ORL face database indicate that the proposed method outperforms Fisherfaces, the standard scatter difference discriminant analysis, not only in the computation efficiency, but also in its recognition performance.

Independent Component Analysis (ICA) is a popular approach for face recognition. However, face recognition is often a small sample size problem, which will weaken the recognition performance of ICA classifier. In this paper, a novel method is proposed to enhance ICA classifier for the small sample size problem. First, we use the random resampling method to generate some random independent subspaces, and a classifier is constructed in each subspace. Then a voting strategy is adopted to integrate these classifiers for discrimination. Experimental results on public available face database show that the proposed method can obvious improve the performance of ICA classifier.KeywordsFace RecognitionIndependent Component AnalysisFace ImageIndependent Component AnalysisFace DatabaseThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In this paper, we propose a new supervised learning algorithm, which is named the Generalized Marginal Fisher Analysis (GMFA), to utilize the advantages of the Marginal Fisher Analysis (MFA) and the Generalized Singular Value Decomposition (GSVD) techniques for face recognition. The experimental results on several standard face databases demonstrate that GMFA outperforms LDA/Fisherface, LDA/GSVD and MFA.

This paper develops a supervised discriminant technique based on graph embedding, called maximum inter-class and marginal discriminant embedding (MIMDE) for feature and classification in high-dimensional data space. In graph embedding, our objective is to find a linear transform matrix to make the samples in the same class as compact as possible and the samples belong to the different classes as dispersed as possible through local property (intra-class compactness graph and inter-class separability graph) and marginal property (margin separability graph). This characteristic makes MIMDE more intuitive and more powerful than marginal Fisher analysis which not considered inter-class separability graph. Experimental results on ORL and AR face databases show the effectiveness of the proposed method.

Kernel quadratic discriminant analysis for small sample size problem

The idea of user-specific multimodal biometrics is pioneered by Jain, A. K., et al. (2002) and further exploited by Toh, K-A, et al. (2004) recently. In this paper, we look into several issues pertaining to user-specific multimodal biometric verification. These issues include the small sample size problem, learning of local decision hyperplanes and setting of local thresholds. For small sample size problem, the noise-injection technique and a feature scaling-space technique are considered. For local learning, we adopt a recently proposed reduced polynomial since it has fast single-step computation and accurate estimation. For setting of local decision thresholds, nine baselines are identified. Extensive experiments are performed on a moderate data set and relatively conclusive results are observed.

In this paper we examine the task of maintenance decision support in classification of the dangerous situations discovered by the monitoring system. This task is reduced to the contextual multi-armed bandit problem. We highlight the small sample size problem appeared in this task due to the rather rare failures. The novel algorithm based on the nearest neighbor search is proposed. An experimental study is provided for several synthetic datasets with the situations described by either simple features or grayscale images. It is shown, that our algorithm outperforms the well-known contextual multi-armed methods with the Upper Confidence Bound and softmax stochastic search strategies.

The small sample size (SSS) problem is one of the major problems encountered when traditional kernel discriminant analysis methods are applied to high-dimensional pattern recognition tasks. Different methods have been proposed to solve this problem. In this paper, we introduce a new kernel discriminant analysis algorithm, which is able to effectively address the SSS problem and extract a set of optimal discriminant vectors without any lose of useful discriminant information. Experiments performed on radar target recognition using range profiles indicate that the proposed method outperforms some existing kernel discriminant algorithms, such as generalized discriminant analysis and kernel direct discriminant analysis, in terms of recognition rate.

Graph regularized non-negative matrix factorization (GNMF) is widely used in feature extraction. In the process of dimensionality reduction, GNMF can retain the internal manifold structure of data by adding a regularizer to non-negative matrix factorization (NMF). Because Ga NMF regularizer is implemented by local preserving projections (LPP), there are small sample size problems (SSS). In view of the above problems, a new algorithm named robust exponential graph regularized non-negative matrix factorization (REGNMF) is proposed in this paper. By adding a matrix exponent to the regularizer of GNMF, the possible existing singular matrix will change into a non-singular matrix. This model successfully solves the problems in the above algorithm. For the optimization problem of the REGNMF algorithm, we use a multiplicative non-negative updating rule to iteratively solve the REGNMF method. Finally, this method is applied to AR, COIL database, Yale noise set, and AR occlusion dataset for performance test, and the experimental results are compared with some existing methods. The results indicate that the proposed method is more significant.

Linear discriminant analysis (LDA) is a popular feature extraction technique in statistical pattern recognition. However, it often suffers from the small sample size problem when dealing with the high dimensional image data. Moreover, while LDA is guaranteed to find the best directions when each class has a Gaussian density with a common covariance matrix, it can fail if the class densities are more general. In this paper, a new feature extraction method, nonparametric maximum margin criterion (NMMC), is proposed. NMMC finds the important discriminant directions without assuming the class densities belong to any particular parametric family, and it does not depend on the nonsingularity of the within-class scatter matrix. Our experimental results on the ATT and FERET face databases demonstrate that NMMC outperforms the existing variant LDA methods and the other state-of-art face recognition approaches.