Dimensionality reduction based on nonparametric discriminant analysis with kernels for feature extraction and recognition

Many state-of-the-art i-vector based voice biometric systems use linear discriminant analysis (LDA) as a post-processing stage to increase the computational efficiency in the back-end via dimensionality reduction, as well as annihilate the undesired (noisy) directions in the total variability subspace. The traditional approach for computing the LDA transform uses parametric representations for both intra- and inter-class scatter matrices that are based on the Gaussian distribution assumption. However, it is known that the actual distribution of i-vectors may not necessarily be Gaussian, and in particular, in the presence of noise and channel distortions. In addition, the rank of the LDA projection (i.e., the maximum number of available discriminant bases) is limited to the number of classes minus 1. Accordingly, language recognition tasks on noisy data that involve only a few language classes receive limited or no benefit from the LDA post-processing. Motivated by this observation, we present an alternative non-parametric discriminant analysis (NDA) technique that measures both the within- and between-language variation on a local basis using the nearest neighbor rule. The effectiveness of the NDA method is evaluated in the context of noisy language recognition tasks using speech material from the DARPA Robust Automatic Transcription of Speech (RATS) program. Experimental results indicate that NDA is more effective than the traditional parametric LDA for language recognition under noisy and channel degraded conditions.

<p>Support vector machine (SVM) is a powerful classification methodology, where the support vectors fully describe the decision surface by incorporating local information. On the other hand, nonparametric discriminant analysis (NDA) is an improvement over LDA where the normality assumption is relaxed. NDA also detects the dominant normal directions to the decision plane. This paper introduces a novel SVM þNDA model which can be viewed as an extension to the SVM by incorporating some partially global information, especially, discriminatory information in the normal direction to the decision boundary. This can also be considered as an extension to the NDA where the support vectors improve the choice of k-nearest neighbors on the decision boundary by incorporating local information. Being an extension to both SVM and NDA, it can deal with heteroscedastic and non-normal data. It also avoids the small sample size problem. Moreover, it can be reduced to the classical SVM model, so that existing softwares can be used. A kernel extension of the model, called KSVM þ KNDA is also proposed to deal with nonlinear problems. We have carried an extensive comparison of the SVM þ NDA to the LDA, SVM, heteroscedastic LDA (HLDA), NDA and the combined SVM and LDA on artificial, real and face recognition data sets. Results for KSVM þ KNDA have also been presented. These comparisons demonstrate the advantages and superiority of our proposed model</p>

3D model classification has many applications in CAD, 3D object retrieval, and so on. The description of 3D model is crucial but difficult, which leads to the difficulty of classification. The traditional classifier has its limitation in classification of 3D model description. In this paper, we present 3D model classification-based nonparametric discriminant analysis with kernels combined with geometry projection-based histogram model for invariable feature extraction. Firstly, we present nonparametric discriminant analysis with kernels, and secondly, we proposed the invariable feature extraction method with geometry projection-based histogram model. Thirdly, we present the framework of 3D model classification using the proposed nonparametric discriminant analysis with kernels and geometry projection-based histogram model. Finally, we testify the feasibility of the proposed algorithm and performance on 3D model classification. The experimental results show that the proposed scheme is feasible and effective on 3D model classification on the public datasets.

Biometrics serves to identify the identity of an individual with the help of mathematical analysis of its biological features. In this paper, we propose a novel classification method based on kernel Nonparametric Discriminant Analysis (KNDA). We use the facial features extracted with Gabor descriptor and ordinal filter, which are encoded in local regions, as visual primitives. The different ordinal measures are derived from various kinds of components of Gabor images such as magnitude, phase, real and imaginary. Then, the statistical distributions of these primitives in face image blocks are concatenated to obtain a feature vector whose dimension is reduced using PCA and variance. Each feature vector is treated as a feature input for the proposed Multi-class KNDA classifier. The proposed method incorporates some near-global variations of the data provided by the KNDA. In fact, this latter is based on flexible non-linear separation between face database classes. Moreover, it relaxes the normality assumption of the Linear Discriminant Analysis (LDA). The proposed method is tested on the well-known ORL face database and the Yale face database. Then, it is evaluatedand compared with non-linear classifier (KFD) and linear classifier (LDA) in term ofclassification accuracy.

This paper presents a novel face recognition method by integrating the Gabor wavelet representation of face images and the enhanced powerful discriminator, complete Kernel Fisher Discriminant (CKFD) with fractional power polynomial (FPP) models. The novelty of this paper comes from (1) Gabor wavelet, is employed to extract desirable facial features characterized by spatial frequency, spatial locality and orientation selectivity to cope with the variations in illumination and facial expressions, which improves the recognition performance; (2) a recently proposed powerful discriminator, namely CKFD, which enhances its discriminating ability using two kinds of discriminant information (i.e., regular and irregular information), is employed to classify the Gabor features; (3) the FPP models, are employed to CKFD analysis to enhance the discriminating ability. Comparing with existing principal component analysis, linear discriminant analysis, kernel principal component analysis, KFD and CKFD methods, the proposed method gives the superior results with the ORL, Yale and UMIST face databases.

This paper presents a novel approach for online subspace learning based on an incremental version of the nonparametric discriminant analysis (NDA). For many real-world applications (like the study of visual processes, for instance) it is impossible to know beforehand the number of total classes or the exact number of instances per class. This motivated us to propose a new algorithm, in which new samples can be added asynchronously, at different time stamps, as soon as they become available. The proposed technique for NDA-eigenspace representation has been used in pattern recognition applications, where classification of data has been performed based on the nearest neighbor rule. Extensive experiments have been carried out both in terms of classification accuracy and execution time. On the one hand, the results show that the Incremental NDA converges towards the classical NDA at the end of the learning process and furthermore. On the other hand, Incremental NDA is suitable to update a large knowledge representation eigenspace in real-time. Finally, the use of our method on a real-world application is presented.

In this paper an ensemble feature extraction algorithm is proposed based on Adaboost.M2 for multiclass classification problem. The proposed algorithm makes no assumption about the distribution of the data and primarily performs by directly selecting the discriminant features with the minimum training error, which can overcome the main drawbacks of the traditional methods, such as Principle Component Analysis (PCA), Fisher Linear Discriminant Analysis (FLD) and Nonparametric Discriminant Analysis (NDA). The proposed method first samples a large number of bootstrap training subsets from the original training set and implements FLD in each subset to get a large number of bootstrap FLD projections. Then at each step of Adaboost.M2, the projection with the minimum weighted K Nearest Neighbor (KNN) classification error is selected from a pool of linear projections to combine the final strong classifier. Experimental results on three real-world data sets demonstrate that the proposed algorithm is superior to other feature extraction techniques.

Face recognition with Symmetric Local Graph Structure (SLGS)

This paper presents a method for recognizing sign language using hand movement trajectory. By applying Kernel Principal Component Analysis (KPCA), the motion trajectory data are firstly mapped into a higher-dimensional feature space for analysis. The advantage of using high dimensionality is that it allows a more flexible decision boundary and thus helps us to achieve better classification accuracy. Then we perform the Nonparametric Discriminant Analysis (NDA) in the higher-dimensional feature space, so that the most helpful information can be extracted. We have tested the proposed method using the Australian Sign Language (ASL) data set. The results demonstrate that our approach outperforms the current state-of-the-art for trajectory-based sign language recognition.

The aim of this study was to determine how the depressor supercilii (DS) connects to the levator labii superioris alaeque nasi (LLSAN) and inferior fibers of the orbicularis oculi (OOc INF) in the human midface. While grimacing, contraction of the DS with fibers connecting to the LLSAN and OOc INF can assist in pulling the medial eyebrow downward more than when these connecting fibers are not present. Contraction of these distinct connecting fibers between the DS and the LLSAN can also slightly elevate the nasal ala and upper lip. The DS was examined in 44 specimens of embalmed adult Korean cadavers. We found that the DS connected to the LLSAN or the OOc INF by muscle fibers or thin aponeuroses in 33 (75.0%) of the 44 specimens. The DS was connected to both the LLSAN and OOc INF by muscle fibers or aponeuroses and had no connection to either in 5 (11.4%) and 11 (25.0%) specimens, respectively. The DS was connected to the LLSAN by the muscle fibers and thin aponeuroses in 6 (13.6%) and 4 (9.1%) specimens, respectively. The DS was connected to the OOc INF by the muscle fibers and thin aponeuroses in 5 (11.4%) and 23 (52.3%) specimens, respectively. Our findings regarding the anatomical connections of the glabellar region DS to the midface LLSAN and OOc INF provide insights on the dynamic balance between the brow depressors such as the DS and brow-elevating muscle and contribute to understanding the anatomical origins of individual variation in facial expressions. These results can also improve the safety, predictability, and aesthetics of treatments for the glabellar region with botulinum toxin type A and can be helpful when performing electromyography.

Owing to the high spectral sampling, the spectral information in hyperspectral imagery (HSI) is often highly correlated and contains redundancy. Motivated by the recent success of sparsity preserving based dimensionality reduction (DR) techniques in both computer vision and remote sensing image analysis community, a novel supervised nonparametric sparse discriminant analysis (NSDA) algorithm is presented for HSI classification. The objective function of NSDA aims at preserving the within-class sparse reconstructive relationship for within-class compactness characterization and maximizing the nonparametric between-class scatter simultaneously to enhance discriminative ability of the features in the projected space. Essentially, it seeks for the optimal projection matrix to identify the underlying discriminative manifold structure of a multiclass dataset. Experimental results on one visualization dataset and three recorded HSI dataset demonstrate NSDA outperforms several state-of-the-art feature extraction methods for HSI classification.

Convolutional neural network (CNN) has shown superior classification results for hyperspectral image classification. Due to high dimensionality of the hyperspectral image, usually a feature reduction method is applied to the hyperspectral data before feeding it to the CNN model. Principal component analysis is the most widely used approach to this end. However, it has two main disadvantages. It uses the mean square error measure and ignores the class separability. Moreover, it is a pixel-based method, which does not utilize the spatial information. To deal with these weaknesses, the patch-based nonparametric discriminant analysis method is proposed for hyperspectral image feature reduction in this work. However, due to the complex and heterogeneous nature of the hyperspectral images, the use of fixed square patches is not so appropriate. To deal with this difficulty, a shaping approach is introduced for the generation of adaptive patches with consistent shapes in each local region. The input patches of the CNN model are also weighted according to the spectral similarity and class boundaries information. Moreover, a simple residual CNN model is suggested for the classification of the extracted shaped input patches. The efficiency of the proposed framework is assessed in different cases using various evaluation measures on six real hyperspectral images.

In this paper, a robust face recognition algorithm is proposed, which is based on the elastic graph matching (EGM) and discriminative feature analysis algorithm. We introduce a cost function for the EGM taking account of variations in face pose and facial expressions, and propose its optimization procedure. Our proposed cost function uses a set of Gabor-wavelet-based features, called robust jet, which are robust against the variations. The robust jet is defined in terms of discrete Fourier transform coefficients of Gabor coefficients. To cope with the difference between face poses of test face and reference faces, 2 x 2 warping matrix is incorporated in the proposed cost function. For the discriminative feature analysis, linear projection discriminant analysis and kernel-based projection discriminant analysis are introduced. These methods are motivated to solve the small-size problem of training samples. The basic idea of PDA is that a class is represented by a subspace spanned by some training samples of the class instead of using sample mean vector, that the distance from a pattern to a class is defined by using the error vector between the pattern and its projection to the subspace representing the class, and that an optimum feature selection rule is developed using the distance concept in a similar way as in the conventional linear discriminant analysis. In order to evaluate the performance of our face recognition algorithm, we carried out some experiments using the well-known FERET face database, and compared the performance with recently developed approaches. We observed that our algorithm outperformed the compared approaches.

We develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression. Taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. We take advantage of the observation that the images of a particular face, under varying illumination but fixed pose, lie in a 3D linear subspace of the high dimensional image space-if the face is a Lambertian surface without shadowing. However, since faces are not truly Lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. Rather than explicitly modeling this deviation, we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation. Our projection method is based on Fisher's linear discriminant and produces well separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expressions. The eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. Yet, extensive experimental results demonstrate that the proposed "Fisherface" method has error rates that are lower than those of the eigenface technique for tests on the Harvard and Yale face databases.

Face identification is the most active area of research in computer vision and biometric authentication. Various face identification methods are developed over the time, still, numerous facial appearances are needed to cope with such as facial expression, pose, and illumination variation. Moreover, faces captured in unrestrained situations also impose immense concern in designing effective face identification methods. It is desirable to extract robust local descriptive features to effectively characterize such facial variations both in unrestrained and restrained situations. This chapter discusses such a face identification method that incorporate a popular local descriptor such as local binary patterns (LBP) based on the improved directional wavelet transform (IDW) method to extract facial features. This designed method is applied to complex face databases such as CASIA-WebFace and LFW which consists of a large number of face images collected under an unrestrained environment with extreme facial variations in expression, pose, and illumination. Experiments and comparison with various methods which include not only the local descriptive methods but also local descriptive-based multiresolution analysis (MRA) based methods demonstrate the efficacy of the LBP-based IDW method.