Online nonparametric discriminant analysis for incremental subspace learning and recognition

Many state-of-the-art i-vector based voice biometric systems use linear discriminant analysis (LDA) as a post-processing stage to increase the computational efficiency in the back-end via dimensionality reduction, as well as annihilate the undesired (noisy) directions in the total variability subspace. The traditional approach for computing the LDA transform uses parametric representations for both intra- and inter-class scatter matrices that are based on the Gaussian distribution assumption. However, it is known that the actual distribution of i-vectors may not necessarily be Gaussian, and in particular, in the presence of noise and channel distortions. In addition, the rank of the LDA projection (i.e., the maximum number of available discriminant bases) is limited to the number of classes minus 1. Accordingly, language recognition tasks on noisy data that involve only a few language classes receive limited or no benefit from the LDA post-processing. Motivated by this observation, we present an alternative non-parametric discriminant analysis (NDA) technique that measures both the within- and between-language variation on a local basis using the nearest neighbor rule. The effectiveness of the NDA method is evaluated in the context of noisy language recognition tasks using speech material from the DARPA Robust Automatic Transcription of Speech (RATS) program. Experimental results indicate that NDA is more effective than the traditional parametric LDA for language recognition under noisy and channel degraded conditions.

Dimensionality reduction is the most popular method for feature extraction and recognition. Recently, Li et al. (IEEE PAMI, 2009) proposed Nonparametric Discriminant Analysis (NDA) based dimensionality reduction for face recognition and reported an excellent recognition performance. However, NDA has its limitations on extracting the nonlinear features of face images for recognition, and owing to the highly nonlinear and complex distribution of face images under a perceivable variation in viewpoint, illumination or facial expression. In order to increase the NDA, we extend the NDA with kernel trick to propose Nonparametric Kernel Discriminant Analysis (NKDA) for feature extraction and recognition. Experimental results on ORL, YALE and UMIST face databases show that NKDA outperforms NDA on recognition, which demonstrates that it is feasible to improve NDA with kernel trick for feature extraction.

Biometrics serves to identify the identity of an individual with the help of mathematical analysis of its biological features. In this paper, we propose a novel classification method based on kernel Nonparametric Discriminant Analysis (KNDA). We use the facial features extracted with Gabor descriptor and ordinal filter, which are encoded in local regions, as visual primitives. The different ordinal measures are derived from various kinds of components of Gabor images such as magnitude, phase, real and imaginary. Then, the statistical distributions of these primitives in face image blocks are concatenated to obtain a feature vector whose dimension is reduced using PCA and variance. Each feature vector is treated as a feature input for the proposed Multi-class KNDA classifier. The proposed method incorporates some near-global variations of the data provided by the KNDA. In fact, this latter is based on flexible non-linear separation between face database classes. Moreover, it relaxes the normality assumption of the Linear Discriminant Analysis (LDA). The proposed method is tested on the well-known ORL face database and the Yale face database. Then, it is evaluatedand compared with non-linear classifier (KFD) and linear classifier (LDA) in term ofclassification accuracy.

<p>Support vector machine (SVM) is a powerful classification methodology, where the support vectors fully describe the decision surface by incorporating local information. On the other hand, nonparametric discriminant analysis (NDA) is an improvement over LDA where the normality assumption is relaxed. NDA also detects the dominant normal directions to the decision plane. This paper introduces a novel SVM þNDA model which can be viewed as an extension to the SVM by incorporating some partially global information, especially, discriminatory information in the normal direction to the decision boundary. This can also be considered as an extension to the NDA where the support vectors improve the choice of k-nearest neighbors on the decision boundary by incorporating local information. Being an extension to both SVM and NDA, it can deal with heteroscedastic and non-normal data. It also avoids the small sample size problem. Moreover, it can be reduced to the classical SVM model, so that existing softwares can be used. A kernel extension of the model, called KSVM þ KNDA is also proposed to deal with nonlinear problems. We have carried an extensive comparison of the SVM þ NDA to the LDA, SVM, heteroscedastic LDA (HLDA), NDA and the combined SVM and LDA on artificial, real and face recognition data sets. Results for KSVM þ KNDA have also been presented. These comparisons demonstrate the advantages and superiority of our proposed model</p>

3D model classification has many applications in CAD, 3D object retrieval, and so on. The description of 3D model is crucial but difficult, which leads to the difficulty of classification. The traditional classifier has its limitation in classification of 3D model description. In this paper, we present 3D model classification-based nonparametric discriminant analysis with kernels combined with geometry projection-based histogram model for invariable feature extraction. Firstly, we present nonparametric discriminant analysis with kernels, and secondly, we proposed the invariable feature extraction method with geometry projection-based histogram model. Thirdly, we present the framework of 3D model classification using the proposed nonparametric discriminant analysis with kernels and geometry projection-based histogram model. Finally, we testify the feasibility of the proposed algorithm and performance on 3D model classification. The experimental results show that the proposed scheme is feasible and effective on 3D model classification on the public datasets.

Feature extraction has been extensively investigated and discussed in computer vision and pattern recognition literature over the past three decades. It has particularly attracted more and more attention in recent years due to the increasing demands for developing real-world human computer interaction systems. While a large number of feature extraction algorithms have been proposed in the literature and some of them have achieved reasonably good performance in many real world applications, such as face recognition, gait recognition, facial expression recognition and human age estimation, there is still some room for further improvement to address the challenges of these methods. In this thesis, we propose various enhanced linear subspace algorithms and apply them to face and gait feature extraction to demonstrate their efficacy and superiority over state-of-the-art methods. Specifically, we propose four new subspace learning approaches, including double weighted subspace learning, parametric regularized subspace learning, cost-sensitive subspace learning, and subspace learning with limited number of training samples. Lastly, we also apply multi-label subspace learning techniques for human age estimation. The above-mentioned methods have been successfully applied to several computer vision applications, such as face recognition, facial expression recognition and gait-based human age estimation. Experimental results are presented to demonstrate the efficacy of the proposed methods.

Convolutional neural network (CNN) has shown superior classification results for hyperspectral image classification. Due to high dimensionality of the hyperspectral image, usually a feature reduction method is applied to the hyperspectral data before feeding it to the CNN model. Principal component analysis is the most widely used approach to this end. However, it has two main disadvantages. It uses the mean square error measure and ignores the class separability. Moreover, it is a pixel-based method, which does not utilize the spatial information. To deal with these weaknesses, the patch-based nonparametric discriminant analysis method is proposed for hyperspectral image feature reduction in this work. However, due to the complex and heterogeneous nature of the hyperspectral images, the use of fixed square patches is not so appropriate. To deal with this difficulty, a shaping approach is introduced for the generation of adaptive patches with consistent shapes in each local region. The input patches of the CNN model are also weighted according to the spectral similarity and class boundaries information. Moreover, a simple residual CNN model is suggested for the classification of the extracted shaped input patches. The efficiency of the proposed framework is assessed in different cases using various evaluation measures on six real hyperspectral images.

Constrained [formula omitted]-optimal sparse representation technique for face recognition

Classification accuracy and execution time are two important parameters in the selection of classification algorithms. In our experiments, 12 different ensemble algorithms, and 11 single classifiers are compared according to their accuracies and train/test time over 36 datasets. The results show that Rotation Forest has the highest accuracy. However, when accuracy and execution time are considered together, Random Forest and Random Committees can be the best choices.

This paper describes a performance evaluation study in which some efficient classifiers are tested in handwritten digit recognition. The evaluated classifiers include a statistical classifier (modified quadratic discriminant function, MQDF), three neural classifiers, and an LVQ (learning vector quantization) classifier. They are efficient in that high accuracies can be achieved at moderate memory space and computation cost. The performance is measured in terms of classification accuracy, sensitivity to training sample size, ambiguity rejection, and outlier resistance. The outlier resistance of neural classifiers is enhanced by training with synthesized outlier data. The classifiers are tested on a large data set extracted from NIST SD19. As results, the test accuracies of the evaluated classifiers are comparable to or higher than those of the nearest neighbor (1-NN) rule and regularized discriminant analysis (RDA). It is shown that neural classifiers are more susceptible to small sample size than MQDF, although they yield higher accuracies on large sample size. As a neural classifier, the polynomial classifier (PC) gives the highest accuracy and performs best in ambiguity rejection. On the other hand, MQDF is superior in outlier rejection even though it is not trained with outlier data. The results indicate that pattern classifiers have complementary advantages and they should be appropriately combined to achieve higher performance.

A demonstrated failure mode for operational solar flare forecasting is the inability to forecast flares that occur near, or just beyond, the solar limb. To address this shortcoming, we develop a “” full‐heliosphere event forecasting framework and evaluate its statistical classification ability against this specific challenge. A magnetic surface flux transport model is used to generate full‐sun maps of the photospheric radial magnetic field from which active regions (ARs) are identified and tracked using a new labeling scheme that is observer‐location agnostic and allows for post‐facto modifications. Flare‐relevant magnetic parameters couple to a “visibility” index that specifies AR location relative to the visible solar limb and expected flare detection. Flare labels are assigned according to peak Soft X‐ray flux, and a statistical classification is performed using nonparametric discriminant analysis. A version where new or emerging ARs on the far (“invisible”) side of the Sun are incorporated into the model by way of far‐side helioseismology, is also tested. We evaluate the new framework by its performance specifically including the limb areas using Brier Skill Score and ROC Skill Score, finding improvement at the level or less. However, we do find that the number of False Negatives, or “missed” forecasts decreases, and find strong evidence that the additional information provided by the far‐side helioseismology can help predict near‐ and just‐beyond‐limb flares, particularly for East‐limb events. While individual components of this framework could be improved, we demonstrate that a known failure mode for solar flare forecasting can be mitigated with available resources.

In this paper an ensemble feature extraction algorithm is proposed based on Adaboost.M2 for multiclass classification problem. The proposed algorithm makes no assumption about the distribution of the data and primarily performs by directly selecting the discriminant features with the minimum training error, which can overcome the main drawbacks of the traditional methods, such as Principle Component Analysis (PCA), Fisher Linear Discriminant Analysis (FLD) and Nonparametric Discriminant Analysis (NDA). The proposed method first samples a large number of bootstrap training subsets from the original training set and implements FLD in each subset to get a large number of bootstrap FLD projections. Then at each step of Adaboost.M2, the projection with the minimum weighted K Nearest Neighbor (KNN) classification error is selected from a pool of linear projections to combine the final strong classifier. Experimental results on three real-world data sets demonstrate that the proposed algorithm is superior to other feature extraction techniques.

: Contents: Density estimation; Edge process models for image reconstruction; Parsimonious additive models; ICM and Annealing; Nonparametric discriminant analysis; Image refinement; Indirectly observed images; Spatial smoothing.

Fatty acid compositional data for Greek virgin olive oils from 24 years of harvest and various regions and cultivars were evaluated using chemometric methods. Non‐parametric discriminant analysis after proper transformation of the data seems to be a suitable approach to characterise the oils according to the geographical origin and may produce a scientific basis for the assignment of an ‘appellation d'origine’ trade mark.

This paper presents a method for recognizing sign language using hand movement trajectory. By applying Kernel Principal Component Analysis (KPCA), the motion trajectory data are firstly mapped into a higher-dimensional feature space for analysis. The advantage of using high dimensionality is that it allows a more flexible decision boundary and thus helps us to achieve better classification accuracy. Then we perform the Nonparametric Discriminant Analysis (NDA) in the higher-dimensional feature space, so that the most helpful information can be extracted. We have tested the proposed method using the Australian Sign Language (ASL) data set. The results demonstrate that our approach outperforms the current state-of-the-art for trajectory-based sign language recognition.