Feature extraction VIA kernel weighted discriminant analysis methods

Linear Discriminant Analysis (LDA) and Multilinear Principal Component Analysis (MPCA) are leading subspace methods for achieving dimension reduction based on supervised learning. Both LDA and MPCA use class labels of data samples to calculate subspaces onto which these samples are projected. Furthermore, both methods have been successfully applied to face recognition. Although LDA and MPCA share common goals and methodologies, in previous research they have been applied separately and independently. In this paper, we propose an extension of LDA to multiple factor frameworks. Our proposed method, Multifactor Discriminant Analysis, aims to obtain multilinear projections that maximize the between-class scatter while minimizing the withinclass scatter, which is the same core fundamental objective of LDA. Moreover, Multifactor Discriminant Analysis (MDA), like MPCA, uses multifactor analysis and calculates subject parameters that represent the characteristics of subjects and are invariant to other changes, such as viewpoints or lighting conditions. In this way, our proposed MDA combines the best virtues of both LDA and MPCA for face recognition.

Kernel discriminant analysis (KDA) and the kernel principal component analysis (KPCA), which are the extension of the linear discriminant analysis (LDA) and the principal component analysis (PCA), respectively, from linear domain to nonlinear domain via the kernel trick, are two very popular nonlinear feature extraction methods. In this paper, we present a new feature extraction algorithm by combing KDA and KPCA, and then apply it to the face recognition task. The experimental results on Yale face dataset show that the proposed method can significantly improve the performance both KDA and KPCA.

Higher order spectral regression discriminant analysis (HOSRDA): A tensor feature reduction method for ERP detection

An automated wood recognition system is designed to classify tropical wood species. The wood features are extracted based on two feature extractors: Basic Grey Level Aura Matrix (BGLAM) technique and statistical properties of pores distribution (SPPD) technique. Due to the nonlinearity of the tropical wood species separation boundaries, a pre classification stage is proposed which consists of Kmeans clustering and kernel discriminant analysis (KDA). Finally, Linear Discriminant Analysis (LDA) classifier and K-Nearest Neighbour (KNN) are implemented for comparison purposes. The study involves comparison of the system with and without pre classification using KNN classifier and LDA classifier. The results show that the inclusion of the pre classification stage has improved the accuracy of both the LDA and KNN classifiers by more than 12%.

To reduce the memory and timing complexity of the Side-Channel Attacks (SCA), dimensionality reduction techniques are usually applied to the measurements. They aim to detect the so-called Points of Interest (PoIs), which are time samples which (jointly) depend on some sensitive information (e.g. secret key sub-parts), and exploit them to extract information. The extraction is done through the use of functions which combine the measurement time samples. Examples of combining functions are the linear combinations provided by the Principal Component Analysis or the Linear Discriminant Analysis. When a masking countermeasure is properly implemented to thwart SCAs, the selection of PoIs is known to be a hard task: almost all existing methods have a combinatorial complexity explosion, since they require an exhaustive search among all possible d-tuples of points. In this paper we propose an efficient method for informative feature extraction in presence of masking countermeasure. This method, called Kernel Discriminant Analysis, consists in completing the Linear Discriminant Analysis with a so-called kernel trick, in order to efficiently perform it over the set of all possible d-tuples of points without growing in complexity with d. We identify and analyse the issues related to the application of such a method. Afterwards, its performances are compared to those of the Projection Pursuit (PP) tool for PoI selection up to a 4th-order context. Experiments show that the Kernel Discriminant Analysis remains effective and efficient for high-order attacks, leading to a valuable alternative to the PP in constrained contexts where the increase of the order d does not imply a growth of the profiling datasets.

In this talk, several feature extraction techniques will be introduced from two cues: algebraic feature extraction andgeometrical feature extraction methods. For algebraic feature extraction, Principal Component Analysis (PCA), LinearDiscriminant Analysis (LDA), Kernel Discriminant Analysis (KDA), Sparse representation and their variants will beintroduced. For geometrical feature extraction, Contour Points Distribution Histogram (CPDH), Active Shape Model (ASM),Active Appearance Model (AAM), ASM+AAM, and their variants will be presented. Typical applications of these featureextraction methods will be presented including face recognition, image fusion, image retrieval, and target tracking, etc. Themain aim of this talk is to introduce my main work in pattern recognition and computer vision, which hopefully ignitesdiscussions, and future development of feature extraction.

A brain-computer interface (BCI) is a system that enables control of devices or communication with other persons, only through cerebral activity, without using muscles. The main application for BCIs is assistive technology for disabled persons. Examples for devices that can be controlled by BCIs are artificial limbs, spelling devices, or environment control systems. BCI research has seen renewed interest in recent years, and it has been convincingly shown that communication via a BCI is in principle feasible. However, present day systems still have shortcomings that prevent their widespread application. In part, these shortcomings are caused by limitations in the functionality of the pattern recognition algorithms used for discriminating brain signals in BCIs. Moreover, BCIs are often tested exclusively with able-bodied persons instead of conducting tests with the target user group, namely disabled persons. The goal of this thesis is to extend the functionality of pattern recognition algorithms for BCI systems and to move towards systems that are helpful for disabled users. We discuss extensions of linear discriminant analysis (LDA), which is a simple but efficient method for pattern recognition. In particular, a framework from Bayesian machine learning, the so-called evidence framework, is applied to LDA. An algorithm is obtained that learns classifiers quickly, robustly, and fully automatically. An extension of this algorithm allows to automatically reduce the number of sensors needed for acquisition of brain signals. More specifically, the algorithm allows to perform electrode selection. The algorithm for electrode selection is based on a concept known as automatic relevance determination (ARD) in Bayesian machine learning. The last part of the algorithmic development in this thesis concerns methods for computing accurate estimates of class probabilities in LDA-like classifiers. These probabilities are used to build a BCI that dynamically adapts the amount of acquired data, so that a preset, approximate bound on the probability of misclassifications is not exceeded. To test the algorithms described in this thesis, a BCI specifically tailored for disabled persons is introduced. The system uses electroencephalogram (EEG) signals and is based on the P300 evoked potential. Datasets recorded from five disabled and four able-bodied subjects are used to show that the Bayesian version of LDA outperforms plain LDA in terms of classification accuracy. Also, the impact of different static electrode configurations on classification accuracy is tested. In addition, experiments with the same datasets demonstrate that the algorithm for electrode selection is computationally efficient, yields physiologically plausible results, and improves classification accuracy over static electrode configurations. The classification accuracy is further improved by dynamically adapting the amount of acquired data. Besides the datasets recorded from disabled and able-bodied subjects, benchmark datasets from BCI competitions are used to show that the algorithms discussed in this thesis are competitive with state-of-the-art electroencephalogram (EEG) classification algorithms. While the experiments in this thesis are uniquely performed with P300 datasets, the presented algorithms might also be useful for other types of BCI systems based on the EEG. This is the case because functionalities such as robust and automatic computation of classifiers, electrode selection, and estimation of class probabilities are useful in many BCI systems. Seen from a more general point of view, many applications that rely on the classification of cerebral activity could possibly benefit from the methods developed in this thesis. Among the potential applications are interrogative polygraphy ("lie detection") and clinical applications, for example coma outcome prognosis and depth of anesthesia monitoring.

Accuracy of discriminant analysis methods to classify activities of subjects with Parkinson's Disease using wearable sensors

Exploring kernel discriminant analysis for speaker verification with limited test data

A new algorithm, Laplacian minmax discriminant projection (LMMDP), is proposed in this paper for supervised dimensionality reduction. LMMDP aims at learning a linear transformation which is an extension of linear discriminant analysis (LDA). Specifically, we define the within-class scatter and the between-class scatter using similarities which are based on pairwise distances in sample space. After the transformation, the considered pairwise within the same class are as close as possible, while those between classes are as far as possible. The structural information of classes is contained in the within-class and the between-class Laplacian matrices. Thus the discriminant projection subspace can be derived by controlling the structural evolution of Laplacian matrices. The performance on several data sets demonstrates the competence of the proposed algorithm.

The early detection of genetic disorders in infants is crucial for the timely management of patients and disease. The particular facial characteristics of patients affected by dysmorphic syndromes, which account to about half of genetic disorders, allows to identify positive cases prior to cytogenetic results, and avoid the overuse of genetic blood tests. However, the diagnostic accuracy by pediatricians is moderate. In this work, we present a general framework for the detection of genetic disorders from facial pictures, combining geometrical and texture features. Based on the 2D extension of Linear Discriminant Analysis, we propose the extraction of optimal landmark-specific Local Binary Pattern-based features. In particular, the proposed framework computes optimal local image filters and soft neighborhood weighting matrices that enhance the discriminative ability of the system. This new framework was tested on a database of 145 cases, including 73 pathological patients with 15 different genetic syndromes, obtaining a detection accuracy of 0.95.

A major and demanding issue in the telecommunications industry is the prediction of churn customers. Churn describes the customer who is attrite from one Telecom service provider to competitors searching for better services offers. Companies from the Telco sector frequently have customer relationship management offices it is the main objective in how to win back defecting clients because preserve long-term customers can be much more beneficial to a company than gain newly recruited customers. Researchers and practitioners are paying great attention and investing more in developing a robust customer churn prediction model, especially in the telecommunication business by proposed numerous machine learning approaches. Many approaches of Classification are established, but the most effective in recent times is a tree-based method. The main contribution of this research is to predict churners/non-churners in the Telecom sector based on project pursuit Random Forest (PPForest) that uses discriminant feature analysis as a novelty extension of the conventional Random Forest approach for learning oblique Project Pursuit tree (PPtree). The proposed methodology leverages the advantage of two discriminant analysis methods to calculate the project index used in the construction of PPtree. The first method used Support Vector Machines (SVM) as a classifier in the construction of PPForest to differentiate between churners and non-churners customers. The second method is a Linear Discriminant Analysis (LDA) to achieve linear splitting of variables node during oblique PPtree construction to produce individual classifiers that are robust and more diverse than classical Random Forest. It found that the proposed methods enjoy the best performance measurements e.g. Accuracy, hit rate, ROC curve, Gini coefficient, Kolmogorov-Smirnov statistic and lift coefficient, H-measure, AUC. Moreover, PPForest based on direct applied of LDA on the raw data delivers an effective evaluator for the customer churn prediction model.

The small sample size (SSS) problem is one of the major problems encountered when traditional kernel discriminant analysis methods are applied to high-dimensional pattern recognition tasks. Different methods have been proposed to solve this problem. In this paper, we introduce a new kernel discriminant analysis algorithm, which is able to effectively address the SSS problem and extract a set of optimal discriminant vectors without any lose of useful discriminant information. Experiments performed on radar target recognition using range profiles indicate that the proposed method outperforms some existing kernel discriminant algorithms, such as generalized discriminant analysis and kernel direct discriminant analysis, in terms of recognition rate.

A modification of kernel discriminant analysis for high-dimensional data—with application to face recognition

Fourier transform is a widely used image processing technology. Kernel discriminant analysis is an effective nonlinear feature extraction technique. Based on them, we propose a novel feature extraction approach for face recognition. First, we perform the Fourier transform on face images and express the Fourier frequency bands in the plural form. By computing the kernel-plural discriminant capability of every frequency band, we choose the bands with strong capabilities and use them to form a new sample set. Then, we extract nonlinear discriminant features from the set and classify it by using the nearest neighbor classifier. Experimental results on AR and Feret face databases demonstrate the effectiveness of the proposed approach.