Linear Embedding by Joint Robust Discriminant Analysis and Inter-class Sparsity

The main goal of discriminant embedding is to extract features that can be compact and informative representations of the original set of features. This paper introduces a hybrid scheme for linear feature extraction for supervised multiclass classification. We introduce a unifying criterion that is able to retain the advantages of robust sparse LDA and Interclass sparsity. Thus, the estimated transformation includes two types of discrimination which are the inter-class sparsity and robust Linear Discriminant Analysis with feature selection. In order to optimize the proposed objective function, we deploy an iterative alternating minimization scheme for estimating the linear transformation and the orthogonal matrix. The introduced scheme is generic in the sense that it can be used for combining and tuning many other linear embedding methods. In the lights of the experiments conducted on six image datasets including faces, objects, and digits, the proposed scheme was able to outperform competing methods in most of the cases.

In recent times, feature extraction was the focus of many researches due to its usefulness in the machine learning and pattern recognition fields. Feature extraction mainly aims to extract informative representations from the original set of features. This can be carried out using various ways. The proposed method is targeting a hybrid linear feature extraction scheme for supervised multi-class classification problems. Inspired by recent robust sparse LDA and Inter-class sparsity frameworks, we will propose a unifying criterion that is able to retain these two powerful linear discriminant method’s advantages. Thus, the obtained transformation encapsulates two different types of discrimination, the inter-class sparsity and robust Linear Discriminant Analysis with feature selection. We will introduce an iterative alternating minimization scheme in order to estimate the linear transform and the orthogonal matrix. The linear transform is efficiently updated via the steepest descent gradient technique. We will also introduce two initialization schemes for the linear transform. The proposed framework is generic in the sense that it allows the combination and tuning of other linear discriminant embedding methods. According to the experiments which have been carried out on several datasets including faces, objects and digits, the proposed method was able to outperform the competing methods in most cases.

A novel feature extraction method for facial expression recognition from sequences of image frames is described and tested. The authors propose HLAC-like features (HLACLF) for feature extraction. The features are extracted using different masks from Grey-scale images for characterising facial texture. Then the most informative features are selected based on mutual information quotient (MIQ) criterion. Multiple linear discriminant analysis (LDA) classifier is adopted. The proposed system is fully automatic and including: face detection, facial detection, feature extraction, feature selection and classification. Experiments on the Cohn–Kanade database illustrate that the HLACLF is efficient for facial expression recognition compared with other feature extraction methods.

Linear discriminant analysis (LDA) is an effective tool in multivariate multigroup data analysis. A standard technique for LDA is to project the data from a high-dimensional space onto a perceivable subspace such that the data can be separated by visual inspection. The criterion of LDA, unfortunately, is extremely susceptible to outliers which commonly occur because of instrument drift and gross errors. This paper proposes a robust discriminant criterion, and based on that criterion, a high-breakdown method for LDA is developed. In an effort to circumvent the local optima trapping, a real genetic algorithm (RGA) was used for the optimization of the criterion. The RGA is capable of locating the global optimal solution with high probability and acceptable computational burden. Classification of one simulated data set and two real chemical ones shows that the developed robust LDA (RLDA) method provides much superior performance to the standard method for outlier-contaminated data and behaves comparably well with the standard one for data without outliers. Copyright © 1999 John Wiley & Sons, Ltd.

An automated wood recognition system is designed to classify tropical wood species. The wood features are extracted based on two feature extractors: Basic Grey Level Aura Matrix (BGLAM) technique and statistical properties of pores distribution (SPPD) technique. Due to the nonlinearity of the tropical wood species separation boundaries, a pre classification stage is proposed which consists of Kmeans clustering and kernel discriminant analysis (KDA). Finally, Linear Discriminant Analysis (LDA) classifier and K-Nearest Neighbour (KNN) are implemented for comparison purposes. The study involves comparison of the system with and without pre classification using KNN classifier and LDA classifier. The results show that the inclusion of the pre classification stage has improved the accuracy of both the LDA and KNN classifiers by more than 12%.

The linear discriminant analysis (LDA) is a very popular linear feature extraction approach. The algorithms of LDA usually perform well under the following two assumptions. The first assumption is that the global data structure is consistent with the local data structure. The second assumption is that the input data classes are Gaussian distributions. However, in real-world applications, these assumptions are not always satisfied. In this paper, we propose an improved LDA framework, the local LDA (LLDA), which can perform well without needing to satisfy the above two assumptions. Our LLDA framework can effectively capture the local structure of samples. According to different types of local data structure, our LLDA framework incorporates several different forms of linear feature extraction approaches, such as the classical LDA and principal component analysis. The proposed framework includes two LLDA algorithms: a vector-based LLDA algorithm and a matrix-based LLDA (MLLDA) algorithm. MLLDA is directly applicable to image recognition, such as face recognition. Our algorithms need to train only a small portion of the whole training set before testing a sample. They are suitable for learning large-scale databases especially when the input data dimensions are very high and can achieve high classification accuracy. Extensive experiments show that the proposed algorithms can obtain good classification results.

The essential task of a Brain-Computer Interface (BCI) is to extract the motor imagery features from Electro-Encephalogram (EEG) signals for classifying the thought process. It is necessary to analyse these obtained signals in both the time domain and frequency domains. It is observed that the combination of multiple algorithms increases the performance of the feature extraction process. This paper identifies combinations that have not been attempted previously and improves the accuracy of the overall process, although other authors implemented different combinations of the techniques. The focus is given more on the feature extraction process and frequency bands, which are user-specific and subject-specific frequency bands. In both time and frequency domains, after analysing EEG signals with the time domain parameter, we select the frequency band and the timing while using the Fisher ratio of the time domain parameter (TDP). We used Fisher discriminant analysis (FDA)-type F-score to simultaneously select the frequency band and time segment for multi-class classification. We extracted subject-specific TDP features from the training trials to train the classifier when optimal time-frequency areas were selected for each subject. In this paper, various methods are explored for obtaining the features, which are Time Domain Parameters (TDP), Fast Fourier Transform (FFT), Principal Component Analysis (PCA), R2, Fast Correlation Based Filter (FCBF), Empirical Mode Decomposition (EMD), and Intrinsic time-scale decomposition (ITD). After the extraction process, PCA is used for dimensionality reduction. An efficient result was obtained with the combination of TDP, FFT, and PCA. We used the multi-class Fisher’s linear discriminant analysis (LDA) as the classifier, which was in line with the FDA-type F-score. It is observed that the combination of feature extraction techniques to the frequency bands that were selected by the Fisher ratio and FDA type F-score along with Fisher’s LDA classifier had higher accuracy than the results obtained other researches. A kappa coefficient accuracy of 0.64 is obtained for the proposed technique. Our method leads to better classification performance when compared to state-of-the-art methods. The novelty of the approach is based on the combination of frequency bands and two feature extraction methods.

A computer vision system for automatic recognition and classification of five varieties of plant leaves under controlled laboratory imaging conditions, comprising: 1–Cydonia oblonga (quince), 2–Eucalyptus camaldulensis dehn (river red gum), 3–Malus pumila (apple), 4–Pistacia atlantica (mt. Atlas mastic tree) and 5–Prunus armeniaca (apricot), is proposed. 516 tree leaves images were taken and 285 features computed from each object including shape features, color features, texture features based on the gray level co-occurrence matrix, texture descriptors based on histogram and moment invariants. Seven discriminant features were selected and input for classification purposes using three classifiers: hybrid artificial neural network–ant bee colony (ANN–ABC), hybrid artificial neural network–biogeography based optimization (ANN–BBO) and Fisher linear discriminant analysis (LDA). Mean correct classification rates (CCR), resulted in 94.04%, 89.23%, and 93.99%, for hybrid ANN–ABC; hybrid ANN–BBO; and LDA classifiers, respectively. Best classifier mean area under curve (AUC), mean sensitivity, and mean specificity, were computed for the five tree varieties under study, resulting in: 1–Cydonia oblonga (quince) 0.991 (ANN–ABC), 95.89% (ANN–ABC), 95.91% (ANN–ABC); 2–Eucalyptus camaldulensis dehn (river red gum) 1.00 (LDA), 100% (LDA), 100% (LDA); 3–Malus pumila (apple) 0.996 (LDA), 96.63% (LDA), 94.99% (LDA); 4–Pistacia atlantica (mt. Atlas mastic tree) 0.979 (LDA), 91.71% (LDA), 82.57% (LDA); and 5–Prunus armeniaca (apricot) 0.994 (LDA), 88.67% (LDA), 94.65% (LDA), respectively.

Fisher linear discriminant analysis (LDA) is widely used to solve classification problems. The classical LDA is developed based on the L2-norm, which is very sensitive to outliers. In this paper, we propose a new LDA with generalized correntropic loss function (GCLF), termed as LDA-GCLF, to improve the robustness of LDA. LDA-GCLF is realized by the alternating direction method of multipliers (ADMM) and the gradient descent method. Experiments results on synthetic and real-world datasets demonstrate the effectiveness of LDA-GCLF.

Classical Linear Discriminant Analysis (LDA) is usually suffers from the small sample size (SSS) problem when dealing with the high dimensional face data. Many methods have been proposed for solving this problem such as Fisherface and Null Space LDA (N-LDA), but these methods are overfitted to the training set and inevitably lose some useful discriminative information in many cases. To effectively utilize nearly all useful discriminative information, a not completely random sampling framework for the integration of multiple features is developed. However, this method has the following main disadvantage: By directly employing feature extraction, the newly constructed variables may contain lots of information originated from those redundant features in the original space. So, in this paper, we introduce a new random sampling LDA by incorporating feature selection for face recognition, that is, some redundant features are removed using the given feature selection methods at first, and then PCA is employed, finally we use random sampling to generate multiple feature subsets. Along this, corresponding weak LDA classifiers are naturally generated and an integrated classifier is developed using a fusion rule. Experiments on 4 face datasets (AR, ORL, Yale, YaleB) show the effectiveness of our algorithm.

Gem-Pro is a new tool for gene mining and functional profiling of bacteria. It initially identifies homologous genes using BLAST and then applies three filtering steps to select orthologous gene pairs. The first one uses BLAST score values to identify trivial paralogs. The second filter uses the shared identity percentages of found trivial paralogs as internal witnesses of non-orthology to set orthology cutoff values. The third filtering step uses conditional probabilities of orthology and non-orthology to define new cutoffs and generate supportive information of orthology assignations. Additionally, a subsidiary tool, called q-GeM, was also developed to mine traits of interest using logistic regression (LR) or linear discriminant analysis (LDA) classifiers. q-GeM is more efficient in the use of computing resources than Gem-Pro but needs an initial classified set of homologous genes in order to train LR and LDA classifiers. Hence, q-GeM could be used to analyze new set of strains with available genome sequences, without the need to rerun a complete Gem-Pro analysis. Finally, Gem-Pro and q-GeM perform a synteny analysis to evaluate the integrity and genomic arrangement of specific pathways of interest to infer their presence. The tools were applied to more than 2 million homologous pairs encoded by Bacillus strains generating statistical supported predictions of trait contents. The different patterns of encoded traits of interest were successfully used to perform a descriptive bacterial profiling.

Automatic age estimation relying on human facial images is a key technology of many real-world applications, which is still a challenging task in the computer vision field. There are three cascade modules for facial age estimation: facial aging feature extraction, dimension reduction (or feature selection) and estimation method. Many existing literatures focus on the first or last module while for an age estimation system, it's also important to construct a reasonable framework. Our work focuses on creating an effective framework by selecting methods for these modules reasonably. Firstly, a BIM (bio-inspired model) is employed to extract facial aging features because it can not only capture discriminative local and global features, but also overcome interferences of some 2D deformations to some extent. Then, LDA (linear discriminant analysis) is used for reducing the BIF (bio-inspired features) to lower dimensions and extracting more discriminative information at the same time. Finally, CS-OHRank (cost-sensitive ordinal hyperplane rank), which tackles with sparse data well and reflects the cumulative attributes of aging, is applied as the estimation method. Experimental results on benchmark dataset FG-NET show that our framework combining BIF, LDA and CS-OHRank is competitive among the state of the art, with MAE (mean absolute error) = 4.72 years..

Background/Objectives: Neurological disorders (ND) are a global health challenge, affecting millions and greatly reducing quality of life. Disorders such as Alzheimer’s disease, mild cognitive impairment (MCI), schizophrenia, and depression often share overlapping symptoms, complicating diagnosis and treatment. Early detection is crucial for timely intervention; however, traditional diagnostic methods rely on subjective assessments and costly imaging, which are not universally accessible. Addressing these challenges, this study investigates the classification of multiple ND using electroencephalography (EEG) signals. Methods: Various feature extraction methods were employed, and the Least Absolute Shrinkage and Selection Operator (Lasso) algorithm was utilized for effective feature selection. Two-class (disease–disease and healthy control–disease), three-class (healthy control and two ND, as well as three ND), and four-class (healthy control and three ND) classifications were conducted using different machine learning algorithms with the selected features. An EEG dataset comprising 40 Alzheimer’s patients, 43 healthy controls, 42 schizophrenia patients, 28 MCI patients, and 28 depression patients served as the experimental benchmark. Results: The Linear Discriminant Analysis (LDA) classifier achieved the highest accuracy, distinguishing between healthy controls and Alzheimer’s with 100% accuracy and demonstrating strong performance in other comparisons. Multi-class classification reached 84.67% accuracy for distinguishing depression, MCI, and schizophrenia, while four-class classification achieved 57.89%, highlighting the complexity of differentiating among multiple ND. The frequent selection of frontal lobe channels across ND indicates their critical role in classification. Conclusions: This study contributes to the literature by emphasizing disease-to-disease classification over the traditional control-versus-patient framework, highlighting the potential for more effective diagnostic tools in clinical settings.

Hyperspectral sensors record the reflectance from the Earth's surface over the full range of solar wavelengths with high spectral resolution. The resulting high-dimensional data contain rich information for a wide range of applications. However, for a specific application, not all the measurements are important and useful. The original feature space may not be the most effective space for representing the data. Feature mining, which includes feature generation, feature selection (FS), and feature extraction (FE), is a critical task for hyperspectral data classification. Significant research effort has focused on this issue since hyperspectral data became available in the late 1980s. The feature mining techniques which have been developed include supervised and unsupervised, parametric and nonparametric, linear and nonlinear methods, which all seek to identify the informative subspace. This paper provides an overview of both conventional and advanced feature reduction methods, with details on a few techniques that are commonly used for analysis of hyperspectral data. A general form that represents several linear and nonlinear FE methods is also presented. Experiments using two widely available hyperspectral data sets are included to illustrate selected FS and FE methods.

Credit risk analysis forms an important criterion for the endurance of the banking sector. Understanding credit liability of customer enhances the security stability of the banker. This research is aimed at comparing the predictive accuracy of data mining techniques with feature selection and extraction techniques in the default of credit card clients and suggesting the most efficient technique for analyzing credit card default. Customer's default payments in Taiwan were used as the dataset. Machine Learning classification algorithms like K-nearest neighbor(KNN), Naive Bayesian(NB), Decision Tree(DT) and Support Vector Machines(SVM) were applied to the dataset. Feature filter methods like information gain (IG), gain ratio (GR) and chi-square (CS) and Wrapper based methods like Best First search(BFS) and Hill Climbing search (HCS) were used to identify minimum features required. Feature extraction methods such as Principal Component Analysis(PCA) and Linear Discriminant Anal-ysis(LDA) were also applied. The minimum features of feature selection algorithms along with PCA and LDA were then applied to Base classifiers and their accuracies were tabulated and compared. SVM with feature selection and extraction techniques had the best predictive accuracy.