Uncorrelated trace ratio linear discriminant analysis for undersampled problems

Face recognition using direct, weighted linear discriminant analysis and modular subspaces

Face recognition using direct, weighted linear discriminant analysis and modular subspaces

The classification of tissue samples based on gene expression data is an important problem in medical diagnosis of diseases such as cancer. In gene expression data, the number of genes is usually very high (in the thousands) compared to the number of data samples (in the tens or low hundreds); that is, the data dimension is large compared to the number of data points (such data is said to be undersampled). To cope with performance and accuracy problems associated with high dimensionality, it is commonplace to apply a preprocessing step that transforms the data to a space of significantly lower dimension with limited loss of the information present in the original data. Linear Discriminant Analysis (LDA) is a well-known technique for dimension reduction and feature extraction, but it is not applicable for undersampled data due to singularity problems associated with the matrices in the underlying representation. This paper presents a dimension reduction and feature extraction scheme, called Uncorrelated Linear Discriminant Analysis (ULDA), for undersampled problems and illustrates its utility on gene expression data. ULDA employs the Generalized Singular Value Decomposition method to handle undersampled data and the features that it produces in the transformed space are uncorrelated, which makes it attractive for gene expression data. The properties of ULDA are established rigorously and extensive experimental results on gene expression data are presented to illustrate its effectiveness in classifying tissue samples. These results provide a comparative study of various state-of-the-art classification methods on well-known gene expression data sets.

Linear Discriminant Analysis (LDA) is a popular statistical method for both feature extraction and dimensionality reduction in face recognition. The major drawback of LDA is the so-called small sample size (3S) problem. This problem always occurs when the total number of training samples is smaller than the dimension of feature space. Under this situation, the within-class scatter matrix Sw becomes singular and LDA approach cannot be implemented directly. To overcome the 3S problem, this paper proposes a novel wavelet-face based subspace LDA algorithm. Wavelet-face feature extraction and dimensionality reduction are based on two-level D4-filter wavelet transform and discarding the null space of total class scatter matrix St. It is shown that our obtained projection matrix satisfies the uncorrelated constraint conditions. Hence in the sense of statistical uncorrelation, this projection matrix is optimal. The proposed method for face recognition has been evaluated with two public available databases, namely ORL and FERET databases. Comparing with existing LDA-based methods to solve the 3S problem, our method gives the best performance.

Analysis of EMG signal has been an interested topic in recent years for classifying surface myoelectric signal patterns. Myoelectric control is an unconventional method to control the upper limb prostheses, human-assisting robots and rehabilitation devices. The aim of present work is to assess the time-domain features of EMG signal for myoelectric control of upper extremity prostheses by utilizing scatter plot. Classification accuracy is calculated using linear discriminant classifier for different combination of feature vectors using principal component analysis (PCA) and uncorrelated linear discriminant analysis (ULDA) feature reduction techniques. Results show that willison amplitude and waveform length are the best features for separating the distinct upper-limb motions.

This paper presents a novel and interesting combination of uncorrelated linear discriminant analysis and wavelets to extract features for face recognition. The proposed algorithm when compared with conventional Fisherface method and ULDA+PCA method has an improved recognition rate and a decrease of computational load for face images with high resolutions. In the proposed technique, the face images are divided into smaller sub-images by 2-D DWT and the uncorrelated linear discriminant analysis is applied to approximations sub-images. The time-cost of the proposed method is greatly reduced and recognition rates ranging between 95% and 97.5% are obtained on the ORL database. An average error rate of 1.4% is obtained with the experiments on the NUST603 database. In addition, the effect of number of discriminant vectors on the recognition system is systematically discussed.

Feature selection plays a crucial role in scientific research and practical applications. In the real world applications, labeling data is time and labor consuming. Thus, unsupervised feature selection methods are desired for many practical applications. Linear discriminant analysis (LDA) with trace ratio criterion is a supervised dimensionality reduction method that has shown good performance to improve classifications. In this paper, we first propose a unified objective to seamlessly accommodate trace ratio formulation and K-means clustering procedure, such that the trace ratio criterion is extended to unsupervised model. After that, we propose a novel unsupervised feature selection method by integrating unsupervised trace ratio formulation and structured sparsity-inducing norms regularization. The proposed method can harness the discriminant power of trace ratio criterion, thus it tends to select discriminative features. Meanwhile, we also provide two important theorems to guarantee the unsupervised feature selection process. Empirical results on four benchmark data sets show that the proposed method outperforms other sate-of-the-art unsupervised feature selection algorithms in all three clustering evaluation metrics.

Orthogonal Linear Discriminant Analysis (OLDA) has been widely studied for dimensionality reduction, as the orthogonality constraints on its projection matrix enable more effective elimination of redundant features compared to conventional Linear Discriminant Analysis (LDA). However, most existing methods for solving OLDA rely on iterative optimization to sequentially construct orthogonal components, incurring massive repeated high-cost matrix operations and thus leading to substantial computational inefficiency. To address this limitation, we propose QR decomposition-based Fast OLDA (QR-FOLDA), a method built upon a theoretical result (Theorem 1) established in this work: the optimal solutions of LDA remain valid under any full-rank linear transformation. By leveraging this property, QR-FOLDA applies QR decomposition directly to the optimal LDA solution, thereby enforcing orthogonality while avoiding the repeated high-cost matrix operations typically involved in iterative optimization procedures. Experimental evaluations conducted on nine real-world datasets across various domains show that QR-FOLDA not only achieves substantial improvements in computational efficiency compared to existing OLDA methods but also delivers superior classification performance. These findings position QR-FOLDA as a theoretically sound and practically efficient solution for orthogonal discriminant analysis.

Academic Information System (Sistem Informasi Akademik aka SIAKAD) Online of Universitas Prima Indonesia (UNPRI) is one of the applications used to facilitate the administration process of lectures which includes the filling process of study plan cards (Kartu Rencana Studi aka KRS), study result cards (Kartu Hasil Studi aka KHS), class schedules, submission of research titles, seminars, and other processes. SIAKAD UNPRI can be accessed by students, lecturers, and academics where every user has a password that has been encrypted to maintain the security of information from people who are not responsible, password security using the encryption method needs to be changed regularly, but there are still many students, lecturers and academic community who are reluctant to change passwords. To improve the security verification stage for SIAKAD users, we propose a face recognition feature approach. Face recognition is a feature that allows the identification of someone from a digital image or video. The way the facial recognition method works is by comparing face data from the camera or images with images that were previously stored in a database. In this study, the Kohonen SOM method is proposed for face identification based on the feature extraction approach of discrete cosine transform (DCT), linear discriminant analysis (LDA) and principal component analysis (PCA) to improve the security of UNPRI SIAKAD users. The analytical framework is done by requiring students to do face taking, where each student will save 5 (five) faces extracted with facial features using the DCT, LDA and PCA model approach, feature extraction results are used as input to the Kohonen SOM network for training and testing facial recognition, then analysis of the effect of DCT, LDA and PCA feature extraction on the Kohonen network on facial recognition accuracy.

This is the author’s version of a work that was submitted/accepted for pub-lication in the following source:

Electronic health records (EHR) provide a rich source of temporal data that present a unique opportunity to characterize disease patterns and risk of imminent disease. While many data-mining tools have been adopted for EHR-based disease early detection, linear discriminant analysis (LDA) is one of the most commonly used statistical methods. However, it is difficult to train an accurate LDA model for early disease diagnosis when too few patients are known to have the target disease. Furthermore, EHR data are heterogeneous with significant noise. In such cases, the covariance matrices used in LDA are usually singular and estimated with a large variance. This article presents Daehr , an extension of the LDA framework using electronic health record data to address these issues. Beyond existing LDA analyzers, we propose Daehr to (1) eliminate the data noise caused by the manual encoding of EHR data and (2) lower the variance of parameter (covariance matrices) estimation for LDA models when only a few patients’ EHR are available for training. To achieve these two goals, we designed an iterative algorithm to improve the covariance matrix estimation with embedded data-noise/parameter-variance reduction for LDA. We evaluated Daehr extensively using the College Health Surveillance Network, a large, real-world EHR dataset. Specifically, our experiments compared the performance of LDA to three baselines (i.e., LDA and its derivatives) in identifying college students at high risk for mental health disorders from 23 U.S. universities. Experimental results demonstrate Daehr significantly outperforms the three baselines by achieving 1.4%--19.4% higher accuracy and a 7.5%--43.5% higher F1-score.

Discrimination of type 2 diabetes mellitus corresponding to different traditional Chinese medicine syndromes based on plasma fatty acid profiles and chemometric methods

Sequence describes the primary structure of a protein, which contains important structural, characteristic, and genetic information and thereby motivates many sequence-based computational approaches to infer protein function. Among them, feature-base approaches attract increased attention because they make prediction from a set of transformed and more biologically meaningful sequence features. However, original features extracted from sequence are usually of high dimensionality and often compromised by irrelevant patterns, therefore dimension reduction is necessary prior to classification for efficient and effective protein function prediction. A protein usually performs several different functions within an organism, which makes protein function prediction a multi-label classification problem. In machine learning, multi-label classification deals with problems where each object may belong to more than one class. As a well-known feature reduction method, linear discriminant analysis (LDA) has been successfully applied in many practical applications. It, however, by nature is designed for single-label classification, in which each object can belong to exactly one class. Because directly applying LDA in multi-label classification causes ambiguity when computing scatters matrices, we apply a new Multi-label Linear Discriminant Analysis (MLDA) approach to address this problem and meanwhile preserve powerful classification capability inherited from classical LDA. We further extend MLDA by l1-normalization to overcome the problem of over-counting data points with multiple labels. In addition, we incorporate biological network data using Laplacian embedding into our method, and assess the reliability of predicted putative functions. Extensive empirical evaluations demonstrate promising results of our methods.

Linear discriminant analysis (LDA) is a popular technique to learn the most discriminative features for multi-class classification. A vast majority of existing LDA algorithms are prone to be dominated by the class with very large deviation from the others, i.e., edge class, which occurs frequently in multi-class classification. First, the existence of edge classes often makes the total mean biased in the calculation of between-class scatter matrix. Second, the exploitation of ℓ2-norm based between-class distance criterion magnifies the extremely large distance corresponding to edge class. In this regard, a novel self-weighted robust LDA with ℓ2,1-norm based pairwise between-class distance criterion, called SWRLDA, is proposed for multi-class classification especially with edge classes. SWRLDA can automatically avoid the optimal mean calculation and simultaneously learn adaptive weights for each class pair without setting any additional parameter. An efficient re-weighted algorithm is exploited to derive the global optimum of the challenging ℓ2,1-norm maximization problem. The proposed SWRLDA is easy to implement and converges fast in practice. Extensive experiments demonstrate that SWRLDA performs favorably against other compared methods on both synthetic and real-world datasets while presenting superior computational efficiency in comparison with other techniques.

Linear Discriminant Analysis (LDA) is a well-known technique for feature extraction and dimension reduction. The performance of classical LDA however, significantly degrades on the High Dimension Low Sample Size (HDLSS) data for the ill-posed inverse problem. Existing approaches for HDLSS data classification typically assume the data in question are with Gaussian distribution and deal the HDLSS classification problem with regularization. However, these assumptions are too strict to hold in many emerging real-life applications, such as enabling personalized predictive analysis using Electronic Health Records (EHRs) data collected from an extremely limited number of patients who have been diagnosed with or without the target disease for prediction. In this paper, we revised the problem of predictive analysis of disease using personal EHR data and LDA classifier. To fill the gap, in this paper, we first studied an analytical model that understands the accuracy of LDA for classifying data with arbitrary distribution. The model gives a theoretical upper bound of LDA error rate that is controlled by two factors: (1) the statistical convergence rate of (inverse) covariance matrix estimators and (2) the divergence of the training/testing datasets to fitted distributions. To this end, we could lower the error rate by balancing the two factors for better classification performance. Hereby, we further proposed a novel LDA classifier De-Sparse that leverages De-sparsified Graphical Lasso to improve the estimation of LDA, which outperforms state-of-the-art LDA approaches developed for HDLSS data. Such advances and effectiveness are further demonstrated by both theoretical analysis and extensive experiments on EHR datasets https://www.overleaf.com/project/5d2728c718f6ff3b2bcf5991 .