QR-FOLDA: A Fast Orthogonal Linear Discriminant Analysis Based on QR Decomposition

Linear discriminant analysis (LDA) is one of the most popular dimension reduction methods and has been widely used in many applications. In the last decades many LDA-based dimension reduction algorithms have been reported. Among these methods, orthogonal LDA (OLDA) is a famous one and several different implementations of OLDA have been proposed. In this paper, we propose a new and fast implementation of OLDA. Compared with the other OLDA implementations, our proposed implementation of OLDA is the fastest one when the dimensionality d is larger than the sample size n. Then, based on our proposed implementation of OLDA, we present an incremental OLDA algorithm which can accurately update the projection matrix of OLDA when new samples are added into the training set. The effectiveness of our proposed new OLDA algorithm and its incremental version are demonstrated by some real-world data sets.

Fast orthogonal linear discriminant analysis with application to image classification

ABSTRACTHyperspectral image classification is among the most frequent topics of research in recent publications. This paper proposes a new supervised linear feature extraction method for classification of hyperspectral images using orthogonal linear discriminant analysis in both spatial and spectral domains. In fact, an orthogonal filter set and a spectral data transformation are designed simultaneously by maximizing the class separability. The important characteristic of the presented approach is that the proposed filter set is supervised and considers the class separability when extracting the features, thus it is more appropriate for feature extraction compared with other filters such as Gabor. In order to compare the proposed method with some existing methods, the extracted spatial–spectral features are fed into a support vector machine classifier. Some experiments on the widely used hyperspectral images, namely Indian Pines, Pavia University, and Salinas data sets, reveal that the proposed approach leads to state-of-the-art performance when compared to other recent approaches.

High-dimensional feature extraction using bit-plane decomposition of local binary patterns for robust face recognition

Uncorrelated trace ratio linear discriminant analysis for undersampled problems

Linear Discriminant Analysis (LDA) has been one of the popular subspace methods for face recognition. But this method suffers from the small sample size (SSS) problem, also known as 'curse of dimensionality'. Various techniques have been proposed in literature to overcome this limitation. But it is still unclear which method provides the best solution to SSS problem. In this paper, we have investigated the performance of some popular subspace methods such as principal component analysis (PCA), PCA + LDA, LDA via QR decomposition, Null-space LDA, Exponential Discriminant Analysis (EDA), PCA+EDA etc. Extensive experiments have been performed on five publically available face datasets viz. AR, CMU-PIE, PIX, Yale and YaleB. The performance is measured in terms of average classification accuracy. Experimental results show that the performance increases with the increase in the number of images per person in training set irrespective of the datasets. There is no clear winner among the subspace methods under investigation. But, the performance of PCA+LDA and PCA+EDA is consistent in tackling SSS problem irrespective of the dataset and can also handle the illumination variation in face recognition.

Research was conducted to investigate the use of optical, air-born acoustic and plasma charge signals for detecting defects in laser welds. Full penetration, partial penetration and gapped-full penetration butt-joint welds were made in 1.4mm cold-rolled steel sheet. A 3 kW CO2 laser was used to produce these three classes of welds while laser power, travel speed and joint fitup were varied to produce the various conditions. Visible light, audible sound and electrical charge near the plume were measured simultaneously in real-time using a silicon photodiode, a condensor microphone and a plasma charge sensing gas nozzle, respectively. The spectra of the sensor signals were analyzed by a statistical technique known as Linear Discriminant Analysis (LDA). Stepwise LDA was used to identify the most important frequency bands and canonical discriminant analysis was employed to develop discriminant functions. Reliability analysis of the training samples revealed that this LDA method classified laser welds with reliability ranging from 85% to 96%, depending on the welds selected for inclusion in the training set. When the linear discriminant function was validated using new (untrained) samples, it distinguished the three different classes of laser welds with approximately 87% accuracy. Among the three signals, sound produced better separation between the classes of welds, although a discriminator based on all three signals combined and one that used only plasma charge and acoustic signals were of almost equal reliability.Research was conducted to investigate the use of optical, air-born acoustic and plasma charge signals for detecting defects in laser welds. Full penetration, partial penetration and gapped-full penetration butt-joint welds were made in 1.4mm cold-rolled steel sheet. A 3 kW CO2 laser was used to produce these three classes of welds while laser power, travel speed and joint fitup were varied to produce the various conditions. Visible light, audible sound and electrical charge near the plume were measured simultaneously in real-time using a silicon photodiode, a condensor microphone and a plasma charge sensing gas nozzle, respectively. The spectra of the sensor signals were analyzed by a statistical technique known as Linear Discriminant Analysis (LDA). Stepwise LDA was used to identify the most important frequency bands and canonical discriminant analysis was employed to develop discriminant functions. Reliability analysis of the training samples revealed that this LDA method classified laser welds with rel...

Analytical capabilities of chemiresistive microsensor arrays in a simulated Martian atmosphere

Linear discriminant analysis (LDA) aims to find a low-dimensional space in which data points in the same class are to be close to each other while keeping data points from different classes apart. To improve the robustness of LDA to non-Gaussian distribution data, most existing discriminant analysis methods extend LDA by approximating the underlying manifold of data. However, these methods suffer from the following problems: 1) local affinity or reconstruction coefficients are learned on the basis of the relationships of all data pairs, which would lead to a sharp increase in the amount of computation and 2) they learn the manifold information in the original space, ignoring the interference of the noise and redundant features. Motivated by these challenges, this article represents a novel discriminant analysis model, called fast and adaptive locality discriminant analysis (FALDA), to improve the efficiency and robustness. First, with the anchor-based strategy, a bipartite graph of each class is constructed to characterize the local structure of data. Since the number of anchor points is far less than that of data points, learning of fuzzy membership relationships between data points and anchor points within each class can save training time. Second, a maximum entropy regularization is introduced to control the uniformity of the weights of graphs and avoid the trivial solution. Third, the above relationships are updated adaptively in the process of dimensionality reduction, which can suppress the interference of the noise and redundant features. Fourth, the whitening constraint is imposed on the projection matrix to remove the relevance between features and restrict the total scatter of data in the subspace. Last but not the least, data with complex distribution can be explicitly divided into sub-blocks according to the learned anchor points (or subclass center points). We test our proposed method on synthetic data, benchmark datasets, and imbalanced datasets. Promising experimental results demonstrate the success of this novel model.

Linear Discriminant Analysis (LDA) and Multilinear Principal Component Analysis (MPCA) are leading subspace methods for achieving dimension reduction based on supervised learning. Both LDA and MPCA use class labels of data samples to calculate subspaces onto which these samples are projected. Furthermore, both methods have been successfully applied to face recognition. Although LDA and MPCA share common goals and methodologies, in previous research they have been applied separately and independently. In this paper, we propose an extension of LDA to multiple factor frameworks. Our proposed method, Multifactor Discriminant Analysis, aims to obtain multilinear projections that maximize the between-class scatter while minimizing the withinclass scatter, which is the same core fundamental objective of LDA. Moreover, Multifactor Discriminant Analysis (MDA), like MPCA, uses multifactor analysis and calculates subject parameters that represent the characteristics of subjects and are invariant to other changes, such as viewpoints or lighting conditions. In this way, our proposed MDA combines the best virtues of both LDA and MPCA for face recognition.

A novel Bayesian logistic discriminant model: An application to face recognition

Electronic health records (EHR) provide a rich source of temporal data that present a unique opportunity to characterize disease patterns and risk of imminent disease. While many data-mining tools have been adopted for EHR-based disease early detection, linear discriminant analysis (LDA) is one of the most commonly used statistical methods. However, it is difficult to train an accurate LDA model for early disease diagnosis when too few patients are known to have the target disease. Furthermore, EHR data are heterogeneous with significant noise. In such cases, the covariance matrices used in LDA are usually singular and estimated with a large variance. This article presents Daehr , an extension of the LDA framework using electronic health record data to address these issues. Beyond existing LDA analyzers, we propose Daehr to (1) eliminate the data noise caused by the manual encoding of EHR data and (2) lower the variance of parameter (covariance matrices) estimation for LDA models when only a few patients’ EHR are available for training. To achieve these two goals, we designed an iterative algorithm to improve the covariance matrix estimation with embedded data-noise/parameter-variance reduction for LDA. We evaluated Daehr extensively using the College Health Surveillance Network, a large, real-world EHR dataset. Specifically, our experiments compared the performance of LDA to three baselines (i.e., LDA and its derivatives) in identifying college students at high risk for mental health disorders from 23 U.S. universities. Experimental results demonstrate Daehr significantly outperforms the three baselines by achieving 1.4%--19.4% higher accuracy and a 7.5%--43.5% higher F1-score.

Convolution helps spiking neural networks (SNNs) capture the spatio-temporal structures of neuromorphic (event) data as evident in the convolution-based SNNs (C-SNNs) with the state-of-the-art classification-accuracies on various datasets. However, the efficacy aside, the efficiency of C-SNN is questionable. In this regard, we propose SNNs with novel trainable dynamic time-surfaces (DTS-SNNs) as efficient alternatives to convolution. The novel dynamic time-surface proposed in this work features its high responsiveness to moving objects given the use of the zero-sum temporal kernel that is motivated by the simple cells’ receptive fields in the early stage visual pathway. We evaluated the performance and computational complexity of our DTS-SNNs on three real-world event-based datasets (DVS128 Gesture, Spiking Heidelberg dataset, N-Cars). The results highlight high classification accuracies and significant improvements in computational efficiency, e.g., merely 1.51% behind of the state-of-the-art result on DVS128 Gesture but a × 18 improvement in efficiency.

Adaptive fast local discriminant analysis with whitening transform.

Linear discriminant analysis (LDA), as an effective feature extraction method, has been widely applied in high-dimensional data analysis. However, its discriminative performance is still severely limited by the following factors. First, the restriction on the total number of features available from LDA has seriously limited its application to problems where the feature dimension is much larger than the number of classes. Second, LDA cannot deal with data containing multiple clusters (or subclasses) within a class because it cannot correctly depict the local structure of the data. To alleviate this issue, we propose a locality-preserving multiprojection discriminant analysis (LPMDA) model to extract more discriminative features preserving local structure. Specifically, LPMDA rephrases the objective function of LDA as a convex discriminant analysis framework from the perspective of metric learning, allowing for extracting more features than the number of classes. Furthermore, an auto-optimized graph technique is also integrated into the discriminant analysis framework to explore the local structure of the data. An efficient iterative optimization algorithm is presented to solve LPMDA. Extensive experiments on several benchmark datasets confirm the effectiveness of the proposed method.