Dimension reduction of multimodal data by auto-weighted local discriminant analysis

Adaptive fast local discriminant analysis with whitening transform.

Linear discriminant analysis (LDA) aims to find a low-dimensional space in which data points in the same class are to be close to each other while keeping data points from different classes apart. To improve the robustness of LDA to non-Gaussian distribution data, most existing discriminant analysis methods extend LDA by approximating the underlying manifold of data. However, these methods suffer from the following problems: 1) local affinity or reconstruction coefficients are learned on the basis of the relationships of all data pairs, which would lead to a sharp increase in the amount of computation and 2) they learn the manifold information in the original space, ignoring the interference of the noise and redundant features. Motivated by these challenges, this article represents a novel discriminant analysis model, called fast and adaptive locality discriminant analysis (FALDA), to improve the efficiency and robustness. First, with the anchor-based strategy, a bipartite graph of each class is constructed to characterize the local structure of data. Since the number of anchor points is far less than that of data points, learning of fuzzy membership relationships between data points and anchor points within each class can save training time. Second, a maximum entropy regularization is introduced to control the uniformity of the weights of graphs and avoid the trivial solution. Third, the above relationships are updated adaptively in the process of dimensionality reduction, which can suppress the interference of the noise and redundant features. Fourth, the whitening constraint is imposed on the projection matrix to remove the relevance between features and restrict the total scatter of data in the subspace. Last but not the least, data with complex distribution can be explicitly divided into sub-blocks according to the learned anchor points (or subclass center points). We test our proposed method on synthetic data, benchmark datasets, and imbalanced datasets. Promising experimental results demonstrate the success of this novel model.

The corrugated layer structure bismuth has been successfully tailored into negative thermal expansion along c axis by size effect. Pair distribution function and extended X-ray absorption fine structure are combined to reveal the local structural distortion for nanosized bismuth. The comprehensive method to identify the local structure of nanomaterials can benefit the regulating and controlling of thermal expansion in nanodivices.

Linear Discriminant Analysis (LDA) is a classical supervised dimensionality reduction algorithm. However, LDA focuses more on global structure and overly depends on reliable data labels. For data with outliers and nonlinear structures, LDA cannot effectively capture the true structure of the data. Moreover, the subspace dimension learned by LDA must be smaller than cluster number, which limits its practical applications. To address these issues, we propose a novel unsupervised LDA method that combines centerless K-means and LDA. This method eliminates the need to calculate cluster centroids and improves model robustness. By fusing centerless K-means and LDA into a unified framework and deducing the connection between K-means and manifold learning, this method captures the local manifold structure and discriminative structure. Additionally, the dimensionality of the subspace is not restricted. This method not only overcomes the limitations of traditional LDA but also improves the model’s adaptability to complex data. Extensive experiments on seven datasets demonstrate the effectiveness of the proposed method.

Local Linear Embedding with Adaptive Neighbors

Dimensionality reduction methods compute a mapping from a high-dimensional space to a space with lower dimensions while preserving important information. The idea of hybridizing dimensionality reduction with evolution strategies is that the search in a space that employs a larger dimensionality than the original solution space may be easier. We propose a dimensionality reduction evolution strategy (DRES) based on a self-adaptive (μ, λ)-ES that generates points in a space with a dimensionality higher than the original solution space. After the population has been generated, it is mapped to the solution space with dimensionality reduction (DR) methods, the solutions are evaluated and the best w.r.t. the fitness in the original space are inherited to the next generation. We employ principal component analysis (PCA) as DR method and show a performance tweak on a small set of benchmark problems.

Local structured feature learning with dynamic maximum entropy graph

The aim of data transformation is to transform the original feature space of data into another space with better properties. This is typically combined with dimensionality reduction, so that the dimensionality of the transformed space is smaller. A widely used method for data transformation and dimensionality reduction is Principal Component Analysis (PCA). PCA finds a subspace that explains most of the data variance. While the new PCA feature space has interesting properties, such as removing linear correlation, PCA is an unsupervised method. Therefore, there is no guarantee that the PCA feature space will be the most appropriate for supervised tasks, such as classification or regression. On the other hand, 3-layer Multi Layer Perceptrons (MLP), which are supervised methods, can also be understood as a data transformation carried out by the hidden layer, followed by a classification/regression operation performed by the output layer. Given that the hidden layer is obtained after a supervised training process, it can be considered that it is performing a supervised data transformation. And if the number of hidden neurons is smaller than the input, also dimensionality reduction. Despite this kind of transformation being widely available (any neural network package that allows access to the hidden layer weights can be used), no extensive experimentation on the quality of 3-layer MLP data transformation has been carried out. The aim of this article is to carry out this research for classification problems. Results show that, overall, this transformation offers better results than the PCA unsupervised transformation method.

Linear Discriminant Analysis (LDA) is a popular statistical method for both feature extraction and dimensionality reduction in face recognition. The major drawback of LDA is the so-called small sample size (3S) problem. This problem always occurs when the total number of training samples is smaller than the dimension of feature space. Under this situation, the within-class scatter matrix Sw becomes singular and LDA approach cannot be implemented directly. To overcome the 3S problem, this paper proposes a novel wavelet-face based subspace LDA algorithm. Wavelet-face feature extraction and dimensionality reduction are based on two-level D4-filter wavelet transform and discarding the null space of total class scatter matrix St. It is shown that our obtained projection matrix satisfies the uncorrelated constraint conditions. Hence in the sense of statistical uncorrelation, this projection matrix is optimal. The proposed method for face recognition has been evaluated with two public available databases, namely ORL and FERET databases. Comparing with existing LDA-based methods to solve the 3S problem, our method gives the best performance.

Dimension reduction has been used to visualise the distribution of multidimensional microbiome data, but the composite variables calculated by the dimension reduction methods have not been widely used to investigate the relationship of the human gut microbiome with lifestyle and disease. In the present study, we applied several dimension reduction methods, including principal component analysis, principal coordinate analysis (PCoA), non-metric multidimensional scaling (NMDS), and non-negative matrix factorization, to a microbiome dataset from 186 subjects with symptoms of allergic rhinitis (AR) and 106 controls. All the dimension reduction methods supported that the distribution of microbial data points appeared to be continuous rather than discrete. Comparison of the composite variables calculated from the different dimension reduction methods showed that the characteristics of the composite variables differed depending on the distance matrices and the dimension reduction methods. The first composite variables calculated from PCoA and NMDS with the UniFrac distance were strongly associated with AR (FDR adjusted P = 2.4 × 10–4 for PCoA and P = 2.8 × 10–4 for NMDS), and also with the relative abundance of Bifidobacterium and Prevotella. The abundance of Bifidobacterium was also linked to intake of several nutrients, including carbohydrate, saturated fat, and alcohol via composite variables. Notably, the association between the composite variables and AR was much stronger than the association between the relative abundance of individual genera and AR. Our results highlight the usefulness of the dimension reduction methods for investigating the association of microbial composition with lifestyle and disease in clinical research.

Linear discriminant analysis (LDA), as an effective feature extraction method, has been widely applied in high-dimensional data analysis. However, its discriminative performance is still severely limited by the following factors. First, the restriction on the total number of features available from LDA has seriously limited its application to problems where the feature dimension is much larger than the number of classes. Second, LDA cannot deal with data containing multiple clusters (or subclasses) within a class because it cannot correctly depict the local structure of the data. To alleviate this issue, we propose a locality-preserving multiprojection discriminant analysis (LPMDA) model to extract more discriminative features preserving local structure. Specifically, LPMDA rephrases the objective function of LDA as a convex discriminant analysis framework from the perspective of metric learning, allowing for extracting more features than the number of classes. Furthermore, an auto-optimized graph technique is also integrated into the discriminant analysis framework to explore the local structure of the data. An efficient iterative optimization algorithm is presented to solve LPMDA. Extensive experiments on several benchmark datasets confirm the effectiveness of the proposed method.

Linear discriminant analysis (LDA) is a classical method for discriminative dimensionality reduction. The original LDA may degrade in its performance for non-Gaussian data, and may be unable to extract sufficient features to satisfactorily explain the data when the number of classes is small. Two prominent extensions to address these problems are subclass discriminant analysis (SDA) and mixture subclass discriminant analysis (MSDA). They divide every class into subclasses and re-define the within-class and between-class scatter matrices on the basis of subclass. In this paper we study the issue of how to obtain subclasses more effectively in order to achieve higher class separation. We observe that there is significant overlap between models of the subclasses, which we hypothesise is undesirable. In order to reduce their overlap we propose an extension of LDA, separability oriented subclass discriminant analysis (SSDA), which employs hierarchical clustering to divide a class into subclasses using a separability oriented criterion, before applying LDA optimisation using re-defined scatter matrices. Extensive experiments have shown that SSDA has better performance than LDA, SDA and MSDA in most cases. Additional experiments have further shown that SSDA can project data into LDA space that has higher class separation than LDA, SDA and MSDA in most cases.

Soft label based Linear Discriminant Analysis for image recognition and retrieval

Linear discriminant analysis (LDA) is a classical statistical machine-learning method, which aims to find a linear data transformation increasing class discrimination in an optimal discriminant subspace. Traditional LDA sets assumptions related to the Gaussian class distributions and single-label data annotations. In this article, we propose a new variant of LDA to be used in multilabel classification tasks for dimensionality reduction on original data to enhance the subsequent performance of any multilabel classifier. A probabilistic class saliency estimation approach is introduced for computing saliency-based weights for all instances. We use the weights to redefine the between-class and within-class scatter matrices needed for calculating the projection matrix. We formulate six different variants of the proposed saliency-based multilabel LDA (SMLDA) based on different prior information on the importance of each instance for their class(es) extracted from labels and features. Our experiments show that the proposed SMLDA leads to performance improvements in various multilabel classification problems compared to several competing dimensionality reduction methods.

Hyperspectral imagery generally contains enormous amounts of data due to hundreds of spectral bands. Classification for these high-dimensional data often requires a large set of training samples and enormous processing time. Therefore, dimension reduction methods for hyperspectral data are catching the attention of researchers lately. In this paper, a dimension reduction method based on sparse penalty regularized linear discriminant analysis was experimented on hyperspectral data. Through imposing sparsity regularization penalty on the Fisher's discriminant analysis projection matrix via the optimal scoring technique, sparse linear discriminant vectors can be achieved. Therefore, interpretability of the spectral bands' physical meaning and effective low dimensional data transforming can be achieved simultaneously in the same model. Experimental analysis on the sparsity and efficacy of low dimensional outputs showed that, sparse linear discriminant analysis can yield good classification results and interpretability in spectral domain.