Multilinear Spatial Discriminant Analysis for Dimensionality Reduction

Linear Discriminant Analysis (LDA) and Multilinear Principal Component Analysis (MPCA) are leading subspace methods for achieving dimension reduction based on supervised learning. Both LDA and MPCA use class labels of data samples to calculate subspaces onto which these samples are projected. Furthermore, both methods have been successfully applied to face recognition. Although LDA and MPCA share common goals and methodologies, in previous research they have been applied separately and independently. In this paper, we propose an extension of LDA to multiple factor frameworks. Our proposed method, Multifactor Discriminant Analysis, aims to obtain multilinear projections that maximize the between-class scatter while minimizing the withinclass scatter, which is the same core fundamental objective of LDA. Moreover, Multifactor Discriminant Analysis (MDA), like MPCA, uses multifactor analysis and calculates subject parameters that represent the characteristics of subjects and are invariant to other changes, such as viewpoints or lighting conditions. In this way, our proposed MDA combines the best virtues of both LDA and MPCA for face recognition.

Because face images are naturally two-dimensional data, there have been several 2D feature extraction methods to deal with facial images while there are few 2D effective classifiers. Meanwhile, there is an increasing interest in the multilinear subspace analysis and many methods have been proposed to operate directly on these tensorial data during the past several years. One of these popular unsupervised multilinear algorithms is Multilinear Principal Component Analysis (MPCA) while another of the supervised multilinear algorithm is Multilinear Discriminant Analysis (MDA). Then a MPCA+MDA method has been introduced to deal with the tensorial signal. However, due to the no convergence of MDA, it is difficult for MPCA+MDA to obtain a precise result. Hence, to overcome this limitation, a new MPCA plus General Tensor Discriminant Analysis (GTDA) solution with well convergence is presented for tensorial face images feature extraction in this paper. Several experiments are carried out to evaluate the performance of MPCA+GTDA on different databases and the results show that this method has the potential to achieve comparative effect as MPCA+MDA.

Although widely used, Multilinear PCA (MPCA), one of the leading multilinear analysis methods, still suffers from four major drawbacks. First, it is very sensitive to outliers and noise. Second, it is unable to cope with missing values. Third, it is computationally expensive since MPCA deals with large multi-dimensional datasets. Finally, it is unable to maintain the local geometrical structures due to the averaging process. This paper proposes a novel approach named Compressed Submanifold Multifactor Analysis (CSMA) to solve the four problems mentioned above. Our approach can deal with the problem of missing values and outliers via SVD-L1. The Random Projection method is used to obtain the fast low-rank approximation of a given multifactor dataset. In addition, it is able to preserve the geometry of the original data. Our CSMA method can be used efficiently for multiple purposes, e.g. noise and outlier removal, estimation of missing values, biometric applications. We show that CSMA method can achieve good results and is very efficient in the inpainting problem as compared to [1], [2]. Our method also achieves higher face recognition rates compared to LRTC, SPMA, MPCA and some other methods, i.e. PCA, LDA and LPP, on three challenging face databases, i.e. CMU-MPIE, CMU-PIE and Extended YALE-B.

Accurate detection of abnormal myocardium regions is essential for differential diagnosis of cardiovascular disease. However, to achieve this goal by image analysis will significantly increase the burden on radiologists who is already overwhelmed. To ease the time and energy-demanding process and enhance the reproducibility, we proposed a novel framework for automatic abnormal shape detection on left ventricular (LV) using MR images. Our proposed approach utilizes the features obtained by large deformation diffeomorphic metric mapping (LDDMM). To take advantage of 3D structural information, we introduce multilinear principal component analysis (MPCA) in the framework to reduce feature dimensions. Then we combine MPCA with linear discriminant analysis (LDA) to perform differential diagnosis. The performance of proposed framework is evaluated on patients’ images. In the classification of three common cardiovascular diseases, our proposed method outperformed traditional classifiers (Global point signature, Random Forest and XGBoost) with an accuracy of 94%. To further automatically detect the dysfunctional heart regions, we did a comparison on 3D morphology between the diseased subjects and healthy controls and performed an automatic visualization of the abnormal myocadiac regions. In conclusion, our proposed framework reserves the spatial information of the features generated through LDDMM registration and enables the 3D visualization of abnormal regions of LV. With the advance of our method, differential diagnosis is successfully performed on patients with different cardiovascular diseases.

Over the years, many dimensionality reduction algorithms have been proposed for learning the structure of high dimensional data by linearly or non-linearly transforming it into a low-dimensional space. Some techniques can keep the local structure of data, while the others try to preserve the global structure. In this paper, we propose a linear dimensionality reduction technique that characterizes the local and global properties of data by firstly applying k-means algorithm on original data, and then finding the projection by simultaneously globally maximizing the between-cluster scatter matrix and locally minimizing the within-cluster scatter matrix, which actually keeps both local and global structure of data. Low complexity and structure preserving are two main advantages of the proposed technique. The experiments on both artificial and real data sets show the effectiveness and novelty of proposed algorithm in visualization and classification tasks.

Subspace clustering has become widely adopted for the unsupervised analysis of hyperspectral images (HSIs). Recent model-aware deep subspace clustering methods often use a two-stage framework, involving the calculation of a self-representation matrix with complexity of O(n2), followed by spectral clustering. However, these methods are computationally intensive, generally incorporating only local or non-local structure constraints, and their structural constraints fall short of effectively supervising the entire clustering process. We propose a scalable, context-preserving deep clustering method based on basis representation, which jointly captures local and non-local structures for efficient HSI clustering. To preserve local structure—i.e., spatial continuity within subspaces—we introduce a spatial smoothness constraint that aligns clustering predictions with their spatially filtered versions. For non-local structure—i.e., spectral continuity—we employ a mini-cluster-based scheme that refines predictions at the group level, encouraging spectrally similar pixels to belong to the same subspace. These two constraints are jointly optimized to reinforce each other. Specifically, our model is designed as a one-stage approach, in which the structural constraints are applied to the entire clustering process. The time and space complexity of our method are O(n), making it applicable to large-scale HSI data. Experiments on real-world datasets show that our method outperforms state-of-the-art techniques.

This paper introduces a pioneering Hybrid Parallel Multi-linear Face Recognition algorithm that capitalizes on multi-linear methodologies, such as Multi-linear Principal Component Analysis (MPCA), Linear Discriminant Analysis (LDA), and Histogram of Oriented Gradients (HOG), to attain exceptional recognition performance. The Hybrid Feature Selection (HFS) algorithm is meticulously crafted to augment the classification performance on the CK+ and FERET datasets by amalgamating the strengths of feature extraction techniques and feature selection methods. HFS seamlessly incorporates Principal Component Analysis (PCA), Local Discriminant Analysis (LDA), and HOG. The primary aim of this algorithm is to autonomously identify a subset of the most distinctive features from the extracted feature pool, thus elevating classification accuracy, precision, recall, and F1-Score. By amalgamating these methodologies, the algorithm adeptly diminishes dimensionality while conserving pivotal features. Experimental trials on facial image datasets, CK+ and FERET, underscore the algorithm's supremacy in terms of accuracy and computational efficiency when contrasted with conventional linear techniques and even certain deep learning approaches. The proposed algorithm proffers an encouraging solution for real-world face recognition applications where precision and operational efficiency are of paramount significance.

Multiple kernel locality-constrained collaborative representation-based discriminant projection for face recognition

This paper presents a novel approach to solve the supervised dimensionality reduction problem and feature extraction by encoding an image object as a general tensor of 2-D/3-D order. In this paper a multilinear principal component analysis (MPCA) for tensor object feature extraction and then a multilinear discriminant analysis (MDA), to find the best subspaces have been proposed. It should be noted that both of the algorithms work with tensor objects so the structure of the objects has been never broken. Therefore we achieve a better result than the result of the traditional methods. The focus of these algorithms is avoiding the curse of dimensionality. Finally, a comprehensive experiments on ORL and FERET databases has been provided by encoding face images as 2-D or 3-D tensors to demonstrate that MPCA+MDA algorithm based on higher order tensors has the potential to outperform the traditional vector-based subspace learning algorithms such as Eigenface and Fisherface, especially in the cases with small sample sizes and curse of dimensionality dilemma.

The linear discriminant analysis (LDA) is a very popular linear feature extraction approach. The algorithms of LDA usually perform well under the following two assumptions. The first assumption is that the global data structure is consistent with the local data structure. The second assumption is that the input data classes are Gaussian distributions. However, in real-world applications, these assumptions are not always satisfied. In this paper, we propose an improved LDA framework, the local LDA (LLDA), which can perform well without needing to satisfy the above two assumptions. Our LLDA framework can effectively capture the local structure of samples. According to different types of local data structure, our LLDA framework incorporates several different forms of linear feature extraction approaches, such as the classical LDA and principal component analysis. The proposed framework includes two LLDA algorithms: a vector-based LLDA algorithm and a matrix-based LLDA (MLLDA) algorithm. MLLDA is directly applicable to image recognition, such as face recognition. Our algorithms need to train only a small portion of the whole training set before testing a sample. They are suitable for learning large-scale databases especially when the input data dimensions are very high and can achieve high classification accuracy. Extensive experiments show that the proposed algorithms can obtain good classification results.

Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approaches that preserve only the global data structure, such as principal component analysis (PCA), are usually sensitive to outliers. Approaches that preserve only the local data structure, such as locality preserving projections, are usually unsupervised (and hence cannot use label information) and uses a fixed similarity graph. We propose a novel linear dimensionality reduction approach, supervised discriminative sparse PCA with adaptive neighbors (SDSPCAAN), to integrate neighborhood-free supervised discriminative sparse PCA and projected clustering with adaptive neighbors. As a result, both global and local data structures, as well as the label information, are used for better dimensionality reduction. Classification experiments on nine high-dimensional datasets validated the effectiveness and robustness of our proposed SDSPCAAN.

The paper deals with the determination of the symmetric model and proper boundary conditions for solving nonlocal elastic symmetric structures. The above concepts, in the context of nonlocal integral elasticity, turn out to be different with respect to the standard ones, classically applied when dealing with local elastic symmetric structures. Indeed, when only a symmetric portion of the structure is analyzed, the nonlocal effects induced by the remaining (cut) portions are lost, this necessitates the consideration of an enlarged symmetric model on which appropriate nonlocal boundary conditions have to be imposed. It has to be pointed out how the width of such an enlarged model depends on the nonlocal material parameters, while the correct unknown nonlocal boundary conditions are here obtained and enforced by an iterative procedure. The accuracy of the proposed approach in solving nonlocal structural symmetric problems is tested with the aid of two numerical examples.

Dimensionality reduction plays a fundamental role in data processing, for which principal component analysis (PCA) is widely used. In this paper, we develop the Laplacian PCA (LPCA) algorithm which is the extension of PCA to a more general form by locally optimizing the weighted scatter. In addition to the simplicity of PCA, the benefits brought by LPCA are twofold: the strong robustness against noise and the weak metric-dependence on sample spaces. The LPCA algorithm is based on the global alignment of locally Gaussian or linear subspaces via an alignment technique borrowed from manifold learning. Based on the coding length of local samples, the weights can be determined to capture the local principal structure of data. We also give the exemplary application of LPCA to manifold learning. Manifold unfolding (non-linear dimensionality reduction) can be performed by the alignment of tangential maps which are linear transformations of tangent coordinates approximated by LPCA. The superiority of LPCA to PCA and kernel PCA is verified by the experiments on face recognition (FRGC version 2 face database) and manifold (Scherk surface) unfolding.

Ridge Regression (RR) is a classical method that is widely used in multiple regression analysis. However, traditional RR does not take the local geometric structure of data into consideration for discriminative learning and it is sensitive to outliers as it is based on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_{2}$</tex-math></inline-formula> -norm. To address this problem, this article proposes a novel method called Joint Sparse Locality Preserving Regression (JSLPR) for discriminative learning. JSLPR not only applies <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_{2,1}$</tex-math></inline-formula> -norm on both loss function and regularization term but also takes the local geometric structure of the data into consideration. The use of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$L_{2,1}$</tex-math></inline-formula> -norm can guarantee the robustness to outliers or noises and the joint sparsity for effective feature selection. Taking the local geometric structure into consideration can improve the performance of the feature extraction and selection method when the data lie on a manifold. To solve the optimization problem of JSLPR, an iterative algorithm is proposed and the convergence of the algorithm is also proven. Experiments on four famous face databases are conducted and the results show the merit of the proposed JSLPR on feature extraction and selection.

Least squares regression (LSR) is a fundamental statistical analysis technique that has been widely applied to feature learning. However, limited by its simplicity, the local structure of data is easy to neglect, and many methods have considered using orthogonal constraint for preserving more local information. Another major drawback of LSR is that the loss function between soft regression results and hard target values cannot precisely reflect the classification ability; thus, the idea of the large margin constraint is put forward. As a consequence, we pay attention to the concepts of large margin and orthogonal constraint to propose a novel algorithm, orthogonal least squares regression with large margin (OLSLM), for multiclass classification in this letter. The core task of this algorithm is to learn regression targets from data and an orthogonal transformation matrix simultaneously such that the proposed model not only ensures every data point can be correctly classified with a large margin than conventional least squares regression, but also can preserve more local data structure information in the subspace. Our efficient optimization method for solving the large margin constraint and orthogonal constraint iteratively proved to be convergent in both theory and practice. We also apply the large margin constraint in the process of generating a sparse learning model for feature selection via joint [Formula: see text]-norm minimization on both loss function and regularization terms. Experimental results validate that our method performs better than state-of-the-art methods on various real-world data sets.