Discriminative low-rank preserving projection for dimensionality reduction

Dimensionality reduction is an essential task in hyperspectral image processing. How to preserve the original intrinsic structure information and enhance the discriminant ability is still a challenge in this area. Recently, with the advantage of preserving global intrinsic structure information, low-rank representation has been applied to dimensionality reduction and achieved promising performance. By exploiting the submanifold information of the original data set, multimanifold learning is effective in enhancing the discriminant ability of the processed data set. In addition, due to the ability of preserving the spatial neighborhood structure information, the tensor analysis has become a popular technique for hyperspectral image processing. Motivated by the above-mentioned analysis, a novel tensor-based low-rank graph with multimanifold regularization (T-LGMR) for dimensionality reduction of hyperspectral images is proposed in this paper. In the T-LGMR, a low-rank constraint is employed to preserve the global data structure while multimanifold information is utilized to enhance the discriminant ability, and tensor representation is used to preserve the spatial neighborhood information. Finally, dimensionality reduction is achieved in the graph embedding framework. Experimental results on three real hyperspectral data sets demonstrate the superiority of the proposed method over several state-of-the-art approaches.

Feature extraction (FE) methods based on low-rank representation (LRR) have become important topics in hyperspectral images (HSIs) data analysis. In this paper, a supervised FE method for HSIs data based on LRR with the ability to preserve the local pairwise constraints information (LRLPC) is proposed. LRLPC does not change the data dimensionality and only employs a technique to enrich the original feature space (OFS) and to obtain enriched feature space, which results in features richer than OFS. To overcome the problem of LRR in lacking the local structure information (LSI) of data, a local discriminative regularization term is imposed on the fitness function of LRR to keep the LSI of data. For nonlinear structure of data, LRLPC is extended to kernel LRLPC (KLRLPC) using kernel trick. Utilization of existing information in the pairwise constraints is useful for limited labeled samples situations as a common problem in HSI data analysis. The obtained experimental results using two well-known HSI data sets confirm the effectiveness of LRLPC and KLRLPC for dimension reduction and classification of HSIs.

Nonnegative low-rank representation based manifold embedding for semi-supervised learning

Discriminative feature extraction based on sparse and low-rank representation

Feature extraction methods in the industrial process may affect the performance of process monitoring and fault assessment. In order to reduce the influence of model preference, a novel fault assessment method integrating global and local structure information is proposed by using the strategy of parallel extraction and weighted combination. First, kernel sparse locally linear embedding and kernel principal component analysis models are constructed to handle nonlinear system states and describe the local and global information in parallel. Second, the fault assessment indicators are obtained through support vector data description models. Then, a comprehensive fault assessment index is established by the weighted fusion method, which can keep the global and local neighbor information of the original sample space, and thus more feature information is included. Finally, the feasibility and effectiveness of the proposed assessment method are illustrated by the application in the simulation process of Tennessee Eastman Process, and the results show that the fusion of two basic models can bring a model with better generalization performance than a single model.

Feature extraction methods have been successfully applied to many real-world applications. The classical Linear Discriminant Analysis (LDA) and its variants are widely used as feature extraction methods. Although they have been used for different classification tasks, these methods have some shortcomings. The main one is that the projection axes obtained are not informative about the relevance of original features. In this paper, we propose a linear embedding method that merges two interesting properties: Robust LDA and inter-class sparsity. Furthermore, the targeted projection transformation focuses on the most discriminant original features, The proposed method is called Robust Discriminant Analysis with Feature Selection and Inter-class Sparsity (RDA FSIS). Two kinds of sparsity are explicitly included in the proposed model. The first kind is obtained by imposing the `2;1 constraint on the projection matrix in order to perform feature ranking. The second kind is obtained by imposing the inter-class sparsity constraint used for getting a common sparsity structure in each class. Comprehensive experiments on five real-world image datasets demonstrate the effectiveness and advantages of our framework over existing linear methods.

Linear Discriminant Analysis (LDA) is a popular statistical method for both feature extraction and dimensionality reduction in face recognition. The major drawback of LDA is the so-called small sample size (3S) problem. This problem always occurs when the total number of training samples is smaller than the dimension of feature space. Under this situation, the within-class scatter matrix Sw becomes singular and LDA approach cannot be implemented directly. To overcome the 3S problem, this paper proposes a novel wavelet-face based subspace LDA algorithm. Wavelet-face feature extraction and dimensionality reduction are based on two-level D4-filter wavelet transform and discarding the null space of total class scatter matrix St. It is shown that our obtained projection matrix satisfies the uncorrelated constraint conditions. Hence in the sense of statistical uncorrelation, this projection matrix is optimal. The proposed method for face recognition has been evaluated with two public available databases, namely ORL and FERET databases. Comparing with existing LDA-based methods to solve the 3S problem, our method gives the best performance.

Low-rank representation with adaptive graph regularization

With the development of cross-modal image fusion in multi-sensor systems, current fusion technologies have made significant progress in feature extraction, facilitating more effective image analysis. However, insufficient fusion information may degrade the correlation between the source and fused images, often resulting in the omission of critical features from the original modalities. Therefore, in order to preserve as much information as possible, especially for the complete extraction of effective feature information in source images, this paper proposes a new cross-modal image fusion method based on low-rank representation and convolutional sparse learning named LRCFuse. Firstly, the learned low-rank representation (LLRR) blocks are employed to perform dimensionality reduction on the source images while simultaneously extracting their low-rank and sparse feature components. Nevertheless, considering that the low-rank representation has insufficient modeling ability for different modal images, we introduce common feature preservation module (CFPM) blocks based on convolutional sparse coding. By leveraging the CFPM module, LRCFuse recovers common features from both source images to mitigate the loss caused by the imperfect assumptions of low-rank representation. Based on this, a multi-level optimization strategy incorporating pixel loss, shallow-level loss, mid-level loss, deep-level loss, and sobel loss is proposed to hierarchically learn and refine diverse image features. Quantitative and qualitative evaluations are conducted across various datasets, revealing that LRCFuse can effectively detect targets infrared salient targets, preserve additional details in visible images, and achieve better fusion results for subsequent downstream tasks.

Graph-based dimensionality reduction (DR) methods have been applied successfully in many practical problems, such as face recognition, where graphs play a crucial role in modeling the data distribution or structure. However, the ideal graph is, in practice, difficult to discover. Usually, one needs to construct graph empirically according to various motivations, priors, or assumptions; this is independent of the subsequent DR mapping calculation. Different from the previous works, in this paper, we attempt to learn a graph closely linked with the DR process, and propose an algorithm called dimensionality reduction with adaptive graph (DRAG), whose idea is to, during seeking projection matrix, simultaneously learn a graph in the neighborhood of a prespecified one. Moreover, the pre-specified graph is treated as a noisy observation of the ideal one, and the square Frobenius divergence is used to measure their difference in the objective function. As a result, we achieve an elegant graph update formula which naturally fuses the original and transformed data information. In particular, the optimal graph is shown to be a weighted sum of the pre-defined graph in the original space and a new graph depending on transformed space. Empirical results on several face datasets demonstrate the effectiveness of the proposed algorithm.

Low-rank inter-class sparsity based semi-flexible target least squares regression for feature representation

The issue of single sample per person (SSPP) face recognition has attracted more and more attention in recent years. Patch/local-based algorithm is one of the most popular categories to address the issue, as patch/local features are robust to face image variations. However, the global discriminative information is ignored in patch/local-based algorithm, which is crucial to recognize the nondiscriminative region of face images. To make the best of the advantage of both local information and global information, a novel two-layer local-to-global feature learning framework is proposed to address SSPP face recognition. In the first layer, the objective-oriented local features are learned by a patch-based fuzzy rough set feature selection strategy. The obtained local features are not only robust to the image variations, but also usable to preserve the discrimination ability of original patches. Global structural information is extracted from local features by a sparse autoencoder in the second layer, which reduces the negative effect of nondiscriminative regions. Besides, the proposed framework is a shallow network, which avoids the over-fitting caused by using multilayer network to address SSPP problem. The experimental results have shown that the proposed local-to-global feature learning framework can achieve superior performance than other state-of-the-art feature learning algorithms for SSPP face recognition.

As an emerging paradigm, Mobile CrowdSensing (MCS) network based fingerprint positioning technology can implement the site survey cheap and fast, and provide a more reliable location-based service for users. However, limited by the differences among individuals, the fingerprints collected from MCS network are troubled by fingerprint ineffectiveness or element missing problems. In order to ensure the availability of fingerprints, in this paper we propose a novel Low-rank and Sparse representation based Tensor Completion (LSTC) fingerprint recovery method to recover the ineffective fingerprints and the missing fingerprint elements accurately in MCS network. Specifically, the tensor low-rank characterization is used to exploit the global structure of the location fingerprints, and the tensor sparsity characterization is used to exploit the local structure of the location fingerprint elements. To promote the global optimal solution solving rapidly, a weight compensation scheme is proposed to fill the convex relaxation gap caused by the approximation of low-rank and sparsity. Meanwhile, to represent the local sparse characterization effectively, orthogonal dictionary learning is introduced and integrated into LSTC. The experimental results show that by using the tensor global and local structure information properly, the proposed LSTC can recover the fingerprint effectively, without jeopardizing the positioning accuracy.

Semisupervised Discriminant Analysis (SDA) aims at dimensionality reduction with both limited labeled data and copious unlabeled data, but it may fail to discover the intrinsic geometry structure when the data manifold is highly nonlinear. The kernel trick is widely used to map the original nonlinearly separable problem to an intrinsically larger dimensionality space where the classes are linearly separable. Inspired by low-rank representation (LLR), we proposed a novel kernel SDA method called low-rank kernel-based SDA (LRKSDA) algorithm where the LRR is used as the kernel representation. Since LRR can capture the global data structures and get the lowest rank representation in a parameter-free way, the low-rank kernel method is extremely effective and robust for kinds of data. Extensive experiments on public databases show that the proposed LRKSDA dimensionality reduction algorithm can achieve better performance than other related kernel SDA methods.

Partial least squares discriminant analysis (PLS-DA) is gaining popularity as a discriminant feature extraction tool. It is often viewed as a “supervised” version of the principle component analysis (PCA) where dimensionality reduction is achieved with full awareness of the class labels. It is well known that as a dimensionality reduction tool, PCA only captures the global geometric structure of a data set and this characteristics of PCA is inherited in PLS-DA. In this paper we propose a locality preserving PLS-DA (LPPLS-DA) in an effort to enhance the discriminant capabilities of PLS-DA. Our proposed LPPLS-DA incorporates neighborhood information of the data by means of a similarity matrix. We investigate the use of LPPLS-DA in spoken letter recognition. Finding relevant features for the classification of spoken letters is affected by high dimensionality of the data and the sound (phonetic) similarities between the letters. To address this problem, we used LPPLS-DA to extract relevant features of the data and discrimination is automatically achieved using the LPPLS-DA method. The effectiveness of LPPLS-DA is demonstrated by experimental results on the ISOLET (Isolated Letter Speech Recognition) data set. The classification accuracies in the experiments are measured using the nearest neighbor classifier. The LPPLS-DA method is shown to give higher accuracies than PLS-DA as well as several other feature extraction methods.