Low-Rank Kernel-Based Semisupervised Discriminant Analysis

Dimensionality reduction is a very important part in the field of face recognition. In view of the problem of the traditional dimensionality reduction methods are inconvenient to select neighbor parameter K and the “dense” characteristic of the low-rank representation coefficient matrix. We presented a method that semi-supervised discriminant analysis via weighted low-rank representation and adaptive neighbor selection (ANSWLR-SDA). First, we uses all the within-class samples to construct the within-class graph which can describe the within-class compactness, and then adaptively chooses the between-class samples to construct the between-class graph which can describe the between-class respectively. On this basis, we use a regularization term by weighted low-rank represented to maintain the global similarity structure of samples. Finally, we carry out the experiments on FERET and yale_faceb databases, and compare this method with the traditional dimensionality reduction methods and the results demonstrate that ANSWLR-SDA method is effectiveness and robust to different types of noise than other state-of-art face recognition method.

BackgroundIdentifying different types of cancer based on gene expression data has become hotspot in bioinformatics research. Clustering cancer gene expression data from multiple cancers to their own class is a significance solution. However, the characteristics of high-dimensional and small samples of gene expression data and the noise of the data make data mining and research difficult. Although there are many effective and feasible methods to deal with this problem, the possibility remains that these methods are flawed.ResultsIn this paper, we propose the graph regularized low-rank representation under symmetric and sparse constraints (sgLRR) method in which we introduce graph regularization based on manifold learning and symmetric sparse constraints into the traditional low-rank representation (LRR). For the sgLRR method, by means of symmetric constraint and sparse constraint, the effect of raw data noise on low-rank representation is alleviated. Further, sgLRR method preserves the important intrinsic local geometrical structures of the raw data by introducing graph regularization. We apply this method to cluster multi-cancer samples based on gene expression data, which improves the clustering quality. First, the gene expression data are decomposed by sgLRR method. And, a lowest rank representation matrix is obtained, which is symmetric and sparse. Then, an affinity matrix is constructed to perform the multi-cancer sample clustering by using a spectral clustering algorithm, i.e., normalized cuts (Ncuts). Finally, the multi-cancer samples clustering is completed.ConclusionsA series of comparative experiments demonstrate that the sgLRR method based on low rank representation has a great advantage and remarkable performance in the clustering of multi-cancer samples.

Classical Linear Discriminant Analysis(LDA) is an effective dimensionality reduction method which preserve class separabiliy by maximizing the between class covariance and simultaneously minimizing the within class covariance. But, the covariance matrix of each class may not be accurately estimated when the given training samples is not sufficient. To overcome this disadvantage of LDA, a novel Semi-supervised Discriminant Analysis(SDA) was introduced, which can effectively utilize the labeled and unlabeled samples. However, SDA still can not fully use the structure information hidden in data, since the samples in a class may be scattered in several regions. Therefore, in this paper, we introduce a novel approach, called Structured Semi-supervised Discriminant Analysis(SSDA), which exploits the data structures hidden in samples in a class via semi-supervised clustering methods and uses them as “different classes” in calculating the between class covariance, hence it is an extension and improvement of SDA. Experiments on 2 artificial datasets and 8 real-life dataset show the effectiveness of our algorithm.

Dimensionality reduction is an essential task in hyperspectral image processing. How to preserve the original intrinsic structure information and enhance the discriminant ability is still a challenge in this area. Recently, with the advantage of preserving global intrinsic structure information, low-rank representation has been applied to dimensionality reduction and achieved promising performance. By exploiting the submanifold information of the original data set, multimanifold learning is effective in enhancing the discriminant ability of the processed data set. In addition, due to the ability of preserving the spatial neighborhood structure information, the tensor analysis has become a popular technique for hyperspectral image processing. Motivated by the above-mentioned analysis, a novel tensor-based low-rank graph with multimanifold regularization (T-LGMR) for dimensionality reduction of hyperspectral images is proposed in this paper. In the T-LGMR, a low-rank constraint is employed to preserve the global data structure while multimanifold information is utilized to enhance the discriminant ability, and tensor representation is used to preserve the spatial neighborhood information. Finally, dimensionality reduction is achieved in the graph embedding framework. Experimental results on three real hyperspectral data sets demonstrate the superiority of the proposed method over several state-of-the-art approaches.

Feature extraction (FE) methods based on low-rank representation (LRR) have become important topics in hyperspectral images (HSIs) data analysis. In this paper, a supervised FE method for HSIs data based on LRR with the ability to preserve the local pairwise constraints information (LRLPC) is proposed. LRLPC does not change the data dimensionality and only employs a technique to enrich the original feature space (OFS) and to obtain enriched feature space, which results in features richer than OFS. To overcome the problem of LRR in lacking the local structure information (LSI) of data, a local discriminative regularization term is imposed on the fitness function of LRR to keep the LSI of data. For nonlinear structure of data, LRLPC is extended to kernel LRLPC (KLRLPC) using kernel trick. Utilization of existing information in the pairwise constraints is useful for limited labeled samples situations as a common problem in HSI data analysis. The obtained experimental results using two well-known HSI data sets confirm the effectiveness of LRLPC and KLRLPC for dimension reduction and classification of HSIs.

In this paper, in terms of pairwise constraints which specify whether a pair of instances belong to the same class (must-link constraints) or different classes (cannot-link constraints), we propose a novel semi-supervised discriminant analysis algorithm which integrates both global and local structures. Specifically, our objective is to learn a smooth as well as discriminative subspace. In order to achieve it, we jointly use both the instances in the cannot-link constraints to maximize the separability between different classes while applying those in the must-link constraints to minimize the distance between the same class and the integration of global and local structures of the data to make nearby instances in the original space close to each other in the embedding space. Experimental results on a collection of real-world data sets demonstrated the effectiveness of the proposed algorithm.

Flexible orthogonal semisupervised learning for dimension reduction with image classification

Recently, low-rank representation (LRR) methods have been widely applied for hyperspectral anomaly detection, due to their potentials in separating the backgrounds and anomalies. However, existing LRR models generally convert 3-D hyperspectral images (HSIs) into 2-D matrices, inevitably leading to the destruction of intrinsic 3-D structure properties in HSIs. To this end, we propose a novel tensor low-rank and sparse representation (TLRSR) method for hyperspectral anomaly detection. A 3-D TLR model is expanded to separate the LR background part represented by a tensorial background dictionary and corresponding coefficients. This representation characterizes the multiple subspace property of the complex LR background. Based on the weighted tensor nuclear norm and the LF,1 sparse norm, a dictionary is designed to make its atoms more relevant to the background. Moreover, a principal component analysis (PCA) method can be assigned as one preprocessing step to exact a subset of HSI bands, retaining enough the HSI object information and reducing computational time of the postprocessing tensorial operations. The proposed model is efficiently solved by the well-designed alternating direction method of multipliers (ADMMs). A comparison with the existing algorithms via experiments establishes the competitiveness of the proposed method with the state-of-the-art competitors in the hyperspectral anomaly detection task.

Low Rank Representation (LRR) based unsupervised clustering methods have achieved great success since these methods could explore low-dimensional subspace structure embedded in original data effectively. The conventional LRR methods generally treat the data as the points in Euclidean space. However, it is no longer suitable for high-dimension data (such as video or imageset). That is because high-dimension data are always considered as non-linear manifold data such as Grassmann manifold. Besides, the typical LRR methods always adopt the traditional single nuclear norm based low rank constraint which can not fully reveal the low rank property of the data representation and often leads to suboptimal solution. In this paper, a new LRR based clustering model is constructed on Grassmann manifold for high-dimension data. In the proposed method, each high-dimension data is formed as a sample on Grassmann manifold with non-linear metric. Meanwhile, a non-convex low rank representation is adopt to reveal the intrinsic property of these high-dimension data and reweighted rank minimization constraint is introduced. The experimental results on several public datasets show that the proposed method outperforms the state-of-the-art clustering methods.

An improved manifold learning method, called enhanced semi-supervised local fisher discriminant analysis (ESELF), for gene expression data classification is proposed. Motivated by the fact that semi-supervised and parameter-free are two desirable and promising characteristics for dimension reduction, a new difference-based optimization objective function with unlabeled samples has been designed. The proposed method preserves the global structure of unlabeled samples in addition to separating labeled samples in different classes from each other. The semi-supervised method has an analytic form of the globally optimal solution and it can be computed based on eigen decompositions. The experimental results and comparisons on synthetic data and two DNA micro array datasets demonstrate the effectiveness of the proposed method.

Despite the widespread adoption of deep learning models, shallow machine learning (SML) algorithms are still used for image classification due to simplicity, interpretability and efficiency. This study aims to bridge this gap by investigating the robustness dynamics of SML techniques under more complex scenarios, such as adversarial perturbations and geometric transformation. Five popular classification algorithms, including k-nearest neighbour and support vector machines, were employed to build classification models. Methodology involves investigating the robustness of proposed methods, first, on original and corrupted data by utilising benchmark datasets across several image domains. To strengthen the investigation, the models were trained using a new low-rank representation (LRR) strategy. This hybrid model simultaneously addresses two key limitations in classical LRR models: overcoming the sequential learning process and effectively capturing both local and global data structures. By introducing a dual regularisation mechanism, it integrates a k-nearest neighbour graph to preserve local consistency, while a global low-rank constraint ensures coherent data representation. Experimental results reveal significant drops in accuracy of most SML methods, especially under adversarial attacks and geometric transformations, but LRR approach mitigates these effects to a notable extent, boosting performance across data variations. The results also show that the proposed method outperforms state-of-the-art LRR techniques in most experiments.

Recently, low-rank and sparse model-based dimensionality reduction (DR) methods have aroused lots of interest. In this paper, we propose an effective supervised DR technique named block-diagonal constrained low-rank and sparse-based embedding (BLSE). BLSE has two steps, i.e., block-diagonal constrained low-rank and sparse representation (BLSR) and block-diagonal constrained low-rank and sparse graph embedding (BLSGE). Firstly, the BLSR model is developed to reveal the intrinsic intra-class and inter-class adjacent relationships as well as the local neighborhood relations and global structure of data. Particularly, there are mainly three items considered in BLSR. First, a sparse constraint is required to discover the local data structure. Second, a low-rank criterion is incorporated to capture the global structure in data. Third, a block-diagonal regularization is imposed on the representation to promote discrimination between different classes. Based on BLSR, informative and discriminative intra-class and inter-class graphs are constructed. With the graphs, BLSGE seeks a low-dimensional embedding subspace by simultaneously minimizing the intra-class scatter and maximizing the inter-class scatter. Experiments on public benchmark face and object image datasets demonstrate the effectiveness of the proposed approach.

Low-rank representation with local constraint for graph construction

Discriminative low-rank preserving projection for dimensionality reduction

Fisher discriminant analysis (FDA) is a classic supervised dimensionality reduction method in statistical pattern recognition. FDA can maximize the scatter between different classes, while minimizing the scatter within each class. As it only utilizes the labeled data and ignores the unlabeled data in the analysis process of FDA, it cannot be used to solve the unsupervised learning problems. Its performance is also very poor in dealing with semi-supervised learning problems in some cases. Recently, several semi-supervised learning methods as an extension of FDA have proposed. Most of these methods solve the semi-supervised problem by using a tradeoff parameter that evaluates the ratio of the supervised and unsupervised methods. In this paper, we propose a general semi-supervised dimensionality learning idea for the partially labeled data, namely the reconstruction probability class of labeled and unlabeled data. Based on the probability class optimizes Fisher criterion function, we propose a novel Semi-Supervised Local Fisher Discriminant Analysis (S2LFDA) method. Experimental results on real-world datasets demonstrate its effectiveness compared to the existing similar correlation methods.