Kernelized multi-view subspace clustering via auto-weighted graph learning

Multi-view data clustering often aims to utilize various representations or views of original data to improve the clustering performance compared to the single-view clustering approach. Most multi-view subspace clustering methods are proposed to construct the affinity matrix of each view individually and then implement with spectral clustering for multi-view data clustering. The multi-view low-rank sparse subspace clustering (MLRSSC) is an effective and popular clustering algorithm among multi-view subspace clustering. This method can explore the joint subspace representation through creating an affinity matrix integrated of all views of input data. In addition, the low-rank and sparsity constraints are introduced into this method to enhance the clustering results. However, the original MLRSSC uses the mean square error as the fidelity term while not consider the complex noise pollution in real situations. Therefore, we introduce a complex noise modeling approach, i.e., independent and piecewise identically distributed (IPID) noise model, for MLRSSC to improve its performance. The related experimental results confirm that this proposed algorithm surpasses many state-of-the-art subspace clustering methods on several real-world datasets.

Multi-view subspace clustering, which aims to partition a dataset into its relevant subspaces based on their multi-view features, has been widely applied to identify various characteristics of datasets. The typical model of multi-view subspace clustering in literature often makes use of the nuclear norm to seek the underlying low-rank representation. However, due to the sum property of the singular values defined by tensor nuclear norm, the existing multi-view subspace clustering does not well handle the noise and the illumination variations embedded in multi-view data. To address and improve the robustness and clustering performance, we propose a new nonconvex multi-view subspace clustering model via tensor minimax concave penalty (MCP) approximation associated with rank minimization (NMSC-MCP), which can simultaneously construct the low-rank representation tensor and affinity matrix in a unified framework. Specifically, the nonconvex MCP approximation rank function is adopted to as a tighter tensor rank approximation to discriminate the dimension of features so that better accuracy can be achieved. In addition, we also address the local structure by including both hyper-Laplacian regularization and auto-weighting scheme into the objective function to promote the clustering performance. A corresponding iterative algorithm is then developed to solve the proposed model and the constructed iterative sequence generated by the proposed algorithm is shown to converge to the desirable KKT critical point. Extensive experiments on benchmark datasets have demonstrate the highly desirable effectiveness of our proposed method.

Multiview subspace clustering is one of the most widely used methods for exploiting the internal structures of multiview data. Most previous studies have performed the task of learning multiview representations by individually constructing an affinity matrix for each view without simultaneously exploiting the intrinsic characteristics of multiview data. In this article, we propose a multiview low-rank representation (MLRR) method to comprehensively discover the correlation of multiview data for multiview subspace clustering. MLRR considers symmetric low-rank representations (LRRs) to be an approximately linear spatial transformation under the new base, that is, the multiview data themselves, to fully exploit the angular information of the principal directions of LRRs, which is adopted to construct an affinity matrix for multiview subspace clustering, under a symmetric condition. MLRR takes full advantage of LRR techniques and a diversity regularization term to exploit the diversity and consistency of multiple views, respectively, and this method simultaneously imposes a symmetry constraint on LRRs. Hence, the angular information of the principal directions of rows is consistent with that of columns in symmetric LRRs. The MLRR model can be efficiently calculated by solving a convex optimization problem. Moreover, we present an intuitive fusion strategy for symmetric LRRs from the perspective of spectral clustering to obtain a compact representation, which can be shared by multiple views and comprehensively represents the intrinsic features of multiview data. Finally, the experimental results based on benchmark datasets demonstrate the effectiveness and robustness of MLRR compared with several state-of-the-art multiview subspace clustering algorithms.

An increasing amount of attention have been attracted in multi-view subspace clustering, whose impressive performance can be achieved by means of the self-expressive property, under the assumption of linear relations between multi-view data samples. However, most of them fail to recover the nonlinear relations between multi-view data for deeper study, and additionally they are incapable of discovering the comprehensiveness and higher-order correlations among multiple views. To deal with these challenges, we propose a novel model termed Coupled Learning for Kernel Representation and Graph Tensor (CLKT) in Multi-view Subspace Clustering, where both nonlinear relations and higher-order correlations among multiple affinity graphs are jointly learned in a unified framework. Optimal solutions of the proposed method can be obtained by an alternative minimizing optimization strategy. Extensive experiments on six real-world datasets indicate the superiority of CLKT compared with the state-of-the-art methods.KeywordsMulti-view clusteringKernel learningLow-rank tensor representation

Multi-view clustering can mine the underlying structure of multi-view data and has attracted increasing attention. Most existing multi-view clustering methods either construct the similarity matrix from the global level through self-representation learning or construct the similarity matrix from the local level through graph learning. Spectral clustering method can be used to yield the clustering results based on the similarity matrix. However, the similarity matrix that only considers global information or local information is not robust. Moreover, separating the similarity matrix learning and clustering as two steps may lead to sub-optimal clustering results. To address these issues, we propose in this paper, a multi-view subspace clustering with local and global information (MVSCLG) method. Our method combines the self-representation learning and graph learning to learn a similarity matrix with global and local information, and simultaneously utilizes the spectral decomposition and the spectral rotation techniques to yield the clustering results. We also develop an effective optimization algorithm to solve the resulting optimization problem. The effectiveness and superiority of this method are verified on four multi-view benchmark data sets.

Multi-view clustering with adaptive anchor and bipartite graph learning

This paper addresses the multi-view subspace clustering problem and proposes the self-paced enhanced low-rank tensor kernelized multi-view subspace clustering (SETKMC) method, which is based on two motivations: (1) singular values of the representations and multiple instances should be treated differently. The reasons are that larger singular values of the representations usually quantify the major information and should be less penalized; samples with different degrees of noise may have various reliability for clustering. (2) many existing methods may cause the degraded performance when multi-view features reside in different nonlinear subspaces. This is because they usually assumed that multiple features lie within the union of several linear subspaces. SETKMC integrates the nonconvex tensor norm, self-paced learning, and kernel trick into a unified model for multi-view subspace clustering. The nonconvex tensor norm imposes different weights on different singular values. The self-paced learning gradually involves instances from more reliable to less reliable ones while the kernel trick aims to handle the multi-view data in nonlinear subspaces. One iterative algorithm is proposed based on the alternating direction method of multipliers. Extensive results on seven real-world datasets show the effectiveness of the proposed SETKMC compared to fifteen state-of-the-art multi-view clustering methods.

Nonlinear subspace clustering for image clustering

Clustering of multi-view relational data based on particle swarm optimization

Subspace clustering groups a set of samples from a union of several linear subspaces into clusters, so that the samples in the same cluster are drawn from the same linear subspace. In the majority of the existing work on subspace clustering, clusters are built based on feature information, while sample correlations in their original spatial structure are simply ignored. Besides, original high-dimensional feature vector contains noisy/redundant information, and the time complexity grows exponentially with the number of dimensions. To address these issues, we propose a tensor low-rank representation (TLRR) and sparse coding-based (TLRRSC) subspace clustering method by simultaneously considering feature information and spatial structures. TLRR seeks the lowest rank representation over original spatial structures along all spatial directions. Sparse coding learns a dictionary along feature spaces, so that each sample can be represented by a few atoms of the learned dictionary. The affinity matrix used for spectral clustering is built from the joint similarities in both spatial and feature spaces. TLRRSC can well capture the global structure and inherent feature information of data and provide a robust subspace segmentation from corrupted data. Experimental results on both synthetic and real-world data sets show that TLRRSC outperforms several established stateof- the-art methods.

With the popularity of multimedia technology, information is always represented or transmitted from multiple views. Most of the existing algorithms are graph-based ones to learn the complex structures within multiview data but overlooked the information within data representations. Furthermore, many existing works treat multiple views discriminatively by introducing some hyperparameters, which is undesirable in practice. To this end, abundant multiview based methods have been proposed for dimension reduction. However, there are still no research to leverage the existing work into a unified framework. To address this issue, in this paper, we propose a general framework for multiview data dimension reduction, named Kernelized Multiview Subspace Analysis (KMSA). It directly handles the multi-view feature representation in the kernel space, which provides a feasible channel for direct manipulations on multiview data with different dimensions. Meanwhile, compared with those graph-based methods, KMSA can fully exploit information from multiview data with nothing to lose. Furthermore, since different views have different influences on KMSA, we propose a self-weighted strategy to treat different views discriminatively according to their contributions. A co-regularized term is proposed to promote the mutual learning from multi-views. KMSA combines self-weighted learning with the co-regularized term to learn appropriate weights for all views. We also discuss the influence of the parameters in KMSA regarding the weights of multi-views. We evaluate our proposed framework on 6 multiview datasets for classification and image retrieval. The experimental results validate the advantages of our proposed method.

With the diversification of data sources, the multi-view data with multiple expressions have been appeared in various application scenarios. These multi-view data generally have high dimensions, large amounts and often lack of label information. Therefore, it is very important to learn multi-view data in an unsupervised way so as to analyze and excavate the potential valuable information. In this paper, we propose a multi-view locality preserving embedding algorithm with view similarity constraint for data dimension reduction. This algorithm not only preserves the local structure into low-dimensional space for each view, but also implements the similarity constraints between different views. On this basis, the algorithm looks for a joint embedding of low-dimensional subspace, so that the neighborhood among samples in original high-dimensional space can be maintained in the subspace, and the structures corresponding to different views are consistent with each other. This algorithm achieves good experimental results both in artificial data sets and multi-view data sets, which prove the correctness and feasibility of the algorithm.

Self-organizing subspace clustering for high-dimensional and multi-view data

In this article, we propose a deep extension of sparse subspace clustering, termed deep subspace clustering with L1-norm (DSC-L1). Regularized by the unit sphere distribution assumption for the learned deep features, DSC-L1 can infer a new data affinity matrix by simultaneously satisfying the sparsity principle of SSC and the nonlinearity given by neural networks. One of the appealing advantages brought by DSC-L1 is that when original real-world data do not meet the class-specific linear subspace distribution assumption, DSC-L1 can employ neural networks to make the assumption valid with its nonlinear transformations. Moreover, we prove that our neural network could sufficiently approximate the minimizer under mild conditions. To the best of our knowledge, this could be one of the first deep-learning-based subspace clustering methods. Extensive experiments are conducted on four real-world data sets to show that the proposed method is significantly superior to 17 existing methods for subspace clustering on handcrafted features and raw data.

Subspace clustering is an effective algorithm refer to the problem which separate the data lying on a union of subspaces. Usually the algorithm consists of two steps. First, an affinity matrix is calculated from the self-representation of the data. Second, spectral clustering is used to cluster the data by the affinity matrix. The paper introduces a new method based on the sparse subspace clustering. The new method used a new objective which considered both the sparseness and grouping effect. And the affinity matrix which is calculated from the self-representation of the data is furthered optimized. It proved to be more efficient than existing subspace clustering methods.