Adaptive multi-view subspace clustering with dynamic prior update

Multi-view data clustering often aims to utilize various representations or views of original data to improve the clustering performance compared to the single-view clustering approach. Most multi-view subspace clustering methods are proposed to construct the affinity matrix of each view individually and then implement with spectral clustering for multi-view data clustering. The multi-view low-rank sparse subspace clustering (MLRSSC) is an effective and popular clustering algorithm among multi-view subspace clustering. This method can explore the joint subspace representation through creating an affinity matrix integrated of all views of input data. In addition, the low-rank and sparsity constraints are introduced into this method to enhance the clustering results. However, the original MLRSSC uses the mean square error as the fidelity term while not consider the complex noise pollution in real situations. Therefore, we introduce a complex noise modeling approach, i.e., independent and piecewise identically distributed (IPID) noise model, for MLRSSC to improve its performance. The related experimental results confirm that this proposed algorithm surpasses many state-of-the-art subspace clustering methods on several real-world datasets.

Multi-view subspace clustering has been an important and powerful tool for partitioning multi-view data, especially multi-view high-dimensional data. Despite great success, most of the existing multi-view subspace clustering methods still suffer from three limitations. First, they often recover the subspace structure in the original space, which can not guarantee the robustness when handling multi-view data with nonlinear structure. Second, these methods mostly regard subspace clustering and affinity matrix learning as two independent steps, which may not well discover the latent relationships among data samples. Third, many of them ignore the different importance of multiple views, whose performance may be badly affected by the low-quality views in multi-view data. To overcome these three limitations, this paper develops a novel subspace clustering method for multi-view data, termed Kernelized Multi-view Subspace Clustering via Auto-weighted Graph Learning (KMSC-AGL). Specifically, the proposed method implicitly maps the multi-view data from linear space into nonlinear space via kernel-induced functions, so as to exploit the nonlinear structure hidden in data. Furthermore, our method aims to enhance the clustering performance by learning a set of view-specific representations and their affinity matrix in a general framework. By integrating the view weighting strategy into this framework, our method can automatically assign the weights to different views, while learning an optimal affinity matrix that is well-adapted to the subsequent spectral clustering. Extensive experiments are conducted on a variety of multi-view data sets, which have demonstrated the superiority of the proposed method.

Multiview subspace clustering (MSC) has attracted growing attention due to the extensive value in various applications, such as natural language processing, face recognition, and time-series analysis. In this article, we are devoted to address two crucial issues in MSC: 1) high computational cost and 2) cumbersome multistage clustering. Existing MSC approaches, including tensor singular value decomposition (t-SVD)-MSC that has achieved promising performance, generally utilize the dataset itself as the dictionary and regard representation learning and clustering process as two separate parts, thus leading to the high computational overhead and unsatisfactory clustering performance. To remedy these two issues, we propose a novel MSC model called joint skinny tensor learning and latent clustering (JSTC), which can learn high-order skinny tensor representations and corresponding latent clustering assignments simultaneously. Through such a joint optimization strategy, the multiview complementary information and latent clustering structure can be exploited thoroughly to improve the clustering performance. An alternating direction minimization algorithm, which owns low computational complexity and can be run in parallel when solving several key subproblems, is carefully designed to optimize the JSTC model. Such a nice property makes our JSTC an appealing solution for large-scale MSC problems. We conduct extensive experiments on ten popular datasets and compare our JSTC with 12 competitors. Five commonly used metrics, including four external measures (NMI, ACC, F-score, and RI) and one internal metric (SI), are adopted to evaluate the clustering quality. The experimental results with the Wilcoxon statistical test demonstrate the superiority of the proposed method in both clustering performance and operational efficiency.

Recently emerging approaches to high-throughput phenotyping have become important tools in unraveling the biological basis of agronomically and medically important phenotypes. These experiments produce very large sets of either low or high-dimensional data. Finding clusters in the entire space of high-dimensional data (HDD) is a challenging task, because the relative distances between any two objects converge to zero with increasing dimensionality. Additionally, real data may not be mathematically well behaved. Finally, many clusters are expected on biological grounds to be "natural" -- that is, to have irregular, overlapping boundaries in different subsets of the dimensions. More precisely, the natural clusters of the data could differ in shape, size, density, and dimensionality; and they might not be disjoint. In principle, clustering such data could be done by dimension reduction methods. However, these methods convert many dimensions to a smaller set of dimensions that make the clustering results difficult to interpret and may also lead to a significant loss of information. Another possible approach is to find subspaces (subsets of dimensions) in the entire data space of the HDD. However, the existing subspace methods don't discover natural clusters. Therefore, in this dissertation I propose a novel data preprocessing method, demonstrating that a group of phenotypes are interdependent, and propose a novel density-based subspace clustering algorithm for high-dimensional data, called Dynamic Locally Density Adaptive Scalable Subspace Clustering (DynaDASC). This algorithm is relatively locally density adaptive, scalable, dynamic, and nonmetric in nature, and discovers natural clusters.

Multi-view subspace clustering, which aims to partition a dataset into its relevant subspaces based on their multi-view features, has been widely applied to identify various characteristics of datasets. The typical model of multi-view subspace clustering in literature often makes use of the nuclear norm to seek the underlying low-rank representation. However, due to the sum property of the singular values defined by tensor nuclear norm, the existing multi-view subspace clustering does not well handle the noise and the illumination variations embedded in multi-view data. To address and improve the robustness and clustering performance, we propose a new nonconvex multi-view subspace clustering model via tensor minimax concave penalty (MCP) approximation associated with rank minimization (NMSC-MCP), which can simultaneously construct the low-rank representation tensor and affinity matrix in a unified framework. Specifically, the nonconvex MCP approximation rank function is adopted to as a tighter tensor rank approximation to discriminate the dimension of features so that better accuracy can be achieved. In addition, we also address the local structure by including both hyper-Laplacian regularization and auto-weighting scheme into the objective function to promote the clustering performance. A corresponding iterative algorithm is then developed to solve the proposed model and the constructed iterative sequence generated by the proposed algorithm is shown to converge to the desirable KKT critical point. Extensive experiments on benchmark datasets have demonstrate the highly desirable effectiveness of our proposed method.

Multiview subspace clustering is one of the most widely used methods for exploiting the internal structures of multiview data. Most previous studies have performed the task of learning multiview representations by individually constructing an affinity matrix for each view without simultaneously exploiting the intrinsic characteristics of multiview data. In this article, we propose a multiview low-rank representation (MLRR) method to comprehensively discover the correlation of multiview data for multiview subspace clustering. MLRR considers symmetric low-rank representations (LRRs) to be an approximately linear spatial transformation under the new base, that is, the multiview data themselves, to fully exploit the angular information of the principal directions of LRRs, which is adopted to construct an affinity matrix for multiview subspace clustering, under a symmetric condition. MLRR takes full advantage of LRR techniques and a diversity regularization term to exploit the diversity and consistency of multiple views, respectively, and this method simultaneously imposes a symmetry constraint on LRRs. Hence, the angular information of the principal directions of rows is consistent with that of columns in symmetric LRRs. The MLRR model can be efficiently calculated by solving a convex optimization problem. Moreover, we present an intuitive fusion strategy for symmetric LRRs from the perspective of spectral clustering to obtain a compact representation, which can be shared by multiple views and comprehensively represents the intrinsic features of multiview data. Finally, the experimental results based on benchmark datasets demonstrate the effectiveness and robustness of MLRR compared with several state-of-the-art multiview subspace clustering algorithms.

An increasing amount of attention have been attracted in multi-view subspace clustering, whose impressive performance can be achieved by means of the self-expressive property, under the assumption of linear relations between multi-view data samples. However, most of them fail to recover the nonlinear relations between multi-view data for deeper study, and additionally they are incapable of discovering the comprehensiveness and higher-order correlations among multiple views. To deal with these challenges, we propose a novel model termed Coupled Learning for Kernel Representation and Graph Tensor (CLKT) in Multi-view Subspace Clustering, where both nonlinear relations and higher-order correlations among multiple affinity graphs are jointly learned in a unified framework. Optimal solutions of the proposed method can be obtained by an alternative minimizing optimization strategy. Extensive experiments on six real-world datasets indicate the superiority of CLKT compared with the state-of-the-art methods.KeywordsMulti-view clusteringKernel learningLow-rank tensor representation

Low-rank self-representation based subspace learning has confirmed its great effectiveness in a broad range of applications. Nevertheless, existing studies mainly focus on exploring the global linear subspace structure, and cannot commendably handle the case where the samples approximately (i.e., the samples contain data errors) lie in several more general affine subspaces. To overcome this drawback, in this paper, we innovatively propose to introduce affine and nonnegative constraints into low-rank self-representation learning. While simple enough, we provide their underlying theoretical insight from a geometric perspective. The union of two constraints geometrically restricts each sample to be expressed as a convex combination of other samples in the same subspace. In this way, when exploring the global affine subspace structure, we can also consider the specific local distribution of data in each subspace. To comprehensively demonstrate the benefits of introducing two constraints, we instantiate three low-rank self-representation methods ranging from single-view low-rank matrix learning to multi-view low-rank tensor learning. We carefully design the solution algorithms to efficiently optimize the proposed three approaches. Extensive experiments are conducted on three typical tasks, including single-view subspace clustering, multi-view subspace clustering, and multi-view semi-supervised classification. The notably superior experimental results powerfully verify the effectiveness of our proposals.

Adaptive multi-view subspace clustering algorithm based on representative features and redundant instances

Multi-view clustering can mine the underlying structure of multi-view data and has attracted increasing attention. Most existing multi-view clustering methods either construct the similarity matrix from the global level through self-representation learning or construct the similarity matrix from the local level through graph learning. Spectral clustering method can be used to yield the clustering results based on the similarity matrix. However, the similarity matrix that only considers global information or local information is not robust. Moreover, separating the similarity matrix learning and clustering as two steps may lead to sub-optimal clustering results. To address these issues, we propose in this paper, a multi-view subspace clustering with local and global information (MVSCLG) method. Our method combines the self-representation learning and graph learning to learn a similarity matrix with global and local information, and simultaneously utilizes the spectral decomposition and the spectral rotation techniques to yield the clustering results. We also develop an effective optimization algorithm to solve the resulting optimization problem. The effectiveness and superiority of this method are verified on four multi-view benchmark data sets.

This paper addresses the multi-view subspace clustering problem and proposes the self-paced enhanced low-rank tensor kernelized multi-view subspace clustering (SETKMC) method, which is based on two motivations: (1) singular values of the representations and multiple instances should be treated differently. The reasons are that larger singular values of the representations usually quantify the major information and should be less penalized; samples with different degrees of noise may have various reliability for clustering. (2) many existing methods may cause the degraded performance when multi-view features reside in different nonlinear subspaces. This is because they usually assumed that multiple features lie within the union of several linear subspaces. SETKMC integrates the nonconvex tensor norm, self-paced learning, and kernel trick into a unified model for multi-view subspace clustering. The nonconvex tensor norm imposes different weights on different singular values. The self-paced learning gradually involves instances from more reliable to less reliable ones while the kernel trick aims to handle the multi-view data in nonlinear subspaces. One iterative algorithm is proposed based on the alternating direction method of multipliers. Extensive results on seven real-world datasets show the effectiveness of the proposed SETKMC compared to fifteen state-of-the-art multi-view clustering methods.

Recently, a growing number of researchers have focused on multi-view subspace clustering (MSC) due to its potential for integrating heterogeneous data. However, current MSC methods remain challenged by limited robustness and insufficient exploitation of cross-view high-order latent information for clustering advancement. To address these challenges, we develop a novel MSC framework termed TMSC-TNNBDR, a tensorized MSC framework that leverages t-SVD based tensor nuclear norm (TNN) regularization and block diagonal representation (BDR) learning to unify view consistency and structural sparsity. Specifically, each subspace representation matrix is constrained by a block diagonal regularizer to enforce cluster structure, while all matrices are aggregated into a tensor to capture high-order interactions. To efficiently optimize the model, we developed an optimization algorithm based on the inexact augmented Lagrange multiplier (ALM). The TMSC-TNNBDR exhibits both optimized block-diagonal structure and low-rank properties, thereby enabling enhanced mining of latent higher-order inter-view correlations while demonstrating greater resilience to noise. To investigate the capability of TMSC-TNNBDR, we conducted several experiments on certain datasets. Benchmarking on circumscribed datasets demonstrates our method’s superior clustering performance over comparative algorithms while maintaining competitive computational overhead.

Multi-view subspace clustering (MVSC) optimally integrates multiple graph structure information to improve clustering performance. Recently, many anchor-based variants are proposed to reduce the computational complexity of MVSC. Though achieving considerable acceleration, we observe that most of them adopt fixed anchor points separating from the subsequential anchor graph construction, which may adversely affect the clustering performance. In addition, post-processing is required to generate discrete clustering labels with additional time consumption. To address these issues, we propose a scalable and parameter-free MVSC method to directly output the clustering labels with optimal anchor graph, termed as Efficient One-pass Multi-view Subspace Clustering with Consensus Anchors (EOMSC-CA). Specially, we combine anchor learning and graph construction into a uniform framework to boost clustering performance. Meanwhile, by imposing a graph connectivity constraint, our algorithm directly outputs the clustering labels without any post-processing procedures as previous methods do. Our proposed EOMSC-CA is proven to be linear complexity respecting to the data size. The superiority of our EOMSC-CA over the effectiveness and efficiency is demonstrated by extensive experiments. Our code is publicly available at https://github.com/Tracesource/EOMSC-CA.

Conventional scalable compressive video sampling cannot sufficiently exploit the structured sparsity within video sequences. This paper proposes a novel quality scalable structured compressive video sampling (SS-CVS) framework with hierarchical subspace learning to support video transmission over heterogeneous network conditions. The proposed framework incorporates the union of data-driven subspaces (UoDS) model to introduce structured sparsity into sensing matrix for progressive acquisition of measurements. Hierarchical subspace learning is developed to generate the subspaces with their bases based on adaptive subspace clustering in a progressive fashion. Consequently, two hierarchical structures are derived to enable progressive mapping of subspaces and bases on the structured sensing matrices to support quality scalability. To guarantee the convergence of hierarchical subspace learning, the training set is updated with the adaptive group set from the reconstructed reference frames. It is demonstrated that SS-CVS can guarantee stable recovery for each quality layer under the constraints of block restricted isometric property (RIP). Furthermore, the proposed SS-CVS model is generalized to tensor subspaces for scalable compressive sampling of high-dimensional signals. Experimental results demonstrate that the proposed algorithm can achieve quality scalabilities with an improved reconstructed performance in comparison to the state-of-the-art scalable compressive video sampling methods.

Adaptive subspace clustering techniques, such as LDA Kmeans (henceforth LDAKM), try to perform clustering on a compact discriminative subspace. However, the subspace extracted by LDAKM may be inferior when encountering noise, so it is with the clustering results. In this paper, we generalize the LDAKM algorithm, and propose the Local Manifold Preserving LDAKM (henceforth LMP–LDAKM) approach, which considers both local manifold structure and discriminative information. A Laplacian similarity matrix is introduced in the subspace extraction subprocess for preserving local manifold information while retaining discrimination of latent clusters, and it is also utilized for the weighted Kmeans clustering subprocess in the subspace to eliminate the influence of distant noise data. Experimental results on both benchmark and artificial datasets indicate the effectiveness and noise-robustness of our method.