Multi-View Utility-Based Clustering: A Mutually Supervised Perspective

Multi-view intact space clustering

Multi-view clustering (MVC) can fuse the information of multiple views for robust clustering result, among it two fusion strategies, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">early-fusion</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">late-fusion</i> are widely adopted. Although they have derived many MVC methods, there are still two crucial questions: (1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">early-fusion</i> forces multiple views to share a consensus latent representation, which compounds the challenge of excavating view-specific diverse local information; (2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">late-fusion</i> generates view-partitions independently and then integrates them in the following clustering procedure, where the two procedures cannot guide each other and lack necessary negotiation. In view of this, we propose a novel Graph Proxy Fusion (GPF) method to preserve and fuse view-specific local information concertedly in one unified framework. Specifically, we first propose anchor-based local information learning to capture view-specific local structural information in bipartite graphs; meanwhile, a view-consensus graph learned through self-expressiveness-based proxy graph learning module is deemed as a higher-order proxy; following, the novel graph proxy fusion module integrally embeds all lower-order bipartite graphs in the higher-order proxy via higher-order correlation theory. As a novel fusion strategy, the proposed GPF efficiently investigates the valuable consensus and diverse information of multiple views. Experiments on various multi-view datasets demonstrate the superiority of our method.

We propose a novel multi-view clustering method by learning auto-regression problems under structural constraints and treating the regression coefficients as new feature representations for the cluster partition. In particular, we take the data intrinsic correlation structure into account. Correlated data under one view tend to be also related under another view and are likely to fall into the same group. Therefore we pair the data matrix from one view and the regression coefficient from a different view together to meet a trace Lasso constraint, which adaptively adjusts the sparsity of regression coefficients in order to promote consistent data correlations across views. Then a joint low-rank constraint is further imposed to encourage similar regression coefficients for the same samples under distinct views. Finally, we develop an effective algorithm to optimize the objective function. And experimental results demonstrate that our method is useful and fairly competitive compared with other state-of-the-art multi-view clustering methods.

With the development of data collection techniques, multi-view clustering (MVC) becomes an emerging research direction to improve the clustering performance. However, most MVC methods assume that the objects are observed on all the views. As a result, existing MVC methods may not achieve satisfactory performance when some views are incomplete. In this paper, we propose a new MVC method, called as partial multi-view clustering via auto-weighting similarity completion (PMVC-ASC). The major contribution lies in jointly learning the consensus similarity matrix, exploring the complementary information among multiple distinct feature sets, quantifying the contribution of each view and splitting the similarity graph into several informative submatrices, each submatrix corresponding to one cluster. The learning process can be modeled via a joint minimization problem, and the corresponding optimization algorithm is given. A series of experiments are conducted on real-world datasets to demonstrate the superiority of PMVC-ASC by comparing with the state-of-the-art methods.

In this article, we address the challenging problem of unpaired multiview clustering (UMC), which aims to achieve effective joint clustering using unpaired samples observed across multiple views. Traditional incomplete multiview clustering (IMC) methods typically rely on paired samples to capture complementary information between views. However, such strategies become impractical in the UMC due to the absence of paired samples. Although some researchers have attempted to address this issue by preserving consistent cluster structures across views, effectively mining such consistency remains challenging when the cluster structures with low confidence. Therefore, we propose a novel method, multilevel reliable guidance for UMC (MRG-UMC), which integrates multilevel clustering and reliable view guidance to learn consistent and confident cluster structures from three perspectives. Specifically, inner view multilevel clustering exploits high-confidence sample pairs across different levels to reduce the impact of boundary samples, resulting in more confident cluster structures. Synthesized-view alignment leverages a synthesized view to mitigate cross-view discrepancies and promote consistency. Cross-view guidance employs a reliable view guidance strategy to enhance the clustering confidence of poorly clustered views. These three modules are jointly optimized across multiple levels to achieve consistent and confident cluster structures. Furthermore, theoretical analyses verify the effectiveness of MRG-UMC in enhancing clustering confidence. Extensive experimental results show that MRG-UMC outperforms state-of-the-art UMC methods, achieving an average NMI improvement of 12.95% on multiview datasets. The source code is available at https://anonymous.4open.science/r/MRG-UMC-5E20.

In recent years, multiview clustering algorithms have achieved promising performance by exploiting the complementarity and consistency of different views. However, many multiview spectral clustering methods only focus on the consistent information of views, and the time cost of feature decomposition is expensive. Moreover, these methods also require postprocessing (e.g., k-means) to obtain the final clustering results. To overcome these limitations simultaneously, we propose a novel multiview clustering algorithm. First, the method removes the inconsistent information of the views through cross-view measurement to maintain consistent information. These inconsistencies may be caused by noise, corruptions, or view-specific properties and will affect the quality of the similarity matrix. Then, we learn a consensus embedding matrix with nonnegative constraints by performing a low-rank decomposition of the consistency information. In this way, we can replace the eigendecomposition of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> Laplacian matrix in spectral clustering with the singular value decomposition of a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> low-rank matrix to reduce the computational burden, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\ll }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> . Furthermore, due to the nonnegative constraint, we can directly obtain the clustering results. Also, to consider the diversity of views, adaptive weighting is applied to different view data. Compared to state-of-the-art multiview clustering methods on five benchmark multiview datasets, we demonstrate the superiority and effectiveness of our approach. We release the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hulu88/FAMvC</uri> .

Deep multi-view subspace clustering has achieved promising performance compared with other multi-view clustering. However, existing deep multi-view subspace clustering only considers the global structure for all views, and they ignore the local geometric structure among each view. In addition, they cannot learn discriminative feature on different clusters of different views, i.e., inter-cluster difference. To solve these problems, in this paper, we propose a novel Deep Multi-view Subspace Clustering with Unified and Discriminative Learning (DMSC-UDL). DMSC-UDL combines global and local structures with self-expression layer. The global and local structures help each other forward and achieve small distance between samples of the same cluster. To make samples in different clusters of different views farther, DMSC-UDL uses a discriminative constraint between different views. In this way, DMSC-UDL makes the same cluster's samples have large weights, while different clusters’ samples have small weights. Thus, it can learn a better shared connection matrix for multi-view clustering. Extensive experimental results reveal that the proposed multi-view clustering method is superior to several state-of-the-art multi-view clustering methods in terms of performance.

Multiview clustering (MVC) aims to partition data into different groups by taking full advantage of the complementary information from multiple views. Most existing MVC methods fuse information of multiple views at the raw data level. They may suffer from performance degradation due to the redundant information contained in the raw data. Graph learning-based methods often heavily depend on one specific graph construction, which limits their practical applications. Moreover, they often require a computational complexity of O(n3) because of matrix inversion or eigenvalue decomposition for each iterative computation. In this paper, we propose a consensus spectral rotation fusion (CSRF) method to learn a fused affinity matrix for MVC at the spectral embedding feature level. Specifically, we first introduce a CSRF model to learn a consensus low-dimensional embedding, which explores the complementary and consistent information across multiple views. We develop an alternating iterative optimization algorithm to solve the CSRF optimization problem, where a computational complexity of O(n2) is required during each iterative computation. Then, the sparsity policy is introduced to design two different graph construction schemes, which are effectively integrated with the CSRF model. Finally, a multiview fused affinity matrix is constructed from the consensus low-dimensional embedding in spectral embedding space. We analyze the convergence of the alternating iterative optimization algorithm and provide an extension of CSRF for incomplete MVC. Extensive experiments on multiview datasets demonstrate the effectiveness and efficiency of the proposed CSRF method.

Nowadays, multi-view dataset have become ubiquitous along with more and more data are gathered from different measuring technologies or various sources, in which various aspects of dataset are formalized as multiple views. Although a variety of multi-view clustering analysis approaches have been put forward to uncover the cluster structure hidden in the data, most of these existing methods are based on such a hypothesis: the relationship between objects and clusters are definite. However, most of the data in our real life may have no clear cluster boundaries but have indistinct or overlapping boundaries. How to effectively reveal the uncertain cluster structure under multiview data is still a big challenge for multi-view clustering analysis. Inspired by the powerful uncertain information modeling and analysis capabilities of rough and fuzzy sets, this paper proposes a new multi-view clustering method to discover the uncertain cluster information. A rough set concept based cluster centroid updating strategy is designed to efficiently describe the uncertain construction of clusters. A view weight is introduced to capture the different importance of various views. A fuzzy-based iterative optimization objective function is developed to fuse different view information. Finally, an efficient iterative optimization algorithm is devised to solve the proposed rough fuzzy objective function. Experiments on widely used benchmark datasets prove that our proposed method is always superior to several latest clustering approaches.

Multi-view clustering via pairwise sparse subspace representation

Recent researches on multiview learning have received widespread attention due to the increasing generalization of multiview data. As an effective probabilistic model, random walk has also shown encouraging performance in various fields. To further exploit the potential of utilizing random walk schemes to address multiview learning problems, this article proposes a simple yet efficient multigraph random walk scheme for both multiview clustering and semisupervised classification tasks. The proposed model integrates random walk with multiview learning, and recursively learns a globally stable probability distribution matrix from multiple views, on the basis of which the label indicator is obtained in the scene of clustering or semisupervised classification. Furthermore, an adaptive weight vector is learned to incorporate the diversity and complementarity of multiview data. Besides, the relationships between the proposed scheme and spectral clustering, neighborhood embedding and manifold embedding are analyzed theoretically. Finally, comprehensive comparative experiments are conducted with several state-of-the-art multiview clustering and semisupervised classification methods on eight real-world datasets. The experimental results demonstrate the superiority of the proposed method in terms of both clustering and classification performance.

Multi-view clustering aims to capture the multiple views inherent information by identifying the data clustering that reflects distinct features of datasets. Since there is a consensus in literature that different views of a dataset share a common latent structure, most existing multi-view subspace learning methods rely on the nuclear norm to seek the low-rank representation of the underlying subspace. However, the nuclear norm often fails to distinguish the variance of features for each cluster due to its convex nature and data tends to fall in multiple non-linear subspaces for multi-dimensional datasets. To address these problems, we propose a new and novel multi-view clustering method (HL-L21-TLD-MSC) that unifies the Hyper-Laplacian (HL) and exclusive ℓ 2,1 (L21) regularization with the Tensor Log-Determinant Rank Minimization (TLD) setting. Specifically, the hyper-Laplacian regularization maintains the local geometrical structure that makes the estimation prune to nonlinearities, and the mixed ℓ 2,1 and ℓ 1,2 regularization provides the joint sparsity within-cluster as well as the exclusive sparsity between-cluster. Furthermore, a log-determinant function is used as a tighter tensor rank approximation to discriminate the dimension of features. An efficient alternating algorithm is then derived to optimize the proposed model, and the construction of a convergent sequence to the Karush-Kuhn-Tucker (KKT) critical point solution is mathematically validated in detail. Extensive experiments are conducted on ten well-known datasets to demonstrate that the proposed approach outperforms the existing state-of-the-art approaches with various scenarios, in which, six of them achieve perfect results under our framework developed in this article, demonstrating highl effectiveness for the proposed approach.

Multi-view clustering has become a widely studied problem in the area of unsupervised learning. It aims to integrate multiple views by taking advantages of the consensus and complimentary information from multiple views. Most of the existing works in multi-view clustering utilize the vector-based representation for features in each view. However, in many real-world applications, instances are represented by graphs, where those vector-based models cannot fully capture the structure of the graphs from each view. To solve this problem, in this paper we propose a Multi-view Clustering framework on graph instances with Graph Embedding (MCGE). Specifically, we model the multi-view graph data as tensors and apply tensor factorization to learn the multi-view graph embeddings, thereby capturing the local structure of graphs. We build an iterative framework by incorporating multi-view graph embedding into the multi-view clustering task on graph instances, jointly performing multi-view clustering and multi-view graph embedding simultaneously. The multi-view clustering results are used for refining the multi-view graph embedding, and the updated multi-view graph embedding results further improve the multi-view clustering. Extensive experiments on two real brain network datasets (i.e., HIV and Bipolar) demonstrate the superior performance of the proposed MCGE approach in multi-view connectome analysis for clinical investigation and application.

In the era of big data, data may come from multiple sources, known as multi-view data. Multi-view clustering aims at generating better clusters by exploiting complementary and consistent information from multiple views rather than relying on the individual view. Due to inevitable system errors caused by data-captured sensors or others, the data in each view may be erroneous. Various types of errors behave differently and inconsistently in each view. More precisely, error could exhibit as noise and corruptions in reality. Unfortunately, none of the existing multi-view clustering approaches handle all of these error types. Consequently, their clustering performance is dramatically degraded. In this paper, we propose a novel Markov chain method for Error-Robust Multi-View Clustering (EMVC). By decomposing each view into a shared transition probability matrix and error matrix and imposing structured sparsity-inducing norms on error matrices, we characterize and handle typical types of errors explicitly. To solve the challenging optimization problem, we propose a new efficient algorithm based on Augmented Lagrangian Multipliers and prove its convergence rigorously. Experimental results on various synthetic and real-world datasets show the superiority of the proposed EMVC method over the baseline methods and its robustness against different types of errors.

Incomplete multiview clustering (IMC) methods have achieved remarkable progress by exploring the complementary information and consensus representation of incomplete multiview data. However, to our best knowledge, none of the existing methods attempts to handle the uncoupled and incomplete data simultaneously, which affects their generalization ability in real-world scenarios. For uncoupled incomplete data, the unclear and partial cross-view correlation introduces the difficulty to explore the complementary information between views, which results in the unpromising clustering performance for the existing multiview clustering methods. Besides, the presence of hyperparameters limits their applications. To fill these gaps, a novel uncoupled IMC (UIMC) method is proposed in this article. Specifically, UIMC develops a joint framework for feature inferring and recoupling. The high-order correlations of all views are explored by performing a tensor singular value decomposition (t-SVD)-based tensor nuclear norm (TNN) on recoupled and inferred self-representation matrices. Moreover, all hyperparameters of the UIMC method are updated in an exploratory manner. Extensive experiments on six widely used real-world datasets have confirmed the superiority of the proposed method in handling the uncoupled incomplete multiview data compared with the state-of-the-art methods.