Bridging Optimization and Neural Networks for Efficient Multi-view Clustering

In the era of Industry 4.0, single‐view clustering algorithm is difficult to play a role in the face of complex data, i.e., multiview data. In recent years, an extension of the traditional single‐view clustering is multiview clustering technology, which is becoming more and more popular. Although the multiview clustering algorithm has better effectiveness than the single‐view clustering algorithm, almost all the current multiview clustering algorithms usually have two weaknesses as follows. (1) The current multiview collaborative clustering strategy lacks theoretical support. (2) The weight of each view is averaged. To solve the above‐mentioned problems, we used the Havrda‐Charvat entropy and fuzzy index to construct a new collaborative multiview fuzzy c‐means clustering algorithm using fuzzy weighting called Co‐MVFCM. The corresponding results show that the Co‐MVFCM has the best clustering performance among all the comparison clustering algorithms.

Multi-view clustering has become a widely studied problem in the area of unsupervised learning. It aims to integrate multiple views by taking advantages of the consensus and complimentary information from multiple views. Most of the existing works in multi-view clustering utilize the vector-based representation for features in each view. However, in many real-world applications, instances are represented by graphs, where those vector-based models cannot fully capture the structure of the graphs from each view. To solve this problem, in this paper we propose a Multi-view Clustering framework on graph instances with Graph Embedding (MCGE). Specifically, we model the multi-view graph data as tensors and apply tensor factorization to learn the multi-view graph embeddings, thereby capturing the local structure of graphs. We build an iterative framework by incorporating multi-view graph embedding into the multi-view clustering task on graph instances, jointly performing multi-view clustering and multi-view graph embedding simultaneously. The multi-view clustering results are used for refining the multi-view graph embedding, and the updated multi-view graph embedding results further improve the multi-view clustering. Extensive experiments on two real brain network datasets (i.e., HIV and Bipolar) demonstrate the superior performance of the proposed MCGE approach in multi-view connectome analysis for clinical investigation and application.

Recently, numerous multi-view clustering (MVC) and multi-view graph clustering (MVGC) methods have been proposed. Despite significant progress, they still face two issues: I) MVC and MVGC are often developed independently for multi-view and multi-graph data. They have redundancy but lack a unified methodology to combine their strengths. II) Contrastive learning is usually adopted to explore the associations across multiple views. However, traditional contrastive losses ignore the neighbor relationship in multi-view scenarios and easily lead to false associations in sample pairs. To address these issues, we propose Graph Embedded Contrastive Learning for Multi-View Clustering. Concretely, we propose a process of view-specific pre-training with adaptive graph convolution to make our method compatible with both multi-view and multi-graph data, which aggregates the graph information into data and leverages autoencoders to learn view-specific representations. Furthermore, to explore the view-cross associations, we introduce the process of view-cross contrastive learning and clustering, where we propose the graph-guided contrastive learning that can generate global graph to mitigate the false association issue as well as the cluster-guided contrastive clustering for improving the model robustness. Finally, extensive experiments demonstrate that our method achieves superior performance on both MVC and MVGC tasks.

Nowadays, multi-view dataset have become ubiquitous along with more and more data are gathered from different measuring technologies or various sources, in which various aspects of dataset are formalized as multiple views. Although a variety of multi-view clustering analysis approaches have been put forward to uncover the cluster structure hidden in the data, most of these existing methods are based on such a hypothesis: the relationship between objects and clusters are definite. However, most of the data in our real life may have no clear cluster boundaries but have indistinct or overlapping boundaries. How to effectively reveal the uncertain cluster structure under multiview data is still a big challenge for multi-view clustering analysis. Inspired by the powerful uncertain information modeling and analysis capabilities of rough and fuzzy sets, this paper proposes a new multi-view clustering method to discover the uncertain cluster information. A rough set concept based cluster centroid updating strategy is designed to efficiently describe the uncertain construction of clusters. A view weight is introduced to capture the different importance of various views. A fuzzy-based iterative optimization objective function is developed to fuse different view information. Finally, an efficient iterative optimization algorithm is devised to solve the proposed rough fuzzy objective function. Experiments on widely used benchmark datasets prove that our proposed method is always superior to several latest clustering approaches.

Multi-view clustering via pairwise sparse subspace representation

In the era of big data, data may come from multiple sources, known as multi-view data. Multi-view clustering aims at generating better clusters by exploiting complementary and consistent information from multiple views rather than relying on the individual view. Due to inevitable system errors caused by data-captured sensors or others, the data in each view may be erroneous. Various types of errors behave differently and inconsistently in each view. More precisely, error could exhibit as noise and corruptions in reality. Unfortunately, none of the existing multi-view clustering approaches handle all of these error types. Consequently, their clustering performance is dramatically degraded. In this paper, we propose a novel Markov chain method for Error-Robust Multi-View Clustering (EMVC). By decomposing each view into a shared transition probability matrix and error matrix and imposing structured sparsity-inducing norms on error matrices, we characterize and handle typical types of errors explicitly. To solve the challenging optimization problem, we propose a new efficient algorithm based on Augmented Lagrangian Multipliers and prove its convergence rigorously. Experimental results on various synthetic and real-world datasets show the superiority of the proposed EMVC method over the baseline methods and its robustness against different types of errors.

Multi-view clustering, as one of the most important methods to analyze multi-view data, has been widely used in many real-world applications. Most existing multi-view clustering methods perform well on the assumption that each sample appears in all views. Nevertheless, in real-world application, each view may well face the problem of the missing data due to noise, or malfunction. In this paper, a new consistent generative adversarial network is proposed for partial multi-view clustering. We learn a common low-dimensional representation, which can both generate the missing view data and capture a better common structure from partial multi-view data for clustering. Different from the most existing methods, we use the common representation encoded by one view to generate the missing data of the corresponding view by generative adversarial networks, then we use the encoder and clustering networks. This is intuitive and meaningful because encoding common representation and generating the missing data in our model will promote mutually. Experimental results on three different multi-view databases illustrate the superiority of the proposed method.

Multi-view clustering has achieved advanced progress over the years, which typically integrates multi-view information to learn discriminative common representations or a unified clustering distribution for clustering. However, existing methods either simply regard each view as equally important or assign fixed weights to each view, which are insufficient to dynamically assess the sample quality variations caused by noise in multi-view data. To address this issue, by effectively modeling the uncertainty of different samples across different views, this paper proposes a novel uncertainty-aware multi-view graph clustering network, termed UMGC-Net, which achieves trusted multi-view clustering in an unsupervised manner. Specifically, by measuring the clustering distribution entropy, an uncertainty-guided common feature learning mechanism is proposed to estimate the uncertainty for each sample of each view, thus learning multi-view features friendly to clustering. Besides, a cross-view trusted distribution fusion module is designed to obtain robust clustering distribution by exploring the trusted consistency among multi-view clustering distributions based on uncertainty. Finally, experimental results on four popular multi-view datasets validate the superior performance of the proposed UMGC-Net.

Multiview clustering (MVC) is a popular research topic in the fields of data mining and pattern recognition, which focuses on fully exploring and employing the correlations between views to jointly discover a consistent cluster structure across data. Typically, weighted MVC is a common clustering method aimed at learning the importance or weights of each view and applying them to explore the complementary information between views. However, current weighted MVCs primarily focus on the quality of each view while overlooking the crucial role of pseudo-label-based self-supervision in weight learning. In addition, most weighted MVCs only use a weighting mechanism to utilize complementary features without sufficiently considering the consistency relationship between the clustering results of individual views and the final clustering result. Aiming to solve the above problems, this article proposes a structure-enhanced self-supervised weighted information bottleneck (S2WIB) method for MVC. Specifically, the S2WIB method establishes a view-weight learning mechanism that leverages both the view-contained information and the self-supervised information to learn view weights, and then integrates the weighted information from different views. Meanwhile, based on the information bottleneck (IB) theory, it explores view correlations from two perspectives, namely complementary information and consistent view cluster structure information, thereby fully exploiting the potential information contained in multiview data. Experiments on various multiview text datasets, multifeature image datasets, multiangle video datasets, multimodal text-image datasets, as well as large-scale datasets and biological multiomics datasets, demonstrate that the S2WIB method performs effectively and exhibits superiority across various fields.

Information in multiple views is typically symmetrical and distinctive. Multi-view clustering generally attempts to address two key issues: (1) how to mine/leverage additional useful information between different views for current view clustering, and (2) how to combine multiple clustering results from multiple views by using compatible and complementary information hidden in multiple views. In order to achieve this, we propose a multi-view clustering method, namely the multi-view utility-based clustering method (MUC), from the novel perspective of utility-based mutual supervision between different views. The proposed method, MUC, has notable merits: (1) It moves multi-view clustering from the level of feature and/or sample side to partition side information. That is, in order to handle the first issue, MUC considers how to use utility-based partition-level side information from all the other views for current view clustering. Such partition-level side information is consistent with human thinking; it is high-caliber and instructional for clustering. (2) The utility-based partition-level side information provided by other views complements current view information in a mutually supervised way. Then, we conduct multi-view clustering via a mutual supervision mode to circumvent the second issue. As a result, using an alternating optimization strategy, the objective function of MUC can be solved in a K-means-like way. Moreover, we leverage multi-view weight learning based on maximum entropy to integrate multi-view clustering results and further improve performance. The extensive experimental results on various multi-view datasets indicate that the proposed method is better than—or at least comparable to—the existing commonly used single- and multi-view clustering methods, in terms of both clustering performance and running speed.

In recent years, multiview clustering algorithms have achieved promising performance by exploiting the complementarity and consistency of different views. However, many multiview spectral clustering methods only focus on the consistent information of views, and the time cost of feature decomposition is expensive. Moreover, these methods also require postprocessing (e.g., k-means) to obtain the final clustering results. To overcome these limitations simultaneously, we propose a novel multiview clustering algorithm. First, the method removes the inconsistent information of the views through cross-view measurement to maintain consistent information. These inconsistencies may be caused by noise, corruptions, or view-specific properties and will affect the quality of the similarity matrix. Then, we learn a consensus embedding matrix with nonnegative constraints by performing a low-rank decomposition of the consistency information. In this way, we can replace the eigendecomposition of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> Laplacian matrix in spectral clustering with the singular value decomposition of a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> low-rank matrix to reduce the computational burden, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\ll }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> . Furthermore, due to the nonnegative constraint, we can directly obtain the clustering results. Also, to consider the diversity of views, adaptive weighting is applied to different view data. Compared to state-of-the-art multiview clustering methods on five benchmark multiview datasets, we demonstrate the superiority and effectiveness of our approach. We release the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hulu88/FAMvC</uri> .

Clustering with multiview data is becoming a hot topic in data mining, pattern recognition, and machine learning. In order to realize an effective multiview clustering, two issues must be addressed, namely, how to combine the clustering result from each view and how to identify the importance of each view. In this paper, based on a newly proposed objective function which explicitly incorporates two penalty terms, a basic multiview fuzzy clustering algorithm, called collaborative fuzzy c-means (Co-FCM), is firstly proposed. It is then extended into its weighted view version, called weighted view collaborative fuzzy c-means (WV-Co-FCM), by identifying the importance of each view. The WV-Co-FCM algorithm indeed tackles the above two issues simultaneously. Its relationship with the latest multiview fuzzy clustering algorithm Collaborative Fuzzy K-Means (Co-FKM) is also revealed. Extensive experimental results on various multiview datasets indicate that the proposed WV-Co-FCM algorithm outperforms or is at least comparable to the existing state-of-the-art multitask and multiview clustering algorithms and the importance of different views of the datasets can be effectively identified.

Recently, a variety of data acquisition methods leads to the description of the same thing from different angles, which induces a large number of multi-view data. Clustering multi-view data in large-scale data analysis is of great significance and very challenging. In this paper, we present a new robust clustering algorithm, the Multi-view Fuzzy K-means clustering (MFKC) algorithm. In MFKC, a fuzzy membership method is used and optimized to describe the probability of a sample belonging to certain clusters. Compared with the hard partition based multi-view clustering algorithms, MFKC can effectively describe the fuzzy relationship between multi-view data and all clusters, and has better interpretability for the clustering results. In addition, MFKC also sets different weights for different views. By optimizing the multi-view clustering objective function, we obtain the optimal weigh of each view to reflect the importance of the view. The feasibility and effectiveness of the proposed algorithm is validated through various experimental comparisons.

Multi-view clustering via high-order bipartite graph fusion

Since advanced technologies via social media, internet, virtual communities and networks and internet of things (IoT), there are more multi-view data to be collected. Multi-view clustering is a substantial tool as a natural design for clustering multi-view data. K-means (KM) clustering for (single-view) data had been extended for handling multi-view data, called multi-view KM (MV-KM). In the literature, most MV-KM clustering algorithms are reported to be influenced by initializations and also need a given number of clusters. In this paper, we propose an unsupervised type of MV-KM clustering algorithm so that it can automatically find an optimal number of clusters without any initialization. We call it unsupervised MV-KM (U-MV-KM). Moreover, we also propose three multi-view cluster validity indices, called multi-view Dunn index (MV-Dunn), multi-view generalized Dunn index (MV-G-Dunn) and multi-view modified Dunn (MV-M-Dunn) indices for MV-KM clustering algorithms. We make experiments on some synthetic and real data sets and also make comparisons with some existing algorithms. Based on the experimental results and comparisons, the proposed U-MV-KM clustering algorithm actually shows good results. We also apply U-MV-KM to real data sets, the results demonstrate the superiority and usefulness of the U-MV-KM algorithm for clustering multi-view data.