Partial Multi-view Clustering via Auto-Weighting Similarity Completion

Multi-view intact space clustering

We propose a novel multi-view clustering method by learning auto-regression problems under structural constraints and treating the regression coefficients as new feature representations for the cluster partition. In particular, we take the data intrinsic correlation structure into account. Correlated data under one view tend to be also related under another view and are likely to fall into the same group. Therefore we pair the data matrix from one view and the regression coefficient from a different view together to meet a trace Lasso constraint, which adaptively adjusts the sparsity of regression coefficients in order to promote consistent data correlations across views. Then a joint low-rank constraint is further imposed to encourage similar regression coefficients for the same samples under distinct views. Finally, we develop an effective algorithm to optimize the objective function. And experimental results demonstrate that our method is useful and fairly competitive compared with other state-of-the-art multi-view clustering methods.

Information in multiple views is typically symmetrical and distinctive. Multi-view clustering generally attempts to address two key issues: (1) how to mine/leverage additional useful information between different views for current view clustering, and (2) how to combine multiple clustering results from multiple views by using compatible and complementary information hidden in multiple views. In order to achieve this, we propose a multi-view clustering method, namely the multi-view utility-based clustering method (MUC), from the novel perspective of utility-based mutual supervision between different views. The proposed method, MUC, has notable merits: (1) It moves multi-view clustering from the level of feature and/or sample side to partition side information. That is, in order to handle the first issue, MUC considers how to use utility-based partition-level side information from all the other views for current view clustering. Such partition-level side information is consistent with human thinking; it is high-caliber and instructional for clustering. (2) The utility-based partition-level side information provided by other views complements current view information in a mutually supervised way. Then, we conduct multi-view clustering via a mutual supervision mode to circumvent the second issue. As a result, using an alternating optimization strategy, the objective function of MUC can be solved in a K-means-like way. Moreover, we leverage multi-view weight learning based on maximum entropy to integrate multi-view clustering results and further improve performance. The extensive experimental results on various multi-view datasets indicate that the proposed method is better than—or at least comparable to—the existing commonly used single- and multi-view clustering methods, in terms of both clustering performance and running speed.

Deep multi-view subspace clustering has achieved promising performance compared with other multi-view clustering. However, existing deep multi-view subspace clustering only considers the global structure for all views, and they ignore the local geometric structure among each view. In addition, they cannot learn discriminative feature on different clusters of different views, i.e., inter-cluster difference. To solve these problems, in this paper, we propose a novel Deep Multi-view Subspace Clustering with Unified and Discriminative Learning (DMSC-UDL). DMSC-UDL combines global and local structures with self-expression layer. The global and local structures help each other forward and achieve small distance between samples of the same cluster. To make samples in different clusters of different views farther, DMSC-UDL uses a discriminative constraint between different views. In this way, DMSC-UDL makes the same cluster's samples have large weights, while different clusters’ samples have small weights. Thus, it can learn a better shared connection matrix for multi-view clustering. Extensive experimental results reveal that the proposed multi-view clustering method is superior to several state-of-the-art multi-view clustering methods in terms of performance.

Multi-view learning techniques are necessary when data is described by multiple distinct feature sets because single-view learning algorithms tend to overfit on these high-dimensional data. Prior successful approaches followed either consensus or complementary principles. Recent work has focused on learning both the shared and private latent spaces of views in order to take advantage of both principles. However, these methods can not ensure that the latent spaces are strictly independent through encouraging the orthogonality in their objective functions. Also little work has explored representation learning techniques for multi-view learning. In this paper, we use the denoising autoencoder to learn shared and private latent spaces, with orthogonal constraints -- disconnecting every private latent space from the remaining views. Instead of computationally expensive optimization, we adapt the backpropagation algorithm to train our model.

Multi-view clustering via pairwise sparse subspace representation

Multi-view clustering aims to capture the multiple views inherent information by identifying the data clustering that reflects distinct features of datasets. Since there is a consensus in literature that different views of a dataset share a common latent structure, most existing multi-view subspace learning methods rely on the nuclear norm to seek the low-rank representation of the underlying subspace. However, the nuclear norm often fails to distinguish the variance of features for each cluster due to its convex nature and data tends to fall in multiple non-linear subspaces for multi-dimensional datasets. To address these problems, we propose a new and novel multi-view clustering method (HL-L21-TLD-MSC) that unifies the Hyper-Laplacian (HL) and exclusive ℓ 2,1 (L21) regularization with the Tensor Log-Determinant Rank Minimization (TLD) setting. Specifically, the hyper-Laplacian regularization maintains the local geometrical structure that makes the estimation prune to nonlinearities, and the mixed ℓ 2,1 and ℓ 1,2 regularization provides the joint sparsity within-cluster as well as the exclusive sparsity between-cluster. Furthermore, a log-determinant function is used as a tighter tensor rank approximation to discriminate the dimension of features. An efficient alternating algorithm is then derived to optimize the proposed model, and the construction of a convergent sequence to the Karush-Kuhn-Tucker (KKT) critical point solution is mathematically validated in detail. Extensive experiments are conducted on ten well-known datasets to demonstrate that the proposed approach outperforms the existing state-of-the-art approaches with various scenarios, in which, six of them achieve perfect results under our framework developed in this article, demonstrating highl effectiveness for the proposed approach.

Incomplete multiview clustering (IMC) methods have achieved remarkable progress by exploring the complementary information and consensus representation of incomplete multiview data. However, to our best knowledge, none of the existing methods attempts to handle the uncoupled and incomplete data simultaneously, which affects their generalization ability in real-world scenarios. For uncoupled incomplete data, the unclear and partial cross-view correlation introduces the difficulty to explore the complementary information between views, which results in the unpromising clustering performance for the existing multiview clustering methods. Besides, the presence of hyperparameters limits their applications. To fill these gaps, a novel uncoupled IMC (UIMC) method is proposed in this article. Specifically, UIMC develops a joint framework for feature inferring and recoupling. The high-order correlations of all views are explored by performing a tensor singular value decomposition (t-SVD)-based tensor nuclear norm (TNN) on recoupled and inferred self-representation matrices. Moreover, all hyperparameters of the UIMC method are updated in an exploratory manner. Extensive experiments on six widely used real-world datasets have confirmed the superiority of the proposed method in handling the uncoupled incomplete multiview data compared with the state-of-the-art methods.

With the widespread existence of multi-view data, many multi-view clustering methods have emerged. The purpose of multi-view clustering is to classify data into multiple clusters based on different views. Existing multi-view clustering methods usually do not sufficiently mine the complementary information of data, which makes the effective information of multi-view data cannot be fully utilized. By considering the diversity of different views, we propose a new multi-view subspace clustering method. Specifically, we first extend single-view self-expression learning to the multi-view domain. Then, based on manifold learning, the public information of multi-view data is obtained. In addition, diversity among the data was measured using the Hilbert-Schmidt Independence Criteria (HSIC). Experimental results on four datasets show that our model has good clustering performance on different evaluation indicators.

In recent years, multiview clustering algorithms have achieved promising performance by exploiting the complementarity and consistency of different views. However, many multiview spectral clustering methods only focus on the consistent information of views, and the time cost of feature decomposition is expensive. Moreover, these methods also require postprocessing (e.g., k-means) to obtain the final clustering results. To overcome these limitations simultaneously, we propose a novel multiview clustering algorithm. First, the method removes the inconsistent information of the views through cross-view measurement to maintain consistent information. These inconsistencies may be caused by noise, corruptions, or view-specific properties and will affect the quality of the similarity matrix. Then, we learn a consensus embedding matrix with nonnegative constraints by performing a low-rank decomposition of the consistency information. In this way, we can replace the eigendecomposition of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> Laplacian matrix in spectral clustering with the singular value decomposition of a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\times }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> low-rank matrix to reduce the computational burden, where <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</i> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\bm {\ll }$</tex-math></inline-formula> <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> . Furthermore, due to the nonnegative constraint, we can directly obtain the clustering results. Also, to consider the diversity of views, adaptive weighting is applied to different view data. Compared to state-of-the-art multiview clustering methods on five benchmark multiview datasets, we demonstrate the superiority and effectiveness of our approach. We release the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hulu88/FAMvC</uri> .

Multiview clustering via binary representation has attracted intensive attention due to its effectiveness in handling large-scale multiple view data. However, these kind of clustering approaches usually ignore a very important potential high-order correlation in discrete representation learning. In this article, we propose a novel all-in collaborative multiview binary representation for clustering (AC-MVBC) framework, where multiview collaborative binary representation and clustering structure are learned in a joint manner. Specifically, using a new type of tensor low-rank constraint, the high-order collaborations, i.e., cross-view and inner view collaborations, can be effectively captured in our model. Moreover, by incorporating the Bregman discrepancy, the projective consistency among different views can be guaranteed to achieve a more powerful binary representation. An efficient optimization algorithm is also proposed to solve the objective function with fast convergence empirically. Experimental results on several challenge datasets demonstrate that the proposed method has achieved highly competent performance compared with the state-of-the-art multiview clustering (MVC) methods while maintaining low computational and memory requirements.

Multiview clustering (MVC) can achieve more accurate results by utilizing complementary information from multiple perspectives, compared to traditional single-view methods. However, current multiview techniques require all views to be available upfront, making them inadequate for dealing with prevalent data sources that arrive as streams, such as stem cell analysis and multicamera surveillance. To address this problem, in this article, we propose a method called lifelong stream-view clustering (LSVC), which comprises an embedding anchor knowledge library and three key components, enabling the capability to perform asynchronous clustering on stream views. These three components are specifically: 1) the knowledge extraction module that extracts the abstract knowledge of the newcome view over time and updates the shared knowledge library; 2) the knowledge transfer module that aligns the newcome view with the historical knowledge library, enabling the transfer of structure information to the knowledge library; and 3) the knowledge rule module that constraints the knowledge library to enjoy a fair amount of anchors for each cluster, improving the discrimination of knowledge. The experimental results show that LSVC outperforms traditional single-view clustering (SVC) and MVC methods as it gradually improves with the accumulation of stream views and tends to be stable over time.

Multi-view clustering (MVC) can fuse the information of multiple views for robust clustering result, among it two fusion strategies, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">early-fusion</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">late-fusion</i> are widely adopted. Although they have derived many MVC methods, there are still two crucial questions: (1) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">early-fusion</i> forces multiple views to share a consensus latent representation, which compounds the challenge of excavating view-specific diverse local information; (2) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">late-fusion</i> generates view-partitions independently and then integrates them in the following clustering procedure, where the two procedures cannot guide each other and lack necessary negotiation. In view of this, we propose a novel Graph Proxy Fusion (GPF) method to preserve and fuse view-specific local information concertedly in one unified framework. Specifically, we first propose anchor-based local information learning to capture view-specific local structural information in bipartite graphs; meanwhile, a view-consensus graph learned through self-expressiveness-based proxy graph learning module is deemed as a higher-order proxy; following, the novel graph proxy fusion module integrally embeds all lower-order bipartite graphs in the higher-order proxy via higher-order correlation theory. As a novel fusion strategy, the proposed GPF efficiently investigates the valuable consensus and diverse information of multiple views. Experiments on various multi-view datasets demonstrate the superiority of our method.

Multi-view clustering, as one of the most important methods to analyze multi-view data, has been widely used in many real-world applications. Most existing multi-view clustering methods perform well on the assumption that each sample appears in all views. Nevertheless, in real-world application, each view may well face the problem of the missing data due to noise, or malfunction. In this paper, a new consistent generative adversarial network is proposed for partial multi-view clustering. We learn a common low-dimensional representation, which can both generate the missing view data and capture a better common structure from partial multi-view data for clustering. Different from the most existing methods, we use the common representation encoded by one view to generate the missing data of the corresponding view by generative adversarial networks, then we use the encoder and clustering networks. This is intuitive and meaningful because encoding common representation and generating the missing data in our model will promote mutually. Experimental results on three different multi-view databases illustrate the superiority of the proposed method.

Anchor-based multi-view clustering has received extensive attention due to its efficient performance. Existing methods only focus on how to dynamically learn anchors from the original data and simultaneously construct anchor graphs describing the relationships between samples and perform clustering, while ignoring the reality of anchors, i.e., high-quality anchors should be generated uniformly from different clusters of data rather than scattered outside the clusters. To deal with this problem, we propose a noval method termed Anchor Learning with Potential Cluster Constraints for Multi-view Clustering (ALPC) method. Specifically, ALPC first establishes a shared latent semantic module to constrain anchors to be generated from specific clusters, and subsequently, ALPC improves the representativeness and discriminability of anchors by adapting the anchor graph to capture the common clustering center of mass from samples and anchors, respectively. Finally, ALPC combines anchor learning and graph construction into a unified framework for collaborative learning and mutual optimization to improve the clustering performance. Extensive experiments demonstrate the effectiveness of our proposed method compared to some state-of-the-art MVC methods.