Essential multi-view graph learning for clustering

Self-taught Multi-view Spectral Clustering

Significant efforts have been focused on enhancing the utilization of multiple node features and topological structures in multi-view graph learning through explicit model-driven and implicit deep learning-based methodologies. The former excels in embedding prior knowledge, thereby offering theoretical interpretability but is limited in application flexibility due to manual parameter selection. In contrast, the latter leverages automatic differentiation, providing greater flexibility but lacking theoretical interpretability due to their opaque nature. Motivated by these observations, we propose an interpretable deep unfolding network for mutual-benefit multi-view graph learning, aiming to combine the strengths of both approaches. Specifically, we employ the Alternating Direction Method of Multipliers (ADMM) to solve a multi-view graph learning model with sparse and low-rank constraints. This solution is then integrated into deep unfolding networks to enhance interpretability. Furthermore, we convert optimization conditions into implicit losses and utilize automatic differentiation to update parameters, reducing the need for manual tuning and increasing flexibility. This integration optimizes multi-view learning for a graph representation that balances interpretability and flexibility. Empirical evaluations on six diverse datasets demonstrate the effectiveness and superiority of the proposed method over state-of-the-art approaches.

Previous multi-view spectral clustering methods are a two-step strategy, which first learns a fixed common representation (or common affinity matrix) of all the views from original data and then conducts k-means clustering on the resulting common affinity matrix. The two-step strategy is not able to output reasonable clustering performance since the goal of the first step (i.e., the common affinity matrix learning) is not designed for achieving the optimal clustering result. Moreover, the two-step strategy learns the common affinity matrix from original data, which often contain noise and redundancy to influence the quality of the common affinity matrix. To address these issues, in this paper, we design a novel One-step Multi-view Spectral Clustering (OMSC) method to output the common affinity matrix as the final clustering result. In the proposed method, the goal of the common affinity matrix learning is designed to achieving optimal clustering result and the common affinity matrix is learned from low-dimensional data where the noise and redundancy of original high-dimensional data have been removed. We further propose an iterative optimization method to fast solve the proposed objective function. Experimental results on both synthetic datasets and public datasets validated the effectiveness of our proposed method, comparing to the state-of-the-art methods for multi-view clustering.

Driven by multi-omics data, some multi-view clustering algorithms have been successfully applied to cancer subtypes prediction, aiming to identify subtypes with biometric differences in the same cancer, thereby improving the clinical prognosis of patients and designing personalized treatment plan. Due to the fact that the number of patients in omics data is much smaller than the number of genes, multi-view spectral clustering based on similarity learning has been widely developed. However, these algorithms still suffer some problems, such as over-reliance on the quality of pre-defined similarity matrices for clustering results, inability to reasonably handle noise and redundant information in high-dimensional omics data, ignoring complementary information between omics data, etc. This paper proposes multi-view spectral clustering with latent representation learning (MSCLRL) method to alleviate the above problems. First, MSCLRL generates a corresponding low-dimensional latent representation for each omics data, which can effectively retain the unique information of each omics and improve the robustness and accuracy of the similarity matrix. Second, the obtained latent representations are assigned appropriate weights by MSCLRL, and global similarity learning is performed to generate an integrated similarity matrix. Third, the integrated similarity matrix is used to feed back and update the low-dimensional representation of each omics. Finally, the final integrated similarity matrix is used for clustering. In 10 benchmark multi-omics datasets and 2 separate cancer case studies, the experiments confirmed that the proposed method obtained statistically and biologically meaningful cancer subtypes.

Multi-view semi-supervised learning for classification on dynamic networks

Adaptive multi-view subspace learning based on distributed optimization

Nonconvex low-rank tensor approximation with graph and consistent regularizations for multi-view subspace learning

This paper discusses about a sparsity based algorithm used for Hyperspectral Image (HSI) classification where the test pixel vectors are sparsely represented as the linear combination of a few number of training samples from a well-organised dictionary matrix. The sparse vector is obtained using Basis Pursuit (BP) which is a constrained l4 minimization problem. This problem is solved by using a simple and powerful iterative algorithm known as Alternating Direction Method of Multipliers (ADMM) which significantly reduces the computational complexity of the problem and thereby speeds up the convergence. The classification accuracy is considerably improved by including efficient preprocessing techniques to remove the unwanted information (noise) present in Hyperspectral images. This paper uses a fast and reliable denoising technique based on Legendre Fenchel Transformation (LFT) to effectively denoise each band of HSI prior to ADMM based classification (proposed method). A comparison of proposed technique with one of the convex optimization tools namely, CVX is given to exhibit the fast convergence of the former method. The experiment is performed on standard Indian Pines dataset captured using AVIRIS sensor. The potential of the proposed method is illustrated by analyzing the classification indices obtained with and without applying any preprocessing methods. With only 10% training set, an overall accuracy of 96.76% is obtained for the proposed method at a much faster rate compared to computation time taken by CVX solver.

Most existing active learning methods often manually label samples and train models with labeled data in an iterative way. Unfortunately, at the early stage of the experiment, few labeled data are available, hence, selecting the most valuable data points to label is necessary and important. To this end, we propose a novel method, called Multi-view Representative and Informative-induced Active Learning (MRI-AL), which selects samples of both representativeness and informativeness with the help of complementarity of multiple views. Specifically, subspace reconstruction with structure sparsity technique is employed to ensure the selected samples to be representative, while the global similarity constraint guarantees the informativeness of the selected samples. The proposed method is solved efficiently by alternating direction method of multipliers (ADMM). We empirically show that our method outperforms existing early experimental design approaches.

The performance of adaptive beamforming suffers from the imprecise knowledge of the array or signal of interest. Many efforts have been made to develop various beamformers and increase their robustness to these unknown errors. This study proposes a new robust adaptive beamformer based on the alternating direction method of multipliers (ADMM). The proposed method is introduced first as an optimisation problem to estimate the steering vector (SV). Owing to its non-convex property, it is converted to a real-valued condition problem by introducing auxiliary variables. Then, the optimisation problem is reformulated as the augmented Lagrangian function. Finally, the estimated SV can be solved by the ADMM. Numerical results prove the effectiveness of the proposed method.

Data collection plays an important role in wireless sensor networks. Recovery of spatio-temporal data from incomplete sensing data is vital to the network lifetime. Many works have utilized the spatial and temporal correlations to achieve satisfactory data recovery results. However, these methods introduce large computational overhead at the fusion center. In this paper, we develop an ADMM-Net framework for correlated spatio-temporal data recovery. Both the spatial correlation and temporal correlation of sensing data are considered in a convex optimization problem, which is solved by the alternating direction method of multipliers (ADMM) algorithm. We then unfold the ADMM algorithm into a fixed-length neural network that reduces the iterations dramatically and does not require additional location information of nodes. Experimental results on a realworld dataset demonstrate that the proposed method can achieve faster convergence speed than the baseline ADMM algorithm with slight accuracy loss.

This paper studies the fairness of achievable energy efficiency (EE) in a multicell multiuser multiple-input single-output downlink. The objective is to maximize the minimum EE among all base stations (BSs) subject to per-BS power constraints. The resulting optimization problem is a max-min fractional program, and, thus, difficult to solve in general. Our goal is to develop a decentralized algorithm for the max-min EE problem which solves the problem locally. The idea behind the proposed method is to combine the framework of successive convex approximation (SCA) and alternative direction method of multipliers (ADMM). We transform the convex program obtained at each step of the SCA procedure into a form that lends itself to the ADMM. The resulting formulation is solved optimally by allowing the BSs to exchange the required information until the ADMM converges. In addition to further reduce the backhaul overhead, the proposed algorithm is modified to enhance the convergence speed. Numerical results are provided to demonstrate the effectiveness of the proposed algorithms.

We discuss an online decentralized decision making problem where the agents are coupled with affine inequality constraints. Alternating Direction Method of Multipliers (ADMM) is used as the computation engine and we discuss the convergence of the algorithm in an online setting. To be specific, when decisions have to be made sequentially with a fixed time step, there might not be enough time for the ADMM to converge before the scenario changes and the decision needs to be updated. In this case, a suboptimal solution is employed and we analyze the optimality gap given the convergence condition. Moreover, in many cases, the decision making problem changes gradually over time. We propose a warm-start scheme to accelerate the convergence of ADMM and analyze the benefit of the warm-start. The proposed method is demonstrated in a decentralized multiagent control barrier function problem with simulation.

Graph learning for latent-variable Gaussian graphical models under laplacian constraints

This paper presents two strategies to speed up the alternating direction method of multipliers (ADMM) for distributed data. In the first method, inspired by stochastic gradient descent, each machine uses only a subset of its data at the first few iterations, speeding up those iterations. A key result is in proving that despite this approximation, our method enjoys the same convergence rate in terms of the number of iterations as the standard ADMM, and hence is faster overall. The second method also follows the idea of sampling a subset of the data to update the model before the communication of each round. It converts an objective to the approximated dual form and performs ADMM on the dual. The method turns out to be a distributed variant of the recently proposed SDCA-ADMM. Yet, compared to the straightforward distributed implementation of SDCA-ADMM, the proposed method enjoys less frequent communication between machines, better memory usage, and lighter computational demand. Experiments demonstrate the effectiveness of our two strategies.