Self-Adjusted Consensus Clustering with Agglomerate Algorithms

Consensus clustering and semi-supervised clustering are important extensions of the standard clustering paradigm. Consensus clustering (also known as aggregation of clustering) can improve clustering robustness, deal with distributed and heterogeneous data sources and make use of multiple clustering criteria. Semi-supervised clustering can integrate various forms of background knowledge into clustering. In this paper, we show how consensus and semi-supervised clustering can be formulated within the framework of nonnegative matrix factorization (NMF). We show that this framework yields NMF-based algorithms that are: (1) extremely simple to implement; (2) provably correct and provably convergent. We conduct a wide range of comparative experiments that demonstrate the effectiveness of this NMF-based approach.

Consensus clustering methods are motivated by the success of combining multiple classifiers in many areas. In this paper, graph-based consensus clustering is used to improve the quality of chemical compound clustering by enhancing the robustness, novelty, consistency and stability of individual clusterings. For this purpose, Hyper-Graph Partitioning Algorithm (HGPA) [1], was applied. The clustering is evaluated based on the ability to separate actives from inactives molecules in each cluster and the results were compared with the Ward's clustering method. The chemical dataset MDL Drug Data Report (MDDR) database has been used for experiments. The MDL Drug Data Report (MDDR) database consists of 102516 molecules. For the experiments, the dataset DS1 was chosen from the MDDR database. This dataset has been used for many virtual screening experiments [2-4]. The dataset DS1contains 10 heterogeneous activity classes (8568 molecules). For the clustering experiments, two 2D fingerprint descriptors will be used which are developed by Scitegic's Pipeline Pilot [5]. These are 120-bit ALOGP and 1024-bit extended connectivity fingerprints (ECFP_4). The results were evaluated based on the effectiveness of the methods to separate actives from non-actives molecules using QPI- (for quality partition index) measure, which was devised by Varin et al. [6]. As defined by [7], an active cluster as a non-singleton cluster for which the percentage of active molecules in the cluster is greater than the percentage of active molecules in the dataset as a whole. Let p be the number of actives in active clusters, q the number of inactives in active clusters, r the number of actives in inactive clusters (i.e., clusters that are not active clusters) and s the number of singleton actives. The high value occurs when the actives are clustered tightly together and separated from the inactive molecules. Then the quality partition index, QPI, is defined to be: QPI=pp+q+r+s (1) Then, the results will be compared with Ward's individual clustering method, the standard clustering method for chemoinformatics applications. The generation process has been done by multiple run of K-means algorithms, each with random initialization of cluster centroids. The number of partitions generated in this step is ranged between n = 5 to n = 50, with 5-times step. Then, all the generated partitions were combined using HGPA to obtain the consensus partition. This process is done for each fingerprint (ALOGP and ECFP_4). The mean of QPI values are averaged over the ten activity classes of the datasets. Tables ​Tables1,1, ​,22 show the effectiveness of MDDR dataset clustering using ALOGP and ECFP_4 fingerprints. The best PQI value of consensus clustering methods for each column has been bold-faced for ease of reference. Table 1 Effectivenss of clustering of high diverse MDDR dataset: ALOGP Fingerprint. Table 2 Effectivenss of clustering of high diverse MDDR dataset: ECFP_4 Fingerprint. Visual inspection of the results enables comparisons to be made between the effectiveness of clustering of MDDR datasets and Ward's method, the best of choice clustering method for chemoinformatics applications. In addition, ten times of consensus clustering, for each fingerprint were observed in order to study the effectiveness of consensus clustering with different ensemble sizes. The results show that HGPA consensus clustering gives robust and novel result when K-means algorithm is run 20-50 times using ALOGP. The performance of consensus clustering outperforms the Wards' method. For consensus of dataset which represented by ECFP_4 fingerprint, the best QPI values of consensus clustering are obtained from ensembles of size n = 20-50. The performance of consensus clustering gives robust results which are better than overall performance of individual clusterings. The values of QPI in both datasets for consensus clustering are close to the Wards method. The consensus clustering, HGPA, provide stable clusters by decreasing the sensitivity to noise and outliers. The average percentages of singleton clusters of individual clusterings compared with consensus clustering for both fingerprints. The results show that consensus clustering partition the datasets with average percentage of singleton equal to zero, which is much better than individual clusterings and Wards' method. For example, 16.72% of molecules of DS1 are clustered as singletons when Wards method is applied on ALOGP fingerprint with number of clusters equal to 1000 clusters. Finally we conclude that graph-based consensus clustering can improve the effectiveness of chemical compounds clustering. The performance of consensus clustering is more robust, novel, stable, consistent, and out-perform Wards' method in case of using ALOGP fingerprint. By using ECFP_4 fingerprint, consensus clustering methods provide more robust, stable, and consistent clustering and close to the Wards clustering results. The experiments reported here suggest that graph-based consensus clustering can improve the quality of individual clustering by using the efficient algorithm, K-means algorithm, to generate the ensemble with size (20-50) for both structurally diverse chemical datasets.

Unsupervised consensus analysis for on-line review and questionnaire data

This study addresses the cluster consensus problem for generic linear heterogeneous multi‐agent systems. The main purpose of this study is to unveil how agents confront the effects of intra‐cluster couplings and inter‐cluster couplings, and reach cluster consensus. For the semi‐heterogeneous case, where agents in the same cluster have identical dynamics, cluster consensus is achieved if each agent suppresses the inter‐cluster couplings. For the heterogeneous case, where all agents have different dynamics, cluster consensus is achieved if each agent confronts both inter‐cluster and intra‐cluster couplings. Two kinds of control protocols using relative output errors are designed for the heterogeneous case. For both semi‐heterogeneous and heterogeneous cases, design procedures are provided to construct controllers to satisfy the relation between the dynamics of agents and the communication graph, which ensure cluster consensus. Finally, numerical examples are provided to verify the theoretical results.

This paper investigates the cluster consensus problem for second-order multi-agent systems by applying the pinning control method to a small collection of the agents. Consensus is attained independently for different agent clusters according to the community structure generated by the group partition of the underlying graph and sufficient conditions for both cluster and general consensus are obtained by using results from algebraic graph theory and the LaSalle Invariance Principle. Finally, some simple simulations are presented to illustrate the technique.

Interactive textual feature selection for consensus clustering

Consensus clustering has been widely used in bioinformatics and other applications to improve the accuracy, stability and reliability of clustering results. This approach ensembles cluster co-occurrences from multiple clustering runs on subsampled observations. For application to large-scale bioinformatics data, such as to discover cell types from single-cell sequencing data, for example, consensus clustering has two significant drawbacks: (i) computational inefficiency due to repeatedly applying clustering algorithms, and (ii) lack of interpretability into the important features for differentiating clusters. In this paper, we address these two challenges by developing IMPACC: Interpretable MiniPatch Adaptive Consensus Clustering. Our approach adopts three major innovations. We ensemble cluster co-occurrences from tiny subsets of both observations and features, termed minipatches, thus dramatically reducing computation time. Additionally, we develop adaptive sampling schemes for observations, which result in both improved reliability and computational savings, as well as adaptive sampling schemes of features, which lead to interpretable solutions by quickly learning the most relevant features that differentiate clusters. We study our approach on synthetic data and a variety of real large-scale bioinformatics data sets; results show that our approach not only yields more accurate and interpretable cluster solutions, but it also substantially improves computational efficiency compared to standard consensus clustering approaches.

A combinatorial necessary and sufficient condition for cluster consensus

There is a growing body of literature demonstrating that immune-related expression signatures predict breast cancer prognosis and chemo-/targeted-therapy responsiveness. However, it is unclear whether these signatures correlate with each other or represent distinct immune-related signals. We evaluated 57 published immune-related expression signatures in four public breast cancer datasets totaling 3295 samples. For each dataset, we used consensus clustering to group signatures together based on their co-expression pattern. Signatures that were in the same consensus cluster across all four datasets were used to define immune modules. Tumors were then classified into immune subtypes based on their module scores using consensus clustering. Survival analysis was conducted using Cox proportional hazards modeling. Consensus clustering consistently yields four distinct co-expression modules across the datasets. These modules appear to represent distinct immune components and signals, where constituent signatures relate to (1) T-cells and/or B-cells (T/B-cell), (2) interferon (IFN), (3) transforming growth factor beta (TGFB), and (4) core-serum response, dendritic cells, and/or macrophages (CSR). Subtyping of tumors based on these co-expression modules consistently yields subsets that fall into five major immune subtypes: T/B-cell/IFN High, IFN/CSR High, CSR High, TGFB High, and Immune Low. Basal and/or triple-negative breast cancer patients with CSR High tumors have significantly worse outcome relative to those within the T/B-cell/IFN High subtype. Our exploratory study identified four distinct immune co-expression modules (T/B-cell, IFN, TGFB, or CSR) from published immune signatures. Using these modules, we identified five immune subtypes with significant outcome differences in basal breast cancers.

On the Approximation of Correlation Clustering and Consensus Clustering

Predicting functions of genes is an important issue in biology. Clustering gene expression profiles has been widely used for gene function prediction, but most clustering methods are unstable and sensitive to input parameters such as starting values and number of clusters. In this article, we develop a novel consensus clustering method to address the instability issue and thus improve the performance of clustering methods. The biological function of an unannotated gene is predicted based on the most enriched functional category in its consensus cluster. The MIPS gene annotations are used to evaluate the predictive performance. It is shown that the consensus clustering-based classification method has a significantly better predictive performance than a previously used clustering-based classification method while performing as well as support vector machines (SVMs). In addition to the obvious applicability of consensus clustering to unsupervised learning, the method's advantages in supervised learning include its being a multiclass classifier that can be trained much faster than SVMs, its generality to include any of the many existing clustering algorithms, and its flexibility to be integrated with other predictive models built with other types of data, suggesting its potential for further improved performance. As a concrete example, we consider its combined use with protein-protein interaction data for gene function prediction. It is shown that the combined analysis has a significantly higher predictive accuracy and a much broader functional coverage than using either data source alone.

This paper introduces a polynomial time approximation scheme for the metric Correlation Clustering problem, when the number of clusters returned is bounded (by k). Consensus Clustering is a fundamental aggregation problem, with considerable application, and it is analysed here as a metric variant of the Correlation Clustering problem. The PTAS exploits a connection between Correlation Clustering and the k-cut problems. This requires the introduction of a new rebalancing technique, based on minimum cost perfect matchings, to provide clusters of the required sizes.Both Consensus Clustering and Correlation Clustering have been the focus of considerable recent study. There is an existing dichotomy between the k-restricted Correlation Clustering problems and the unrestricted versions. The former, in general, admit a PTAS, whereas the latter are, in general, APX-hard. This paper extends the dichotomy to the metric case, responding to the result that Consensus Clustering is APX-hard to approximate.

Consensus clustering provides a framework to ensemble multiple clustering results to obtain a consensus and robust result. Most existing consensus clustering methods usually apply all data to ensemble learning, whereas ignoring the side effects caused by some difficult or unreliable instances. To tackle this problem, we propose a novel self-paced consensus clustering method to gradually involve instances from more reliable to less reliable ones into the ensemble learning. We first construct an initial bipartite graph from the multiple base clustering results, where the nodes represent the instances and clusters and the edges indicate that an instance belongs to a cluster. Then, we learn a structured bipartite graph from the initial one by self-paced learning, i.e., we automatically decide the reliability of each edge and involves the edges into graph learning in order of their reliability. At last, we obtain the final consensus clustering result from the learned bipartite graph. The extensive experimental results demonstrate the effectiveness and superiority of the proposed method.

Observer-based cluster consensus control of high-order multi-agent systems

In this paper, cluster consensus problem in continuous-time networks of multi-agents with external inputs is studied. Here, following [29], cluster consensus is defined in two aspects: intra-cluster synchronization, which means that the state differences among agents in the same cluster asymptotically converge to zero, and inter-cluster separation, which means the states of the agents in different clusters are separated. Under the inter-cluster common influence condition and with adapted inter-cluster identical inputs, the stability of cluster consensus is proved by extending the existing concepts and approaches in the matrix and graph theories to the cluster cases. The adapted inputs are also required to separate different clusters (or called inter-cluster separation). For the graph of the network, both static and switching topologies are considered. Numerical simulation to verify the theoretical results is given, too.