Adaptive Noise Immune Cluster Ensemble Using Affinity Propagation

Ensemble clustering based on evidence extracted from the co-association matrix

Background: Advances in technology have facilitated the generation of gene expression data from large numbers of samples and the development of “Big Data” approaches to analysing gene expression in basic and biomedical systems. That being said, the data still includes relatively small numbers of samples and tens of thousands of variables/gene expression. A variety of different approaches have been developed for searching these gene spaces in order to select the most informative variables that can accurately distinguish one class of subjects/ samples from another. However, there is still a need for new approaches that can accurately distinguish biologically different classes of subjects with similar gene expression profiles. We describe a new and promising approach for selecting the most informative differentially expressed genes that addresses this problem. We describe a method for identifying significant differentially expressed clusters of genes using a process of Recursive Cluster Elimination (RCE) that is based on an ensemble clustering approach. We refer to this approach as SVM-RCE-EC (Ensemble Clustering). We show that SVM-RCE-EC improves gene selection, classification accuracy as compared to other methods including the traditional SVM-RCE approach, and that this is particularly evident when applied to difficult data sets that are poorly resolved by other approaches. Methods: To implement SVM-RCE-EC we first applied an ensemble-clustering method, to identify robust gene clusters. We then applied Support Vector Machines (SVMs), with cross validation to score (rank) those clusters of genes based on their contributions to classification accuracy. The clusters of genes that are least significant are progressively removed by the procedure of RCE with the most significant clusters being retained until one identifies the most robust, significantly differentially expressed genes between the two classes. We compare the classification performance of SVM-RCE-EC to a variety of published classification algorithms. Results and Conclusion: Utilization of gene clusters selected using the ensemble method enhances classification performance as compared to other methods and identifies sets of significant genes that appear to be more biologically meaningful to the system being analyzed. We show that SVM-RCE-EC outperforms several other methods on data that represent highly similar sample classes that are difficult to distinguish and is comparable to other methods when applied to data where the classes are more easily separated. The improved performance of SVM-RCE-EC on difficult data sets is likely due to the fact that the significant clusters, as determined by the ensemble approach, capture the native structure of the data while SVM-RCE leaves that determination to the user. This hypothesis is supported by the observations that the performance of the clusters generated by SVM-RCE-EC is more robust. Availability: The Matlab version of SVM-RCE-EC is available upon request to the first author and at GitHub (https://github.com/malikyousef/svm-rce-ec).

Class discovery from gene expression data is an important task for cancer diagnosis. In this paper, we present a new framework for class discovery. The new framework integrates the perturbation technique, the cluster ensemble approach, and the cluster validity index. Specifically, it first generates a set of perturbed datasets from the original microarray data. Then, the Neural Gas, which serves as the basic clustering algorithm, is applied to obtain the partitions from the original dataset and the perturbed datasets. Finally, a new cluster validity index called disagreement/agreement (DA) index (DAI) is designed to identify the number of classes in the dataset by considering the difference between the partition obtained from the original dataset and the partitions obtained from the perturbed datasets. The experiments in three synthetic datasets and four cancer datasets show that: 1) DAI successfully discovers the underlying structure from all the synthetic datasets and most of the cancer datasets and 2) DAI outperforms most of the state-of-the-art cluster validity indexes when applied to gene expression data.

Many stability measures, such as Normalized Mutual Information NMI, have been proposed to validate a set of partitionings. It is highly possible that a set of partitionings may contain one or more high quality clusters but is still adjudged a bad cluster by a stability measure, and as a result, is completely neglected. Inspired by evaluation approaches measuring the efficacy of a set of partitionings, researchers have tried to define new measures for evaluating a cluster. Thus far, the measures defined for assessing a cluster are mostly based on the well-known NMI measure. The drawback of this commonly used approach is discussed in this paper, after which a new asymmetric criterion, called the Alizadeh--Parvin--Moshki--Minaei criterion APMM, is proposed to assess the association between a cluster and a set of partitionings. We show that the APMM criterion overcomes the deficiency in the conventional NMI measure. We also propose a clustering ensemble framework that incorporates the APMM's capabilities in order to find the best performing clusters. The framework uses Average APMM AAPMM as a fitness measure to select a number of clusters instead of using all of the results. Any cluster that satisfies a predefined threshold of the mentioned measure is selected to participate in an elite ensemble. To combine the chosen clusters, a co-association matrix-based consensus function by which the set of resultant partitionings are obtained is used. Because Evidence Accumulation Clustering EAC can not derive the co-association matrix from a subset of clusters appropriately, a new EAC-based method, called Extended EAC EEAC, is employed to construct the co-association matrix from the chosen subset of clusters. Empirical studies show that our proposed approach outperforms other cluster ensemble approaches.

Cluster ensemble is one of the hot topics in the machine learning area. Though plenty of cluster ensemble methods and frameworks have been proposed, many cluster ensemble methods are easily faded by noisy datasets and local optimal problems. In this article, we introduced a novel cluster ensemble method, named as Double Partition Around Medoids based Cluster Ensemble (PAM2CE). PAM2CE will effectively weaken or even eliminate the effect of noisy datasets and local optimal problems via clustering attributes and selecting the representative attributes. The experimental results reveal the better robustness and effectiveness of proposed method.

In recent years, the clustering ensembles emerged as a problem solver for extracting the data points into clusters in an efficient way. However, still clustering poses a serious issue due to the presence of imperfect information while partitioning the data into clusters. This creates a serious issue in creating an efficient cluster with cluster ensembles. In this paper, we propose a solution to solve the degradation in clustering during data partitioning. The initial clusters are generated using firefly algorithm. A linked cluster ensemble approach uses similarity measurement using multi-viewpoint and weighted triple quality using entropy measurements that ensembles the data points into clusters. This avoids the problem of local optimum and avoids the issues arise from high-dimensional datasets and improve the quality of clustering. Here, the data partitioning is done with bipartite spectral algorithm and similarity measurement. Finally, the artificial neural network is used to generate classified results from the optimized clustered datasets. The experimental results are carried out over UCI repository datasets and the results show that the proposed method attains an effective ensemble clustering with higher clustering accuracy than the conventional ones.

Image segmentation is a classical problem in the area of image processing, multimedia, medical image, and so on. Although there exist a lot of approaches to perform image segmentation, few of them study the image segmentation by the cluster ensemble approach. In this paper, we propose a new algorithm called the cluster ensemble algorithm (CEA) for image segmentation. Specifically, CEA first obtains two set of segmented regions which are partitioned by EM according to the color feature and the texture feature respectively. Then, it integrates these regions to k segmented regions based on the similarity measure and the fuzzy membership function. Finally, CEA performs the denoise algorithm on the segmented regions to remove the noise. The experiments show that CEA works well during the process of image segmentation.KeywordsImage SegmentationTexture FeatureCore RegionColor FeatureExpectation Maximization AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

ObjectiveThe objective of this study was to employ ensemble clustering and tree-based risk model approaches to identify interactions between clinicogenomic features for colorectal cancer using the 100,000 Genomes Project.ResultsAmong the 2211 patients with colorectal cancer (mean age of diagnosis: 67.7; 59.7% male), 16.3%, 36.3%, 39.0% and 8.4% had stage 1, 2, 3 and 4 cancers, respectively. Almost every patient had surgery (99.7%), 47.4% had chemotherapy, 7.6% had radiotherapy and 1.4% had immunotherapy. On average, tumour mutational burden (TMB) was 18 mutations/Mb and 34.4%, 31.3% and 25.7% of patients had structural or copy number mutations in KRAS, BRAF and NRAS, respectively. In the fully adjusted Cox model, patients with advanced cancer [stage 3 hazard ratio (HR) = 3.2; p < 0.001; stage 4 HR = 10.2; p < 0.001] and those who had immunotherapy (HR = 1.8; p < 0.04) or radiotherapy (HR = 1.5; p < 0.02) treatment had a higher risk of dying. The ensemble clustering approach generated four distinct clusters where patients in cluster 2 had the best survival outcomes (1-year: 98.7%; 2-year: 96.7%; 3-year: 93.0%) while patients in cluster 3 (1-year: 87.9; 2-year: 70.0%; 3-year: 53.1%) had the worst outcomes. Kaplan–Meier analysis and log rank test revealed that the clusters were separated into distinct prognostic groups (p < 0.0001). Survival tree or recursive partitioning analyses were performed to further explore risk groups within each cluster. Among patients in cluster 2, for example, interactions between cancer stage, grade, radiotherapy, TMB, BRAF mutation status were identified. Patients with stage 4 cancer and TMB ≥ 1.6 mutations/Mb had 4 times higher risk of dying relative to the baseline hazard in that cluster.

Clustering ensembles combine multiple partitions of data into a single clustering solution. It is an effective technique for improving the quality of clustering results. Current clustering ensemble algorithms are usually built on the pairwise agreements between clusterings that focus on the similarity via consensus functions, between data objects that induce similarity measures from partitions and re-cluster objects, and between clusters that collapse groups of clusters into meta-clusters. In most of those models, there is a strong assumption on IIDness (i.e., independent and identical distribution), which states that base clusterings perform independently of one another and all objects are also independent. In the real world, however, objects are generally likely related to each other through features that are either explicit or even implicit. There is also latent but definite relationship among intermediate base clusterings because they are derived from the same set of data. All these demand a further investigation of clustering ensembles that explores the interdependence characteristics of data. To solve this problem, a new coupled clustering ensemble (CCE) framework that works on the interdependence nature of objects and intermediate base clusterings is proposed in this article. The main idea is to model the coupling relationship between objects by aggregating the similarity of base clusterings, and the interactive relationship among objects by addressing their neighborhood domains. Once these interdependence relationships are discovered, they will act as critical supplements to clustering ensembles. We verified our proposed framework by using three types of consensus function: clustering-based, object-based, and cluster-based. Substantial experiments on multiple synthetic and real-life benchmark datasets indicate thatCCEcan effectively capture the implicit interdependence relationships among base clusterings and among objects with higher clustering accuracy, stability, and robustness compared to 14 state-of-the-art techniques, supported by statistical analysis. In addition, we show that the final clustering quality is dependent on the data characteristics (e.g., quality and consistency) of base clusterings in terms of sensitivity analysis. Finally, the applications in document clustering, as well as on the datasets with much larger size and dimensionality, further demonstrate the effectiveness, efficiency, and scalability of our proposed models.

A cluster ensemble method for clustering categorical data

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

High-dimensional data with many features present a significant challenge to current clustering algorithms.Sparsity, noise, and correlation of features are common properties of high-dimensional data.Another essential aspect is that clusters in such data often exist in various subspaces. Ensemble clusteringis emerging as a leading technique for improving robustness, stability, and accuracy of high-dimensional data clusterings. In this paper, we propose FastMap projection for generating subspace component data sets from high-dimensional data. By using component data sets, we create component clusterings and provides a new objective function that ensembles them by maximizing the average similarity between component clusterings and final clustering. Compared with the random sampling and random projection methods, the component clusterings by FastMap projection showed high average clustering accuracy without sacrificing clustering diversity in synthetic data analysis. We conducted a series of experiments on real-world data sets from microarray, text, and image domains employing three subspace component data generation methods, three consensus functions, and a proposed objective function for ensemble clustering. The experiment results consistently demonstrated that the FastMap projection method with the proposed objection function provided the best ensemble clustering results for all data sets.

Data clustering is an important task and has found applications in numerous real-world problems. Since no single clustering algorithm is able to identify all different types of cluster shapes and structures, ensemble clustering was proposed to combine different partitions of the same data generated by multiple clustering algorithms. The key idea of most ensemble clustering algorithms is to find a partition that is consistent with most of the available partitions of the input data. One problem with these algorithms is their inability to handle uncertain data pairs, i.e. data pairs for which about half of the partitions put them into the same cluster and the other half do the opposite. When the number of uncertain data pairs is large, they can mislead the ensemble clustering algorithm in generating the final partition. To overcome this limitation, we propose an ensemble clustering approach based on the technique of matrix completion. The proposed algorithm constructs a partially observed similarity matrix based on the data pairs whose cluster memberships are agreed upon by most of the clustering algorithms in the ensemble. It then deploys the matrix completion algorithm to complete the similarity matrix. The final data partition is computed by applying an efficient spectral clustering algorithm to the completed matrix. Our empirical studies with multiple real-world datasets show that the proposed algorithm performs significantly better than the state-of-the-art algorithms for ensemble clustering.

Ensemble clustering has recently emerged as a powerful tool for aggregating multiple clustering results into a probably better and more robust clustering result. While many ensemble clustering techniques have been developed in recent years, there is still a potential drawback in most of the existing algorithms. They generally neglect the possible noise inside the ensemble of base clusterings, which may exhibit a negative influence on the robustness of the consensus result. To address this, this paper proposes a new ensemble clustering algorithm based on noise-aware graph decomposition. The co-association matrix is first constructed to accumulate information from multiple base clusterings, and then sparsified by means of K-nearest neighbors. By treating the sparsified co-association matrix as the similarity matrix, the corresponding graph is further decomposed into two parts, i.e., the corruption(noise) graph and the good(denoised) graph. In particular, the denoised graph as well as its spectral embedding can be learned simultaneously by solving a trace minimization problem. Finally, upon the learned spectral embedding, the k-means discretization is performed to obtain the final consensus clustering. Experiments on multiple real-world datasets have shown the superiority of the proposed algorithm.

Ensemble Clustering has been developed to provide an alternative way of obtaining more stable and accurate clustering results. It aims to avoid the biases of individual clustering algorithms. However, it is still a challenge to develop an efficient and robust method for Ensemble Clustering. Based on an existing ensemble clustering method, Consensus Clustering (CC), this paper introduces an advanced Consensus Clustering algorithm called Multi-Optimisation Consensus Clustering (MOCC), which utilises an optimised Agreement Separation criterion and a Multi-Optimisation framework to improve the performance of CC. Fifteen different data sets are used for evaluating the performance of MOCC. The results reveal that MOCC can generate more accurate clustering results than the original CC algorithm.KeywordsConsensus ClusteringSimulated AnnealingMulti-optimisation