Class Discovery From Gene Expression Data Based on Perturbation and Cluster Ensemble

There are two notoriously hard problems in cluster analysis, estimating the number of clusters, and checking whether the population to be clustered is not actually homogeneous. Given a dataset, a clustering method and a cluster validation index, this paper proposes to set up null models that capture structural features of the data that cannot be interpreted as indicating clustering. Artificial datasets are sampled from the null model with parameters estimated from the original dataset. This can be used for testing the null hypothesis of a homogeneous population against a clustering alternative. It can also be used to calibrate the validation index for estimating the number of clusters, by taking into account the expected distribution of the index under the null model for any given number of clusters. The approach is illustrated by three examples, involving various different clustering techniques (partitioning around medoids, hierarchical methods, a Gaussian mixture model), validation indexes (average silhouette width, prediction strength and BIC), and issues such as mixed type data, temporal and spatial autocorrelation.

Cluster ensemble is one of the main branches in the ensemble learning area which is an important research focus in recent years. The objective of cluster ensemble is to combine multiple clustering solutions in a suitable way to improve the quality of the clustering result. In this paper, we design a new noise immune cluster ensemble framework named as $AP^{2}CE$ to tackle the challenges raised by noisy datasets. $AP^{2}CE$ not only takes advantage of the affinity propagation algorithm (AP) and the normalized cut algorithm (Ncut), but also possesses the characteristics of cluster ensemble. Compared with traditional cluster ensemble approaches, $AP^{2}CE$ is characterized by several properties. ( $1$ ) It adopts multiple distance functions instead of a single Euclidean distance function to avoid the noise related to the distance function. ( $2$ ) $AP^{2}CE$ applies AP to prune noisy attributes and generate a set of new datasets in the subspaces consists of representative attributes obtained by AP. ( $3$ ) It avoids the explicit specification of the number of clusters. ( $4$ ) $AP^{2}CE$ adopts the normalized cut algorithm as the consensus function to partition the consensus matrix and obtain the final result. In order to improve the performance of $AP^{2}CE$ , the adaptive $AP^{2}CE$ is designed, which makes use of an adaptive process to optimize a newly designed objective function. The experiments on both synthetic and real datasets show that ( $1$ ) $AP^{2}CE$ works well on most of the datasets, in particular the noisy datasets; ( $2$ ) $AP^{2}CE$ is a better choice for most of the datasets when compared with other cluster ensemble approaches; ( $3$ ) $AP^{2}CE$ has the capability to provide more accurate, stable and robust results.

DNA microarray technology is a fundamental tool in gene expression data analysis. The collection of datasets from the technology has underscored the need for quantitative analytical tools to examine such data. Due to the large number of genes and complex gene regulation networks, clustering is a useful exploratory technique for analyzing these data. Many clustering algorithms have been proposed to analyze microarray gene expression data, but very few of them evaluate the quality of the clusters. In this paper, a novel cluster analysis technique has been proposed without considering number of clusters a priori. The method computes a similarity measurement function based on which the clusters are merged and subsequently splits a cluster by computing the degree of separation of the cluster. The process of splitting and merging performs iteratively until the cluster validity index (i.e. DB index) degrades. The experimental result shows that the proposed cluster analysis technique gives comparable results on gene cancer dataset with existing methods. This study may help raise relevant issues in the extraction of meaningful biological information from microarray expression data. Keywords- DNA microarray data, Clustering technique, Cluster validity index, Similarity measurement, Splitting and Merging, Cancer data analysis

Clustering is a useful exploratory technique for the analysis of gene expression data. Many different heuristic clustering algorithms have been proposed in this context. Clustering algorithms based on probability models offer a principled alternative to heuristic algorithms. In particular, model-based clustering assumes that the data is generated by a finite mixture of underlying probability distributions such as multivariate normal distributions. The issues of selecting a 'good' clustering method and determining the 'correct' number of clusters are reduced to model selection problems in the probability framework. Gaussian mixture models have been shown to be a powerful tool for clustering in many applications. We benchmarked the performance of model-based clustering on several synthetic and real gene expression data sets for which external evaluation criteria were available. The model-based approach has superior performance on our synthetic data sets, consistently selecting the correct model and the number of clusters. On real expression data, the model-based approach produced clusters of quality comparable to a leading heuristic clustering algorithm, but with the key advantage of suggesting the number of clusters and an appropriate model. We also explored the validity of the Gaussian mixture assumption on different transformations of real data. We also assessed the degree to which these real gene expression data sets fit multivariate Gaussian distributions both before and after subjecting them to commonly used data transformations. Suitably chosen transformations seem to result in reasonable fits. MCLUST is available at http://www.stat.washington.edu/fraley/mclust. The software for the diagonal model is under development. kayee@cs.washington.edu. http://www.cs.washington.edu/homes/kayee/model.

IMI2: A fuzzy clustering validity index for multiple imbalanced clusters

Background: Advances in technology have facilitated the generation of gene expression data from large numbers of samples and the development of “Big Data” approaches to analysing gene expression in basic and biomedical systems. That being said, the data still includes relatively small numbers of samples and tens of thousands of variables/gene expression. A variety of different approaches have been developed for searching these gene spaces in order to select the most informative variables that can accurately distinguish one class of subjects/ samples from another. However, there is still a need for new approaches that can accurately distinguish biologically different classes of subjects with similar gene expression profiles. We describe a new and promising approach for selecting the most informative differentially expressed genes that addresses this problem. We describe a method for identifying significant differentially expressed clusters of genes using a process of Recursive Cluster Elimination (RCE) that is based on an ensemble clustering approach. We refer to this approach as SVM-RCE-EC (Ensemble Clustering). We show that SVM-RCE-EC improves gene selection, classification accuracy as compared to other methods including the traditional SVM-RCE approach, and that this is particularly evident when applied to difficult data sets that are poorly resolved by other approaches. Methods: To implement SVM-RCE-EC we first applied an ensemble-clustering method, to identify robust gene clusters. We then applied Support Vector Machines (SVMs), with cross validation to score (rank) those clusters of genes based on their contributions to classification accuracy. The clusters of genes that are least significant are progressively removed by the procedure of RCE with the most significant clusters being retained until one identifies the most robust, significantly differentially expressed genes between the two classes. We compare the classification performance of SVM-RCE-EC to a variety of published classification algorithms. Results and Conclusion: Utilization of gene clusters selected using the ensemble method enhances classification performance as compared to other methods and identifies sets of significant genes that appear to be more biologically meaningful to the system being analyzed. We show that SVM-RCE-EC outperforms several other methods on data that represent highly similar sample classes that are difficult to distinguish and is comparable to other methods when applied to data where the classes are more easily separated. The improved performance of SVM-RCE-EC on difficult data sets is likely due to the fact that the significant clusters, as determined by the ensemble approach, capture the native structure of the data while SVM-RCE leaves that determination to the user. This hypothesis is supported by the observations that the performance of the clusters generated by SVM-RCE-EC is more robust. Availability: The Matlab version of SVM-RCE-EC is available upon request to the first author and at GitHub (https://github.com/malikyousef/svm-rce-ec).

Several authors have addressed learning as a classifier given by a mixed labeled/unlabeled training set. These works assumes the unlabeled sample originates from one of the (known) classes. This work considers the scenario in which unlabeled points may belong either to known/predefined or to here-to-fore undiscovered classes. There are several practical situations where such data may arise. We earlier proposed a novel statistical mixture model to fit in this mixed data. In this paper we review the method and introduce an alternative model. Our fundamental strategy is to view as observed the data not only the feature vector and the class label, but also the fact of label presence/absence for each point. Two types of mixture components are used to explain label presence/absence. "Predefined" components generate both labeled and unlabeled points and assume the labels that are missing at random. These components represent the known classes. "Non-predefined" components only generate unlabeled points. In localized regions, the data subsets are captured exclusively unlabeled. Such subsets may represent an outlier distribution, or new classes. The components' predefined/non-predefined natures are data-driven, learned with the other parameters via an algorithm based on expectation-maximization (EM). There are three natural applications presented: 1) robust classifier design, given by a mixed training set with outliers; 2) classification with rejections; and 3) identification of the unlabeled points (and their representative components) originated from unknown classes, i.e. new class discovery. The effectiveness of our models in discovering purely unlabeled data components (potential new classes) is evaluated both by synthetic and real data sets. Although each of our models has its own advantages, the original model is found is achieved by the best class discovery results.

Breast cancer outcome can be predicted using models derived from gene expression data or clinical data. Only a few studies have created a single prediction model using both gene expression and clinical data. These studies often remain inconclusive regarding an obtained improvement in prediction performance. We rigorously compare three different integration strategies (early, intermediate, and late integration) as well as classifiers employing no integration (only one data type) using five classifiers of varying complexity. We perform our analysis on a set of 295 breast cancer samples, for which gene expression data and an extensive set of clinical parameters are available as well as four breast cancer datasets containing 521 samples that we used as independent validation.mOn the 295 samples, a nearest mean classifier employing a logical OR operation (late integration) on clinical and expression classifiers significantly outperforms all other classifiers. Moreover, regardless of the integration strategy, the nearest mean classifier achieves the best performance. All five classifiers achieve their best performance when integrating clinical and expression data. Repeating the experiments using the 521 samples from the four independent validation datasets also indicated a significant performance improvement when integrating clinical and gene expression data. Whether integration also improves performances on other datasets (e.g. other tumor types) has not been investigated, but seems worthwhile pursuing. Our work suggests that future models for predicting breast cancer outcome should exploit both data types by employing a late OR or intermediate integration strategy based on nearest mean classifiers.

When estimating the optimal value of the number of clusters, C, of a given data set, one typically uses, for each candidate value of C, a single (final) result of the clustering algorithm. If distributional data of size T are used, these data come from Tdata sets obtained, e.g., by a bootstrapping technique. Here a new approach is introduced that utilizes distributional data generated by clustering the original data T times in the framework of cost function optimization and cluster validity indices. Results of this method are reported for model data (100 realizations) and gene expression data. The probability of correctly estimating the number of clusters was often higher compared to recently published results of several classical methods and a new statistical approach (Clest).

Fuzzy C-Means (FCM) algorithm has been widely used in cluster analysis of gene expression data. It can converge rapidly and provide more information regarding relationships within the data thanks to the usage of fuzzy sets to represent the degrees of cluster membership for every data point. However, FCM has a shortcoming in that it requires a priori specification of cluster number and cluster validation. Most cluster validity indices are based on the data themselves and may not be applicable for gene expression data. In this paper, we propose a Bayesian method using Gene Ontology (GO) annotations for gene expression cluster validation. We show that our method outperforms popular validity indices on gene expression datasets.

In recent years, the clustering ensembles emerged as a problem solver for extracting the data points into clusters in an efficient way. However, still clustering poses a serious issue due to the presence of imperfect information while partitioning the data into clusters. This creates a serious issue in creating an efficient cluster with cluster ensembles. In this paper, we propose a solution to solve the degradation in clustering during data partitioning. The initial clusters are generated using firefly algorithm. A linked cluster ensemble approach uses similarity measurement using multi-viewpoint and weighted triple quality using entropy measurements that ensembles the data points into clusters. This avoids the problem of local optimum and avoids the issues arise from high-dimensional datasets and improve the quality of clustering. Here, the data partitioning is done with bipartite spectral algorithm and similarity measurement. Finally, the artificial neural network is used to generate classified results from the optimized clustered datasets. The experimental results are carried out over UCI repository datasets and the results show that the proposed method attains an effective ensemble clustering with higher clustering accuracy than the conventional ones.

Ensemble clustering based on evidence extracted from the co-association matrix

Many stability measures, such as Normalized Mutual Information NMI, have been proposed to validate a set of partitionings. It is highly possible that a set of partitionings may contain one or more high quality clusters but is still adjudged a bad cluster by a stability measure, and as a result, is completely neglected. Inspired by evaluation approaches measuring the efficacy of a set of partitionings, researchers have tried to define new measures for evaluating a cluster. Thus far, the measures defined for assessing a cluster are mostly based on the well-known NMI measure. The drawback of this commonly used approach is discussed in this paper, after which a new asymmetric criterion, called the Alizadeh--Parvin--Moshki--Minaei criterion APMM, is proposed to assess the association between a cluster and a set of partitionings. We show that the APMM criterion overcomes the deficiency in the conventional NMI measure. We also propose a clustering ensemble framework that incorporates the APMM's capabilities in order to find the best performing clusters. The framework uses Average APMM AAPMM as a fitness measure to select a number of clusters instead of using all of the results. Any cluster that satisfies a predefined threshold of the mentioned measure is selected to participate in an elite ensemble. To combine the chosen clusters, a co-association matrix-based consensus function by which the set of resultant partitionings are obtained is used. Because Evidence Accumulation Clustering EAC can not derive the co-association matrix from a subset of clusters appropriately, a new EAC-based method, called Extended EAC EEAC, is employed to construct the co-association matrix from the chosen subset of clusters. Empirical studies show that our proposed approach outperforms other cluster ensemble approaches.

Image segmentation is a classical problem in the area of image processing, multimedia, medical image, and so on. Although there exist a lot of approaches to perform image segmentation, few of them study the image segmentation by the cluster ensemble approach. In this paper, we propose a new algorithm called the cluster ensemble algorithm (CEA) for image segmentation. Specifically, CEA first obtains two set of segmented regions which are partitioned by EM according to the color feature and the texture feature respectively. Then, it integrates these regions to k segmented regions based on the similarity measure and the fuzzy membership function. Finally, CEA performs the denoise algorithm on the segmented regions to remove the noise. The experiments show that CEA works well during the process of image segmentation.KeywordsImage SegmentationTexture FeatureCore RegionColor FeatureExpectation Maximization AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

ObjectiveThe objective of this study was to employ ensemble clustering and tree-based risk model approaches to identify interactions between clinicogenomic features for colorectal cancer using the 100,000 Genomes Project.ResultsAmong the 2211 patients with colorectal cancer (mean age of diagnosis: 67.7; 59.7% male), 16.3%, 36.3%, 39.0% and 8.4% had stage 1, 2, 3 and 4 cancers, respectively. Almost every patient had surgery (99.7%), 47.4% had chemotherapy, 7.6% had radiotherapy and 1.4% had immunotherapy. On average, tumour mutational burden (TMB) was 18 mutations/Mb and 34.4%, 31.3% and 25.7% of patients had structural or copy number mutations in KRAS, BRAF and NRAS, respectively. In the fully adjusted Cox model, patients with advanced cancer [stage 3 hazard ratio (HR) = 3.2; p < 0.001; stage 4 HR = 10.2; p < 0.001] and those who had immunotherapy (HR = 1.8; p < 0.04) or radiotherapy (HR = 1.5; p < 0.02) treatment had a higher risk of dying. The ensemble clustering approach generated four distinct clusters where patients in cluster 2 had the best survival outcomes (1-year: 98.7%; 2-year: 96.7%; 3-year: 93.0%) while patients in cluster 3 (1-year: 87.9; 2-year: 70.0%; 3-year: 53.1%) had the worst outcomes. Kaplan–Meier analysis and log rank test revealed that the clusters were separated into distinct prognostic groups (p < 0.0001). Survival tree or recursive partitioning analyses were performed to further explore risk groups within each cluster. Among patients in cluster 2, for example, interactions between cancer stage, grade, radiotherapy, TMB, BRAF mutation status were identified. Patients with stage 4 cancer and TMB ≥ 1.6 mutations/Mb had 4 times higher risk of dying relative to the baseline hazard in that cluster.