SU‐E‐J‐98: Radiogenomics: Correspondence Between Imaging and Genetic Features Based On Clustering Analysis

Combining hierarchical clustering approaches using the PCA method

To the Editor: We read with great interest the article by Hadanny et al,1 published in Neurosurgery. In this study, the authors used an approach which augmented unsupervised and supervised machine learning methods to predict response to spinal cord stimulators. This paper is an excellent example of how both approaches can be used together to prognosticate on patient outcomes, and the methodological implications span beyond the impact on patients with spinal cord stimulators. Unsupervised machine learning (UML) operates on the premise that the ground truth is not known—for this reason, there are no training data involved in the model. The outcomes of UML algorithms are usually unique clusters identified in the data set. This varies from supervised machine learning (SML) algorithms, which construct a model based on known input and output data. In the literature, UML methods are commonly used to identify novel and emergent phenotypes, whereas SML has commonly been used to perform outcome prediction.2,3 When using UML methods, there are several important parameters to consider which may influence the outcomes seen. First parameter is the cluster number identification. In this paper, the authors used the K-nearest neighbor algorithm (KNN), preceded by an elbow-based cluster optimization step. The KNN algorithm itself is not a completely unsupervised algorithm and requires the inclusion of the number of preset clusters. It requires a cluster optimization step to identify this parameter. In their supplementary methods, they argue that, using the elbow method, a pseudoelbow was created at K = 3. As a result, the decision is made that two clusters exist in the data. We encourage the authors to consider the use of other approaches to validate this finding, including the silhouette method, which has been shown to be more robust and quantifiable in some cases and may be less prone to subjective examination of the analyst.4 The number of clusters is also not readily apparent in the reprojections to the principal component analysis domain space. We suggest the use of other dimensionality reduction approaches to visualize the clusters, namely t-distributed stochastic network embedding or uniform manifold approximation and projection. While the clustering should remain in the multidimensional space, dimensionality reduction approaches have been useful in visualizing multidimensional data otherwise difficult to appreciate. Second parameter is the algorithm itself. KNN is a very well-documented partitioning algorithm. However, it has several limitations. First, the algorithm requires the use of numerical data, which may distort the categorical data, such as patient biological sex, diagnosis, smoking status, and present inherent to the data set. In addition, the sample size required for use of the KNN method is also relatively large, with some studies suggesting the need for a sample size of 2^n, where n is the number of features (24 features in this study).5,6 Other clustering algorithms, such as hierarchical clustering, have been shown to be far more effective for smaller data sets, and we encourage the authors to explore the use of more robust algorithms that would better fit the constraints of the data. As more neurosurgical studies use machine learning, specifically UML, approaches for the analysis of data, it becomes very important for researchers to consider the aforementioned parameters to strengthen their observations and findings. We applaud Hadanny et al1 for an excellent presentation of how both UML and SML approaches can be augmented, and we look forward to additional novel contributions using both approaches in the future to better help understand complex disease processes and ultimately help patients.

The two most widely studied clustering algorithms are partitional and hierarchical clustering. These algorithms have been heavily used in a wide range of applications primarily due to their simplicity and ease of implementation relative to other clustering algorithms. Partitional methods need to be provided with a set of initial seeds which are then improved iteratively. Hierarchical clustering can be achieved in two different ways, namely, bottom-up and top-down clustering. Hierarchical methods can start off with the individual data points in single clusters and build the clustering. Hierarchical clustering algorithms approach the problem of clustering by developing a binary tree-based data structure called the dendrogram. Once the dendrogram is constructed, one can automatically choose the right number of clusters by splitting the tree at different levels to obtain different clustering solutions for the same dataset without rerunning the clustering algorithm again.

In this work, the agglomerative hierarchical clustering and K-means clustering algorithms are implemented on small datasets. Considering that the selection of the similarity measure is a vital factor in data clustering, two measures are used in this study - cosine similarity measure and Euclidean distance - along with two evaluation metrics - entropy and purity - to assess the clustering quality. The datasets used in this work are taken from UCI machine learning depository. The experimental results indicate that k-means clustering outperformed hierarchical clustering in terms of entropy and purity using cosine similarity measure. However, hierarchical clustering outperformed k-means clustering using Euclidean distance. It is noted that performance of clustering algorithm is highly dependent on the similarity measure. Moreover, as the number of clusters gets reasonably increased, the clustering algorithms’ performance gets higher.

Similarity and cluster analysis are important aspects for analyzing microarray data. Based on our perspective of viewing microarrays as time series data, both similarity analysis and cluster analysis are carried out through indexing on time series data using R*-Trees. We have developed algorithms for similarity and cluster analysis on microarray data, and conducted experimental studies and comparative studies. First, our study shows that principle components analysis (PCA) has superiority over several other methods (such as DFT and PAA) as far as distance conservation is concerned. A similarity analysis tool based on PCA has been developed, which is able to explore less R*-Tree nodes before finding its similar counterparts and returns less false positives than other methods. In addition, we also extend R*-Tree's application to cluster analysis. With the aid of R*-Tree indexing, two clustering algorithms. KMeans-R and Hierarchy-R, are proposed as an improved version of K-Means and hierarchical clustering, respectively. Experiments for similarity search and cluster analysis based on proposed algorithms have been carried out and have shown favorable results. Experiments related to yeast cell cycle dataset are reported in this paper.

Spatial clustering analysis is an important spatial data mining technique. It divides objects into clusters according to their similarities in both location and attribute aspects. It plays an essential role in density distribution identification, hot-spot detection, and trend discovery. Spatial clustering algorithms in the Euclidean space are relatively mature, while those in the network space are less well researched. This study aimed to present a well-known clustering algorithm, named density-based spatial clustering of applications with noise (DBSCAN), to network space and proposed a new clustering algorithm named network space DBSCAN (NS-DBSCAN). Basically, the NS-DBSCAN algorithm used a strategy similar to the DBSCAN algorithm. Furthermore, it provided a new technique for visualizing the density distribution and indicating the intrinsic clustering structure. Tested by the points of interest (POI) in Hanyang district, Wuhan, China, the NS-DBSCAN algorithm was able to accurately detect the high-density regions. The NS-DBSCAN algorithm was compared with the classical hierarchical clustering algorithm and the recently proposed density-based clustering algorithm with network-constraint Delaunay triangulation (NC_DT) in terms of their effectiveness. The hierarchical clustering algorithm was effective only when the cluster number was well specified, otherwise it might separate a natural cluster into several parts. The NC_DT method excessively gathered most objects into a huge cluster. Quantitative evaluation using four indicators, including the silhouette, the R-squared index, the Davis–Bouldin index, and the clustering scheme quality index, indicated that the NS-DBSCAN algorithm was superior to the hierarchical clustering and NC_DT algorithms.

This article deals with the development of an improved clustering technique for categorical data that is based on the identification of points having significant membership to multiple classes. Cluster assignments of such points are difficult, and they often affect the actual partitioning of the data. As a consequence, it may be more effective if the points that are associated with maximum confusion regarding their cluster assignments are first identified and excluded from consideration at the first stage of algorithm and these points may be assigned to one of the identified clusters based on an ANN classifier at the second stage of this algorithm. At the first stage of this algorithm we are using our developed genetic algorithm and simulated annealing based fuzzy clustering and well known Fuzzy C-Medoids algorithm when the number of clusters is known a priori. The performance of the proposed clustering algorithms has been compared with the average linkage hierarchical clustering algorithm, in addition to the genetic algorithm based fuzzy clustering, simulated annealing based fuzzy clustering and Fuzzy C-Medoids with ANN for a variety of artificial and real life categorical data sets. Also statistical significance test have been performed to establish the superiority of the proposed algorithm.

A fundamental problem in data clustering is how to determine the correct number of clusters. The k-adaptive medoid set ant colony optimization (ACO) clustering (METACOC-K) algorithm is superior in solving clustering problems. However, METACOC-K does not guarantee in finding the best number of clusters. It assumed the number of clusters based on an adaptive parameter strategy that lacks feedback learning. This has restrained the algorithm in producing compact clusters and the optimal number of clusters. In this paper, a self-adaptive ACO clustering (S-ACOC) algorithm is proposed to produce the optimal number of clusters by incorporating a self-adaptive parameter strategy. The S-ACOC algorithm is a centroid-based algorithm that automatically adjusts the number of clusters during the algorithm run. The selection of the number of clusters is based on a construction graph that reflects the influence of a pheromone in algorithm learning. Experiments were conducted on real-world datasets to evaluate the performance of the proposed algorithm. The external evaluation metrics (purity, F-measure, and entropy) were used to compare the results of the proposed algorithm with other swarm clustering algorithms, including a genetic algorithm (GA), particle swarm optimization (PSO), and METACOC-K. Results showed that S-ACOC provides higher purity (50%) and lower entropy (40%) than GA, PSO, and METACOC-K. Experiments were also performed on several predefined clusters, and results demonstrate that the S-ACOC algorithm is superior to GA, PSO, and METACOC-K. Based on the superior performance, S-ACOC can be used to solve clustering problems in various application domains.

In this study, the concepts of competitive agglomeration clustering algorithm is incorporated into fuzzy c-means (FCM) clustering algorithm for symbolic interval-values data. In the proposed approach, called as IFCMwUNC clustering algorithm, the problems of the unknown clusters number and the initialization of prototypes in the FCM clustering algorithm for symbolic interval-values data are overcome and discussed. Due to the competitive agglomeration clustering algorithm possess the advantages of the hierarchical clustering algorithm and the partitional clustering algorithm, IFCMwUNC clustering algorithm can be fast converges in a few iterations regardless of the initial number of clusters. Moreover, it is also converges to the same optimal partition regardless of its initialization. Experiments results show the merits and usefulness of IFCMwUNC clustering algorithm for the symbolic interval-values data.

PurposeThe study aims to cluster the travellers based on their social media interactions as well as to find the different segments with similar and dissimilar categories according to traveller's choice. The study also aims to understand the behaviour of clusters of the travellers towards destination selection and accordingly make the tour packages in order to improve tourists' satisfaction and gain viable benefits.Design/methodology/approachAgglomerative hierarchical clustering with Ward's minimum variance linkage algorithm and model-based clustering with parameterized finite Gaussian mixture models has been implemented to achieve the respective goals. The dimension reduction (DR) technique was introduced for better visualizing clustering structure obtained from a finite mixture of Gaussian densities.FindingsA total of 980 travellers have been clustered into 8 different interest groups according to their tourism destinations selection across East Asia based on individual social media feedback. For selecting the optimal number of clusters as well as the behaviour of the interested travellers groups, both these proposed methods have shown remarkable similarities. DR technique ensures the reduction in dimensionality with seven directions, of which the first two directions explained 95% of total variability.Practical implicationsTourism organizations focus on marketing efforts to promote the most attractive benefits to the clusters of travellers. By segmenting travellers of East Asia into homogeneous groups, it is feasible to choose a similar area to test different marketing techniques. Finally, it can be identified to which segments, new respondents or potential clients belong; consequently, the tourism organizations can design the tour packages.Originality/valueThe study has uniqueness in two aspects. Firstly, the study empirically revealed tourists' experience and behavioural intention to select tourism destinations and secondly, it finds quantifiable insights into the tourism phenomenon in East Asia, which helps tourism organizations to understand the buying behaviours of tourists' segments. Finally, the application of clustering algorithms to achieve the purpose of this study and the findings are very new in the literature on tourism, to understand the tourist behaviour towards destination selection based on social media reviews.

Clustering microarray gene expression data is a powerful tool for elucidating co-regulatory relationships among genes. Many different clustering techniques have been successfully applied and the results are promising. However, substantial fluctuation contained in microarray data, lack of knowledge on the number of clusters and complex regulatory mechanisms underlying biological systems make the clustering problems tremendously challenging. We devised an improved model-based Bayesian approach to cluster microarray gene expression data. Cluster assignment is carried out by an iterative weighted Chinese restaurant seating scheme such that the optimal number of clusters can be determined simultaneously with cluster assignment. The predictive updating technique was applied to improve the efficiency of the Gibbs sampler. An additional step is added during reassignment to allow genes that display complex correlation relationships such as time-shifted and/or inverted to be clustered together. Analysis done on a real dataset showed that as much as 30% of significant genes clustered in the same group display complex relationships with the consensus pattern of the cluster. Other notable features including automatic handling of missing data, quantitative measures of cluster strength and assignment confidence. Synthetic and real microarray gene expression datasets were analyzed to demonstrate its performance. A computer program named Chinese restaurant cluster (CRC) has been developed based on this algorithm. The program can be downloaded at http://www.sph.umich.edu/csg/qin/CRC/.

A Bayesian deep restricted Boltzmann-Kohonen architecture for data clustering termed deep restricted Boltzmann machine (DRBM)-ClustNet is proposed. This core-clustering engine consists of a DRBM for processing unlabeled data by creating new features that are uncorrelated and have large variance with each other. Next, the number of clusters is predicted using the Bayesian information criterion (BIC), followed by a Kohonen network (KN)-based clustering layer. The processing of unlabeled data is done in three stages for efficient clustering of the nonlinearly separable datasets. In the first stage, DRBM performs nonlinear feature extraction by capturing the highly complex data representation by projecting the feature vectors of d dimensions into n dimensions. Most clustering algorithms require the number of clusters to be decided a priori; hence, here, to automate the number of clusters in the second stage, we use BIC. In the third stage, the number of clusters derived from BIC forms the input for the KN, which performs clustering of the feature-extracted data obtained from the DRBM. This method overcomes the general disadvantages of clustering algorithms, such as the prior specification of the number of clusters, convergence to local optima, and poor clustering accuracy on nonlinear datasets. In this research, we use two synthetic datasets, 15 benchmark datasets from the UCI Machine Learning repository, and four image datasets to analyze the DRBM-ClustNet. The proposed framework is evaluated based on clustering accuracy and ranked against other state-of-the-art clustering methods. The obtained results demonstrate that the DRBM-ClustNet outperforms state-of-the-art clustering algorithms.

Determining number of clusters and prototype locations via multi-scale clustering

Experimentally determined protein structures have been classified in different public databases according to their structural and evolutionary relationships. Frequently, alternative structural models, determined using X-ray crystallography or NMR spectroscopy, are available for a protein. These models can present significant structural dissimilarity. Currently there is no classification available for these alternative structures. In order to classify them, we developed STRuster, an automated method for clustering ensembles of structural models according to their backbone structure. The method is based on the calculation of carbon alpha (Calpha) distance matrices. Two filters are applied in the calculation of the dissimilarity measure in order to identify both large and small (but significant) backbone conformational changes. The resulting dissimilarity value is used for hierarchical clustering and partitioning around medoids (PAM). Hierarchical clustering reflects the hierarchy of similarities between all pairs of models, while PAM groups the models into the 'optimal' number of clusters. The method has been applied to cluster the structures in each SCOP species level and can be easily applied to any other sets of conformers. The results are available at: http://bioinf.mpi-sb.mpg.de/projects/struster/.

Consensus clustering algorithm based on the automatic partitioning similarity graph