A Data Distribution View of Clustering Algorithms

Ant clustering algorithm with K-harmonic means clustering

Imaging of colorectal adenocarcinoma using FT-IR microspectroscopy and cluster analysis.

While Chap. 2 presented the broadness of the spectrum of clustering methods, this chapter focusses on clustering of objects that are embedded in some metric space and gives an insight into the potential variations of clustering algorithms driven by peculiarities of data at hand. It concentrates essentially around k-means clustering algorithm and its variants, which serve as an illustration on the kinds of issues the algorithms need to accommodate to. After presenting the basic variants of k-means, implementational issues are discussed, including initialisation and efficiency (e.g. k-means++). Then accommodations to special forms of data, e.g. text data (spherical k-means), or data with clusters separated by complicated boundary lines (kernel k-means), or ones residing in non-Euclidean space (k-medoids, k-modes) are presented. Subsequently non-sharp separation areas between clusters are discussed. First the EM algorithm is presented, then fuzzy k-means, with fuzzified variants of the aforementioned algorithms. k-means application presumes apriorical knowledge of the number of clusters. To overcome this limitation, so-called affinity propagation algorithm is presented. Finally approaches to handling data of non-typical relation between the number of objects and the number of dimensions, that is residing (predominantly) in a narrow section of the feature space are discussed (projective clustering, random projection, k q-flats, manifold clustering), or with too large data sets (subsampling), or clustering changing over time. The latter issue is closely connected to clustering under partial pre-labelling of objects. Also methods for simultaneous clustering of both objects and their features (co-clustering) are briefly outlined.

K-means is a widely used partitional clustering method. While there are considerable research efforts to characterize the key features of K-means clustering, further investigation is needed to reveal whether and how the data distributions can have the impact on the performance of K-means clustering. Indeed, in this paper, we revisit the K-means clustering problem by answering three questions. First, how the true cluster sizes can make impact on the performance of K-means clustering? Second, is the entropy an algorithm-independent validation measure for K-means clustering? Finally, what is the distribution of the clustering results by K-means? To that end, we first illustrate that K-means tends to generate the clusters with the relatively uniform distribution on the cluster sizes. In addition, we show that the entropy measure, an external clustering validation measure, has the favorite on the clustering algorithms which tend to reduce high variation on the cluster sizes. Finally, our experimental results indicate that K-means tends to produce the clusters in which the variation of the cluster sizes, as measured by the Coefficient of Variation(CV), is in a specific range, approximately from 0.3 to 1.0.

Data clustering is a collection of data objects that are similar to one another within in the same cluster and are dissimilar to the objects in other clusters. In this paper we proposed new algorithm called black hole optimization with a heuristic search algorithm to overcome the problem of the BHO algorithm. The black hole optimization is used to produce an initial solution to the clustering problem and then a heuristic search algorithm is applied to improve the quality of this solution by searching around it. The black hole optimization algorithm starts with an initial population of candidate solutions to an optimization problem and for each star, evaluate the objective function. Then at each iteration of the black hole optimization algorithm, the best star is selected to be the black hole and all other candidate forms the normal stars. If a star moves too close to the black hole, it will be absorbed by the black hole and permanently disappear. The black hole optimization algorithm converges rapidly during the starting stage of the search process, but it goes near global optimal solution, the convergence speed also will decrease. So, it may get trapped in local optimal over a certain number of iterations. To evaluate the performance of BHOHS can be compared with k-means, particle swarm optimization (PSO) algorithm and black hole optimization algorithm (BHO).

The Agave genus is composed of about 200 species, but the cultivation of sisal for fiber production in Brazil is restricted to two species: Agave sisalana and Agave fourcroydes, both have several accessions with wide variability. The collection of Agave of Embrapa has 37 accessions maintained in situ and periodically evaluated agronomical traits. Most of these accessions have phenotypic similarities, although they differ in fiber quality, which are widely used for commercial purposes. The identification of promising accesses contributes to the advance in improvement works, focusing on commercial indication. In order to estimate the genetic divergence of this collection, a cluster analyses was performed based on Inter-Simple Sequence Repeat (ISSR) markers and phenotypic traits. Genomic DNA from these accessions were used in polymerase chain reaction (PCR) with thirty ISSR oligonucleotides. For phenotypic characterization, twelve descriptors were adopted based on morphological and agronomic data. The Unweighted Pair Group Method with Arithmetic Mean (UPGMA) and Tocher multivariate methods were adopted. Thirteen groups were formed by the Tocher Method and six by UPGMA; however UPGMA method was more representative in the group formation. The comparison of the band patterns among accessions derived from the shoots showed that genetic variability is generated during asexual reproduction in these plants. The four lines generated from Tatui were the most divergent accessions. These plants are tallest, with higher mass values of fresh and dry mucilage, fresh and dry fiber mass, fiber length and presence of spines at the edges. The accessions from Instituto Agronômico de Campinas (IAC) showed the lowest genetic distances, indicating a possible narrow genetic base and high kinship degree. The crossings between H-RN, H-Kenya, H-400 fls, and H-11648 with Tatuí 1, 2, 3, and 4 can be a valorous strategy to broadening genetic diversity among commercial and native sisal germplasm.

An experiment was conducted at University of Agricultural Sciences, Dharwad, Karnataka to detect molecular variation in Puccinia sorghi Schw. through molecular tool viz., random amplified Polymorphic DNA (RAPD). In this study, molecular variation in the pathogen served as a guideline for breeding suitable maize varieties against common rust disease which seriously affects maize productivity. It was reported that losses in total yield ranged from 0 in the more resistant entries to nearly 50% in more susceptible entries. Losses in total yield in late-planted sweet corn were 18, 26, and 49% for cv. Sugarloaf (most resistant), cv. Jubilee (intermediate) and cv. Style Pak (most susceptible), respectively. Results revealed that a variety of polymorphic bands were found from the specimens by polymerase chain reaction (PCR) amplification using 10 primers: OPA-19, OPB-08, OPB-15, OPB-17, OPC-08, OPD-07, OPD-13, OPD-18, OPF-02 and OPG-05. These bands were used to construct unweighted pair group method with arithmetic mean (UPGMA) dendogram, which cluster analysis divided the 15 specimens into major 2 groups: Groups A and B. Out of 10 primers used, 3 primers viz., OPA-19 (11.11%), OPB-17 (20%) and OPF-2 (14%) showed polymorphism. The similarity coefficient values revealed that, the least similarity (81%) was between Tamil Nadu-Andhra Pradesh, Haryana-Tamil Nadu and Madhya Pradesh-Haveri isolates. The maximum similarity (100%) was found between Andhra Pradesh-Haryana and Belgaum-Jammu and Kashmir isolates. Bihar isolate formed entirely separate cluster within Group A, whereas Tamil Nadu-Maharashtra and Bijapur-Haveri isolates formed separate clusters within Group A. In Group B, Sikkim, Uttar Pradesh, West Bengal and Bengaluru isolates formed entirely separate clusters, respectively. Key words: Genetic variability, Zea mays, common rust, Puccinia sorghi, random amplified polymorphic DNA (RAPD).

The paper presents the theoretical justification and software for an effective method of data identification in intelligent control systems, which is based on the use of the stages of clustering and classification of objects represented by their parametric description. The author proposed a method of data clustering with fixation of cluster boundary points based on the principle of finding their surface tension. The application of this method of cluster analysis allows to increase the efficiency of identification of objects in control systems using the SVM type classifier. The SVM (Support Vector Machine) type classifier is currently the most advanced and, due to the use of generalized decision functions, is able to operate with both linear and non-linearly distributed images of objects in the parameter space. Today, a significant number of cluster analysis methods and algorithms are known. They can be divided into classical, based on the general algorithm of Fu K.S., and the latest, based on taking into account the nature and structure of data and the purpose of their use. Examples of such methods include: k-means, density-based (DBSCAN), Mean-Shift, using Gaussian mixture models (GMM), methods of hierarchical agglomerative clustering (HAC) . Their common drawback is that they form clusters of points in the parameter space, but do not fix their boundary points. The last characteristic (the presence of marked cluster boundary points) is very useful when building classifiers that implement the task of object identification. In this paper, the author proposed and theoretically substantiated the method of clustering with the definition of surface points by analogy with the physical principle of surface tension of a liquid. The work developed an algorithm for the implementation of this method, as well as software in the Python language in the form of the ClusterBorderFinder application. The author's certificate was obtained for the developed program.

Tris- and water-soluble seed protein profiling was used to characterize, identify and differentiate 13 genotypes of pigeonpea via SDS–PAGE based electrophoresis of seed protein. Electropherograms, RM-value and UPGMA based cluster (dendrograms) analysis was used to analyze tris- and water-soluble protein banding patterns of these genotypes. It revealed that RM-value of protein bands have differed from all the genotypes for both soluble proteins. Tris-soluble protein banding pattern exhibited 17 bands including 12-15 variate of RM-value (0.083 to 0.98) and two common bands with RM-value 0.083 and 0.78. However, water-soluble protein banding pattern revealed the presence of 16 bands including 8-12 variate of RM-value (0.16 to 0.95) with four common bands of RM-value 0.43, 048, 0.23 and 0.65, respectively. UPGMA cluster analysis was used to group the 13 genotypes into six distinct clusters. The banding pattern of tris-soluble protein was found to be more distinct than water-soluble protein. The result demonstrated that the electrophoretic profile of tris-soluble protein through SDS-PAGE was more effective than the water-soluble protein.

The security and integrity of a computer system is compromised when an intrusion occurs. It becomes impossible for legitimate users to access different network services when network-based attacks purposely occupy or sabotage network resources and services. Our proposed method is a scalable detection method for network based anomalies. We use Support Vector Machines (SVM) for classification. This paper presents a method for enhancing the training time of SVM, particularly when dealing with large data sets, using hierarchical clustering technique. We use the Dynamically Growing Self-Organizing Tree (DGSOT) algorithm for clustering because it has proved to overcome the problems of traditional hierarchical clustering algorithms (e.g., hierarchical agglomerative clustering). Clustering analysis helps to find the boundary points, which are the most qualified data points to train SVM, between any two classes. We present a new approach of combination of SVM and DGSOT, which begins with an initial training set and expands it gradually using the clustering structure produced by the DGSOT algorithm. We show that our proposed variations contribute significantly in improving the training process of SVM with high percentage of detection accuracy. Keywords: SVM, classification, Intrusion detection, Intrusion detection System, Network Security

Recently, many fields have widely used cluster analysis: psychology, biology, statistics, pattern recognition, information retrieval, machine learning, and data mining. Diagnosis of histopathological images of prostate cancer is one of the routine tasks for pathologists and it is challenging for pathologists to analyze the formation of glands and tumors based on the Gleason grading system. In this study, unsupervised classification has been performed for differentiating malignant (cancerous) from benign (non-cancerous) tumors. Therefore, the unsupervised-based computer-aided diagnosis (CAD) technique would be of great benefit in easing the workloads of pathologists. This technique is used to find meaningful clustering objects (i.e., individuals, entities, patterns, or cases) and identify useful patterns. Radiomic-based features were extracted for cluster analysis using the gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), and gray-level size zone matrix (GLSZM) techniques. Multi-clustering techniques used for the unsupervised classification are K-means clustering, K-medoids clustering, Agglomerative Hierarchical (AH) clustering, Gaussian mixture model (GMM) clustering, and Spectral clustering. The quality of the clustering algorithms was determined using Purity, Silhouettes, Adjusted Rand, Fowlkes Mallows, and Calinski Harabasz (CH) scores. However, the best-performing algorithm (i.e., K-means) has been applied to predict and annotate the cancerous regions in the whole slide image (WSI) to compare with the pathologist annotation.

In this paper we study the use of a semi-supervised agglomerative hierarchical clustering (ssAHC) algorithm to text categorization, which consists of assigning text documents to predefined categories. ssAHC is (i) a clustering algorithm that (ii) uses a finite design set of labeled data to (iii) help agglomerative hierarchical clustering (AHC) algorithms partition a finite set of unlabeled data and then (iv) terminates without the capability to label other objects. We first describe the text representation method we use in this work; we then present a feature selection method that is used to reduce the dimensionality of the feature space. Finally, we apply the ssAHC algorithm to the Reuters database of documents and show that its performance is superior to the Bayes classifier and to the Expectation-Maximization algorithm combined with Bayes classifier. We showed also that ssAHC helps AHC techniques to improve their performance. © 2000 John Wiley & Sons, Inc.

An image segmentation method based on a modified local-information weighted intuitionistic Fuzzy C-means clustering and Gold-panning Algorithm

Modified FDP cluster algorithm and its application in protein conformation clustering analysis

Introduction: The discovery of tumor subtypes helps to explore tumor pathogenesis, determine the operability of clinical treatment, and improve patient survival. Clustering analysis is increasingly applied to multi-genomic data. However, due to the diversity and complexity of multi-omics data, developing a complete clustering algorithm for tumor molecular typing is still challenging. Methods: In this study, we present an adaptive density-aware spectral clustering method based on a variational autoencoder (ADSVAE). ADSVAE first learns the underlying spatial information of each omics data using a variational autoencoder (VAE) based on the Wasserstein distance metric. Secondly, a similarity matrix is built for each gene set using an adaptive density-aware kernel. Thirdly, tensor product graphs (TPGs) are used to merge different data sources and reduce noise. Finally, ADSVAE employs a spectral clustering algorithm and utilizes the Gaussian mixture model (GMM) to cluster the final eigenvector matrix to identify cancer subtypes. Results: We tested ADSVAE on 5 TCGA datasets, all with good performance in comparison with several advanced multi-omics clustering algorithms. Compared with the existing multi-group clustering algorithms, the variational autoencoder based on the Wasserstein distance measure in the ADSVAE algorithm can learn the underlying spatial information on each omics data, which has a better effect on learning complex data distribution. The self-tuning density-aware kernel used by the ADSVAE algorithm enhances the similarity between shared near neighbor points, and the process of tensor product plot data integration and diffusion can better reduce the noise and reveal the underlying structure, improving the performance. Conclusion: Due to the inherent pitfalls of computational biology in the study of cancer subtype identification, although some research conclusions have been made in this paper on the related issues, as the research in related fields continues to deepen, the clustering study of cancer subtype identification based on genomic data needs further improvement and refinement.