A framework for classification using genetic algorithm based clustering

The two most widely studied clustering algorithms are partitional and hierarchical clustering. These algorithms have been heavily used in a wide range of applications primarily due to their simplicity and ease of implementation relative to other clustering algorithms. Partitional methods need to be provided with a set of initial seeds which are then improved iteratively. Hierarchical clustering can be achieved in two different ways, namely, bottom-up and top-down clustering. Hierarchical methods can start off with the individual data points in single clusters and build the clustering. Hierarchical clustering algorithms approach the problem of clustering by developing a binary tree-based data structure called the dendrogram. Once the dendrogram is constructed, one can automatically choose the right number of clusters by splitting the tree at different levels to obtain different clustering solutions for the same dataset without rerunning the clustering algorithm again.

Clustering is a complex unsupervised method used to group most similar observations of a given dataset within the same cluster. To guarantee high efficiency, the clustering process should ensure high accuracy and low complexity. Many clustering methods were developed in various fields depending on the type of application and the data type considered. Categorical clustering considers segmenting a dataset in which the data are categorical and were widely used in many real-world applications. Thus several methods were developed including hard, fuzzy and rough set-based methods. In this survey, more than 30 categorical clustering algorithms were investigated. These methods were classified into hierarchical and partitional clustering methods and classified in terms of their accuracy, precision and recall to identify the most prominent ones. Experimental results show that rough set-based clustering methods provided better efficiency than hard and fuzzy methods. Besides, methods based on the initialization of the centroids also provided good results.

The purpose of data clustering algorithm is to form clusters (groups) of data points such that there is high intra-cluster and low inter-cluster similarity. There are different types of clustering methods such as hierarchical, partitioning, grid and density based. Hierarchical clustering is a method of cluster analysis which seeks to build a hierarchy of clusters. A hierarchical clustering method can be thought of as a set of ordinary (flat) clustering methods organized in a tree structure. These methods construct the clusters by recursively partitioning the objects in either a top-down or bottom-up fashion. In this paper we present a new hierarchical clustering algorithm using Euclidean distance. To validate this method we have performed some experiments with low dimensional artificial datasets and high dimensional fMRI dataset. Finally the result of our method is compared to some of existing clustering methods.

Abstract. Classification is beneficial for understanding flood variabilities and their formation mechanisms from massive flood event samples for both flood scientific research and management purposes. Our study investigates comprehensive manageable flood event classes from 1446 unregulated flood events in 68 headstream catchments of China using hierarchical and partitional clustering methods. Control mechanisms of meteorological and physio-geographical factors (e.g., meteorology or land cover and catchment attributes) on spatial and temporal variabilities of individual flood event classes are explored using constrained rank analysis and a Monte Carlo permutation test. We identify five robust flood event classes, i.e., moderately, highly, and slightly fast floods as well as moderately and highly slow floods, which account for 24.0 %, 21.2 %, 25.9 %, 13.5 %, and 15.4 %, respectively, of the total number of events. All of the classes are evenly distributed in the entire period, but the spatial distributions are quite distinct. The fast flood classes are mainly in southern China, and the slow flood classes are mainly in northern China and the transition region between southern and northern China. The meteorological category plays a dominant role in flood event variabilities, followed by catchment attributes and land covers. Precipitation factors, such as volume and intensity, and the aridity index during the events are the significant control factors. Our study provides insights into flood event variabilities and aids in flood prediction and control.

Photovoltaic is attracting attention as one of the renewable energy sources that replace existing fossil fuel-based power generation. Photovoltaic power generation data clustering is becoming increasingly popular because it allows us to identify PV systems without extensive analysis and simulation. Generally, clustering method of PV power data analyzes only the meaning of each cluster, so it is difficult to grasp the similarity between clusters. In this paper, clustering of PV power data through hierarchical clustering algorithm is performed to calculate the relationship between clusters. The hierarchical clustering method was used to analyze the similarity of PV power data between clusters. As a result, PV power data were classified into seasonal clusters and defective clusters.

Community welfare is one of the important points for a region and is also the essence of national development. The welfare of the people in Indonesia is fairly unequal, especially in East Java. To be able to map an area to the welfare of its people in East Java, one way that can be used is to use clustering. The hierarchical clustering method is one of the clustering methods for grouping data. In hierarchical clustering, single linkage, complete linkage, and average linkage methods are suitable methods for grouping data, which will compare the best method to use. The results of the calculation show that the average linkage method with three clusters is the best calculation with a silhouette index value of 0.6054, with the 1st cluster there are 23 regions, namely the city/district with the highest community welfare, the 2nd cluster there are 11 regions, namely cities/districts with moderate social welfare, and in the third cluster there are 4 regions, namely cities/districts with the lowest community welfare.

Supplementary Material for the manuscript "Drivers of carbon dioxide emissions: An empirical investigation using hierarchical and non- hierarchical clustering methods"

Clustering is a group of unsupervised statistical techniques commonly used in many disciplines. Considering their applications to fish abundance data, many technical details need to be considered to ensure reasonable interpretation. However, the reliability and stability of the clustering methods have rarely been studied in the contexts of fisheries. This study presents an intensive evaluation of three common clustering methods, including hierarchical clustering (HC), K-means (KM), and expectation-maximization (EM) methods, based on fish community surveys in the coastal waters of Shandong, China. We evaluated the performances of these three methods considering different numbers of clusters, data size, and data transformation approaches, focusing on the consistency validation using the index of average proportion of non-overlap (APN). The results indicate that the three methods tend to be inconsistent in the optimal number of clusters. EM showed relatively better performances to avoid unbalanced classification, whereas HC and KM provided more stable clustering results. Data transformation including scaling, square-root, and log-transformation had substantial influences on the clustering results, especially for KM. Moreover, transformation also influenced clustering stability, wherein scaling tended to provide a stable solution at the same number of clusters. The APN values indicated improved stability with increasing data size, and the effect leveled off over 70 samples in general and most quickly in EM. We conclude that the best clustering method can be chosen depending on the aim of the study and the number of clusters. In general, KM is relatively robust in our tests. We also provide recommendations for future application of clustering analyses. This study is helpful to ensure the credibility of the application and interpretation of clustering methods.

Hierarchical clustering, a traditional clustering method, has been getting attention again. Among several reasons, a credit goes to a recent paper by Dasgupta in 2016 that proposed a cost function that quantitatively evaluates hierarchical clustering trees. An important question is how to combine this recent advance with existing successful clustering methods. In this paper, we propose a hierarchical clustering method to minimize the cost function of clustering tree by incorporating existing clustering techniques. First, we developed an ensemble tree-search method that finds an integrated tree with reduced cost by integrating multiple existing hierarchical clustering methods. Second, to operate on large and arbitrary shape data, we designed an efficient hierarchical clustering framework, called integrating divisive and ensemble-agglomerate (IDEA) by combining it with advanced clustering techniques such as nearest neighbor graph construction, divisive-agglomerate hybridization, and dynamic cut tree. The IDEA clustering method showed better performance in minimizing Dasgupta's cost and improving accuracy (adjusted rand index) over existing cost-minimization-based, and density-based hierarchical clustering methods in experiments using arbitrary shape datasets and complex biology-domain datasets.

In Indonesia, Dengue Fever (DF) is a contagious disease that is a significant issue in public health. The Kediri Regency in East Java, as reported by the Ministry of Health in 2019, had the highest number of DF cases. If not addressed promptly, DF can lead to outbreaks, creating a health emergency. The lack of a thorough investigation into the diversity of risk within a spatial and temporal region exacerbates this issue. Therefore, spatial-temporal analysis is crucial in developing a warning system to prevent and control DF. This paper proposes a method that combines the Euclidean Distance calculation with the Hierarchical Clustering method. We collected data from the Kediri Regency health department and conducted pre-processing and classification processes, considering the number of DF victims, death rate, population, rainfall, and public facilities. The hierarchical clustering algorithm was used to categorize the 344 village analyses into low, medium, and high vulnerability categories. This method allows for a comparison of yearly single, average, complete, and centroid linkage in DF vulnerability levels. We also employed spatial-temporal visualization based on cellular applications to create a clear picture of areas vulnerable to DF. The experimental results in clustering showed a satisfactory level of matching, with variant values calculated using the hierarchical clustering method. The variants for single linkages were 0.113; for average linkages, they were 0.120; for complete linkages, they were 0.178; and for centroid linkages, they were 0.106. The grouping validation results indicated that the centroid linkage method produced the best variant level. We suggest further enhancing the methods with better process steps using other pre-processing methods to improve the validation quality.

We present a novel pairwise clustering method. Given a proximity matrix of pairwise relations (i.e. pairwise similarity or dissimilarity estimates) between data points, our algorithm extracts the two most prominent clusters in the data set. The algorithm, which is completely nonparametric, iteratively employs a two-step transformation on the proximity matrix. The first step of the transformation represents each point by its relation to all other data points, and the second step re-estimates the pairwise distances using a statistically motivated proximity measure on these representations. Using this transformation, the algorithm iteratively partitions the data points, until it finally converges to two clusters. Although the algorithm is simple and intuitive, it generates a complex dynamics of the proximity matrices. Based on this bipartition procedure we devise a hierarchical clustering algorithm, which employs the basic bipartition algorithm in a straightforward divisive manner. The hierarchical clustering algorithm copes with the model validation problem using a general cross-validation approach, which may be combined with various hierarchical clustering methods. We further present an experimental study of this algorithm. We examine some of the algorithm's properties and performance on some synthetic and ‘standard’ data sets. The experiments demonstrate the robustness of the algorithm and indicate that it generates a good clustering partition even when the data is noisy or corrupted.

In this study, universities are clustered based on the quantity and title of the academics employed by Turkish Universities. The data comes from both state and private universities and vocational schools. For clustering, partitional and hierarchical clustering methods were used. The dataset was obtained from the YOK academic search website using “web scraping” techniques implemented by the authors. Agglomerative clustering technique was found to yield better results regarding cluster sizes and intracluster distances. The Silhouette coefficient is used for clustering quality measurement. As a result of the study, Turkish universities were mainly composed of four clusters. These are instructor dominant universities, just instructor based vocational schools, professor and associate professor dominated universities and research assistant dominated universities. The results of this study may be used for academic staff employment planning in Turkish Universities.

DBSCAN is a well-known density-based clustering algorithm which offers advantages for finding clusters of arbitrary shapes compared to partitioning and hierarchical clustering methods. However, there are few papers studying the DBSCAN algorithm under the privacy preserving distributed data mining model, in which the data is distributed between two or more parties, and the parties cooperate to obtain the clustering results without revealing the data at the individual parties. In this paper, we address the problem of two-party privacy preserving DBSCAN clustering. We first propose two protocols for privacy preserving DBSCAN clustering over horizontally and vertically partitioned data respectively and then extend them to arbitrarily partitioned data. We also provide analysis of the performance and proof of privacy of our solution.

DBSCAN is a well-known density-based clustering algorithm which offers advantages for finding clusters of arbitrary shapes compared to partitioning and hierarchical clustering methods. However, the...

Application of hierarchical clustering, singularity mapping, and Kohonen neural network to identify Ag-Au-Pb-Zn polymetallic mineralization associated geochemical anomaly in Pangxidong district