MGR: An information theory based hierarchical divisive clustering algorithm for categorical data

Partition-and-merge based fuzzy genetic clustering algorithm for categorical data

Clustering categorical data poses two challenges defining an inherently meaningful similarity measure, and effectively dealing with clusters which are often embedded in different subspaces. In this paper, we propose a novel divisive hierarchical clustering algorithm for categorical data, named DHCC. We view the task of clustering categorical data from an optimization perspective, and propose effective procedures to initialize and refine the splitting of clusters. The initialization of the splitting is based on multiple correspondence analysis (MCA). We also devise a strategy for deciding when to terminate the splitting process. The proposed algorithm has five merits. First, due to its hierarchical nature, our algorithm yields a dendrogram representing nested groupings of patterns and similarity levels at different granularities. Second, it is parameter-free, fully automatic and, in particular, requires no assumption regarding the number of clusters. Third, it is independent of the order in which the data is processed. Fourth, it is scalable to large data sets. And finally, our algorithm is capable of seamlessly discovering clusters embedded in subspaces, thanks to its use of a novel data representation and Chi-square dissimilarity measures. Experiments on both synthetic and real data demonstrate the superior performance of our algorithm.

Knowledge discovery is the process of extracting useful and understandable patterns in data. A knowledge discovery process involves different learning methods in the data mining step or different model visualization methods in the post-processing phase. Among them, categorical data clustering has attracted much attention recently. While many algorithms for clustering categorical data have been proposed, some have low clustering accuracy while others have high computational complexity. In this paper, a visualization method of structured partial ordered attribute diagram for clustering categorical data is proposed based on the formal concept analysis theory. The structured partial ordered attribute diagram is constructed based on covering operations. Knowledge can be extracted from the hierarchies and clusters in a structured partial ordered attribute diagram. The method is applied to a famous formal context in formal concept analysis and a categorical dataset in UCI Machine Learning Repository. The results show that the structured partial ordered attribute diagram provides a new way for visual knowledge mining.

A cluster ensemble method for clustering categorical data

k-ANMI: A mutual information based clustering algorithm for categorical data

This paper focuses on solving the problem of clustering for categorical data with missing values. Specifically, we design a new framework that can impute missing values and assign objects into appropriate clusters. For the imputation step, we use a decision tree-based method to fill in missing values. For the clustering step, we use a kernel density estimation approach to define cluster centers and an information theoretic-based dissimilarity measure to quantify the differences between objects. Then, we propose a center-based algorithm for clustering categorical data with missing values, namely k-CCM. An experimental evaluation was performed on real-life datasets with missing values to compare the performance of the proposed algorithm with other popular clustering algorithms in terms of clustering quality. Generally, the experimental result shows that the proposed algorithm has a comparative performance when compared to other algorithms for all datasets.

The University of California at Irvine (UCI), in a 29-page audit of its Willed Body Program that took more than a year and a half to complete, has faulted its former director, Christopher S Brown. The audit uncovered lack of supervision, poor record keeping, failure to return remains to families in accordance with their wishes, and financial irregularities. The last included supplying 7 cadaver spines to a research group in Phoenix, Arizona, for $5,125. The payment was made to a company, University Health Services, not affiliated with UCI and owned by Jeffrey Frazier, a business associate of Brown's. The audit also found overcharges of several hundred dollars by Harry's Transportation, a company owned by Brown and Frazier for carrying cadavers. Donated cadavers were used, without UCI's permission, in a private anatomy class run in UCI's Willed Body morgue by Frazier's company Replica, which charged $175 per student. Brown charged families who desired a return of their cremated donors' ashes a $600 fee. Three families paid to have ashes returned, but there is no record that UCI received the money. Poor record keeping prevented the UCI auditors from accounting for 320 of the 441 cadavers donated to the Willed Body Program from January 1, 1995, through August 11, 1999. As the result of alleged misplaced bodies, about 20 lawsuits have been filed by survivors against UCI, Brown, and Frazier. UCI fired Brown in September 1999. No charges have been brought against him, but the Orange County District Attorney's office is investigating the allegations. Corrective measures taken by UCI include replacing Brown, a former funeral director, with a more experienced mortician as program director and written policies and procedures. Thomas C Cesario, UCI College of Medicine Dean, has appointed Peter Lawrence, a professor of surgery, to oversee the program, in conjunction with an advisory committee of faculty members. Willed Body programs based at US medical schools collect about 15,000 bodies annually. The demand far outstrips the supply, according to Philip Guyett, former manager of the Willed Body Program at Western University of Health Sciences in Pomona, California. After the audit began last year, Cesario said, The people who will their body to medical science and education have made one of the most noble gifts possible, and it is absolutely essential that our Willed Body Program operate not only in accordance with the highest professional standards, but also with the highest standards of respect for these donors and their families.

This research aims to assess the similarity between determining the value of the similarity of objects by using various measures. Particularly for categorical data, the used similarity measures may behave differently depending on whether the analyzed data are binary, non-binary, or mixed. In the case of categorical data, it begins to matter how many attributes we have and how many different values they can take. Cluster analysis is an essential technique in unsupervised machine learning. Object-clustering cannot be determined without determining similarities between objects in the set and searching for clusters with the most significant internal consistency and external separatory. Therefore, we need knowledge about which measure behaves similarly and which differs depending on the nature of the analyzed data. According to our knowledge, there is no such research in the literature. In research, we analyzed six different measures for eight different datasets (differing in size and data type). The results say, among others, which measures take the longest time for calculations and which are the most correlated with others, so they can be used interchangeably without fear of losing valuable information about the mutual similarity of objects.

This chapter aims to explore important clustering applications for categorical datasets and explains benefits and drawbacks of existing categorical clustering algorithms. It presents the goals of categorical clustering algorithms in general and provides gives an overview of similarity measures for categorical data. The chapter describes categorical clustering algorithms from the literature and discusses the scalability of the algorithms. Categorical clustering algorithms have various features, which make them suitable for applications with different requirements. A growing number of clustering algorithms for categorical data have been proposed, along with interesting applications, such as partitioning large software systems and protein interaction data. Hierarchical algorithms for categorical clustering take the approach of building a hierarchy representing a dataset’s entire underlying cluster structure. Hierarchical algorithms require few user-specified parameters and are insensitive to object ordering. The simplest categorical similarity measure is the Hamming distance, which measures the overlap between two categorical data objects by counting the number of matching attributes.

Clustering is a widely used technique in data mining application for discovering patterns in underlying data. Most traditional clustering algorithms are limited in handling datasets that contain categorical attributes. Howerver, datasets with categorical types of attributes are common in real life data mining problem. For these data sets, no inherent distance measure, like the Euclidean distance, would work to compute the distance between two catgorical objects. In this article, we have described differential evolution, genetic algorithm and simulated annealing based fuzzy clustering. The performance of the proposed algorithms have been compared with that of different well known categorical data clustering algorithms and demonstrated for a variety of artificial and real life categorical data sets. Statistical significance test has been performed to establish the superiority of the proposed algorithms.

In this paper, we propose an incremental clustering algorithm called INCLUS for high dimensional sparse transactional data using a newly defined similarity measure and a notion of cluster representatives based on locally frequent items of each cluster. INCLUS seeks structures in transactional data with respect to the support and similarity constraints specified by the users. The effectiveness and the order-independence property of INCLUS are empirically studied and compared with two state- of-art algorithms. Though it is a one-pass algorithm without any iterative refinement, INCLUS is not only effective and scalable, but also insensitive to the order of transactions, which is a crucial property for an incremental algorithm. globally frequent items are neglected during clustering, clusters embedded in such transactions cannot be discovered. In this paper, we define a new notion of cluster representative based on frequent items within each cluster that is easy to comprehend by users, and introduce a new similarity measure especially suitable for sparse transactional data. Using these, an incremental clustering algorithm INCLUS is proposed to cope with high cardinality of transactional datasets. INCLUS is a structure seeking algorithm as it produces clusters based on the support and similarity constraints specified by the users. The domains of input parameters (support and similarity thresholds) are well defined and can be easily chosen by users. INCLUS is significantly different from LargeItem, SUMMARY and TrK-Means which also employ frequent items for clustering. LargeItem uses global optimization to minimize both overlapping of frequent items among clusters and the number of infrequent items for a clustering. INCLUS uses frequent items for local optimization such that similarities for a clustering are maximized with respect to the similarity constraints. In SUMMARY, the frequent items used are globally frequent in the whole data set while in INCLUS they are only locally frequent within a cluster. Unlike INCLUS, TrK-Means does not use similarity constraints and it produces k or more clusters based on the user input of k. Squeezer (7) is an incremental clustering algorithm for categorical data. It uses a different similarity function from INCLUS, which is defined as the number of matched attribute values in a pair of tuples.

In model-based cluster analysis, the expectation-maximization (EM) algorithm has a number of desirable properties, but in some situations, this algorithm can be slow to converge. Some variants are proposed to speed-up EM in reducing the time spent in the E-step, in the case of Gaussian mixture. The main aims of such methods is first to speed-up convergence of EM, and second to yield same results (or not so far) than EM itself. In this paper, we compare these methods from categorical data, with the latent class model, and we propose a new variant that sustains better results on synthetic and real data sets, in terms of convergence speed-up and number of misclassified objects.

This paper introduces a self-organizing map dedicated to clustering, analysis and visualization of categorical data. Usually, when dealing with categorical data, topological maps use an encoding stage: categorical data are changed into numerical vectors and traditional numerical algorithms (SOM) are run. In the present paper, we propose a novel probabilistic formalism of Kohonen map dedicated to categorical data where neurons are represented by probability tables. We do not need to use any coding to encode variables. We evaluate the effectiveness of our model in four examples using real data. Our experiments show that our model provides a good quality of results when dealing with categorical data.

Categorical data clustering has been attracted a lot of attention recently due to its necessary in the real-world applications. Many clustering methods have been proposed for categorical data. However, most of the existing algorithms require the predefined number of clusters which is usually unavailable in real-world problems. Only a few works focused on automatic clustering, but mainly handled for numerical data. This study develops a novel automatic fuzzy clustering using non-dominated sorting particle swarm optimization (AFC-NSPSO) algorithm for categorical data. The proposed AFC-NSPSO algorithm can automatically identify the optimal number of clusters and exploit the clustering result with the corresponding selected number of clusters. In addition, a new technique is investigated to identify the maximum number of clusters in a dataset based on the local density. To select a final solution in the first Pareto front, some internal validation indices are used. The performance of the proposed AFC-NSPSO on the real-world datasets collected from the UCI machine learning repository exhibits effectiveness compared with some other existing automatic categorical clustering algorithms. Besides, this study also applies the proposed algorithm to analyze a real-world case study with an unknown number of clusters.

Clustering techniques are used to split data into clusters where each cluster contains elements that look more similar to elements in the same cluster than elements in other clusters. Some of these techniques are capable of handling clustering process uncertainty, while other techniques may have stability issues. In this paper, a novel method, called Minimum Information Gain Roughness (MIGR), is proposed to select the clustering attribute based on information entropy with rough set theory. To evaluate its performance, three benchmark UCI datasets are chosen to be clustered by using MIGR. Then, the resulting clusters are compared to those which are resulted from applying Min-Min-Rough (MMR) and information-theoretic dependency roughness (ITDR) algorithms. Both last-mentioned techniques were already compared with a variety of clustering algorithms like k-modes, fuzzy centroids, and fuzzy k-modes. The Global purity, the overall purity, and F-measure are considered here as performance measures to compare the quality of the resulting clusters. The experimental results show that the MIGR algorithm outperforms both MMR and ITDR algorithms for clustering categorical data.