Frequency tree for clustering categorical data with similarity measure based on items' weights

Tasks such as clustering and classification assume the existence of a similarity measure to assess the similarity (or dissimilarity) of a pair of observations or clusters. The key difference between most clustering methods is in their similarity measures. This article proposes a new similarity measure function called PWO “Probability of the Weights between Overlapped items ”which could be used in clustering categorical dataset; proves that PWO is a metric; presents a framework implementation to detect the best similarity value for different datasets; and improves the F-tree clustering algorithm with Semi-supervised method to refine the results. The experimental evaluation on real categorical datasets, such as “Mushrooms, KrVskp, Congressional Voting, Soybean-Large, Soybean-Small, Hepatitis, Zoo, Lenses, and Adult-Stretch” shows that PWO is more effective in measuring the similarity between categorical data than state-of-the-art algorithms; clustering based on PWO with pre-defined number of clusters results a good separation of classes with a high purity of average 80% coverage of real classes; and the overlap estimator perfectly estimates the value of the overlap threshold using a small sample of dataset of around 5% of data size.

This paper introduces a self-organizing map dedicated to clustering, analysis and visualization of categorical data. Usually, when dealing with categorical data, topological maps use an encoding stage: categorical data are changed into numerical vectors and traditional numerical algorithms (SOM) are run. In the present paper, we propose a novel probabilistic formalism of Kohonen map dedicated to categorical data where neurons are represented by probability tables. We do not need to use any coding to encode variables. We evaluate the effectiveness of our model in four examples using real data. Our experiments show that our model provides a good quality of results when dealing with categorical data.

One of the challenges in clustering categorical data is the curse of dimensionality caused by the inherent sparsity of high-dimensional data, the records of which include a large number of attributes. The latent class model (LCM) assumes local independence between the variables in clusters, and is a parsimonious model-based clustering approach that has been used to circumvent the problem. The mixture of a log-linear model is more flexible but requires more parameters to be estimated. In this research, we recognize that each categorical observation can be conceived as a network with pairwise linked nodes, which are the response levels of the observation attributes. Therefore, the categorical data for clustering is considered a finite mixture of different component layer networks with distinct patterns. We apply a penalized composite likelihood approach to a finite mixture of networks for sparse multivariate categorical data to reduce the number of parameters, implement the EM algorithm to estimate the model parameters, and show that the estimates are consistent and satisfy asymptotic normality. The performance of the proposed approach is shown to be better in comparison with the conventional methods for both synthetic and real datasets.

A cluster ensemble method for clustering categorical data

k-ANMI: A mutual information based clustering algorithm for categorical data

In this paper, we present a framework for categorical data analysis which allows such data sets to be explored using a rich set of techniques that are only applicable to continuous data sets. We introduce the concept of separability statistics in the context of exploratory categorical data analysis. We show how these statistics can be used as a way to map categorical data to continuous space given a labeled reference data set. This mapping enables visualization of categorical data using techniques that are applicable to continuous data. We show that in the transformed continuous space, the performance of the standard k-nn based outlier detection technique is comparable to the performance of the k-nn based outlier detection technique using the best of the similarity measures designed for categorical data. The proposed framework can also be used to devise similarity measures best suited for a particular type of data set.

MGR: An information theory based hierarchical divisive clustering algorithm for categorical data

Knowledge discovery is the process of extracting useful and understandable patterns in data. A knowledge discovery process involves different learning methods in the data mining step or different model visualization methods in the post-processing phase. Among them, categorical data clustering has attracted much attention recently. While many algorithms for clustering categorical data have been proposed, some have low clustering accuracy while others have high computational complexity. In this paper, a visualization method of structured partial ordered attribute diagram for clustering categorical data is proposed based on the formal concept analysis theory. The structured partial ordered attribute diagram is constructed based on covering operations. Knowledge can be extracted from the hierarchies and clusters in a structured partial ordered attribute diagram. The method is applied to a famous formal context in formal concept analysis and a categorical dataset in UCI Machine Learning Repository. The results show that the structured partial ordered attribute diagram provides a new way for visual knowledge mining.

This paper focuses on solving the problem of clustering for categorical data with missing values. Specifically, we design a new framework that can impute missing values and assign objects into appropriate clusters. For the imputation step, we use a decision tree-based method to fill in missing values. For the clustering step, we use a kernel density estimation approach to define cluster centers and an information theoretic-based dissimilarity measure to quantify the differences between objects. Then, we propose a center-based algorithm for clustering categorical data with missing values, namely k-CCM. An experimental evaluation was performed on real-life datasets with missing values to compare the performance of the proposed algorithm with other popular clustering algorithms in terms of clustering quality. Generally, the experimental result shows that the proposed algorithm has a comparative performance when compared to other algorithms for all datasets.

Clustering categorical data poses two challenges defining an inherently meaningful similarity measure, and effectively dealing with clusters which are often embedded in different subspaces. In this paper, we propose a novel divisive hierarchical clustering algorithm for categorical data, named DHCC. We view the task of clustering categorical data from an optimization perspective, and propose effective procedures to initialize and refine the splitting of clusters. The initialization of the splitting is based on multiple correspondence analysis (MCA). We also devise a strategy for deciding when to terminate the splitting process. The proposed algorithm has five merits. First, due to its hierarchical nature, our algorithm yields a dendrogram representing nested groupings of patterns and similarity levels at different granularities. Second, it is parameter-free, fully automatic and, in particular, requires no assumption regarding the number of clusters. Third, it is independent of the order in which the data is processed. Fourth, it is scalable to large data sets. And finally, our algorithm is capable of seamlessly discovering clusters embedded in subspaces, thanks to its use of a novel data representation and Chi-square dissimilarity measures. Experiments on both synthetic and real data demonstrate the superior performance of our algorithm.

During the last one decade, rapid proliferation of categorical data attracts computer scientists and engineers to analyse the categorical data. In this regard, unsupervised technique, such as clustering has been used by inception of new algorithms or modification of the existing ones. These methods are basically optimizing single objective function to get the partitions. However, optimization of multiple conflicting objectives simultaneously may evolve better clustering results as the multiobjective optimization techniques have been successfully applied in various fields of engineering and science. Hence, in this article, Multiobjective Differential Evolution based Fuzzy Clustering for Categorical Data is proposed. For this purpose, differential evolution is used as an underlying optimization technique. Moreover, the index encoding scheme is used to encode the vector in differential evolution and after the mutation operation in differential evolution, scaling is introduced to adjust the encoded index value within the permissible range of categorical objects. The performance of the proposed method has been demonstrated by comparing it with the widely used state-of-the-art methods for two synthetic and two real life data sets. Finally, statistical test has been conducted to judge the superiority of the proposed method.

Random testing (RT) is a fundamental testing technique to assess software reliability, by simply selecting test cases in a random manner from the whole input domain. As an enhancement of RT, adaptive random testing (ART) has better failure-detection capability and has been widely applied in different scenarios, such as numerical programs, some object-oriented programs, and mobile applications. However, not much work has been done on the effectiveness of ART for the programs with combinatorial input domain (i.e., the set of categorical data). To extend the ideas to the testing for combinatorial input domain, we have adopted different similarity measures that are widely used for categorical data in data mining and have proposed two similarity measures based on interaction coverage. Then, we propose a new version named ART-CID as an extension of ART in combinatorial input domain, which selects an element from categorical data as the next test case such that it has the lowest similarity against already generated test cases. Experimental results show that ART-CID generally performs better than RT, with respect to different evaluation metrics.

The k-modes clustering algorithm is undoubtedly one of the most widely used partitional algorithms for categorical data. Unfortunately, due to its gradient descent nature, this algorithm is highly sensitive to the initialization of clustering. Categorical initialization methods have been proposed to address this problem. In this paper, we present an overview of initialization methods of clustering for numerical data and categorical data respectively with an emphasis on their computational efficiency. We then propose a new initialization method for categorical data, which can obtain the good initial cluster centers using the new distance base on the RD, and explore the methods of density and grid. Finally, proposed method has been tested on diagnosis dataset, a real world data set from UCI Machine Learning Repository, and been analyzed the experimental results, which illustrates that the proposed method is effective and efficient for initializing categorical data.

We present a new similarity measure for texture images based on the gray level aura matrices (GLAM), originally proposed by Elfadel and Picard for modeling textures. With the new similarity measure, a support vector machine (SVM) is used to learn pattern similarities for texture image retrieval. In our approach, a texture image is first segmented into clusters of gray level sets. Defined based on the aura measures, a normalized aura matrix is calculated between the gray level sets of the image. The similarity between two texture images computed by the distance of their corresponding normalized aura matrices is defined as the aura matrix distance. The smaller the distance, the more similar are the two textures. To enable the learning of similarity for texture image retrieval, an existing SVM method is adapted to our application, but with a different similarity measure function, different texture feature vectors, and a different similarity ranking scheme for the final retrieved images based on the GLAM. We compare our approach experimentally with existing approaches by performing texture image retrieval from the Brodatz database and the Vistex database. The experimental results show that the proposed approach has performance significantly better than existing approaches with an average successful retrieval rate of 99% - 100% vs 89% - 92% using other approaches.

Clustering refers to the method grouping the large data into the smaller groups based on the similarity measure. Clustering techniques have been applied on numerical, categorical and mix data. One of the categorical data clustering technique based on the soft set theory is Maximum Attribute Relation (MAR). The MAR technique allows calculating all of pair multi soft set made. However, the computational complexity is still an issue of the technique. To overcome the drawback, the paper proposes the alternative algorithm to decrease the complexity so get the faster response time. In this paper, to get the similar results as MAR without calculating all pair of soft set is proved. The alternative algorithm is implemented in MATLAB Software, and then experimental is run on the 10 benchmark datasets. The results show that the alternative algorithm improves the computational complexity in term of response time up to 36.46%