Initialization of k-modes clustering for categorical data

This paper introduces a self-organizing map dedicated to clustering, analysis and visualization of categorical data. Usually, when dealing with categorical data, topological maps use an encoding stage: categorical data are changed into numerical vectors and traditional numerical algorithms (SOM) are run. In the present paper, we propose a novel probabilistic formalism of Kohonen map dedicated to categorical data where neurons are represented by probability tables. We do not need to use any coding to encode variables. We evaluate the effectiveness of our model in four examples using real data. Our experiments show that our model provides a good quality of results when dealing with categorical data.

A cluster ensemble method for clustering categorical data

Data clustering is crucial when it comes to data processing and analytics. The new clustering method overcomes the challenge of evaluating and extracting data from big data. Numerical or categorical data can be grouped. Existing clustering methods favor numerical data clustering and ignore categorical data clustering. Until recently, the only way to cluster categorical data was to convert it to a numeric representation and then cluster it using current numeric clustering methods. However, these algorithms could not use the concept of categorical data for clustering. Following that, suggestions for expanding traditional categorical data processing methods were made. In addition to expansions, several new clustering methods and extensions have been proposed in recent years. ROCK is an adaptable and straightforward algorithm for calculating the similarity between data sets to cluster them. This paper aims to modify the algorithm by creating a parameterized version that takes specific algorithm parameters as input and outputs satisfactory cluster structures. The parameterized ROCK algorithm is the name given to the modified algorithm (P-ROCK). The proposed modification makes the original algorithm more flexible by using user-defined parameters. A detailed hypothesis was developed later validated with experimental results on real-world datasets using our proposed P-ROCK algorithm. A comparison with the original ROCK algorithm is also provided. Experiment results show that the proposed algorithm is on par with the original ROCK algorithm with an accuracy of 97.9%. The proposed P-ROCK algorithm has improved the runtime and is more flexible and scalable.

Clustering is an important data mining technique that groups similar data records. In this research, the problem of clustering categorical (or transactional) data is studied. An algorithm for clustering transactional data, called F-Tree (frequent tree), is proposed, and a new similarity measure function, called PWO (probability of the weights of overlapped items), is introduced. Then, a framework to detect the best similarity value for different datasets is developed. Based on the items' frequencies in transactions, the F-Tree algorithm first generates small but highly pure clusters. It then merges similar clusters using the PWO similarity measure function, which depends on the probability of weights of overlapped items. One advantage of PWO is that it outputs the percentage of similarity between items; this is a clearer and more accurate measure of similarity than other similarity measures. Our experimental evaluation on real categorical datasets such as (Mushrooms, KrVskp, Congressional Voting, Soybean-Large, Soybean-Small, Hepatitis, Zoo, Lenses, and Adult-Stretch) shows that: Firstly, F-Tree is effective in finding rare or interesting clusters, and produces clusters with higher purity than LargeItem, WCD and CLOPE. Secondly, PWO is more effective in measuring the similarity between categorical data than LargeItem and CLOPE. Thirdly, clustering using the similarity measure PWO with pre-defined number of clusters results in separate classes with a good purity of average 80% coverage of real classes. Fourthly, the overlap estimator perfectly estimates the value of the overlap threshold using a small sample of dataset of around 5%. Finally, it is seen that the process of merging pure and small clusters increases the purity of resulted clusters as well as reduces time of clustering better than the process of generating clusters directly from the dataset then refining clusters. Based on the items' frequencies in transactions, The F-Tree algorithm first generates small but highly pure clusters. It then merges similar clusters using the PWO similarity measure function, which depends on the probability of weights of overlapped items. One advantage of PWO is that it outputs the percentage of similarity between items; this is a clearer and more accurate measure of similarity than other similarity measures. Our experimental evaluation on real categorical datasets such as "Mushrooms, KrVskp, Congressional Voting, Soybean-Large, Soybean-Small, Hepatitis, Zoo, Lenses, and Adult-Stretch" shows that: Firstly, F-Tree is effective in finding rare or interesting clusters, and produces clusters with higher purity than LargeItem, WCD and CLOPE. Secondly, PWO is more effective in measuring the similarity between categorical data than LargeItem and CLOPE. Thirdly, Clustering using the similarity measure PWO with pre-defined number of clusters results in separate classes with a good purity of average 80% coverage of real classes. Fourthly, the overlap estimator perfectly estimates the value of the overlap threshold using a small sample of dataset of around 5 percent. Finally, It is seen that the process of merging pure and small clusters increases the purity of resulted clusters as well as reduces time of clustering better than the process of generating clusters directly from the dataset then refining clusters.

One of the challenges in clustering categorical data is the curse of dimensionality caused by the inherent sparsity of high-dimensional data, the records of which include a large number of attributes. The latent class model (LCM) assumes local independence between the variables in clusters, and is a parsimonious model-based clustering approach that has been used to circumvent the problem. The mixture of a log-linear model is more flexible but requires more parameters to be estimated. In this research, we recognize that each categorical observation can be conceived as a network with pairwise linked nodes, which are the response levels of the observation attributes. Therefore, the categorical data for clustering is considered a finite mixture of different component layer networks with distinct patterns. We apply a penalized composite likelihood approach to a finite mixture of networks for sparse multivariate categorical data to reduce the number of parameters, implement the EM algorithm to estimate the model parameters, and show that the estimates are consistent and satisfy asymptotic normality. The performance of the proposed approach is shown to be better in comparison with the conventional methods for both synthetic and real datasets.

k-ANMI: A mutual information based clustering algorithm for categorical data

MGR: An information theory based hierarchical divisive clustering algorithm for categorical data

Knowledge discovery is the process of extracting useful and understandable patterns in data. A knowledge discovery process involves different learning methods in the data mining step or different model visualization methods in the post-processing phase. Among them, categorical data clustering has attracted much attention recently. While many algorithms for clustering categorical data have been proposed, some have low clustering accuracy while others have high computational complexity. In this paper, a visualization method of structured partial ordered attribute diagram for clustering categorical data is proposed based on the formal concept analysis theory. The structured partial ordered attribute diagram is constructed based on covering operations. Knowledge can be extracted from the hierarchies and clusters in a structured partial ordered attribute diagram. The method is applied to a famous formal context in formal concept analysis and a categorical dataset in UCI Machine Learning Repository. The results show that the structured partial ordered attribute diagram provides a new way for visual knowledge mining.

During the last one decade, rapid proliferation of categorical data attracts computer scientists and engineers to analyse the categorical data. In this regard, unsupervised technique, such as clustering has been used by inception of new algorithms or modification of the existing ones. These methods are basically optimizing single objective function to get the partitions. However, optimization of multiple conflicting objectives simultaneously may evolve better clustering results as the multiobjective optimization techniques have been successfully applied in various fields of engineering and science. Hence, in this article, Multiobjective Differential Evolution based Fuzzy Clustering for Categorical Data is proposed. For this purpose, differential evolution is used as an underlying optimization technique. Moreover, the index encoding scheme is used to encode the vector in differential evolution and after the mutation operation in differential evolution, scaling is introduced to adjust the encoded index value within the permissible range of categorical objects. The performance of the proposed method has been demonstrated by comparing it with the widely used state-of-the-art methods for two synthetic and two real life data sets. Finally, statistical test has been conducted to judge the superiority of the proposed method.

Model-based clustering methods for continuous data are well established and commonly used in a wide range of applications. However, model-based clustering methods for categorical data are less standard. Latent class analysis is a commonly used method for model-based clustering of binary data and/or categorical data, but due to an assumed local independence structure there may not be a correspondence between the estimated latent classes and groups in the population of interest. The mixture of latent trait analyzers model extends latent class analysis by assuming a model for the categorical response variables that depends on both a categorical latent class and a continuous latent trait variable; the discrete latent class accommodates group structure and the continuous latent trait accommodates dependence within these groups. Fitting the mixture of latent trait analyzers model is potentially difficult because the likelihood function involves an integral that cannot be evaluated analytically. We develop a variational approach for fitting the mixture of latent trait models and this provides an efficient model fitting strategy. The mixture of latent trait analyzers model is demonstrated on the analysis of data from the National Long Term Care Survey (NLTCS) and voting in the U.S. Congress. The model is shown to yield intuitive clustering results and it gives a much better fit than either latent class analysis or latent trait analysis alone.

In many real world data analysis tasks, it is expected to get much more useful knowledge by utilizing multiple databases stored in different organizations, such as cooperation groups, state organs and allied countries. However, in many such organizations, they often hesitate to publish their databases because of privacy and security issues although they believe the advantages of collaborative analysis. To utilize such multiple databases without fear of information leaks, a privacy preserving procedure was introduced to fuzzy clustering for categorical multivariate data (FCCM). This method deals with vertically partitioned databases, and several experimental results demonstrated that the method can contribute to revealing global intrinsic co-cluster structure rather than individual site-wise analysis. In this paper, site-wise confidence is introduced to the method to weaken harmful effects of insignificant sites for estimating stable intrinsic co-cluster structures. Site-wise confidence is measured with a degree of coincidence between site-wise object memberships and the global ones. Several experimental results show that site-wise confidence is useful for estimating reliable co-cluster structures.

Estimation models in software engineering are used to predict some important attributes of future entities such as software development effort, software reliability and programmers productivity. Among these models, those estimating software effort have motivated considerable research in recent years. Estimation by analogy is one of the attractive techniques in software effort estimation field. However, the procedure used in estimation by analogy is not yet able to handle correctly categorical data. In this paper, we propose a new approach based on reasoning by analogy, fuzzy logic and linguistic quantifiers to estimate effort when the software project is described either by categorical or numerical data. Fuzzy logic-based cost estimation models are more appropriate when vague and imprecise information is to be accounted for. Fuzzy systems try to behave just like the processes of the brain with a rule base. In our proposed method the effort and the cost can be estimated using the Fuzzy logic based analogy method and the cost can be calculated. The performance of the proposed system is calculated using MARE (Mean Absolute Relative Error).

Partition-and-merge based fuzzy genetic clustering algorithm for categorical data

Clustering is the problem of identifying the distribution of patterns and intrinsic correlations in large data sets by partitioning the data points into similarity classes. In this paper, we study on the problem of clustering categorical data, where data objects are made up of non-numerical attributes. We propose MECC (Minimum Error Classification Clustering), an alternative technique for categorical data clustering using VPRS taking into account minimum error classification. The technique is implemented in MATLAB. Experimental results on two benchmark UCI datasets show that MECC technique is better than the baseline categorical data clustering techniques with respect to selecting the clustering attribute.

Categorical data clustering has been attracted a lot of attention recently due to its necessary in the real-world applications. Many clustering methods have been proposed for categorical data. However, most of the existing algorithms require the predefined number of clusters which is usually unavailable in real-world problems. Only a few works focused on automatic clustering, but mainly handled for numerical data. This study develops a novel automatic fuzzy clustering using non-dominated sorting particle swarm optimization (AFC-NSPSO) algorithm for categorical data. The proposed AFC-NSPSO algorithm can automatically identify the optimal number of clusters and exploit the clustering result with the corresponding selected number of clusters. In addition, a new technique is investigated to identify the maximum number of clusters in a dataset based on the local density. To select a final solution in the first Pareto front, some internal validation indices are used. The performance of the proposed AFC-NSPSO on the real-world datasets collected from the UCI machine learning repository exhibits effectiveness compared with some other existing automatic categorical clustering algorithms. Besides, this study also applies the proposed algorithm to analyze a real-world case study with an unknown number of clusters.