Comparison of Different Fuzzy Clustering Algorithms: A Replicated Case Study

The identification of fuzzy models for classification is a very complex task. Often, real world databases have a large number of features and the most relevant ones must be chosen. Recently, a new automatic feature selection for classification problems was proposed to construct compact fuzzy classification models. This technique used the classical fuzzy c-means algorithm. However, other fuzzy clustering algorithms, such as possibilistic c-means, fuzzy possibilistic c-means or possibilistic fuzzy c-means can be used to cluster the data. An open topic of research is what clustering algorithms can be used to derive fuzzy models for classification. This paper addresses this topic, by comparing fuzzy clustering algorithms in terms of computational efficiency and accuracy in classification problems. The algorithms were tested in well-known data sets: iris plant, wine, hepatitis, breast cancer and in a difficult real-world problem: the prediction of bankruptcy.

<span id="docs-internal-guid-94842888-7fff-2ae1-cd5c-026943b95b7f"><span>Cancer is one of the diseases with the highest mortality rate in the world. Cancer is a disease when abnormal cells grow out of control that can attack the body's organs side by side or spread to other organs. Lung cancer is a condition when malignant cells form in the lungs. To diagnose lung cancer can be done by taking x-ray images, CT scans, and lung tissue biopsy. In this modern era, technology is expected to help research in the field of health. Therefore, in this study feature extraction from CT images was used as data to classify lung cancer. We used CT scan image data from SPIE-AAPM Lung CT challenge 2015. Fuzzy C-Means and fuzzy kernel C-Means were used to classify the lung nodule from the patient into benign or malignant. Fuzzy C-Means is a soft clustering method that uses Euclidean distance to calculate the cluster center and membership matrix. Whereas fuzzy kernel C-Means uses kernel distance to calculate it. In addition, the support vector machine was used in another study to obtain 72% average AUC. Simulations were performed using different k-folds. The score showed fuzzy kernel C-Means had the highest accuracy of 74%, while fuzzy C-Means obtained 73% accuracy. </span></span>

Fuzzy C-means (FCM) is an important clustering algorithm with broad applications such as retail market data analysis, network monitoring, web usage mining, and stock market prediction. Especially, parameters in FCM have influence on clustering results. However, a lot of FCM algorithm did not solve the problem, that is, how to set parameters. In this study, we present a kind of method for computing parameters values according to role of parameters in the clustering process. New parameters are assigned to membership and typicality so as to modify objective function, on the basis of which Lagrange equation is constructed and iterative equation of membership is acquired, so does the typicality and center equation. At last, a new possibilistic fuzzy C-means based on the weight parameter algorithm (WPFCM) was proposed. In order to test the efficiency of the algorithm, some experiments on different datasets are conducted to compare WPFCM with FCM, possibilistic C-means (PCM), and possibilistic fuzzy C-means (PFCM). Experimental results show that iterative times of WPFCM are less than FCM about 25% and PFCM about 65% on dataset X12. Resubstitution errors of WPFCM are less than FCM about 19% and PCM about 74% and PFCM about 10% on the IRIS dataset.

Possibilistic c-means (PCM) algorithm was proposed to overcome the noise sensitivity of fuzzy c-means (FCM). However, the performance of PCM depends heavily on the initialization, and often deteriorates due to the coincident clustering problem. To overcome these problems, we propose a new hybrid clustering algorithm that incorporates an ACO-based clustering into PCM, namely ACOPCM for noisy image segmentation. Our ACOPCM solves the coincident clustering problem using pre-classified pixel information and provides the near optimal initialization of the number of clusters and their centroids. Experimental results demonstrate that our proposed approach achieves higher segmentation accuracy than PCM and hybrid fuzzy clustering approaches.

Fuzzy c-means (FCM) is one of the prominent method utilized for medical image segmentation. In literature intuitionistic fuzzy c-means (IFCM) is suggested which is based on intuitionistic fuzzy sets (IFSs) theory to handle uncertainty and vagueness associated with real data. The objective function of which is defined using the hesitation degree along with membership degree. However, instead of solving the objective function analytically, the approximate solution is obtained using FCM. In this paper, we have proposed a modified intuitionistic fuzzy c-means algorithm (MIFCM) and solved analytically the objective function of the MIFCM method using Lagrange method of undetermined multiplier. To incorporate hesitation degree, two parametric intuitionistic fuzzy complements namely Sugeno’s negation function and Yager’s negation function are investigated. The performance of the MIFCM method is compared with three intuitionistic fuzzy clustering methods and the FCM on two publicly available MRI dataset and a synthetic dataset. The performance measures (average segmentation accuracy, dice score, jaccard score, false negative ratio and false positive ratio) are used to compare the performance of the MIFCM method with three variants of intuitionistic fuzzy clustering methods and the FCM. Experimental results demonstrate the superior performance of the MIFCM method over others.

Possibilistic fuzzy c-means (PFCM) clustering is a kind of hybrid clustering method based on fuzzy c-means (FCM) and possibilistic c-means (PCM), which not only has the stability of FCM but also partly inherits the robustness of PCM. However, as an extension of FCM on the objective function, PFCM tends to find a suboptimal local minimum, which affects its performance. In this paper, we rederive PFCM using the majorization-minimization (MM) method, which is a new derivation approach not seen in other studies. In addition, we propose an effective optimization method to solve the above problem, called MMPFCM. Firstly, by eliminating the variable V∈Rp×c, the original optimization problem is transformed into a simplified model with fewer variables but a proportional term. Therefore, we introduce a new intermediate variable s∈Rc to convert the model with the proportional term into an easily solvable equivalent form. Subsequently, we design an iterative sub-problem using the MM method. The complexity analysis indicates that MMPFCM and PFCM share the same computational complexity. However, MMPFCM requires less memory per iteration. Extensive experiments, including objective function value comparison and clustering performance comparison, demonstrate that MMPFCM converges to a better local minimum compared to PFCM.

In this paper, a new Hopfield-model net based on fuzzy possibilistic reasoning is proposed for the classification of multispectral images. The main purpose is to modify the Hopfield network embedded with fuzzy possibilistic C-means (FPCM) method to construct a classification system named fuzzy-possibilistic Hopfield net (FPHN). The classification system is a paradigm for the implementation of fuzzy logic systems in neural network architecture. Instead of one state in a neuron for the conventional Hopfield nets, each neuron occupies 2 states called membership state and typicality state in the proposed FPHN. The proposed network not only solves the noise sensitivity fault of Fuzzy C-means (FCM) but also overcomes the simultaneous clustering problem of possibilistic C-means (PCM) strategy. In addition to the same characteristics as the FPCM algorithm, the simple features of this network are clear potential in optimal problem. The experimental results show that the proposed FPHN can obtain better solutions in the classification of multispectral images.

Questions: Does fuzzy clustering provide an appropriate numerical framework to manage vegetation classifications? What is the best fuzzy clustering method to achieve this? Material: We used 531 releves from Catalonia (Spain), belonging to two syntaxonomic alliances of mesophytic and xerophytic montane pastures, and originally classified by experts into nine and 13 associations, respectively. Methods: We compared the performance of fuzzy C-means (FCM), noise clustering (NC) and possibilistic C-means (PCM) on four different management tasks: (1) assigning new releve data to existing types; (2) updating types incorporating new data; (3) defining new types with unclassified releves; and (4) reviewing traditional vegetation classifications. Results: As fuzzy classifiers, FCM fails to indicate when a given releve does not belong to any of the existing types; NC might leave too many releves unclassified; and PCM membership values cannot be compared. As unsupervised clustering methods, FCM is more sensitive than NC to transitional releves and therefore produces fuzzier classifications. PCM looks for dense regions in the space of species composition, but these are scarce when vegetation data contain many transitional releves. Conclusions: All three models have advantages and disadvantages, although the NC model may be a good compromise between the restricted FCM model and the robust but impractical PCM model. In our opinion, fuzzy clustering might provide a suitable framework to manage vegetation classifications using a consistent operational definition of vegetation type. Regardless of the framework chosen, national/regional vegetation classification panels should promote methodological standards for classification practices with numerical tools.

Bias-correction fuzzy clustering algorithms

Clustering is the process of dividing data clusters into fragmented clusters so that the same set of data is similar, but the data of different clusters is different. The basis of the fuzzy clustering algorithms is the c-means families and the strongest algorithm is the fuzzy c-means algorithm. However, the fuzzy c-means algorithm is sensitive to outliers. In this study, on the real data set we examined three different algorithms -possibilistic c-means algorithm (PCM), fuzzy possibilistic c-means (FPCM) and possibilistic fuzzy c- means algorithm (PFCM)- which are developed to overcome the unfavorable side of the FCM algorithm. To compare these algorithms, iteration numbers and completion times were calculated.

ABSTRACTA process of splitting the image into pixel bands is the image segmentation. As medical imaging contain uncertainties, there are difficulties in classification of images into homogeneous regions. There is a need for segmentation algorithm for removing the noise from the medical image segmentation. The very popular algorithm is Fuzzy C‐Means (FCM) algorithm used for image segmentation. Fuzzy sets, rough sets, and the combination of fuzzy and rough sets play a prominent role in formalizing uncertainty, vagueness, and incompleteness in diagnosis. But it will use intensity values only which will be highly sensitive to noise. In this article, an Intuitionistic FCM (IFCM) algorithm is presented for clustering. Intuitionistic fuzzy (IF) sets are generalized sets and their elements are characterized by a membership value as well as nonmembership value. This IFCM has an uncertainty parameter which is called hesitation degree and a new objective function is integrated in the standard FCM based on IF entropy. The IFCM will provide better performance than FCM for image segmentation.

Intuitionistic Fuzzy C-means (IFCM) is a robust clustering method which is based upon intuitionistic fuzzy set theory. It uses Euclidean distance as a distance metric, hence can only cluster hyper spherically distributed data-sets in data space or in feature space. FCM and KFCM with a new distance measure (FCM-σ and KFCM-σ) can detect non-hyperspherical clusters in data space and feature space but they are sensitive to noise and produce inefficient results in the presence of noise. This paper present a robust Intuitionistic Fuzzy c-means(IFCM-σ) and a robust kernel Intutitionistic Fuzzy C-Means(KIFCM-σ) with a new distance metric that incorporates the distance variation in a cluster to regularize the distance between data point and the cluster centroid. Propose algorithms are the hybridization of IFCM, kernel function, and new distance metric in the data space and in the feature space which avoid various problems of IFCM and FCM-σ. Experiments are done using two-dimensional synthetic data-sets and noisy digital images, and results are compared with IFCM, KIFCM, FCM-σ and KFCM-σ. The results show that our proposed algorithms, especially KIFCM-σ are more effective.

In this work the development of an interval type-2 fuzzy clustering algorithm, combining the Fuzzy C-Means (FCM) and Possibilistic C-Means (PCM) clustering algorithms is presented. The process of data clustering is carried out with a fuzzification exponent of m = 2. The development of the interval fuzzy clustering algorithm with a fixed fuzzification exponent (e.g. m = 2), instead of a fuzzification interval [m1, m2] consists of the combination of the FCM and PCM algorithms. This interval fuzzy clustering algorithm is possible because the computation of the used fuzzy partition matrices for each fuzzy clustering algorithm is different. This was proposed to overcome the disadvantages of not properly managing uncertainty in data clustering.

Clustering is a potential tool of data mining. A clustering method analyzes the pattern of a data set and groups the data into several clusters based on the similarity among themselves. Clusters may be either crisp or fuzzy in nature. The present chapter deals with clustering of some data sets using Fuzzy C-Means (FCM) algorithm and Entropy-based Fuzzy Clustering (EFC) algorithm. In FCM algorithm, the nature and quality of clusters depend on the pre-defined number of clusters, level of cluster fuzziness and a threshold value utilized for obtaining the number of outliers (if any). On the other hand, the quality of clusters obtained by the EFC algorithm is dependent on a constant used to establish the relationship between the distance and similarity of two data points, a threshold value of similarity and another threshold value used for determining the number of outliers. The clusters should ideally be distinct and at the same time compact in nature. Moreover, the number of outliers should be as minimum as possible. Thus, the above problem may be posed as an optimization problem, which will be solved using a Genetic Algorithm (GA). The best set of multi-dimensional clusters will be mapped into 2-D for visualization using a Self-Organizing Map (SOM).

A fuzzy minimax clustering model and its applications