A Bi-directional Fuzzy C-Means Clustering Ensemble Algorithm Considering Local Information

目的 随着实际应用场景中海量数据采集技术的发展和数据标注成本的不断增加，自监督学习成为海量数据分析的一个重要策略。然而，如何从海量数据中抽取有用的监督信息，并该监督信息下开展有效的学习仍然是制约该方向发展的研究难点。为此，提出了一个基于共识图学习的自监督集成聚类框架。方法 框架主要包括3个功能模块。首先，利用集成学习中多个基学习器构建共识图；其次，利用图神经网络分析共识图，捕获节点优化表示和节点的聚类结构，并从聚类中挑选高置信度的节点子集及对应的类标签生成监督信息；再次，在此标签监督下，联合其他无标注样本更新集成成员基学习器。交替迭代上述功能块，最终提高无监督聚类的性能。结果 为验证该框架的有效性，在标准数据集（包括图像和文本数据）上设计了一系列实验。实验结果表明，所提方法在性能上一致优于现有聚类方法。尤其是在MNIST-Test （modified national institute of standards and technology database）上，本文方法实现了97.78%的准确率，比已有最佳方法高出3.85%。结论 该方法旨在利用图表示学习提升自监督学习中监督信息捕获的能力，监督信息的有效获取进一步强化了集成学习中成员构建的能力，最终提升了无监督海量数据本质结构的挖掘性能。;Objective Clustering is focused on machine learning-related data segmentation for multiple datasets. Its applications are in relevant to such domains like image segmentation and anomaly detection. In addition，to simplify complex tasks optimize its performance，clustering is used in data preprocessing tasks of those are data sub-blocks segmentation， pseudo-labels generation，and abnormal points-removal. Self-supervised learning has become an essential technique for massive data analysis. However，it is challenged to extract effective supervision information and analyze the input data. Method A consensus graph learning based self-supervised ensemble clustering（CGL-SEC）framework is developed. It consists of three main modules：1）to construct the consensus graph based on several ensemble components（i. e. ，the basic clustering methods）. 2）to extract the supervision information by learning the consensus graph representation，and 3）its node clustering results，where the subset of nodes with the high-confidence are selected as labeled samples. To optimize the ensemble components and the corresponding consensus graph，t basic clustering methods are re-trained in related the option of samples-labeled and other samples-unlabeled. The final clustering results can be optimized iteratively until the learning process converges. Result A series of experiments are carried out on benchmarks，including both image and textual datasets. Especially，CGL-SEC is 3. 85% over baseline in terms of clustering evaluation metric on themodified national institute of standards and technology database（MNIST-Test）. First，to optimize data representation and cluster assignment at the same time，deep embedding clustering can be focused on data itself as the supervision information and auto-encoder with the reconstruction loss is pre-trained. The soft cluster assignment of features-embedded is then calculated，and the KL（Kullback-Leibler）divergence is minimized between the soft cluster assignment and the auxiliary target distribution. To improve the performance of the model further，following deep clustering network（DCN）can use hard clustering instead of soft allocation，and local constraints are applied by improved deep embedding clustering（IDEC）. The pseudo-label strategy is implemented as a self-supervised learning method that uses the prediction results of the neural network as the label to simulate the supervision information compared to using data itself as the supervision information. Deepcluster-based K-means clustering is used to generate pseudo-labels to guide the training of convolutional networks. However，the generated pseudo-labels have lower confidence and are prone to trivial solutions in the initial stage of network training. Deep embedding clustering with data augmentation（DEC-DA）and MixMatch-based prediction of data-enhanced samples are used as the supervision information of the original data，which improves the accuracy of the supervision information to a certain extent，but this method is difficult to extend to text and other fields. Deep adaptive clustering-based high-confidence pseudo-label subsets-selected are iteratively trained the network in the prediction results，but lowconfidence samples-involved data distribution information is ignored. Pseudo-semi-supervised clustering votes are used to select a subset of high-confidence pseudo-labels，and all samples are used to train semi-supervised neural network. Although the ensemble strategy can improve the confidence of the pseudo-label，the voting strategy is concerned of category representation only without the feature representation of the sample itself，which can reduce the clustering performance in some cases. The ensemble learning is regarded as a representative machine learning method that reflects the ability of group intelligence，whereas a learning method can improve the overall prediction performance via multiple base learners training and their coordinated prediction results. In pseudo-label-based clustering tasks，it can coordinate multiple base learners to obtain high-confidence pseudo-labels. However，the effectiveness of the supervision information acquisition is still to be resolved. The category information of the sample is considered for current pseudo-label-based ensemble clustering method only when the label is captured and some effective information are ignored like the feature representation of the sample itself and the clustering structure between samples. Conclusion Graph neural network is composed of content information of nodes and the structural information between nodes at the same time. To design a self-supervised ensemble clustering method based on consensus graph representation learning，it is required to make full use of sample features and relationships between samples in ensemble learning. To obtain higher confidence pseudo-labels as supervised information and improve the performance of self-supervised clustering，it is necessary to mine global and local information at the same time. We illustrate a learnable data ensemble representation through graph neural network. The confidence of pseudo-labels is improved，and the entire model is trained in self-supervision iteratively. To be summarized：1）Commonly-used consensus graph learning-integrated clustering framework is developed，which can use multi-level information like clusteringintegrated sample characteristics and category structure. 2）Self-supervision method is proposed，which uses graph neural network to mine the global and local information of the consensus graph，and high-confidence pseudo-labels are obtained as supervised information. 3）Experiments are demonstrated that the consensus graph learning ensemble clustering method has its potentials on image and text datasets.

Cluster ensemble is a popular method in data mining to discover hidden structures in unlabeled datasets. Some studies have indicated that choosing base clusters with more significant distinction and higher quality can effectively improve the final clustering results. However, the existing base clustering optimization methods consider the diversity and the quality of base clustering separately. Inspired by the three-way decisions, we introduce a three-way optimization model that considers both quality and diversity (3W_CBQD). With two thresholds α and β where α>β, the model calculates the product of the diversity and quality among base clusters as the quality metric and evaluates the quality metric by the thresholds to make decisions. A base cluster will be retained if the quality metric is greater than or equal to the threshold α, removed if the quality metric is less than or equal to the threshold, and recalculated otherwise. This procedure will be repeated until either no base clusters are removed or the number of repeats reaches a specified threshold. Finally, by comparing with some traditional clustering algorithms and clustering ensemble methods, our model shows an effective improvement in the final clustering results with good scalability.

Evaluation measures for cluster ensembles based on a fuzzy generalized Rand index

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT AUTHOR'S REQUEST.] The possibilistic c-means (PCM) was developed as an extension of the fuzzy c-means (FCM) clustering algorithm by abandoning the membership sum-to-one constraint. In the PCM, each cluster is independent of the other clusters and can be processed separately. Because of this separability, the sequential possibilistic one-mean (SP1M) was proposed to find clusters sequentially by running the possibilistic one-mean (P1M) c times. One critical problem in both PCM and SP1M is how to determine the parameter [subscript]. The sequential possibilistic one-mean with adaptive eta (SP1M-AE) was developed to allow [subscript] to change during iterations. In this thesis, a new dynamic adaptation mechanism for the parameter [subscript] in each cluster is inserted into SP1M. The resultant algorithm, called the sequential possibilistic one-mean with dynamic eta (SP1M-DE) is shown to provide superior performance over PCM, SP1M, and SP1M-AE in determining correct clustering results. In this thesis, the family of the SP1M clustering algorithm is also extended to combine local spatial information in an image. The resultant algorithm, called sequential possibilistic local information one-mean (SPLI1M) is shown to have a better performance on image segmentation over the fuzzy local information c-means (FLICM), the possibilistic local information c-means (PLICM), and the family of SP1M without combing local spatial information in image.

In this work, a new image segmentation algorithm is introduced. The proposed algorithm combines the results of an hybrid clustering ensemble. The ensemble clustering is composed of fuzzy c-means (FCM) algorithm and fuzzy local information c-means (FCM_S1 and FCM_S2) algorithms with different values of the neighbors effect. The consensus technique is performed by the ordered weighted averaging (OWA) method. The weight attributed to each classifier can be modified during the process of classification and is determined by the classifier results of the pixel neighbors classification. Dynamic weights give the method the ability to select the temporal best performance during the classification process. Experiments performed on a synthetic image, and real images show that the proposed algorithm is effective and efficient and provides good noise elimination effect.

The damaged areas of brain tissues can be extracted by using segmentation methods, most of which are based on the integration of machine learning and data mining techniques. An important segmentation method is to utilize clustering techniques, especially the fuzzy C-means (FCM) clustering technique, which is sufficiently accurate and not overly sensitive to imaging noise. Therefore, the FCM technique is appropriate for multiple sclerosis diagnosis, although the optimal selection of cluster centers can affect segmentation. They are difficult to select because this is an NP-hard problem. In this study, the Harris Hawks optimization (HHO) algorithm was used for the optimal selection of cluster centers in segmentation and FCM algorithms. The HHO is more accurate than other conventional algorithms such as the genetic algorithm and particle swarm optimization. In the proposed method, every membership matrix is assumed as a hawk or an HHO member. The next step is to generate a population of hawks or membership matrices, the most optimal of which is selected to find the optimal cluster centers to decrease the multiple sclerosis clustering error. According to the tests conducted on a number of brain MRIs, the proposed method outperformed the FCM clustering and other techniques such as the k-NN algorithm, support vector machine, and hybrid data mining methods in accuracy.

In fuzzy C-means (FCM) clustering, each data point belongs to a cluster to a degree specified by a membership grade. FCM partitions a collection of vectors in c fuzzy groups and finds a cluster center in each group such that the dissimilarity measure is minimized. This paper presents a training algorithm for the radial basis function (RBF) network using symmetry-based Fuzzy C-means (SFCM) clustering method which is the modified version of FCM clustering method based on point symmetry distance measure. The training algorithm which uses SFCM clustering method to train the network has a number of advantages such as faster training time, more accurate predictions and reduced network architecture compared to the standard RBF networks. The proposed training algorithm has been implemented in the RBF networks created by the newrb function of MATLAB which uses gradient based iterative method as learning strategy, therefore the new network will undergo a hybrid learning process. The networks called Symmetry-based Fuzzy C-means Clustering–Radial Basis Function Network (SFCM/RBF) has been tested against the standard RBF network and the networks called standard Fuzzy C-means Clustering (FCM)-RBF network (FCM/RBF) in forecasting. The experimental models has been tested on three real world application problems, particularly in Air pollutant problem, Biochemical Oxygen Demand (BOD) problem, and Phytoplankton problem. Keywords: Fuzzy c-means clustering; SFCM; Radial basis function network; point symmetry distance; forecasting.

This article is focused in developing an improved cluster ensemble method based cluster forests. Cluster forests (CF) is considered as a version of clustering inspired from Random Forests (RF) in the context of clustering for massive data. It aggregates intermediate Fuzzy C-Means (FCM) clustering results via spectral clustering since pseudo-clustering results are presented in the spectral space in order to classify these data sets in the multidimensional data space. One of the main advantages is the use of FCM, which allows building fuzzy membership to all partitions of the datasets due to the fuzzy logic whereas the classical algorithms as K-means permitted to build just hard partitions. In the first place, we ameliorate the CF clustering algorithm with the integration of fuzzy FCM and we compare it with other existing clustering methods. In the second place, we compare K-means and FCM clustering methods with the agglomerative hierarchical clustering (HAC) and other theory presented methods using data benchmarks from UCI repository.

Fast and robust fuzzy c-means clustering algorithms incorporating local information for image segmentation

Multiple SVR based on ensemble learning could be enhanced from the viewpoint of the performance, but the performance of modeling closely depends on the initial condition of the partitioning method and they are easily affected by noise and outliers. In this study, a multi-linear fuzzy support vector regression (MFSVR) robust to noise is proposed with the aid of the composite kernel function and $\varepsilon $ -fuzzy c-means (FCM) clustering based on insensitive data information. Here insensitive data information stands for the interval data information of “$\varepsilon $ ” which stands for insensitive loss parameter used in the $\varepsilon $ - insensitive loss function. The objective of this study is to reduce the effect of noise and to alleviate the overfitting problem through the synergistic effect of the following methods: First, $\varepsilon $ -FCM clustering based on insensitive data information is used for considering more impact on decision boundary and reducing the effect of noise. Second, the composite kernel based on multiple linear kernel expression is proposed for implementing multi-linear decision boundary to alleviate overfitting problem. In more detail, each training data point is assigned with corresponding membership degrees in the $\varepsilon $ -FCM clustering. Some data which are potentially to be noise or outlier are assigned with lower membership degrees and given small contribution (compensation) considered in composite kernel function. Then, the composite kernel function for multiple local SVRs is constructed according to the distribution characteristics of $\varepsilon $ -FCM clustering. The proposed MFSVR is tested with both synthetic and UCI data sets in order to verify the effectiveness as well as efficient performance improvement. Experimental results demonstrate that the proposed method shows the better performance when compared to other some methods studied so far.

Magnetotelluric (MT) inversion algorithms are an essential tool in exploration geophysics because they provide us with resistivity models of the subsurface. Inverse problems are ill-posed and non-unique. Resistivity models produced by geologically unconstrained MT inversions are typically smooth, which means that the geological interpretations obtained based on the resistivity models will be ambiguous and uncertain. To solve the above problems, geophysicists have made many efforts over the years to introduce any prior information (such as geological structure information and physical property data) into models, which could help to better constrain the inverted model with the inversion algorithms. In the last few years, there has been renewed interest in machine learning techniques. Various machine learning methods have been applied to inversions. Using the fuzzy c-means (FCM) clustering method, geophysicists proposed new inversion algorithms that are capable of building petrophysical information into the inversion. Although the FCM has been used and advanced in previous work, geophysicists concluded that a priori information of the correct number and value of cluster centers is very important for a successful FCM inversion. In fact, it is difficult to obtain the appropriate clustering information in some geophysical survey areas. In this study, we present an effective way to build the petrophysical information for the MT inversion based on FCM clustering algorithm. When the actual petrophysical information is insufficient, considering the characteristics of MT data, we perform a one-dimensional blocky inversion that can clearly identify the interface with distinct electrical property contrast; then, we obtain the number and value of cluster centers from 1D blocky inversion for the further MT inversion with FCM clustering. The algorithm uses guided FCM clustering to improve the model within the iterative minimization during MT inversion. In our MT inversion method, we integrate the geophysical inversion and geological differentiation into a unified scheme, which interact and enhance each other. The resistivity models obtained from the inversion respect the geophysical and petrophysical data and are easy to geologically interpret. We tested the algorithm using two synthetic examples and a field data example.

From the objective function of a generalized entropy-based fuzzy c-means (FCM) clustering, an algorithm was derived, which is a counterpart of Gaussian mixture models clustering. A drawback of the iterative clustering method is the slow convergence of the algorithm. Miyamoto et al. derived a hard clustering algorithm by defuzzifying the FCM clustering in which covariance matrices were introduced as decision variables. Taking into account this method, for quick and stable convergence of FCM type clustering, we propose the semi-hard clustering approach. The clustering result is used for a classifier and the free parameters of the membership function of fuzzy clusters are selected by particle swarm optimization (PSO). A high classification performance is achieved on a vehicle detection problem for outdoor parking lots.KeywordsParticle Swarm OptimizationMembership FunctionCluster CenterGaussian Mixture ModelMahalanobis DistanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The combined use of self-organizing map technique and fuzzy c-means clustering to evaluate urban groundwater quality in Seoul metropolitan city, South Korea

Region-of-interest extraction in low depth of field images using ensemble clustering and difference of Gaussian approaches

As one of the best image clustering methods, fuzzy local information C-means is often used for image segmentation. The effects of noise are avoided by utilizing the spatial relationship among pixels, but it often generates boundary zones for the mix pixels around the edges. This letter presents an image spatial clustering method, called fuzzy C-means with edge and local information (FELICM), which reduces the edge degradation by introducing the weights of pixels within local neighbor windows. The edges are extracted at first by Canny edge detection. During detection, two adaptive thresholds obtained by multi-Otsu method are used. Then, different weights are set according to whether the window center and the local neighbors are separated by an edge or not. Pixels, together with different weighted local neighbors, are clustered iteratively, until the final clustering result is obtained. The method can be directly applied to the image without any filter preprocessing, and the experimental results over remote sensing images show that FELICM not only effectively solves the problem of isolated and random distribution of pixels inside regions but also obtains high edge accuracies.