A Novel Semi-supervised Subtractive Clustering Method Using Pairwise Constraints

Semi-supervised clustering can be viewed as a clustering paradigm that exploits both labeled and unlabeled data to steer learning accurate data clusters and avoid local minimum solutions. Nonetheless, the attempts to refine existing semi-supervised clustering methods are relatively limited when compared to the advancements witnessed in the current benchmark methods in fully unsupervised clustering. This research introduces a novel semi-supervised method for deep clustering that leverages deep neural networks and fuzzy memberships to better capture the data partitions. In particular, the proposed Dual-Constraint-based Semi-Supervised Deep Clustering (DC-SSDEC) method utilizes two sets of pairwise soft constraints; "should-link" and "shouldNot-link", to guide the clustering process. The intended clustering task is expressed as an optimization of a newly designed objective function. Additionally, DC-SSDEC performance was evaluated through comprehensive experiments using three real-world and benchmark datasets. Moreover, a comparison with related state-of-the-art clustering techniques was conducted to showcase the DC-SSDEC outperformance. In particular, DC-SSDEC significance consists of the proposed dual-constraint formulation and its integration into a novel objective function. This contribution yielded an improvement in the resulting clustering performance compared to relevant state-of-the-art approaches. In addition, the assessment of the proposed model using real-world datasets represents another contribution of this research. In fact, increases of 3.25%, 1.44%, and 1.82% in the clustering accuracy were gained by DC-SSDEC over the best performing single-constraint-based approach, using MNIST, STL-10, and USPS datasets, respectively.

Semi-supervised clustering methods are applied in various fields. However, these methods have the limitations when the size of data is large. Thus, distributed clustering models are proposed. By separating data set into some parts, distributed models overcome the challenges of big size dataset. In this paper, a novel model, based on semi-supervised fuzzy clustering and distributed mechanism, is proposed. In our research, the method to define distributed additional information is introduced. This additional information is used to implement distributed semi-supervised fuzzy clustering model on three datasets. The main contribution in this paper is the proposal of distributed semi-supervised fuzzy clustering algorithm (denoted as DSSFCM) that performs on Master – Slave model on homogeneous datasets.

A simple clustering method is proposed for extracting representative subsets from lengthy data sets. The main purpose of the extracted subset of data is to use it to build prediction models (of the form of approximating functional relationships) instead of using the entire large data set. Such smaller subsets of data are often required in exploratory analysis stages of studies that involve resource consuming investigations. A few recent studies have used a subtractive clustering method (SCM) for such data extraction, in the absence of clustering methods for function approximation. SCM, however, requires several parameters to be specified. This study proposes a clustering method, which requires only a single parameter to be specified, yet it is shown to be as effective as the SCM. A method to find suitable values for the parameter is also proposed. Due to having only a single parameter, using the proposed clustering method is shown to be orders of magnitudes more efficient than using SCM. The effectiveness of the proposed method is demonstrated on phase space prediction of three univariate time series and prediction of two multivariate data sets. Some drawbacks of SCM when applied for data extraction are identified, and the proposed method is shown to be a solution for them.

Support vector machines combining subtractive clustering method are proposed in this paper. Subtractive clustering method is used to select a set of cluster centers which are the data samples themselves as the representation of original massive set of training data. The new training set then is used to construct support vector machines. Two benchmarks on two-class recognition and multi-class problem are tested, and the results show that the support vector machines based on subtractive clustering have better or equal classification accuracy and generalization ability with smaller set of training data and cost less optimization computation time than conventional support vector machines.

Semisupervised clustering extends standard clustering methods to the semisupervised setting, in some cases considering situations when clusters are associated with a given outcome variable that acts as a “noisy surrogate,” that is a good proxy of the unknown clustering structure. In this article, a novel approach to semisupervised clustering associated with an outcome variable named network‐based semisupervised clustering (NeSSC) is introduced. It combines an initialization, a training and an agglomeration phase. In the initialization and training a matrix of pairwise affinity of the instances is estimated by a classifier. In the agglomeration phase the matrix of pairwise affinity is transformed into a complex network, in which a community detection algorithm searches the underlying community structure. Thus, a partition of the instances into clusters highly homogeneous in terms of the outcome is obtained. We consider a particular specification of NeSSC that uses classification or regression trees as classifiers and the Louvain, Label propagation and Walktrap as possible community detection algorithm. NeSSC's stopping criterion and the choice of the optimal partition of the original data are also discussed. Several applications on both real and simulated data are presented to demonstrate the effectiveness of the proposed semisupervised clustering method and the benefits it provides in terms of improved interpretability of results with respect to three alternative semisupervised clustering methods.

Deformability of rock masses influencing their behavior is an important geomechanical property for the rock structures design. Due to the problems in determining the deformability of jointed rock masses at the laboratory-scale, various in situ test methods such as plate loading tests, dilatometer etc. have been developed. Although these methods are currently the best techniques, they are expensive and time consuming, and present operational problems. Furthermore, the influence of the test volume on modulus of deformation depending on the technique used is also important. For these reasons, in this paper, the adaptive network-based fuzzy inference system (ANFIS) was used to build a prediction model for the indirect estimation of deformation modulus of a rock mass. Three ANFIS models were implemented by grid partitioning (GP), subtractive clustering method (SCM) and fuzzy c-means clustering method (FCM). The estimation abilities offered using three ANFIS models were presented by using field data of achieved from road and railway construction sites in Korea. In these models, rock mass rating (RMR), depth, uniaxial compressive strength of intact rock (UCS) and elastic modulus of intact rock (Ei) were utilized as the input parameters, while the deformation modulus of a rock mass was the output parameter. Various statistical performance indexes were utilized to compare the performance of those estimation models. The results achieved indicate that the ANFIS-SCM model has strong potential to indirect estimation of deformation modulus of a rock mass with high degree of accuracy and robustness.

A kernel-based subtractive clustering method

In this paper, a subtractive clustering identification algorithm is introduced to model type-2 Takagi-Sugeno-Kang (TSK) fuzzy logic systems (FLS). The type-2 TSK FLS identification algorithm is an extension of the type-1 TSK FLS modeling algorithm proposed in (S. L. Chiu, 1994), (S. L. Chiu, 1997). In the type-2 algorithm, subtractive clustering method is combined with least squares estimation algorithms to pre-identify a type-1 FLS form input/output data. Then using type-2 TSK FLS theory (J. M. Mendel, 2001), expand the type-1 FLS to a type-2 TSK FLS. Minimum error models are obtained through enumerative search of optimum values for spreading percentage of cluster centers and consequence parameters. By doing so, fuzzy modeling of type-2 TSK FLS is found to be more effective than that of type-1 TSK FLS. Experimental results confirm the effectiveness of this method. A comparison of the Type-1 and -2 TSK FLSs is presented and the limitations of this method are discussed

Dental X-ray image segmentation has an important role in practical dentistry and is widely used in the discovery of odontological diseases, tooth archeology and in automated dental identification systems. Enhancing the accuracy of dental segmentation is the main focus of researchers, involving various machine learning methods to be applied in order to gain the best performance. However, most of the currently used methods are facing problems of threshold, curve functions, choosing suitable parameters and detecting common boundaries among clusters. In this paper, we will present a new semi-supervised fuzzy clustering algorithm named as SSFC-FS based on Interactive Fuzzy Satisficing for the dental X-ray image segmentation problem. Firstly, features of a dental X-Ray image are modeled into a spatial objective function, which are then to be integrated into a new semi-supervised fuzzy clustering model. Secondly, the Interactive Fuzzy Satisficing method, which is considered as a useful tool to solve linear and nonlinear multi-objective problems in mixed fuzzy-stochastic environment, is applied to get the cluster centers and the membership matrix of the model. Thirdly, theoretically validation of the solutions including the convergence rate, bounds of parameters, and the comparison with solutions of other relevant methods is performed. Lastly, a new semi-supervised fuzzy clustering algorithm that uses an iterative strategy from the formulae of solutions is designed. This new algorithm was experimentally validated and compared with the relevant ones in terms of clustering quality on a real dataset including 56 dental X-ray images in the period 2014–2015 of Hanoi Medial University, Vietnam. The results revealed that the new algorithm has better clustering quality than other methods such as Fuzzy C-Means, Otsu, eSFCM, SSCMOO, FMMBIS and another version of SSFC-FS with the local Lagrange method named SSFC-SC. We also suggest the most appropriate values of parameters for the new algorithm.

This paper presents semi-supervised agglomerative hierarchical clustering algorithm using clusterwise tolerance based pairwise constraints. In semi-supervised clustering, pairwise constraints, that is, must-link and cannot-link, are frequently used in order to improve clustering properties. From that sense, we will propose another way named clusterwise tolerance based pairwise constraints to handle must-link and cannot-link constraints inL2-space. In addition, we will propose semi-supervised agglomerative hierarchical clustering algorithm based on it. We will, moreover, show the effectiveness of the proposed method through numerical examples.

Although semi-supervised classification has widely been studied by many researchers, semi-supervised agglomerative hierarchical clustering is not popular. Two methods to introduce pairwise constraint to agglomerative hierarchical clustering have been proposed so far. The first method is to modify distance between two objects that should be in difference clusters by using a kernel function. The second method is to add a penalty term to similarity measure. In addition, difference of linkage methods of agglomerative hierarchical clustering should be compared to observe how the difference affects the resulting clusters. In this paper, we compare different linkage methods with the above two methods for pairwise constraints. Moreover asymmetric similarity measure is considered. Effects of pairwise constraints are shown by simple examples.

Objective: Churn prediction process on telecom industry is performed using background knowledge such as user information and application oriented constraints with the aim of clustering churn items of telecom communication users. Methods: This paper proposes Semi-supervised Constraint based Churn Clustering (SCCC) method. Semi-supervised learning method identifies different types of churns for labelled data items in telecom industry. PCK Mean based Clusteringwith Expectation Maximization (EM) algorithm finds cluster labels and distance metric for easy grouping of faults set. Similar types of issues are mapped with fast traversal procedure. Constraint based Cluster Membership achieves effective churns clustering by considering relative pairing of users. Findings: The proposed Semi-supervised Constraint based Churn Clustering (SCCC) method is implemented by using JAVA language. This JAVA language uses the code to effectively identify the churns in telecom industry. The SCCC method uses a churn dataset for conducting the experimental work.Experiment is conducted on the factors such as execution time, accuracy, clustering efficiency and support rate on predicting the churns in telecom industry. Experimental results show that the proposed Semi-supervised Constraint based Churn Clustering (SCCC) method out performs than the existing methods. Improvement: Proposed method is able to minimize the execution time, improve accuracy and clustering efficiency and also increase the support rate on predicting churns in telecom industry.Keywords: Cluster Membership, Churn Prediction, Expectation Maximization, Semi-Supervised Clustering

As a variant of nonnegative matrix factorization (NMF), symmetric NMF (SNMF) has shown to be effective for capturing the cluster structure embedded in the graph representation. In contrast to the existing SNMF-based clustering methods that empirically construct the similarity matrix and rigidly introduce the supervisory information to the assignment matrix, in this paper, we propose a novel SNMF-based semisupervised clustering method, namely, pairwise constraint propagation-induced SNMF (PCPSNMF). By formulating a single-constrained optimization problem, PCPSNMF is capable of learning the similarity and assignment matrices adaptively and simultaneously, in which a small amount of supervisory information in the form of pairwise constraints is introduced in a flexible way to guide the construction of the similarity matrix, and the two matrices communicate with each other to achieve mutual refinement until convergence. In addition, we propose an efficient alternating iterative algorithm to solve the optimization problem, whose convergence is theoretically proven. Experimental results over several benchmark image data sets demonstrate that PCPSNMF is less sensitive to initialization and produces higher clustering performance, compared with the state-of-the-art methods.

During the past few years, semi-supervised clustering has emerged as a new interesting direction in machine learning research. In a semi-supervised clustering algorithm, the clustering results can be significantly improved by using side information, which is available or collected from users. There are two main kinds of side information that can be learned in semi-supervised clustering algorithms: the class labels - called seeds or the pairwise constraints. The first semi-supervised clustering was introduced in 2000, and since that, many algorithms have been presented in literature. However, it is not easy to use both types of side information in the same algorithm. To address the problem, this paper proposes a semi-supervised graph based clustering algorithm that tries to use seeds and constraints in the clustering process, called MCSSGC. Moreover, we introduces a simple but efficient active learning method to collect the constraints that can boost the performance of MCSSGC, named KMMFFQS. In order to verify effectiveness of the proposed algorithm, we conducted a series of experiments not only on real data sets from UCI, but also on a document data set applied in an Information Extraction of Vietnamese documents. These obtained results show that the proposed algorithm can significantly improve the clustering process compared to some recent algorithms.

Currently, breast cancer is one of the most common cancers among women. To aid clinicians in diagnosis, lesion regions in mammography pictures can be segmented using an artificial intelligence system. This has significant clinical implications. Clustering algorithms, as unsupervised models, are widely used in medical image segmentation. However, due to the different sizes and shapes of lesions in mammography images and the low contrast between lesion areas and the surrounding pixels, it is difficult to use traditional unsupervised clustering methods for image segmentation. In this study, we try to apply the semisupervised fuzzy clustering algorithm to lesion segmentation in mammography molybdenum target images and propose semisupervised fuzzy clustering based on the cluster centres of labelled samples (called SFCM_V, where V stands for cluster centre). The algorithm refers to the cluster centre of the labelled sample dataset during the clustering process and uses the information of the labelled samples to guide the unlabelled samples during clustering to improve the clustering performance. We compare the SFCM_V algorithm with the current popular semisupervised clustering algorithm and an unsupervised clustering algorithm and perform experiments on real patient mammogram images using DICE and IoU as evaluation metrics; SFCM_V has the highest evaluation metric coefficient. Experiments demonstrate that SFCM_V has higher segmentation accuracy not only for larger lesion regions, such as tumours, but also for smaller lesion regions, such as calcified spots, compared with existing clustering algorithms.