An Efficient Semi-Supervised Clustering Algorithm with Sequential Constraints

During the past few years, semi-supervised clustering has emerged as a new interesting direction in machine learning research. In a semi-supervised clustering algorithm, the clustering results can be significantly improved by using side information, which is available or collected from users. There are two main kinds of side information that can be learned in semi-supervised clustering algorithms: the class labels - called seeds or the pairwise constraints. The first semi-supervised clustering was introduced in 2000, and since that, many algorithms have been presented in literature. However, it is not easy to use both types of side information in the same algorithm. To address the problem, this paper proposes a semi-supervised graph based clustering algorithm that tries to use seeds and constraints in the clustering process, called MCSSGC. Moreover, we introduces a simple but efficient active learning method to collect the constraints that can boost the performance of MCSSGC, named KMMFFQS. In order to verify effectiveness of the proposed algorithm, we conducted a series of experiments not only on real data sets from UCI, but also on a document data set applied in an Information Extraction of Vietnamese documents. These obtained results show that the proposed algorithm can significantly improve the clustering process compared to some recent algorithms.

This paper presents semi-supervised agglomerative hierarchical clustering algorithm using clusterwise tolerance based pairwise constraints. In semi-supervised clustering, pairwise constraints, that is, must-link and cannot-link, are frequently used in order to improve clustering properties. From that sense, we will propose another way named clusterwise tolerance based pairwise constraints to handle must-link and cannot-link constraints inL2-space. In addition, we will propose semi-supervised agglomerative hierarchical clustering algorithm based on it. We will, moreover, show the effectiveness of the proposed method through numerical examples.

Clustering is often considered as an unsupervised data analysis method, but making full use of the prior information in the process of clustering will significantly improve the performance of the clustering algorithm. Spectral clustering algorithm can well use the prior pairwise constraint information to cluster and has become a new hot spot of machine learning research in recent years. In this paper, we propose an effective clustering algorithm, called a semi-supervised spectral clustering algorithm based on pairwise constraints, in which the similarity matrix of data points is adjusted and optimized by pairwise constraints. The experiments on real-world data sets demonstrate the effectiveness of this algorithm.

In unsupervised clustering tasks, incorporating pairwise constraints as prior information has become an effective approach to improve clustering quality. Building upon the efficient clustering algorithm k-sums, which unifies the frameworks of K-means and Ratio-cut, this paper introduces a novel semi-supervised clustering algorithm by integrating prior pairwise constraints. On one hand, the proposed method inherits the ability of k-sums to handle large-scale data and complex cluster structures with linear time complexity. On the other hand, it enhances clustering discriminability and robustness through constraint-based optimization under semi-supervised settings. Experimental results on multiple benchmark datasets demonstrate that the proposed method significantly outperforms existing mainstream semi-supervised clustering approaches in both clustering accuracy and computational efficiency, showing strong practical value and promising scalability.

Semi-supervised clustering has been widely explored in the last years. In this paper, we present HCAC-ML (Hierarchical Confidence-based Active Clustering with Metric Learning), an innovative approach for this task which employs distance metric learning through cluster-level constraints. HCAC-ML is based on the HCAC algorithm, an state-of-the-art algorithm for hierarchical semi-supervised clustering that uses an active learning approach for inserting cluster-level constraints. These constraints are presented to a variation of ITML (Information-theoretic Metric Learning) algorithm to learn a Mahalanobis-like distance function. We compared HCAC-ML with other semi-supervised clustering algorithms in 26 different datasets. Results indicate that HCAC-ML outperforms other algorithms in most of the scenarios, but specially when the number of constraints is small. This makes HCAC-ML useful in practical applications.

Semi-supervised cluster ensemble usually introduces a small amount of supervision in the first stage of cluster ensemble, i.e., ensemble generation, by performing many runs of semi-supervised clustering algorithms. However, it is neither efficient in terms of computational complexity, nor flexible in a dynamic learning environment where limited supervision changes over time. In this article we propose a new framework which generates base partitions in an unsupervised manner and attributes different weights to each cluster of the base partitions. The weighting scheme considers both the internal validation measures of clustering and the degrees of satisfaction of pairwise constraints. A weighted co-association matrix based consensus approach is then applied to achieve a final partition. To handle high-dimensional data, we generate base partitions using k-means with both random sampling and random subspace techniques. The new framework retains a high accuracy, and is efficient since it avoids performing semi-supervised clustering in ensemble generation and the complexity of the weighting scheme is independent of the number of instances in a dynamic environment. It is more adaptive than the traditional approach because it does not require rerunning semi-supervised clustering algorithms when the limited supervision changes. Empirical results on 12 datasets demonstrate that it is also more robust to noisy constraints.

Dental X-ray image segmentation is a necessary and important process in medical diagnosis, which assists clinicians to make decisions about possible dental diseases of a patient from a dental X-ray image. It is a multi-objective optimization problem which involves basic components of fuzzy clustering, spatial structures of a dental image, and additional information of experts expressed through a pre-defined membership matrix. In our previous work, the authors presented a semi-supervised fuzzy clustering algorithm using interactive fuzzy satisficing named as SSFC-FS for this problem. An important issue of SSFC-FS is that the pre-defined membership matrix is a fixed function in the sense that it uses the same structure and parameters for all dental images. This is a shortcoming of SSFC-FS since each image has its own structure and morphology so that it needs different membership matrices. In this paper, the authors propose another new dynamic semi-supervised fuzzy clustering called SSFC-FSAI that extends SSFC-FS by employing a collection of pre-defined membership matrices for dental images. A procedure to choose a suitable pre-defined membership matrix for a given dental X-ray image is proposed and attached to SSFC-FSAI. Experimental results on a real dataset of 56 dental X-ray images from Hanoi University of Medical in 2014 - 2015 show that SSFC-FSAI has better accuracy than SSFC-FS and the relevant algorithms.

Dental X-ray image segmentation has an important role in practical dentistry and is widely used in the discovery of odontological diseases, tooth archeology and in automated dental identification systems. Enhancing the accuracy of dental segmentation is the main focus of researchers, involving various machine learning methods to be applied in order to gain the best performance. However, most of the currently used methods are facing problems of threshold, curve functions, choosing suitable parameters and detecting common boundaries among clusters. In this paper, we will present a new semi-supervised fuzzy clustering algorithm named as SSFC-FS based on Interactive Fuzzy Satisficing for the dental X-ray image segmentation problem. Firstly, features of a dental X-Ray image are modeled into a spatial objective function, which are then to be integrated into a new semi-supervised fuzzy clustering model. Secondly, the Interactive Fuzzy Satisficing method, which is considered as a useful tool to solve linear and nonlinear multi-objective problems in mixed fuzzy-stochastic environment, is applied to get the cluster centers and the membership matrix of the model. Thirdly, theoretically validation of the solutions including the convergence rate, bounds of parameters, and the comparison with solutions of other relevant methods is performed. Lastly, a new semi-supervised fuzzy clustering algorithm that uses an iterative strategy from the formulae of solutions is designed. This new algorithm was experimentally validated and compared with the relevant ones in terms of clustering quality on a real dataset including 56 dental X-ray images in the period 2014–2015 of Hanoi Medial University, Vietnam. The results revealed that the new algorithm has better clustering quality than other methods such as Fuzzy C-Means, Otsu, eSFCM, SSCMOO, FMMBIS and another version of SSFC-FS with the local Lagrange method named SSFC-SC. We also suggest the most appropriate values of parameters for the new algorithm.

Semi-supervised clustering methods are applied in various fields. However, these methods have the limitations when the size of data is large. Thus, distributed clustering models are proposed. By separating data set into some parts, distributed models overcome the challenges of big size dataset. In this paper, a novel model, based on semi-supervised fuzzy clustering and distributed mechanism, is proposed. In our research, the method to define distributed additional information is introduced. This additional information is used to implement distributed semi-supervised fuzzy clustering model on three datasets. The main contribution in this paper is the proposal of distributed semi-supervised fuzzy clustering algorithm (denoted as DSSFCM) that performs on Master – Slave model on homogeneous datasets.

Semi-supervised clustering takes advantage of some labeled data called seeds to bring a great benefit to the clustering of unlabeled data. This paper presents a novel semi-supervised clustering method based on one-class support vector machine, which applies a novel decision rule to assigning the class label to one data point. To investigate the effectiveness of our approach, experiments are done on one artificial data set and two real datasets. Experimental results show that the proposed method can improve the clustering performance significantly compared to other semi-supervised clustering algorithms when using a very small amount of seeds.

Semi-supervised fuzzy clustering with metric learning and entropy regularization

An efficient semi-supervised balanced cut with hard pairwise constraints and partial labels

The multi-model method is used in complex system modeling or industrial data monitoring, and its goal is to establish the sub-models corresponding to different conditions. How to divide the modeling data into datasets corresponding to different conditions in the case of insufficient prior knowledge of the process is an important problem to be solved. Machine learning provides many excellent algorithms for condition identification. However, many algorithms are easily affected by abnormal data, but making full use of prior knowledge can effectively solve this problem. Based on the semi-supervised clustering, mixed constraints which include composite distance and the pairwise constraint are introduced to distinguish strongly and weakly dependent data and clarify boundary data to realize a multi-model condition division of thermal process data. Radial Basis Function neural network is used to realize feature learning, and an online condition identification is constructed. The influence of network structure and parameters on the generalization ability of the recognizer is analyzed. On the premise of not significantly increasing the amount of calculation, the generalization ability is improved by adjusting the weight coefficient in the composite distance, and the weight coefficient under low error rate is given by particle swarm optimization. Compared with classic the methods, such as Sliding Windows, Bottom-Up and Top-Down, the proposed method has better performance in segmentation results.Article HighlightsThe thermal process data in a segment is divided into strongly and weakly dependent data, and different segmentation methods are given.The definition of composite distance parameter, including time component, value component and velocity component, is given to distinguish different types of data.The semi-supervised clustering with mixed constraints, superimposing composite distance constraint and pairwise constraint, is given to segment the thermal process data.

Semi-supervised clustering ensemble has emerged as an important elaboration of classical clustering problem that improves quality and robustness in clustering by combining the results of different clustering components with user provided constraints. In this paper, we propose a novel semi-supervised consensus clustering algorithm based on multi-ant colonies. Our method incorporates pairwise constraints not only in each ant colony clustering process, but also in computing new similarity matrix during the multi-ant colonies ensemble. Experimental results demonstrate the effectiveness of the proposed method.

Swarm evolution learning is to design machine learning algorithms based on optimization tools provided by evolutionary algorithms. Evolutionary learning is also called evolutionary algorithms. It is an algorithm cluster inspired by the biological evolution of nature. With the impact of global warming and climate change in recent years, a large number of cold-water fishes have been migrating, and the planning and prediction of migration paths has become a major research hot-spot. In this paper, a semi-supervised clustering neural network is used to build a model to predict the most likely migration direction of fish in Scotland, and a simulated annealing algorithm is used to calculate the maximum profit. This study will have certain implications for the development of fisheries and the allocation and planning of future marine resources reference value.