An Improved Semi-Supervised Clustering Algorithm for Multi-Density Datasets with Fewer Constraints

Semi-supervised cluster ensemble usually introduces a small amount of supervision in the first stage of cluster ensemble, i.e., ensemble generation, by performing many runs of semi-supervised clustering algorithms. However, it is neither efficient in terms of computational complexity, nor flexible in a dynamic learning environment where limited supervision changes over time. In this article we propose a new framework which generates base partitions in an unsupervised manner and attributes different weights to each cluster of the base partitions. The weighting scheme considers both the internal validation measures of clustering and the degrees of satisfaction of pairwise constraints. A weighted co-association matrix based consensus approach is then applied to achieve a final partition. To handle high-dimensional data, we generate base partitions using k-means with both random sampling and random subspace techniques. The new framework retains a high accuracy, and is efficient since it avoids performing semi-supervised clustering in ensemble generation and the complexity of the weighting scheme is independent of the number of instances in a dynamic environment. It is more adaptive than the traditional approach because it does not require rerunning semi-supervised clustering algorithms when the limited supervision changes. Empirical results on 12 datasets demonstrate that it is also more robust to noisy constraints.

During the past few years, semi-supervised clustering has emerged as a new interesting direction in machine learning research. In a semi-supervised clustering algorithm, the clustering results can be significantly improved by using side information, which is available or collected from users. There are two main kinds of side information that can be learned in semi-supervised clustering algorithms: the class labels - called seeds or the pairwise constraints. The first semi-supervised clustering was introduced in 2000, and since that, many algorithms have been presented in literature. However, it is not easy to use both types of side information in the same algorithm. To address the problem, this paper proposes a semi-supervised graph based clustering algorithm that tries to use seeds and constraints in the clustering process, called MCSSGC. Moreover, we introduces a simple but efficient active learning method to collect the constraints that can boost the performance of MCSSGC, named KMMFFQS. In order to verify effectiveness of the proposed algorithm, we conducted a series of experiments not only on real data sets from UCI, but also on a document data set applied in an Information Extraction of Vietnamese documents. These obtained results show that the proposed algorithm can significantly improve the clustering process compared to some recent algorithms.

Semi-supervised clustering leverages side information such as pairwise constraints to guide clustering procedures. Despite promising progress, existing semi-supervised clustering approaches overlook the condition of side information being generated sequentially, which is a natural setting arising in numerous real-world applications such as social network and e-commerce system analysis. Given emerged new constraints, classical semi-supervised clustering algorithms need to re-optimize their objectives over all data samples and constraints in availability, which prevents them from efficiently updating the obtained data partitions. To address this challenge, we propose an efficient dynamic semi-supervised clustering framework that casts the clustering problem into a search problem over a feasible convex set, i.e., a convex hull with its extreme points being an ensemble of m data partitions. According to the principle of ensemble clustering, the optimal partition lies in the convex hull, and can thus be uniquely represented by an m-dimensional probability simplex vector. As such, the dynamic semi-supervised clustering problem is simplified to the problem of updating a probability simplex vector subject to the newly received pairwise constraints. We then develop a computationally efficient updating procedure to update the probability simplex vector in O(m2) time, irrespective of the data size n. Our empirical studies on several real-world benchmark datasets show that the proposed algorithm outperforms the state-of-the-art semi-supervised clustering algorithms with visible performance gain and significantly reduced running time.

Clustering is often considered as an unsupervised data analysis method, but making full use of the prior information in the process of clustering will significantly improve the performance of the clustering algorithm. Spectral clustering algorithm can well use the prior pairwise constraint information to cluster and has become a new hot spot of machine learning research in recent years. In this paper, we propose an effective clustering algorithm, called a semi-supervised spectral clustering algorithm based on pairwise constraints, in which the similarity matrix of data points is adjusted and optimized by pairwise constraints. The experiments on real-world data sets demonstrate the effectiveness of this algorithm.

This paper presents semi-supervised agglomerative hierarchical clustering algorithm using clusterwise tolerance based pairwise constraints. In semi-supervised clustering, pairwise constraints, that is, must-link and cannot-link, are frequently used in order to improve clustering properties. From that sense, we will propose another way named clusterwise tolerance based pairwise constraints to handle must-link and cannot-link constraints inL2-space. In addition, we will propose semi-supervised agglomerative hierarchical clustering algorithm based on it. We will, moreover, show the effectiveness of the proposed method through numerical examples.

An efficient semi-supervised balanced cut with hard pairwise constraints and partial labels

In unsupervised clustering tasks, incorporating pairwise constraints as prior information has become an effective approach to improve clustering quality. Building upon the efficient clustering algorithm k-sums, which unifies the frameworks of K-means and Ratio-cut, this paper introduces a novel semi-supervised clustering algorithm by integrating prior pairwise constraints. On one hand, the proposed method inherits the ability of k-sums to handle large-scale data and complex cluster structures with linear time complexity. On the other hand, it enhances clustering discriminability and robustness through constraint-based optimization under semi-supervised settings. Experimental results on multiple benchmark datasets demonstrate that the proposed method significantly outperforms existing mainstream semi-supervised clustering approaches in both clustering accuracy and computational efficiency, showing strong practical value and promising scalability.

The multi-model method is used in complex system modeling or industrial data monitoring, and its goal is to establish the sub-models corresponding to different conditions. How to divide the modeling data into datasets corresponding to different conditions in the case of insufficient prior knowledge of the process is an important problem to be solved. Machine learning provides many excellent algorithms for condition identification. However, many algorithms are easily affected by abnormal data, but making full use of prior knowledge can effectively solve this problem. Based on the semi-supervised clustering, mixed constraints which include composite distance and the pairwise constraint are introduced to distinguish strongly and weakly dependent data and clarify boundary data to realize a multi-model condition division of thermal process data. Radial Basis Function neural network is used to realize feature learning, and an online condition identification is constructed. The influence of network structure and parameters on the generalization ability of the recognizer is analyzed. On the premise of not significantly increasing the amount of calculation, the generalization ability is improved by adjusting the weight coefficient in the composite distance, and the weight coefficient under low error rate is given by particle swarm optimization. Compared with classic the methods, such as Sliding Windows, Bottom-Up and Top-Down, the proposed method has better performance in segmentation results.Article HighlightsThe thermal process data in a segment is divided into strongly and weakly dependent data, and different segmentation methods are given.The definition of composite distance parameter, including time component, value component and velocity component, is given to distinguish different types of data.The semi-supervised clustering with mixed constraints, superimposing composite distance constraint and pairwise constraint, is given to segment the thermal process data.

Dental X-ray image segmentation has an important role in practical dentistry and is widely used in the discovery of odontological diseases, tooth archeology and in automated dental identification systems. Enhancing the accuracy of dental segmentation is the main focus of researchers, involving various machine learning methods to be applied in order to gain the best performance. However, most of the currently used methods are facing problems of threshold, curve functions, choosing suitable parameters and detecting common boundaries among clusters. In this paper, we will present a new semi-supervised fuzzy clustering algorithm named as SSFC-FS based on Interactive Fuzzy Satisficing for the dental X-ray image segmentation problem. Firstly, features of a dental X-Ray image are modeled into a spatial objective function, which are then to be integrated into a new semi-supervised fuzzy clustering model. Secondly, the Interactive Fuzzy Satisficing method, which is considered as a useful tool to solve linear and nonlinear multi-objective problems in mixed fuzzy-stochastic environment, is applied to get the cluster centers and the membership matrix of the model. Thirdly, theoretically validation of the solutions including the convergence rate, bounds of parameters, and the comparison with solutions of other relevant methods is performed. Lastly, a new semi-supervised fuzzy clustering algorithm that uses an iterative strategy from the formulae of solutions is designed. This new algorithm was experimentally validated and compared with the relevant ones in terms of clustering quality on a real dataset including 56 dental X-ray images in the period 2014–2015 of Hanoi Medial University, Vietnam. The results revealed that the new algorithm has better clustering quality than other methods such as Fuzzy C-Means, Otsu, eSFCM, SSCMOO, FMMBIS and another version of SSFC-FS with the local Lagrange method named SSFC-SC. We also suggest the most appropriate values of parameters for the new algorithm.

Semi-supervised clustering has been widely explored in the last years. In this paper, we present HCAC-ML (Hierarchical Confidence-based Active Clustering with Metric Learning), an innovative approach for this task which employs distance metric learning through cluster-level constraints. HCAC-ML is based on the HCAC algorithm, an state-of-the-art algorithm for hierarchical semi-supervised clustering that uses an active learning approach for inserting cluster-level constraints. These constraints are presented to a variation of ITML (Information-theoretic Metric Learning) algorithm to learn a Mahalanobis-like distance function. We compared HCAC-ML with other semi-supervised clustering algorithms in 26 different datasets. Results indicate that HCAC-ML outperforms other algorithms in most of the scenarios, but specially when the number of constraints is small. This makes HCAC-ML useful in practical applications.

Although semi-supervised classification has widely been studied by many researchers, semi-supervised agglomerative hierarchical clustering is not popular. Two methods to introduce pairwise constraint to agglomerative hierarchical clustering have been proposed so far. The first method is to modify distance between two objects that should be in difference clusters by using a kernel function. The second method is to add a penalty term to similarity measure. In addition, difference of linkage methods of agglomerative hierarchical clustering should be compared to observe how the difference affects the resulting clusters. In this paper, we compare different linkage methods with the above two methods for pairwise constraints. Moreover asymmetric similarity measure is considered. Effects of pairwise constraints are shown by simple examples.

Semi-supervised clustering takes advantage of some labeled data called seeds to bring a great benefit to the clustering of unlabeled data. This paper presents a novel semi-supervised clustering method based on one-class support vector machine, which applies a novel decision rule to assigning the class label to one data point. To investigate the effectiveness of our approach, experiments are done on one artificial data set and two real datasets. Experimental results show that the proposed method can improve the clustering performance significantly compared to other semi-supervised clustering algorithms when using a very small amount of seeds.

This paper presents a novel pairwise constraint propagation approach by decomposing the challenging constraint propagation problem into a set of independent semi-supervised learning subproblems which can be solved in quadratic time using label propagation based on k-nearest neighbor graphs. Considering that this time cost is proportional to the number of all possible pairwise constraints, our approach actually provides an efficient solution for exhaustively propagating pairwise constraints throughout the entire dataset. The resulting exhaustive set of propagated pairwise constraints are further used to adjust the similarity matrix for constrained spectral clustering. Other than the traditional constraint propagation on single-source data, our approach is also extended to more challenging constraint propagation on multi-source data where each pairwise constraint is defined over a pair of data points from different sources. This multi-source constraint propagation has an important application to cross-modal multimedia retrieval. Extensive results have shown the superior performance of our approach.

Swarm evolution learning is to design machine learning algorithms based on optimization tools provided by evolutionary algorithms. Evolutionary learning is also called evolutionary algorithms. It is an algorithm cluster inspired by the biological evolution of nature. With the impact of global warming and climate change in recent years, a large number of cold-water fishes have been migrating, and the planning and prediction of migration paths has become a major research hot-spot. In this paper, a semi-supervised clustering neural network is used to build a model to predict the most likely migration direction of fish in Scotland, and a simulated annealing algorithm is used to calculate the maximum profit. This study will have certain implications for the development of fisheries and the allocation and planning of future marine resources reference value.

Semi-supervised clustering takes advantage of a small amount of labeled data to bring a great benefit to the clustering of unlabeled data. Based on a locality sensitive k-means clustering method, this paper presents two novel semi-supervised clustering algorithms inspired by the semi-supervised variants of the k-means clustering by seeding. To investigate the effectiveness of our approaches, experiments are done on one artificial dataset and three real datasets. Experimental results show that two proposed methods can improve the clustering performance significantly compared to other unsupervised and semi-supervised clustering algorithms.