Research of semi-supervised spectral clustering algorithm based on pairwise constraints

In unsupervised clustering tasks, incorporating pairwise constraints as prior information has become an effective approach to improve clustering quality. Building upon the efficient clustering algorithm k-sums, which unifies the frameworks of K-means and Ratio-cut, this paper introduces a novel semi-supervised clustering algorithm by integrating prior pairwise constraints. On one hand, the proposed method inherits the ability of k-sums to handle large-scale data and complex cluster structures with linear time complexity. On the other hand, it enhances clustering discriminability and robustness through constraint-based optimization under semi-supervised settings. Experimental results on multiple benchmark datasets demonstrate that the proposed method significantly outperforms existing mainstream semi-supervised clustering approaches in both clustering accuracy and computational efficiency, showing strong practical value and promising scalability.

During the past few years, semi-supervised clustering has emerged as a new interesting direction in machine learning research. In a semi-supervised clustering algorithm, the clustering results can be significantly improved by using side information, which is available or collected from users. There are two main kinds of side information that can be learned in semi-supervised clustering algorithms: the class labels - called seeds or the pairwise constraints. The first semi-supervised clustering was introduced in 2000, and since that, many algorithms have been presented in literature. However, it is not easy to use both types of side information in the same algorithm. To address the problem, this paper proposes a semi-supervised graph based clustering algorithm that tries to use seeds and constraints in the clustering process, called MCSSGC. Moreover, we introduces a simple but efficient active learning method to collect the constraints that can boost the performance of MCSSGC, named KMMFFQS. In order to verify effectiveness of the proposed algorithm, we conducted a series of experiments not only on real data sets from UCI, but also on a document data set applied in an Information Extraction of Vietnamese documents. These obtained results show that the proposed algorithm can significantly improve the clustering process compared to some recent algorithms.

This paper presents semi-supervised agglomerative hierarchical clustering algorithm using clusterwise tolerance based pairwise constraints. In semi-supervised clustering, pairwise constraints, that is, must-link and cannot-link, are frequently used in order to improve clustering properties. From that sense, we will propose another way named clusterwise tolerance based pairwise constraints to handle must-link and cannot-link constraints inL2-space. In addition, we will propose semi-supervised agglomerative hierarchical clustering algorithm based on it. We will, moreover, show the effectiveness of the proposed method through numerical examples.

Semi-supervised clustering leverages side information such as pairwise constraints to guide clustering procedures. Despite promising progress, existing semi-supervised clustering approaches overlook the condition of side information being generated sequentially, which is a natural setting arising in numerous real-world applications such as social network and e-commerce system analysis. Given emerged new constraints, classical semi-supervised clustering algorithms need to re-optimize their objectives over all data samples and constraints in availability, which prevents them from efficiently updating the obtained data partitions. To address this challenge, we propose an efficient dynamic semi-supervised clustering framework that casts the clustering problem into a search problem over a feasible convex set, i.e., a convex hull with its extreme points being an ensemble of m data partitions. According to the principle of ensemble clustering, the optimal partition lies in the convex hull, and can thus be uniquely represented by an m-dimensional probability simplex vector. As such, the dynamic semi-supervised clustering problem is simplified to the problem of updating a probability simplex vector subject to the newly received pairwise constraints. We then develop a computationally efficient updating procedure to update the probability simplex vector in O(m2) time, irrespective of the data size n. Our empirical studies on several real-world benchmark datasets show that the proposed algorithm outperforms the state-of-the-art semi-supervised clustering algorithms with visible performance gain and significantly reduced running time.

An efficient semi-supervised balanced cut with hard pairwise constraints and partial labels

In the image segmentation, the spectral clustering algorithm are facing the problems, that the similarity matrix and Laplace matrix requires a lot of computing and storage, which limits its expansion in the large-scale image. Aiming at these problems, a semi-supervised spectrum clustering algorithm based on the Nystrom approximation was proposed. In the algorithm, adopting the Nystrom approximation method to estimate the similarity matrix, which effectively reduces the amount of computation and storage of the spectral clustering algorithm. Finally, on the Brodatz texture library, the image segmentation experiments showed the feasibility and effectiveness of the algorithm.

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

Semi-supervised clustering has been widely explored in the last years. In this paper, we present HCAC-ML (Hierarchical Confidence-based Active Clustering with Metric Learning), an innovative approach for this task which employs distance metric learning through cluster-level constraints. HCAC-ML is based on the HCAC algorithm, an state-of-the-art algorithm for hierarchical semi-supervised clustering that uses an active learning approach for inserting cluster-level constraints. These constraints are presented to a variation of ITML (Information-theoretic Metric Learning) algorithm to learn a Mahalanobis-like distance function. We compared HCAC-ML with other semi-supervised clustering algorithms in 26 different datasets. Results indicate that HCAC-ML outperforms other algorithms in most of the scenarios, but specially when the number of constraints is small. This makes HCAC-ML useful in practical applications.

Simulated annealing spectral clustering algorithm for image segmentation

An Improved Semi-Supervised Clustering Algorithm for Multi-Density Datasets with Fewer Constraints

Semi-supervised cluster ensemble usually introduces a small amount of supervision in the first stage of cluster ensemble, i.e., ensemble generation, by performing many runs of semi-supervised clustering algorithms. However, it is neither efficient in terms of computational complexity, nor flexible in a dynamic learning environment where limited supervision changes over time. In this article we propose a new framework which generates base partitions in an unsupervised manner and attributes different weights to each cluster of the base partitions. The weighting scheme considers both the internal validation measures of clustering and the degrees of satisfaction of pairwise constraints. A weighted co-association matrix based consensus approach is then applied to achieve a final partition. To handle high-dimensional data, we generate base partitions using k-means with both random sampling and random subspace techniques. The new framework retains a high accuracy, and is efficient since it avoids performing semi-supervised clustering in ensemble generation and the complexity of the weighting scheme is independent of the number of instances in a dynamic environment. It is more adaptive than the traditional approach because it does not require rerunning semi-supervised clustering algorithms when the limited supervision changes. Empirical results on 12 datasets demonstrate that it is also more robust to noisy constraints.

The multi-model method is used in complex system modeling or industrial data monitoring, and its goal is to establish the sub-models corresponding to different conditions. How to divide the modeling data into datasets corresponding to different conditions in the case of insufficient prior knowledge of the process is an important problem to be solved. Machine learning provides many excellent algorithms for condition identification. However, many algorithms are easily affected by abnormal data, but making full use of prior knowledge can effectively solve this problem. Based on the semi-supervised clustering, mixed constraints which include composite distance and the pairwise constraint are introduced to distinguish strongly and weakly dependent data and clarify boundary data to realize a multi-model condition division of thermal process data. Radial Basis Function neural network is used to realize feature learning, and an online condition identification is constructed. The influence of network structure and parameters on the generalization ability of the recognizer is analyzed. On the premise of not significantly increasing the amount of calculation, the generalization ability is improved by adjusting the weight coefficient in the composite distance, and the weight coefficient under low error rate is given by particle swarm optimization. Compared with classic the methods, such as Sliding Windows, Bottom-Up and Top-Down, the proposed method has better performance in segmentation results.Article HighlightsThe thermal process data in a segment is divided into strongly and weakly dependent data, and different segmentation methods are given.The definition of composite distance parameter, including time component, value component and velocity component, is given to distinguish different types of data.The semi-supervised clustering with mixed constraints, superimposing composite distance constraint and pairwise constraint, is given to segment the thermal process data.

Semi-supervised clustering is a special form of classification that uses a large amount of unlabeled data together with labeled data to achieve better classification results. This paper introduces a semi-supervised fuzzy clustering algorithm of relational data with multiple prototype representation (SS-CLAMP) that aims to furnish a partition and a set of prototypes for each fuzzy cluster as well as to learn a relevance weight for each dissimilarity matrix by optimizing an adequacy criterion that measures the fit between the fuzzy clusters and their representatives in a competitive way and that takes into account pairwise constraints must-link and cannot-link. Experiments with real-valued data sets show the usefulness of the proposed algorithm.

Semi-supervised clustering takes advantage of a small amount of labeled data to bring a great benefit to the clustering of unlabeled data. Based on a locality sensitive k-means clustering method, this paper presents two novel semi-supervised clustering algorithms inspired by the semi-supervised variants of the k-means clustering by seeding. To investigate the effectiveness of our approaches, experiments are done on one artificial dataset and three real datasets. Experimental results show that two proposed methods can improve the clustering performance significantly compared to other unsupervised and semi-supervised clustering algorithms.

A new interactive semi-supervised clustering model for large image database indexing