Pairwise-Constrained Deep Document Clustering

Adaptive semi-supervised dimensionality reduction with sparse representation using pairwise constraints

Subspace methods have been successfully applied to face recognition tasks. It is well-studied in both unsupervised learning and supervised learning, such as Eigenface and Fisherface. In practice, besides abundant unlabeled examples, domain knowledge in the form of pairwise constraints is commonly available, which specifies whether a pair of instances belong to the same class or different classes. In this study, we propose a face recognition method based on semi-supervised locality preserving learning together with pairwise constraints and unlabeled data, called Semi-supervised Laplacianface (S-Laplacianface). It tries to preserve the local geometric structure of the face manifold as Laplacianface, also requires the subspace to satisfy the pairwise constraints defined by the user. Experimental results on two face databases demonstrate the effectiveness of proposed algorithm.

Due to the demand for performance improvement and the existence of prior information, semi-supervised community detection with pairwise constraints becomes a hot topic. Most existing methods have been successfully encoding the must-link constraints, but neglect the opposite ones, i.e., the cannot-link constraints, which can force the exclusion between nodes. In this paper, we are interested in understanding the role of cannot-link constraints and effectively encoding pairwise constraints. Towards these goals, we define an integral generative process jointly considering the network topology, must-link and cannot-link constraints. We propose to characterize this process as a Multi-variance Mixed Gaussian Generative (MMGG) Model to address diverse degrees of confidences that exist in network topology and pairwise constraints and formulate it as a weighted nonnegative matrix factorization problem. The experiments on artificial and real-world networks not only illustrate the superiority of our proposed MMGG, but also, most importantly, reveal the roles of pairwise constraints. That is, though the must-link is more important than cannot-link when either of them is available, both must-link and cannot-link are equally important when both of them are available. To the best of our knowledge, this is the first work on discovering and exploring the importance of cannot-link constraints in semi-supervised community detection.

Data clustering is an important task in many disciplines. A large number of studies have attempted to improve clustering by using the side information that is often encoded as pairwise constraints. However, these studies focus on designing special clustering algorithms that can effectively exploit the pairwise constraints. We present a boosting framework for data clustering,termed as BoostCluster, that is able to iteratively improve the accuracy of any given clustering algorithm by exploiting the pairwise constraints. The key challenge in designing a boosting framework for data clustering is how to influence an arbitrary clustering algorithm with the side information since clustering algorithms by definition are unsupervised. The proposed framework addresses this problem by dynamically generating new data representations at each iteration that are, on the one hand, adapted to the clustering results at previous iterations by the given algorithm, and on the other hand consistent with the given side information. Our empirical study shows that the proposed boosting framework is effective in improving the performance of a number of popular clustering algorithms (K-means, partitional SingleLink, spectral clustering), and its performance is comparable to the state-of-the-art algorithms for data clustering with side information.

As a variant of nonnegative matrix factorization (NMF), symmetric NMF (SNMF) has shown to be effective for capturing the cluster structure embedded in the graph representation. In contrast to the existing SNMF-based clustering methods that empirically construct the similarity matrix and rigidly introduce the supervisory information to the assignment matrix, in this paper, we propose a novel SNMF-based semisupervised clustering method, namely, pairwise constraint propagation-induced SNMF (PCPSNMF). By formulating a single-constrained optimization problem, PCPSNMF is capable of learning the similarity and assignment matrices adaptively and simultaneously, in which a small amount of supervisory information in the form of pairwise constraints is introduced in a flexible way to guide the construction of the similarity matrix, and the two matrices communicate with each other to achieve mutual refinement until convergence. In addition, we propose an efficient alternating iterative algorithm to solve the optimization problem, whose convergence is theoretically proven. Experimental results over several benchmark image data sets demonstrate that PCPSNMF is less sensitive to initialization and produces higher clustering performance, compared with the state-of-the-art methods.

Recent work has focused the incorporation of a priori knowledge into the data clustering process, in the form of pairwise constraints, aiming to improve clustering quality and find appropriate clustering solutions to specific tasks or interests. In this work, we integrate must-link and cannot-link constraints into the cluster ensemble framework. Two algorithms for combining multiple data partitions with instance level constraints are proposed. The first one consists of a modification to Evidence Accumulation Clustering and the second one maximizes both the similarity between the cluster ensemble and the target consensus partition, and constraint satisfaction using a genetic algorithm. Experimental results shown that the proposed constrained clustering combination methods performances are superior to the unconstrained Evidence Accumulation Clustering.

The maintenance of Case-Based Reasoning (CBR) systems has attracted increasing interest within current research since they proved high-quality results in different real-world domains. This kind of systems stores previous experiences, which are described by a vocabulary (e.g., attributes), incrementally in a case base. Actually, the vocabulary presents one among the most important maintenance targets, since it highly contributes in providing accurate solutions and in improving systems’ performance, especially within high-dimensional domains. However, there is no policy, in the literature, that offers the ability to exploit prior knowledge (e.g., given by domain-experts) during the maintenance of features describing cases. In this paper, we propose a flexible policy for the most relevant attribute selection based on the attribute clustering concept. This new policy is able, on the one hand, to manage uncertainty using the belief function theory based tools, and on the other hand, to make use of domain-experts knowledge in form of pairwise constraints: If two attributes offer the same information without any added-value, then a Must-link constraint between them is generated. Otherwise, if there is no relation between them and they offer different information, then a Cannot-link constraint between them is created.

The research on community structure is a key to analyze the network functionality and topology, and thus it is significant to detect and analysis the community structure. During the abstract process from an actual system to a network, especially for a large-scale network, it is inevitable to have mistaken connections between nodes or have connection missing. In addition, in real applications, from time to time we can obtain prior information in the form of pairwise constraints between nodes besides topology information, although they may be inaccurate or conflicted. These noises in the network-related information will dramatically reduce the accuracy of community detection. Hence, in this paper, we introduce a dissimilarity index to determine the trustworthiness of pairwise constraints and settle the conflict of pairwise constraints. Then, focusing on the community detection with false connections or conflicted connections, we propose a pairwise constrained structure-enhanced extremal optimization-based semi-supervised algorithm (PCSEO-SS algorithm). Compared with existing semi-supervised community detection approaches, the experimental results executed on real networks and synthetic networks, show that PCSEO-SS can solve the problem of false connections or conflicted connections to some extent and detect the community structure more precisely.

We consider the problem of deep semi-supervised classification, where label information is obtained in the form of pairwise constraints. Existing approaches to this problem begin with a clustering network and utilize custom loss functions to encourage the learned representations to conform to the obtained constraints. We present a novel framework that seamlessly integrates pairwise constrained clustering, semi-supervised classification, and supervised classification. This approach leverages advances in unsupervised learning by jointly training a Siamese network and autoencoder to learn a representation that is amenable for both clustering and classification. The resulting framework outperforms existing approaches on common image recognition datasets.

In this paper, we consider a general problem of multiple view learning from pairwise constraints in the form of must-link and cannot-link. As one kind of side information, the must-link constraints imply that a pair of instances belongs to the same class, while the cannot-link constraints compel them to be different classes. Given must-link and cannot-link constraints, we propose a new locality preserving projections approach for multiple view semi-supervised dimensionality reduction. The goal of the proposed approach uses pairwise constraints to derive embedding in each view together with the unlabeled instances. Hence, the consensus pattern can be learned from multiple embeddings of multiple representations. Experimental results on real-world data sets show the effectiveness of the proposed algorithm.

Improving clustering with constrained communities

In this paper, we focus on face clustering in videos. To promote the performance of video clustering by multiple intrinsic cues, i.e., pairwise constraints and multiple views, we propose a constrained multi-view video face clustering method under a unified graph-based model. First, unlike most existing video face clustering methods which only employ these constraints in the clustering step, we strengthen the pairwise constraints through the whole video face clustering framework, both in sparse subspace representation and spectral clustering. In the constrained sparse subspace representation, the sparse representation is forced to explore unknown relationships. In the constrained spectral clustering, the constraints are used to guide for learning more reasonable new representations. Second, our method considers both the video face pairwise constraints as well as the multi-view consistence simultaneously. In particular, the graph regularization enforces the pairwise constraints to be respected and the co-regularization penalizes the disagreement among different graphs of multiple views. Experiments on three real-world video benchmark data sets demonstrate the significant improvements of our method over the state-of-the-art methods.

An efficient semi-supervised balanced cut with hard pairwise constraints and partial labels

Scalable semi-supervised clustering by spectral kernel learning

Feature selection is an important pre-processing stage in many machine learning and pattern recognition tasks, which eliminates irrelevant and redundant features and improves learning performance. Regularized sparse feature selection methods like Lasso and its variants using l 1 -norm regularization term in their optimization problem have received much attention in recent years. Prior information could be represented as the class labels or pairwise constraints, i.e., must-link (positive) and cannot-link (negative) constraints. In this paper, besides the l 1 -norm regularization term, a normalized adapted affinity matrix is applied to embed the pairwise constraints in the affinity matrix. In the proposed affinity matrix, the weights are strengthened/weakened according to the positive/negative constraints. The experimental results on several data sets from University of California-Irvine (UCI) machine learning repository and a high dimensional data set, show the effectiveness of the proposed method in the classification tasks compared to some similar feature selection methods.