Near-Optimal Algorithms for Instance-Level Constrained k-Center Clustering.

Semi-supervised or constrained clustering refers to clustering data instances in the presence of very limited supervisory information. Although it has been widely investigated in traditional clustering algorithms such as k-means, hierarchical and spectral clustering, little research has addressed the problem of incorporating such knowledge into swarm-intelligence based clustering algorithms. In this study, we present a new Constrained Ant Clustering Algorithm (CACA) with its application to the task of aspect category identification in product reviews, a central clustering task in Aspect-Based Sentiment Analysis (ABSA). We validate our CACA on benchmark datasets, and we show its effectiveness to real-world datasets for ABSA.

Improving clustering with constrained communities

Active Constraint Learning (ACL) is continuously gaining popularity in the area of constrained clustering due to its ability to achieve performance gains via incorporating minimal feedback from a human annotator for selected instances. For constrained clustering algorithms, such instances are integrated in the form of Must-Link (ML) and Cannot-Link (CL) constraints. Existing iterative uncertainty reduction schemes, introduce high computational burden particularly when they process larger datasets that are usually present in computer vision and visual learning applications. For scenarios that multiple agents (i.e., robots) require user feedback for performing recognition tasks, minimizing the interaction between the user and the agents, without compromising performance, is an essential task. In this study, a non-iterative ACL scheme with proven performance benefits is presented. We select to demonstrate the effectiveness of our methodology by building on the well known K-Means algorithm for clustering; one can easily extend it to alternative clustering schemes. The proposed methodology introduces the use of the Silhouette values, conventionally used for measuring clustering performance, in order to rank the degree of information content of the various samples. In addition, an efficient greedy selection scheme was devised for selecting the most informative samples for human annotation. To the best of our knowledge, this is the first active constrained clustering methodology with the ability to process computer vision datasets that this study targets. Performance results are shown on various computer vision benchmarks and support the merits of adopting the proposed scheme.

Recent work has looked at extending clustering algorithms with instance level must-link (ML) and cannot-link (CL) background information. Our work introduces ? and ? cluster level constraints that influence inter-cluster distances and cluster composition. The addition of background information, though useful at providing better clustering results, raises the important feasibility question: Given a collection of constraints and a set of data, does there exist at least one partition of the data set satisfying all the constraints? We study the complexity of the feasibility problem for each of the above constraints separately and also for combinations of constraints. Our results clearly delineate combinations of constraints for which the feasibility problem is computationally intractable (i.e., NP-complete) from those for which the problem is efficiently solvable (i.e., in the computational class P). We also consider the ML and CL constraints in conjunctive and disjunctive normal forms (CNF and DNF respectively). We show that for ML constraints, the feasibility problem is intractable for CNF but efficiently solvable for DNF. Unfortunately, for CL constraints, the feasibility problem is intractable for both CNF and DNF. This effectively means that CL-constraints in a non-trivial form cannot be efficiently incorporated into clustering algorithms. To overcome this, we introduce the notion of a choice-set of constraints and prove that the feasibility problem for choice-sets is efficiently solvable for both ML and CL constraints. We also present empirical results which indicate that the feasibility problem occurs extensively in real world problems.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2010 SIAM International Conference on Data Mining (SDM)A SAT-based Framework for Efficient Constrained ClusteringIan Davidson, S. S. Ravi, and Leonid ShamisIan DavidsonDepartment of Computer Science, University of California – Davis, Davis, CADepartment of Computer Science, University at Albany – Albany, NYDepartment of Computer Science, University of California – Davis, CASearch for more papers by this author, S. S. RaviDepartment of Computer Science, University of California – Davis, Davis, CADepartment of Computer Science, University at Albany – Albany, NYDepartment of Computer Science, University of California – Davis, CASearch for more papers by this author, and Leonid ShamisDepartment of Computer Science, University of California – Davis, Davis, CADepartment of Computer Science, University at Albany – Albany, NYDepartment of Computer Science, University of California – Davis, CASearch for more papers by this authorpp.94 - 105Chapter DOI:https://doi.org/10.1137/1.9781611972801.9PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract The area of clustering under constraints has recently received much attention in the data mining community. However, most work involves adding constraints to existing algorithms which, although being quite pragmatic, raises several difficulties. Examples of these difficulties include creating intractable constraint satisfaction sub-problems and constrained clustering algorithms that are easily over-constrained so they may not converge or converge to a poor clustering solution. In this paper we show how both instance and cluster-level constraints can be expressed as instances of the 2SAT problem and how multiple calls to a 2SAT solver can be used to construct algorithms that are guaranteed to satisfy all the constraints and converge to a global optimum for a number of intuitive objective functions. Our approach provides two additional advantages. Firstly, it leads to polynomial time algorithms for the k = 2 case for several objective functions. Secondly, one can specify large sets of constraints without fear of over-constraining the problem: if one or more solutions satisfying all constraints exist, our algorithm is guaranteed to find a best such solution. We present experimental results to show that our approach outperforms several popular algorithms particularly for large constraint sets, where these algorithms are over-constrained and fair poorly. Previous chapter Next chapter RelatedDetails Published:2010ISBN:978-0-89871-703-7eISBN:978-1-61197-280-1 https://doi.org/10.1137/1.9781611972801Book Series Name:ProceedingsBook Code:PR136Book Pages:1-953

This paper discusses the clustering quality and complexities of the hierarchical data clustering algorithm based on gravity theory. The gravitybased clustering algorithm simulates how the given N nodes in a K-dimensional continuous vector space will cluster due to the gravity force, provided that each node is associated with a mass. One of the main issues studied in this paper is how the order of the distance term in the denominator of the gravity force formula impacts clustering quality. The study reveals that, among the hierarchical clustering algorithms invoked for comparison, only the gravity-based algorithm with a high order of the distance term neither has a bias towards spherical clusters nor suffers the well-known chaining effect. Since bias towards spherical clusters and the chaining effect are two major problems with respect to clustering quality, eliminating both implies that high clustering quality is achieved. As far as time complexity and space complexity are concerned, the gravitybased algorithm enjoys either lower time complexity or lower space complexity, when compared with the most well-known hierarchical data clustering algorithms except single-link.Keywords:data clusteringagglomerative hierarchical clusteringgravity force

Database clustering is a preprocess technology for multi-database mining.Existing algorithms for database clustering are successful in terms of having a cluster, but the time complexity is high or excessive pursuit in respect of the non-trivial complete clustering, which may lead to a bad clustering or application-dependent.The practical application of these algorithms have high time instability and complexity.In this article, we put forward the application-independent database clustering methodology by using hierarchical clustering method to avoid instability and reduce time complexity.This methodology is called Database Hierarchical Clustering.We firstly construct a multi-objective optimization problem, and then use hierarchical clustering algorithm to find the optimal cluster structure thought.We also use the cophenetic correlation coefficient to evaluate the best cluster.Experiments on the synthetic databases and the real-world databases show that our method of clustering stability features lower time complexity than that of the BestClassification while also highlighting strong generalization ability.

Clustering of gene expression data is a standard exploratory technique used to identify closely related genes. Many other sources of data are also likely to be of great assistance in the analysis of gene expression data. This data provides a mean to begin elucidating the large-scale modular organization of the cell. The authors consider the challenging task of developing exploratory analytical techniques to deal with multiple complete and incomplete information sources. The Multi-Source Clustering (MSC) algorithm developed performs clustering with multiple, but complete, sources of data. To deal with incomplete data sources, the authors adopted the MPCK-means clustering algorithms to perform exploratory analysis on one complete source and other potentially incomplete sources provided in the form of constraints. This paper presents a new clustering algorithm MSC to perform exploratory analysis using two or more diverse but complete data sources, studies the effectiveness of constraints sets and robustness of the constrained clustering algorithm using multiple sources of incomplete biological data, and incorporates such incomplete data into constrained clustering algorithm in form of constraints sets.

Clustering is a common problem in the analysis of large data sets. Streaming algorithms, which make a single pass over the data set using small working memory and produce a clustering comparable in cost to the optimal offline solution, are especially useful. We develop the first streaming algorithms achieving a constant-factor approximation to the cluster radius for two variations of the k-center clustering problem. We give a streaming (4 + ε)-approximation algorithm using O(ε − 1 kz) memory for the problem with outliers, in which the clustering is allowed to drop up to z of the input points; previous work used a random sampling approach which yields only a bicriteria approximation. We also give a streaming (6 + ε)-approximation algorithm using O(ε − 1 ln (ε − 1) k + k 2) memory for a variation motivated by anonymity considerations in which each cluster must contain at least a certain number of input points.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2003 SIAM International Conference on Data Mining (SDM)On the Techniques for Data Clustering with Numerical ConstraintsBi-Ru Dai, Cheng-Ru Lin, and Ming-Syan ChenBi-Ru Dai, Cheng-Ru Lin, and Ming-Syan Chenpp.264 - 268Chapter DOI:https://doi.org/10.1137/1.9781611972733.27PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract In this paper, the attributes employed to model the constraints are called constraint attributes and those attributes involved in the objective function to be optimized are called cost-optimal attributes. The constrained clustering considered is conducted in such a way that the objective function of cost-optimal attributes is optimized subject to the condition that the imposed constraint is satisfied. Explicitly, we address the problem of constrained clustering with numerical constraints, in which the constraint attribute values of any two data items in the same cluster are required to be within the corresponding constraint range. We devise an effective and efficient algorithm with complete-link to solve this clustering problem. It is noted that due to the intrinsic nature of the numerical constrained clustering, there is an order dependency on the process of attaining the clustering, which in many cases degrades the clustering results. In view of this, we devise a progressive constraint relaxation technique to remedy this drawback and improve the overall performance of clustering results. Explicitly, by using a smaller (tighter) constraint range in earlier iterations of merge, we will have more room to relax the constraint and seek for better solutions in subsequent iterations. It is empirically shown that the progressive constraint relaxation technique is able to improve not only the execution efficiency but also the clustering quality. Previous chapter Next chapter RelatedDetails Published:2003ISBN:978-0-89871-545-3eISBN:978-1-61197-273-3 https://doi.org/10.1137/1.9781611972733Book Series Name:ProceedingsBook Code:PR112Book Pages:xiv + 347Key words:Data Mining, data clustering, constrained clustering

Typical constrained clustering algorithms incorporate a set of must-link and cannot-link constraints into the clustering process. These instance level constraints specify relationships between pairs of data items and are generally derived by a domain expert. Generating these constraints is considered as a cumbersome and expensive task. In this paper we describe an incremental constrained clustering framework to discover clusters using a decision theoretic approach. Our framework is novel since we provide an overall evaluation of the clustering in terms of quality in decision making and use this evaluation to “generate” instance level constraints. We do not assume any domain knowledge to start with. We show empirical validation of this approach on several test domains and show that we achieve better performance than a feature selection based approach.

The problem of clustering with constraints has received considerable attention in the last decade. Indeed, several algorithms have been proposed, but only a few studies have (partially) compared their performances. In this work, three well-known algo

The grid-based clustering algorithm is an efficient clustering algorithm, but its effect is seriously influenced by the size of the predefined grids and the threshold of the significant cells. The data space will be partitioned into a finite number of cells to form a grid system and then performs all clustering operations on this obtained grid system. A critical problem needed to be noticed here, no one knows the distribution of the clustering data in each dimension. So, a method to reduce the influence of the size of predefined cells and the density threshold of significant cells is a must. To cluster efficiently and simultaneously, to reduce the influences of the size of the cells and inherits the advantage with the low time complexity, we propose “A Crossover-Imaged Clustering Algorithm with Bottom-up Tree Architecture” in this dissertation. The data space is separating into cells which are organized the first grid system, and the data space is transformed into the new grid system. The chosen density threshold and some predefined cells are utilized to identify the significant cells and semi-significant cells. With our clustering algorithms, the size of significant cell and its density threshold are easy to select to group the significant cells into ideal clusters. Next, the data cells are shifted to define the new grid system. Then the semi-significant cells are selected to group the significant cells and other semi-significant cells to be clusters. Then, we build the bottom-up tree architecture with four levels, root (set of clusters), combination (semi-significant cells), connection (pre-clusters), leaf nodes (significant cells), and define the semi-significant cells to group the clusters. Finally we will verify by experiment that the results of our proposed algorithms can reduce the influence of the size of predefined cells and the density threshold of significant cells. And the proposed algorithms still inherit the advantage with the low time complexity and require at most one single scan through the data and one cell clustering.

Optimality conditions and duality in continuous programming II. The linear problem revisited

Redistricting is the process of dividing a geographic area into districts or zones. This process has been considered in the past as a problem that is computationally too complex for an automated system to be developed that can produce unbiased plans. In this paper we present a novel method for redistricting a geographic area using a heuristic-based approach for polygonal spatial clustering. While clustering geospatial polygons several complex issues need to be addressed - such as: removing order dependency, clustering all polygons assuming no outliers, and strategically utilizing domain knowledge to guide the clustering process. In order to address these special needs, we have developed the constrained polygonal spatial clustering (CPSC) algorithm that holistically integrates do-main knowledge in the form of cluster-level and instance-level constraints and uses heuristic functions to grow clusters. In order to illustrate the usefulness of our algorithm we have applied it to the problem of formation of unbiased congressional districts. Furthermore, we compare and contrast our algorithm with two other approaches proposed in the literature for redistricting, namely-graph partitioning and simulated annealing.