Approximation Algorithms for Min-Sum k-Clustering and Balanced k-Median

In this paper, we study the following vehicle routing problem: given n vertices in a metric space, a specified root vertex r (the depot), and a length bound D, find a minimum cardinality set of r-paths that covers all vertices, such that each path has length at most D. This problem is $\mathcal{NP}$-complete, even when the underlying metric is induced by a weighted star. We present a 4-approximation for this problem on tree metrics. On general metrics, we obtain an O(logD) approximation algorithm, and also an $(O(\log \frac{1}{\epsilon}),1+\epsilon)$ bicriteria approximation. All these algorithms have running times that are almost linear in the input size. On instances that have an optimal solution with one r-path, we show how to obtain in polynomial time, a solution using at most 14 r-paths. We also consider a linear relaxation for this problem that can be solved approximately using techniques of Carr & Vempala [7]. We obtain upper bounds on the integrality gap of this relaxation both in tree metrics and in general.

Approximation schemes for Min-Sum [formula omitted]-Clustering

Approximation algorithms for Hamming clustering problems

The nonlinear model predictive control method is applied to design emergency voltage controller. Based on quasi-steady-state approximation, the receding dynamic optimization model is established which subjected to differential-algebraic equations with continuous-discrete time variables. To enhance computational efficiency and control precision of solving this dynamic optimization model, direct dynamic optimization approach combined with moving finite elements are proposed to solve the problem of dynamic optimization. Firstly, the research interval is divided into finite elements. Radau collocation method is used to convert the optimization model into a nonlinear programming by approximating state variable, algebraic variable and control variable profiles by a family of polynomials on each element. Then moving finite elements is introduced to adjust the length of each interval dynamically, achieve the precise positioning of control variable and improve the accuracy of the algorithm. The interior-point algorithm given by AMPL as a mathematical optimization solver is used to solve the problem. Simulation results on New England 10-machine 39-bus system verify effectiveness of the proposed method.

Data clustering is a fundamental problem in the field of machine learning. Among the numerous clustering techniques, matrix factorization-based methods have achieved impressive performances because they are able to provide a compact and interpretable representation of the input data. However, most of the existing works assume that each class has a global centroid, which does not hold for data with complicated structures. Besides, they cannot guarantee that the sample is associated with the nearest centroid. In this work, we present a concept factorization with the local centroids (CFLCs) approach for data clustering. The proposed model has the following advantages: 1) the samples from the same class are allowed to connect with multiple local centroids such that the manifold structure is captured; 2) the pairwise relationship between the samples and centroids is modeled to produce a reasonable label assignment; and 3) the clustering problem is formulated as a bipartite graph partitioning task, and an efficient algorithm is designed for optimization. Experiments on several data sets validate the effectiveness of the CFLC model and demonstrate its superior performance over the state of the arts.

Motivated by the increasing need for fast distributed processing of large-scale graphs such as the Web graph and various social networks, we study a number of fundamental graph problems in the message-passing model, where we have k machines that jointly perform computation on an arbitrary n-node (typically, n > k) input graph. The graph is assumed to be randomly partitioned among the k ≥ 2 machines (a common implementation in many real world systems). The communication is point-to-point, and the goal is to minimize the time complexity i.e., the number of communication rounds, of solving various fundamental graph problems.We present lower bounds that quantify the fundamental time limitations of distributively solving graph problems. We first show a lower bound of Ω(n/k) rounds for computing a spanning tree (ST) of the input graph. This result also implies the same bound for other fundamental problems such as computing a minimum spanning tree (MST), breadth-first tree (BFS), and shortest paths tree (SPT). We also show an Ω(n/k2) lower bound for connectivity ST verification and other related problems. Our lower bounds develop and use new bounds in random-partition communication complexity.To complement our lower bounds, we also give algorithms for various fundamental graph problems, e.g., PageRank, MST, connectivity, ST verification, shortest paths, cuts, spanners, covering problems, densest subgraph, subgraph isomorphism, finding triangles, etc. We show that problems such as PageRank, MST, connectivity, and graph covering can be solved in O(n/k) time (the notation O hides polylog(n) factors and an additive polylog(n) term); this shows that one can achieve almost linear (in k) speedup, whereas for shortest paths, we present algorithms that run in O(n/Ok) time (for (1 + e)-factor approximation) and in O(n/k) time (for O(log n)-factor approximation) respectively.Our results are a step towards understanding the complexity of distributively solving large-scale graph problems.

Completion of tree metrics and rank 2 matrices

We study a version of the graph $$2$$ -clustering problem and the related semi-supervised problem. In these problems, given an undirected graph, we have to find a nearest $$2 $$ -cluster graph, i.e. a graph on the same vertex set with exactly two nonempty connected components each of which is a complete graph. The distance between two graphs is the number of noncoinciding edges. The problems under consideration are NP-hard. In 2008, Coleman, Saunderson, and Wirth presented a polynomial time $$2 $$ -approximation algorithm for the analogous problem in which the number of clusters does not exceed $$2$$ . Unfortunately, the method of proving the performance guarantee of the Coleman, Saunderson, and Wirth algorithm is inappropriate for the graph $$2$$ -clustering problem in which the number of clusters equals $$2$$ . We propose a polynomial time $$2 $$ -approximation algorithm for the $$2 $$ -clustering problem on an arbitrary graph. In contrast to the proof by Coleman, Saunderson, and Wirth, our proof of the performance guarantee of this algorithm does not use switchings. Moreover, we propose an analogous $$2 $$ -approximation algorithm for the related semi-supervised problem.

The Johnson--Lindenstrauss lemma states that n points in a high-dimensional Hilbert space can be embedded with small distortion of the distances into an O (log n ) dimensional space by applying a random linear transformation. We show that similar (though weaker) properties hold for certain random linear transformations over the Hamming cube. We use these transformations to solve NP-hard clustering problems in the cube as well as in geometric settings.More specifically, we address the following clustering problem. Given n points in a larger set (e.g., ℝ d ) endowed with a distance function (e.g., L 2 distance), we would like to partition the data set into k disjoint clusters, each with a "cluster center," so as to minimize the sum over all data points of the distance between the point and the center of the cluster containing the point. The problem is provably NP-hard in some high-dimensional geometric settings, even for k = 2. We give polynomial-time approximation schemes for this problem in several settings, including the binary cube {0,1} d with Hamming distance, and ℝ d either with L 1 distance, or with L 2 distance, or with the square of L 2 distance. In all these settings, the best previous results were constant factor approximation guarantees.We note that our problem is similar in flavor to the k -median problem (and the related facility location problem), which has been considered in graph-theoretic and fixed dimensional geometric settings, where it becomes hard when k is part of the input. In contrast, we study the problem when k is fixed, but the dimension is part of the input.

We study Hamming versions of two classical clustering problems. The Hamming radius p-clustering problem (HRC) for a set S of k binary strings, each of length n, is to find p binary strings of length n that minimize the maximum Hamming distance between a string in S and the closest of the p strings; this minimum value is termed the p-radius of S and is denoted by ϱ The related Hamming diameter p-clustering problem (HDC) is to split S into p groups so that the maximum of the Hamming group diameters is minimized; this latter value is called the p-diameter of S. First, we provide an integer programming formulation of HRC which yields exact solutions in polynomial time whenever k and p are constant. We also observe that HDC admits straightforward polynomial-time solutions when k = O(log n) or p = 2. Next, by reduction from the corresponding geometric p-clustering problems in the plane under the L1 metric, we show that neither HRC nor HDC can be approximated within any constant factor smaller than two unless P=NP. We also prove that for any Ɛ > 0 it is NP-hard to split S into at most pk1/7-Ɛ clusters whose Hamming diameter doesn't exceed the p-diameter. Furthermore, we note that by adapting Gonzalez' farthest-point clustering algorithm [6], HRC and HDC can be approximated within a factor of two in time O(pkn). Next, we describe a 2O(pϱ/Ɛ)kO(p/Ɛ)n2-time (1+ Ɛ)- approximation algorithm for HRC. In particular, it runs in polynomial time when p = O(1) and ϱ = O(log(k+n)): Finally, we show how to find in O((n/Ɛ + kn log n + k2 log n)(2ϱk)2/Ɛ) time a set L of O(p log k) strings of length n such that for each string in S there is at least one string in L within distance (1 + Ɛ)ϱ, for any constant 0 < Ɛ < 1.

A fundamental problem in data clustering is how to determine the correct number of clusters. The k-adaptive medoid set ant colony optimization (ACO) clustering (METACOC-K) algorithm is superior in solving clustering problems. However, METACOC-K does not guarantee in finding the best number of clusters. It assumed the number of clusters based on an adaptive parameter strategy that lacks feedback learning. This has restrained the algorithm in producing compact clusters and the optimal number of clusters. In this paper, a self-adaptive ACO clustering (S-ACOC) algorithm is proposed to produce the optimal number of clusters by incorporating a self-adaptive parameter strategy. The S-ACOC algorithm is a centroid-based algorithm that automatically adjusts the number of clusters during the algorithm run. The selection of the number of clusters is based on a construction graph that reflects the influence of a pheromone in algorithm learning. Experiments were conducted on real-world datasets to evaluate the performance of the proposed algorithm. The external evaluation metrics (purity, F-measure, and entropy) were used to compare the results of the proposed algorithm with other swarm clustering algorithms, including a genetic algorithm (GA), particle swarm optimization (PSO), and METACOC-K. Results showed that S-ACOC provides higher purity (50%) and lower entropy (40%) than GA, PSO, and METACOC-K. Experiments were also performed on several predefined clusters, and results demonstrate that the S-ACOC algorithm is superior to GA, PSO, and METACOC-K. Based on the superior performance, S-ACOC can be used to solve clustering problems in various application domains.

Motivated by the increasing need for fast distributed processing of large-scale graphs such as the Web graph and various social networks, we study a message-passing distributed computing model for graph processing and present lower bounds and algorithms for several graph problems. This work is inspired by recent large-scale graph processing systems (e.g., Pregel and Giraph) which are designed based on the message-passing model of distributed computing. Our model consists of a point-to-point communication network of $k$ machines interconnected by bandwidth-restricted links. Communicating data between the machines is the costly operation (as opposed to local computation). The network is used to process an arbitrary $n$-node input graph (typically $n \gg k > 1$) that is randomly partitioned among the $k$ machines (a common implementation in many real world systems). Our goal is to study fundamental complexity bounds for solving graph problems in this model. We present techniques for obtaining lower bounds on the distributed time complexity. Our lower bounds develop and use new bounds in random-partition communication complexity. We first show a lower bound of $\Omega(n/k)$ rounds for computing a spanning tree (ST) of the input graph. This result also implies the same bound for other fundamental problems such as computing a minimum spanning tree (MST). We also show an $\Omega(n/k^2)$ lower bound for connectivity, ST verification and other related problems. We give algorithms for various fundamental graph problems in our model. We show that problems such as PageRank, MST, connectivity, and graph covering can be solved in $\tilde{O}(n/k)$ time, whereas for shortest paths, we present algorithms that run in $\tilde{O}(n/\sqrt{k})$ time (for $(1+\epsilon)$-factor approx.) and in $\tilde{O}(n/k)$ time (for $O(\log n)$-factor approx.) respectively.

In the era of data-driven decision-making, the Gaussian Mixture Model (GMM) stands as a cornerstone in statistical modeling, particularly in clustering and density estimation. The Improved GMM presents a robust solution to a fundamental problem in clustering: the determination of the optimal number of clusters. Unlike its predecessor, it does not rely on a predetermined cluster count but employs model selection criteria, such as the Bayesian Information Criterion (BIC) or Akaike Information Criterion (AIC), to automatically identify the most suitable cluster count for the given data. This inherent adaptability is a hallmark of the Improved GMM, making it a versatile tool in a broad spectrum of applications, from market segmentation to image processing. Furthermore, the Improved GMM revolutionizes parameter estimation and model fitting. It leverages advanced optimization techniques, such as the Expectation-Maximization (EM) algorithm or variational inference, to achieve convergence to more favorable local optima. This results in precise and reliable parameter estimates, including cluster means, covariances, and component weights. The Improved GMM is particularly invaluable when dealing with data of varying complexities, non-standard data distributions, and clusters with differing shapes and orientations. It excels at capturing the nuanced relationships within the data, providing a powerful framework for understanding complex systems. One of the key differentiators of the Improved GMM is its accommodation of full covariance matrices for each component. This feature empowers the model to account for intricate interdependencies between variables, which is essential for modeling real-world data effectively. It is capable of handling data that exhibits non-spherical or irregular cluster shapes, a significant limitation of the traditional GMM.

Given a set of points P ⊂ Rd and value ∊ > 0, an ∊-core-set S ⊂ P has the property that the smallest ball containing S is an ∊-approximation of the smallest ball containing P. This paper shows that any point-set has an ∊-core-set of size [2/∊]. We also give a fast algorithm that finds this core-set. These results imply the existence of small core-sets for solving approximate k-center clustering and related problems. The sizes of these core-sets are considerably smaller than the previously known bounds, and imply faster algorithms; one such algorithm needs O(dn/∊ + (l/∊)5) time to compute an ∊-approximate minimum enclosing ball (1-center) of n points in d dimensions. A simple gradient-descent algorithm is also given, for computing the minimum enclosing ball in O(dn/∊2) time. This algorithm also implies slightly faster algorithms for computing approximately the smallest radius k-flat fitting a set of points.

Nowadays, along with the development of the malware writing techniques, the diversity and amount of malware variants are constantly increasing and proliferation of these malware has posed major threats to the network computer users. However, variants of malware families always share typical characteristics which reflect their origin and purpose. Therefore, categorizing malware to different families is one of the computer security topics that are of great interest. A fundamental and difficult problem for malware clustering is the determination of the “true” number of malware families on real data collection. To address this issue, in this paper, resting on the analysis of the extracted instruction of malware samples, we propose an improved clustering validity index VNFS for determining the number of malware clusters based on hierarchical clustering algorithm for malware categorization. V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NFS</inf> is defined based on the concepts of the compactness within each cluster and distances of the clusters representatives. By using a novel function proposed in this paper for total separation between clusters, V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">NFS</inf> can deal with different density and irregular data sets more robustly. A comprehensive experimental study on real daily collections of PE malware samples is performed to compare various clustering validation measure methods, such as RS-RMSSTD, PST2, V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FS</inf> and V <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">XB</inf> . Promising experimental results demonstrate that our proposed clustering validity index VNFS always leads to better cluster number than other clustering validity indices for hierarchical clustering and the number of malware clusters it recommends is almost the same as the virus analysts suggest.