Genetic algorithms for graph partitioning and incremental graph partitioning

We investigate the problem of partitioning finite difference meshes in two dimensions among the processors of a parallel computer. The objective is to achieve a perfect load balance while minimizing the communication cost. There are well-known graph, hypergraph, and geometry-based partitioning algorithms for this problem. The known geometric algorithms have linear running time and obtain the best results for very special mesh sizes and processor numbers. We propose another geometric algorithm. The proposed algorithm is linear, is applicable to much more cases than some well-known alternatives, obtains better results than the graph partitioning algorithms, obtains better results than the hypergraph partitioning algorithms almost always. Our algorithm also obtains better results than a known asymptotically-optimal algorithm for some small number of processors. We also catalog related theoretical results.

The efficient execution of scientific simulations on HPC systems requires a partitioning of the underlying mesh among the processors such that the load is balanced and the inter-processor communication is minimized. Graph partitioning algorithms have been applied with much success for this purpose. However, the parallelization of multi-phase and multi-physics computations poses new challenges that require fundamental advances in graph partitioning technology. In addition, most existing graph partitioning algorithms are not suited for the newer heterogeneous high-performance computing platforms. This talk will describe research efforts in our group that are focused on developing novel multi-constraint and multi-objective graph partitioning algorithms that can support the advancing state-of-the-art in numerical simulation technologies. In addition, we will present our preliminary work on new partitioning algorithms that are well suited for heterogeneous architectures.

In parallel computing based on finite element analysis, domain decomposition is a key technique for its preprocessing. Generally, a domain decomposition of a mesh can be realized through partitioning of a graph which is converted from a finite element mesh. This paper discusses the method for graph partitioning and the way to actualize mesh partitioning. Relevant softwares are introduced, and the data structure and key functions of Metis and ParMetis are introduced. The writing, compiling, and testing of the mesh partitioning interface program based on these key functions are performed. The results indicate some objective law and characteristics to guide the users who use the graph partitioning algorithm and software to write PFEM program, and ideal partitioning effects can be achieved by actualizing mesh partitioning through the program. The interface program can also be used directly by the engineering researchers as a module of the PFEM software. So that it can reduce the application of the threshold of graph partitioning algorithm, improve the calculation efficiency, and promote the application of graph theory and parallel computing.

An important application of graph partitioning is data clustering using a graph model - the pairwise similarities between all data objects form a weighted graph adjacency matrix that contains all necessary information for clustering. In this paper, we propose a new algorithm for graph partitioning with an objective function that follows the min-max clustering principle. The relaxed version of the optimization of the min-max cut objective function leads to the Fiedler vector in spectral graph partitioning. Theoretical analyses of min-max cut indicate that it leads to balanced partitions, and lower bounds are derived. The min-max cut algorithm is tested on newsgroup data sets and is found to out-perform other current popular partitioning/clustering methods. The linkage-based refinements to the algorithm further improve the quality of clustering substantially. We also demonstrate that a linearized search order based on linkage differential is better than that based on the Fiedler vector, providing another effective partitioning method.

We propose a heuristic for parallel partitioning of graphs into equi-sized components. In particular, we identify a relationship between the graph partitioning problem (GPP) and the traveling saleman problem (TSP), and use that to reduce partitioning to TSP. Given that better performing heuristics are known for TSP than are for GPP, this reduction also leads to improved GPP heuristics. What is more, a good GPP solution can also be used to speed up computation of TSP.We first derive a good bi-partition from a cut of the TSP cycle in time proportional to the number of edges in the graph. We then continue this bi-partitioning recursively until the required number of partitions are left. Further, in order to speed up the computation of TSP, which we use as a subroutine, we perform an initial rough partitioning of the graph into K parts, compute TSP tours in each of these smaller partitions and then merge these local tours to solve the full TSP.We then use this full TSP solution to obtain the final partitioning in parallel. Our empirical analysis shows that for partition count k ≥ 32, our parallel algorithm gives a cut better in size than that of algorithms known for low cut-size (e.g., KaBaPE), and when time is of concern, it finishes in significantly less time with comparable cuts. We also show that our algorithm gives much smaller cuts in comparable time than those known for fast computation (e.g., PT-Scotch).

With ever increasing data volumes, large compute clusters that process data in a distributed manner have become prevalent in industry. For distributed stream processing platforms (such as Storm) the question of how to distribute workload to available machines, has important implications for the overall performance of the system. We present a workload scheduling strategy that is based on a graph partitioning algorithm. The scheduler is application agnostic: it collects the communication behavior of running applications and creates the schedules by partitioning the resulting communication graph using the METIS graph partitioning software. As we build upon graph partitioning algorithms that have been shown to scale to very large graphs, our approach can cope with topologies with millions of tasks. While the experiments in this paper assume static data loads, our approach could also be used in a dynamic setting. We implemented our proposed algorithm for the Storm stream processing system and evaluated it on a commodity cluster with up to 80 machines. The evaluation was conducted on four different use cases - three using synthetic data loads and one application that processes real data. We compared our algorithm against two state-of-the-art scheduler implementations and show that our approach offers significant improvements in terms of resource utilization, enabling higher throughput at reduced network loads. We show that these improvements can be achieved while maintaining a balanced workload in terms of CPU usage and bandwidth consumption across the cluster. We also found that the performance advantage increases with message size, providing an important insight for stream-processing approaches based on micro-batching.

SummaryThis paper presents how parallel advanced graph partitioning algorithms can be designed and improved with the use of global application states monitoring of distributed programs. The proposed algorithms have been implemented inside a novel distributed program design framework Program Execution Governed by Asynchronous Supervision of States in Distributed Applications (PEGASUS DA). This framework provides system support for the design of execution control in distributed applications based on automated global state monitoring. Two strategies for the control design of advanced parallel/distributed graph partitioning algorithms are presented and discussed. In the first one, the parallel algorithm control runs on top of the popular graph partitioning METIS tool. The second control strategy is based on genetic programming so that partitioning primitives and the overall algorithmic control can be freely designed by the user. Advanced partitioning methods have been conveniently embedded inside the global state monitoring driven PEGASUS DA framework which controls partitioning distributed actions at the level of processes and threads. The use of the framework allowed easy design and testing of different graph optimization strategies. The presented graph partitioning methods are illustrated by experiments with benchmark graphs. The experiments have comparatively assessed the obtained graph partitioning quality (visible improvement has been observed) and have identified benefits of the proposed approach for programmers.

The balanced graph partitioning consists in dividing the vertices of an undirected graph into a given number of subsets of approximately equal size, such that the number of edges crossing the subsets is minimized. In this work, we present a multilevel memetic algorithm for this NP-hard problem that relies on a powerful grouping recombination operator and a dedicated local search procedure. The proposed operator tends to preserve the backbone with respect to a set of parent individuals, i.e. the grouping of vertices which is same throughout each parent individual. Although our approach requires significantly longer computing time compared to some current state-of-art graph partitioning algorithms such as SCOTCH, METIS, CHACO, JOSTLE, etc., it competes very favorably with these approaches in terms of solution quality. Moreover, it easily reaches or improves on the best partitions ever reported in the literature.

Clustering algorithms divide data into meaningful or useful groups, called clusters, such that the intra-cluster similarity is maximized and the inter-cluster similarity is minimized.These discovered clusters can be used to explain the characteristics of the underlying data distribution and thus serve as the foundation for various data mining and analysis techniques.The applications of clustering include characterization of different customer groups based upon purchasing patterns, categorization of documents on the World Wide Web, grouping of genes and proteins that have similar functionality, grouping of spatial locations prone to earth quakes from seismological data, etc. What is CLUTOCLUTO is a software package for clustering low and high dimensional datasets and for analyzing the characteristics of the various clusters.CLUTO provides three different classes of clustering algorithms that operate either directly in the object's feature space or in the object's similarity space.These algorithms are based on the partitional, agglomerative, and graphpartitioning paradigms.A key feature in most of CLUTO's clustering algorithms is that they treat the clustering problem as an optimization process which seeks to maximize or minimize a particular clustering criterion function defined either globally or locally over the entire clustering solution space.CLUTO provides a total of seven different criterion functions that can be used to drive both partitional and agglomerative clustering algorithms, that are described and analyzed in [6,5].Most of these criterion functions have been shown to produce high quality clustering solutions in high dimensional datasets, especially those arising in document clustering.In addition to these criterion functions, CLUTO provides some of the more traditional local criteria (e.g., single-link, complete-link, and UPGMA) that can be used in the context of agglomerative clustering.Furthermore, CLUTO provides graph-partitioning-based clustering algorithms that are well-suited for finding clusters that form contiguous regions that span different dimensions of the underlying feature space.An important aspect of partitional-based criterion-driven clustering algorithms is the method used to optimize this criterion function.CLUTO uses a randomized incremental optimization algorithm that is greedy in nature, has low computational requirements, and has been shown to produce high-quality clustering solutions [6].CLUTO's graphpartitioning-based clustering algorithms utilize high-quality and efficient multilevel graph partitioning algorithms derived from the METIS and hMETIS graph and hypergraph partitioning algorithms [4,3].CLUTO also provides tools for analyzing the discovered clusters to understand the relations between the objects assigned to each cluster and the relations between the different clusters, and tools for visualizing the discovered clustering solutions.CLUTO can identify the features that best describe and/or discriminate each cluster.These set of features can be used to gain a better understanding of the set of objects assigned to each cluster and to provide concise summaries about the cluster's contents.Moreover, CLUTO provides visualization capabilities that can be used to see the relationships between the clusters, objects, and features.CLUTO's algorithms have been optimized for operating on very large datasets both in terms of the number of objects as well as the number of dimensions.This is especially true for CLUTO's algorithms for partitional clustering.These algorithms can quickly cluster datasets with several tens of thousands objects and several thousands of dimensions.Moreover, since most high-dimensional datasets are very sparse, CLUTO directly takes into account this sparsity and requires memory that is roughly linear on the input size.CLUTO's distribution consists of both stand-alone programs (vcluster and scluster) for clustering and analyzing these clusters, as well as, a library via which an application program can access directly the various clustering and analysis algorithms implemented in CLUTO. Outline of CLUTO's ManualCLUTO's manual is organized as follows.Section 2 describe the major changes from the previous release.Section 3 describes the stand-alone programs provided by CLUTO, and discusses its various options and analysis capabilities.Section 4 describes the type of clusters that CLUTO's algorithms can find, and discusses their scalability.Section 5 de-

With the rapid expansion of social networks, efficiently mining and analyzing massive graph data has become a fundamental challenge in social network research. Graph partitioning plays a pivotal role in enhancing the performance of such analyses. However, conventional graph partitioning methods predominantly rely on local structural information and often overlook the rich attribute information associated with vertices in social network graphs. To overcome this limitation, this paper introduces GP-DQN (Graph Partitioning via Double Deep Q-Network), a large-scale graph partitioning algorithm that jointly considers structural correlations, attribute disparities among user vertices, and partition load balancing. GP-DQN encodes partition load metrics and vertex attributes into vector representations and employs a Graph Convolutional Network (GCN) to aggregate both vertex features and neighborhood structures, thereby improving the accuracy and scalability of the partitioning process. A tailored reward function is designed to guide partitioning actions, where a Double Deep Q-Network (DDQN) predicts the expected partitioning rewards based on GCN-extracted features for assigning each vertex to different partitions. The partitioning strategy is iteratively optimized using both immediate and expected rewards, ultimately achieving balanced load distribution while minimizing the number of edge cuts. Experimental results demonstrate that GP-DQN produces well-balanced partitions with significantly fewer edge cuts, leading to enhanced computational efficiency within each partition.

Big graph analytics is gaining a widespread momentum across different fields, including biology, computer vision, social networks, recommendation systems and transportation logistics, to mention just a few. Distributed systems for graph analytics are utilized as a mean to process big graphs. To distribute and balance computation and communication loads within a distributed graph analytics system, graph partitioning algorithms can be leveraged. In this paper, we propose Revolver, a machine learning-based graph partitioning algorithm. In particular, Revolver uses reinforcement learning and label propagation to efficiently and effectively carry out the task of graph partitioning. It employs a vertex-centric approach where each vertex in a graph is associated with an autonomous agent responsible for assigning a suitable partition to the vertex. In addition, it uses label propagation to evaluate the decency of partitioning. Evaluation results show that Revolver can produce highly balanced and localized partitions compared to three popular and state-of-the-art graph partitioning algorithms.

Over the last few years, graph partitioning has been recognized as a suitable technique for optimizing cellular network structure. For example, in a recent paper, the authors proposed a classical graph partitioning algorithm to optimize the assignment of cells to Packet Control Units (PCUs) in GSM-EDGE Radio Access Network. Based on this approach, the quality of packet data services in a live environment was increased by reducing the number of cell re-selections between different PCUs. To learn more about the potential of graph partitioning in cellular networks, in this paper, a more sophisticated, yet computationally efficient, partitioning algorithm is proposed for the same problem. The new method combines multi-level refinement and adaptive multi-start techniques with algorithms to ensure the connectivity between cells under the same PCU. Performance assessment is based on an extensive set of graphs constructed with data taken from a live network. During the tests, the new method is compared with classical graph partitioning approaches. Results show that the proposed method outperforms classical approaches in terms of solution quality at the expense of a slight increase in computing time, while providing solutions that are easier to check by the network operator.

Network segmentation has a variety of applications, including computer network security. A well segmented computer network is less likely to result in information leaks and more resilient to adversarial traversal. Conventionally network segmentation approaches rely on graph partitioning algorithms. However, general-purpose graph partitioning solutions are just that, general purpose. These approaches do not exploit specific topological characteristics present in certain classes of networks. Tailored partition methods can be developed to target specific domains, but this process can be time consuming and difficult. This work builds on previous research employing generative hyper-heuristics in the form of genetic programming for automating the development of customized graph partitioning heuristics by incorporating a dynamic approach to controlling the granularity of the heuristic search. The potential of this approach is demonstrated using two real-world complex network applications. Results show that the automated design process is capable of fine tuning graph partitioning heuristics that sacrifice generality for improved performance on targeted networks.

Graph partitioning is one of the most important problems in solving many large-scale scientific computations. It forms the basis of many wide ranging problems in VLSI CAD ranging from high-level applications like hardware-software partitioning on one hand, to physical design methods like min-cut, on the other. In this paper we describe a novel method of using distributed computing in the form of loosely coupled independent processors on a network to solve the k-way graph partitioning problem. We have implemented this system on a network of workstations using PVM (Parallel Virtual Machine). We have tested the method with benchmarks designs of the Partitioning '93 suite with encouraging results.

The graph partitioning problem is that of dividing the vertices of a graph into sets of specified sizes such that few edges cross between sets. This NP-complete problem arises in many important scientific and engineering problems. Prominent examples include the decomposition of data structures for parallel computation, the placement of circuit elements and the ordering of sparse matrix computations. We present a multilevel algorithm for graph partitioning in which the graph is approximated by a sequence of increasingly smaller graphs. The smallest graph is then partitioned using a spectral method, and this partition is propagated back through the hierarchy of graphs. A variant of the Kernighan-Lin algorithm is applied periodically to refine the partition. The entire algorithm can be implemented to execute in time proportional to the size of the original graph. Experiments indicate that, relative to other advanced methods, the multilevel algorithm produces high quality partitions at low cost.