GraVF: A vertex-centric distributed graph processing framework on FPGAs

The importance of high-performance graph processing to solve big data problems targeting high-impact applications is greater than ever before. Recent graph processing frameworks target different hardware platforms (e.g., shared memory systems, accelerators such as GPUs, and distributed systems) and differ with respect to the programming model they adopt (e.g., based on linear algebra formulations of graph algorithms or enabling direct access to the graph structure). To better understand the impact of these choices, this paper, presents a comparative study of five state-of-the-art graph processing frameworks: two CPU-only frameworks - GraphMat and Galois, two GPU-based frameworks - Nvgraph and Gunrock; and Totem, a hybrid (CPU+GPU) framework. We use three popular graph algorithms (PageRank, Single Source Shortest Path, and Breadth-First Search), and massive scale graphs with up to billions of edges. Our evaluation focuses on three performance metrics: (i) execution time, (ii) scalability and (iii) energy consumption.

On external-memory MST, SSSP and multi-way planar graph separation

Many emerging real-world applications require fast processing of large-scale data represented in the form of graphs. In this paper, we design a Field-Programmable Gate Array (FPGA) framework to accelerate graph algorithms based on the edge-centric paradigm. Our design is flexible for accelerating general graph algorithms with various vertex attributes and update propagation functions, such as Sparse Matrix Vector Multiplication (SpMV), PageRank (PR), Single Source Shortest Path (SSSP), and Weakly Connected Component (WCC). The target platform consists of large external memory to store the graph data and FPGA to accelerate the processing. By taking an edge-centric graph algorithm and hardware resource constraints as inputs, our framework can determine the optimal design parameters and produce an optimized Register-Transfer Level (RTL) FPGA accelerator design. To improve data locality and increase parallelism, we partition the input graph into non-overlapping partitions. This enables our framework to efficiently buffer vertex data in the on-chip memory of FPGA and exploit both inter-partition and intra-partition parallelism. Further, we propose an optimized data layout to improve external memory performance and reduce data communication between FPGA and external memory. Based on our design methodology, we accelerate two fundamental graph algorithms for performance evaluation: Sparse Matrix Vector Multiplication (SpMV) and PageRank (PR). Experimental results show that our accelerators sustain a high throughput of up to 2250 Million Traversed Edges Per Second (MTEPS) and 2487 MTEPS for SpMV and PR, respectively. Compared with several highly-optimized multi-core designs, our FPGA framework achieves up to 20.5× speedup for SpMV, and 17.7× speedup for PR, respectively; compared with two state-of-the-art FPGA frameworks, our designs demonstrate up to 5.3× and 1.8× throughput improvement for SpMV and PR, respectively.

Breadth-first search (BFS) and single source shortest paths (SSSP) are two fundamental graph problems with countless real-world applications. It is of major interest to develop efficient parallel algorithms and implementations to solve these problems on modern multicore architectures. The challenge is that these computations are often irregular, and there is no known algorithm that guarantees polylogarithmic depth for all graphs. For BFS, this paper describe several performance engineering methods to develop a practical parallel implementation for all classes of real-world graphs. For SSSP, we introduce a unique contraction based preprocessing method which significantly speeds up queries on high diameter graphs, like road networks. Our method is generic and can be used with any algorithm for SSSP queries.

With continuously growing data, clusters also need to grow periodically to accommodate the increased demand of data processing. This is usually done by addition of newer hardware, whose configuration might differ from the existing nodes. As a result, clusters are becoming heterogeneous in nature. For many real world machine learning and data mining applications, data is represented in the form of graphs. Most of the existing distributed graph processing frameworks such as Pregel and Graphlab assume that the computational nodes are homogeneous. These frameworks split the graph into approximately equal subgraphs, which is appropriate for homogeneous clusters. In heterogeneous clusters, these frameworks perform poorly in most of the scenarios. To the best of our knowledge, GraphIVE is the only heterogeneity-aware graph processing framework. It learns the relative capabilities of the nodes based on runtime metrics of previous jobs and partitions the graph proportionally. However, it may not perform well if a new job differs drastically in terms of resource requirements when compared to previous jobs executed on the cluster. To overcome this limitation, we propose GraphSteal, a dynamic graph re-partitioning policy for vertex-cut based graph processing frameworks on heterogeneous clusters. GraphSteal dynamically re-partitions the graph based on the runtime characteristics of the job. To avoid computational skew in the cluster, it migrates edges from slow nodes to fast nodes. To demonstrate our approach, we modify the source code of Graphlab to incorporate dynamic graph re-partitioning strategy. Experimental results show that GraphSteal significantly improves the performance over Graphlab.

Thanks to their massive computational power and their SIMT computational model, Graphics Processing Units (GPUs) have been successfully used to accelerate a wide variety of regular applications (linear algebra, stencil computations, image processing and bioinformatics algorithms, among others). However, many established and emerging problems are based on irregular data structures, such as graphs. Examples can be drawn from different application domains: networking, social networking, machine learning, electrical circuit modeling, discrete event simulation, compilers, and computational sciences. It has been shown that irregular applications based on large graphs do exhibit runtime parallelism; moreover, the amount of available parallelism tends to increase with the size of the datasets. In this work, we explore an implementation space for deploying a variety of graph algorithms on GPUs. We show that the dynamic nature of the parallelism that can be extracted from graph algorithms makes it impossible to find an optimal solution. We propose a runtime system able to dynamically transition between different implementations with minimal overhead, and investigate heuristic decisions applicable across algorithms and datasets. Our evaluation is performed on two graph algorithms: breadth-first search and single-source shortest paths. We believe that our proposed mechanisms can be extended and applied to other graph algorithms that exhibit similar computational patterns.

Graphs are used in a wide variety of applicationdomains, from social science to machine learning. Graphalgorithms present large numbers of irregular accesses with littledata reuse to amortize the high cost of memory accesses, requiring high memory bandwidth. Processing in memory (PIM) implemented through 3D die-stacking can deliver this highmemory bandwidth. In a system with multiple memory moduleswith PIM, the in-memory compute logic has low latency and highbandwidth access to its local memory, while accesses to remotememory introduce high latency and energy consumption. Ideally, in such a system, computation and data are partitioned amongthe PIM devices to maximize data locality. But the irregularmemory access patterns present in graph applications make itdifficult to guarantee that the computation in each PIM devicewill only access its local data. A large number of remote memoryaccesses can negate the benefits of using PIM. In this paper, we examine the feasibility and potential of finegrainedwork migration to reduce remote data accesses insystems with multiple PIM devices. First, we propose a datadrivenimplementation of our study algorithms: breadth-firstsearch (BFS), single source shortest path (SSSP) and betweennesscentrality (BC) where each PIM has a queue where the verticesthat it needs to process are held. New vertices that need to beprocessed are enqueued at the PIM device co-located with thememory that stores those vertices. Second, we propose hardwaresupport that takes advantage of PIM to implement highlyefficient queues that improve the performance of the queuingframework by up to 16.7%. Third, we develop a timing model forthe queueing framework to explore the benefits of workmigration vs. remote memory accesses. And, finally, our analysisusing the above framework shows that naive task migration canlead to performance degradations and identifies trade-offsamong data locality, redundant computation, and load balanceamong PIM devices that must be taken into account to realize thepotential benefits of fine-grain task migration.

We present linear-I/O algorithms for fundamental graph problems on embedded outerplanar graphs. We show that breadth-first search, depth-first search, single-source shortest paths, triangulation, and computing an � separator of size O(1/� )t akeO(scan(N)) I/Os on embedded outerplanar graphs. We also show that it takes O(sort(N)) I/Os to test whether a given graph is outerplanar and to compute an outerplanar embedding of an outerplanar graph, thereby providing O(sort(N))-I/O algorithms for the above problems if no embedding of the graph is given. As all these problems have linear-time algorithms in internal memory, a simple simulation technique can be used to improve the I/O-complexity of our algorithms from O(sort(N)) to O(perm(N)). We prove matching lower bounds for embedding, breadth-first search, depth-first search, and singlesource shortest paths if no embedding is given. Our algorithms for the above problems use a simple linear-I/O time-forward processing algorithm for rooted trees whose vertices are stored in preorder.

GraphX is a distributed graph processing framework build on top of Spark Core. This work investigates the two questions, whether GraphX is an appropriate environment for the implementation of graph algorithms and how the computation of graph algorithms based on GraphX scales. This paper examines a graph algorithm for semi-clustering as used in social network analysis. We describe the implementation process of this algorithm beginning with a graph-oriented modeling tailored for GraphX up to an executable program. Based on our implementation, we have performed empirical evaluations regarding the scalability of our implementation and the GraphX platform. The experiments evidence that different kind of graph algorithms are supported by GraphX and that the execution of our algorithm can scale almost linearly when properly designed.

Replicas 1 of a vertex play an important role in existing distributed graph processing systems which make a single vertex to be parallel processed by multiple machines and access remote neighbors locally without any remote access. However, replicas of vertices introduce data coherency problem. Existing distributed graph systems treat replicas of a vertex v as an atomic and indivisible vertex, and use an eager data coherency approach to guarantee replicas atomicity. In eager data coherency approach, any changes to vertex data must be immediately communicated to all replicas of v, thus leading to frequent global synchronizations and communications. In this paper, we propose a lazy data coherency approach, called LazyAsync, which treats replicas of a vertex as independent vertices and maintains the data coherency by computations, rather than communications in existing eager approach. Our approach automatically selects some data coherency points from the graph algorithm, and maintains all replicas to share the same global view only at such points, which means the replicas are enabled to maintain different local views between any two adjacent data coherency points. Based on PowerGraph, we develop a distributed graph processing system LazyGraph to implement the LazyAsync approach and exploit graph-aware optimizations. On a 48-node EC2-like cluster, LazyGraph outperforms PowerGraph on four widely used graph algorithms across a variety of real-world graphs, with a speedup ranging from 1.25x to 10.69x.

Query planning is essential for graph query execution performance. In distributed graph processing, data partitioning and messaging significantly influence performance. However, these aspects are difficult to model analytically, which makes query planning especially challenging. This paper introduces scouting queries, a lightweight mechanism to gather runtime information about different query plans, which can then be used to choose the "best" plan. In a pipelined, depth-first-oriented graph processing engine, scouting queries typically execute for a brief amount of time with negligible overhead. Partial results can be reused to avoid redundant work. We evaluate scouting queries and show that they bring speedups of up to 8.7× for heavy queries, while adding low overhead for those queries that do not benefit.

Graphs are a natural fit for modeling concepts used in solving diverse problems in science, commerce, engineering, and governance. Responding to the variety of graph data and algorithms, many parallel and distributed graph processing systems exist. However, until now these platforms use a static model of deployment: they only run on a pre-defined set of machines. This raises many conceptual and pragmatic issues, including misfit with the highly dynamic nature of graph processing, and could lead to resource waste and high operational costs. In contrast, in this work we explore a dynamic model of deployment. We first characterize workload dynamicity, beyond mere active-vertex variability. Then, to conduct an in-depth elasticity study of distributed graph processing, we build a prototype, JoyGraph, which is the first such system that implements complex, policy-based, and fine-grained elasticity. Using the state-of-the-art LDBC Graphalytics benchmark and the SPEC Cloud Group's elasticity metrics, we show the benefits of elasticity in graph processing: (i) improved resource utilization, (ii) reduced operational costs, and (iii) aligned operation-workload dynamicity. Furthermore, we explore the cost of elasticity in graph processing. We identify a key drawback: although elasticity does not degrade application throughput, graph-processing workloads are sensitive to data movement while leasing or releasing resources.

Graph algorithms are used to implement data mining tasks on graph data-sets. Besides conducting the algorithms by the default deterministic manner, some graph processing frameworks, especially those supporting asynchronous execution model, provide interfaces for the algorithms to be executed in nondeterministic manner, which can improve the scalability and performance of the algorithm's executions. However, is the graph algorithm eligible for nondeterministic execution, and will the execution produce expected results? The literature gives few answers to these questions. In this paper, we study the nondeterministic execution of graph algorithms by considering the scenario where data dependences happen in the edges in graph processing frameworks that employ asynchronous execution model. Our study reveals that only by guaranteeing the atomicity of individual reads and writes, some algorithms (e.g., Graph traversal algorithms) can converge by recovering from corrupted intermediate results with nondeterministic execution, and thus tolerate even write-write conflicts, while some other algorithms (e.g., Fixed point iteration algorithms) can converge but tolerate only read-write conflicts. By conducting graph algorithms on real-world graphs in Graph Chi, and comparing their performances and results with deterministic executions, we find that their performance gains are generally scalable to the available processors with nondeterministic executions, and the results at convergence of fixed point iteration algorithms from nondeterministic executions exhibit larger variances from one run to another than their deterministic executions.

In many global businesses of multinational enterprises, graph-structure data is usually geographically distributed in different regions to support low-latency services. Geo-distributed graph processing suffers from the Wide Area Networks (WANs) with scarce and heterogeneous bandwidth, thus essentially differs from traditional distributed graph processing. In this paper, we propose RAGraph, aRegion-Aware framework for geo-distributed graph processing.At the core of RAGraph, we design a region-aware graph processing framework that allows advancing inefficient global updates locally and enables sensible coordination-free message interactions. RAGraph also contains an adaptive hierarchical message interaction engine to switch interaction modes adaptively based on network heterogeneity and fluctuation, and a discrepancy-aware message filtering strategy to filter important messages. Finally, the experiments show that RAGraph can achieve 1.69X - 40.53X speedup and 20.9% - 97% WAN cost reduction compared with state-of-the-art systems.

With Moore's law in effect, as the complexity of digital electronic circuits increases, the amount of time spent by the electronic design automation (EDA) tools to design such circuits also increases. It brings us to the point, where we need to improve the performance of EDA algorithms to fulfil the present and the future requirements of the EDA industry. Out of many algorithms used by these tools, Breadth First Traversal (BFT) is one of the most commonly used algorithms to traverse the gates of electronic circuits. In this paper, we present a new simple, fast and parallelizable BFT algorithm for sparse graphs, named H-BFT. We show that the CPU implementation of H-BFT is about 75× faster than the CPU implementation of the state of the art, the Sparse Matrix-Vector Product (SMVP) based BFT. Further, with the new features introduced by NVIDIA in their GPUs, we have accelerated both the state of the art SMVP based BFT implementation and our new H-BFT implementation. The best speedups we achieved via these accelerations are 180× and 25× for the SMVP-BFT and H-BFT respectively.