Temporal Pattern Recognition in Graph Data Structures

This thesis delves into critical challenges associated with subgraph analysis in large-scale graphs. The proliferation of data in domains such as social networks, Internet applications, and biological networks necessitates a deep understanding of relationships modeled as graphs. The complexity and scale of these graphs pose challenges for efficient extraction of meaningful information. The thesis focuses on three interconnected research problems: subgraph counting in fully-dynamic graph streams, k-clique listing in large general graphs, and maximal clique enumeration in large general graphs. The first research problem addresses subgraph counting in fully-dynamic graph streams, where graphs evolve dynamically with edge insertions and deletions. Traditional ap- proaches sample edges uniformly, overlooking the varying importance of edges. The thesis introduces a novel weighted sampling framework, WSD, for fully-dynamic graph streams, which samples the edges based on their weights that indicate their importance and reflect their properties. It determines the weights of edges in a data-driven fashion, using a novel method based on reinforcement learning. We conduct extensive experi- ments to verify that our technique can produce estimates with smaller errors while often running faster compared with existing algorithms. The second research problem focuses on k-clique listing in large general graphs, a critical task for applications such as community detection and network analysis. The state-of-the-art algorithms all adopt a branch-and-bound (BB) framework with a vertex-oriented branching strategy (called VBBkC), which forms a sub-branch by expanding a partial k-clique with a vertex. These algorithms have the time complexity of O(km(\delta/2)^(k-2)), where m is the number of edges in the graph and \delta is the degeneracy of the graph. In this thesis, we propose a BB framework with a new edge-oriented branching (called EBBkC), which forms a sub-branch by expanding a partial k-clique with two vertices that connect each other (which correspond to an edge). We explore various edge orderings for EBBkC such that it achieves a time complexity of O(m\delta + km(\tau/2)^(k-2), where \tau is an integer related to the maximum truss number of the graph and we have \tau<\delta. The time complexity of EBBkC is better than that of VBBkC algorithms for k>3 since both O(m\delta) and O(km(\tau/2)^(k-2) are bounded by O(km(\delta/2)^(k-2)). Furthermore, we develop specialized algorithms for sub-branches on dense graphs so that we can early-terminate them and apply the specialized algorithms. We conduct extensive experiments on 19 real graphs, and the results show that our newly developed EBBkC based algorithms with the early termination technique consistently and largely outperform the state-of-the- art (VBBkC based) algorithms. The third research problem centers on maximal clique enumeration (MCE) in large general graphs, crucial for tasks like community detection and biological network analysis. Existing algorithms typically adopt the branch-and-bound framework with the vertex-oriented Bron-Kerbosch (BK) branching strategy, which forms the sub-branches by expanding the partial clique with a vertex. The state-of-the-arts have a time complexity no better than O(n\delta 3^(\delta/3)), where n is the number of vertices in the graph and \delta is the degeneracy of the graph. We introduce a novel approach, HBBMC, a hybrid framework combining vertex-oriented and edge-oriented BK branching, where the latter adopts a new branch-and-bound framework which forms the sub-branches by expanding the partial clique with a edge. This hybrid strategy enables more efficient pruning of sub-branches, resulting in a worst-case time complexity of O(m\delta + m\tau 3^(\tau/3)), where \tau is related to the maximum truss number. This theoretical result is better than that of the state-of-the-arts when \delta > \tau + (3/ln3) \rho, a condition satisfied by many real-world graphs, where \rho is the edge density of the graph defined as \rho = m/n. To further enhance efficiency, we introduce an early termination technique applicable to all branch-and-bound frameworks, which leverages the topological information of the graphs within the branches. This technique accelerates the enumeration process without affecting worst-case time complexity. Extensive experiments on real and synthetic graphs demonstrate the superior performance of our approach. The proposed HBBMC algorithm with the early termination technique consistently outperforms existing vertex-oriented algorithms. In summary, this thesis makes significant contributions by introducing novel algo- rithms, frameworks, and methodologies to address subgraph-related problems. The pro- posed approaches show superior performance across various real-world and synthetic datasets, providing efficient solutions for graph analysis in large-scale, dynamic, and complex scenarios.

Next generation analytics will be all about graphs, though performance has been a fundamental challenge for large scale graph processing. In this paper, we present an industrial graph processing engine for exploring various large scale linked data, which exhibits superior performance due to the several innovations. This engine organizes a graph as a set of edge-sets, compatible with the traditional edge-centric sharding for graphs, but becomes more amenable for large scale processing. Each time only a portion of the sets are needed for computation and the data access patterns can be highly predictable for prefetch for many graph computing algorithms. Due to the sparsity of large scale graph structure, this engine differentiates logical edge-sets from the edge-sets physically stored on the disk, where multiple logical edge-sets can be organized into a same physical edge-set to increase the data locality. Besides, in contrast to existing solution, the data structures utilized for the physical edge-sets can vary from one to another. Such heterogeneous edge-set representation explores the best graph processing performance according to local data access patterns. We conduct experiments on a representative set of property graphs on multiple platforms, where the proposed system outperform the baseline systems consistently.

Large-scale graph processing has become a popular research topic in domain graph partitioning and community detection. FastCD is a community-based detection algorithm based on modularity optimization capable of detecting communities on large-scale graphs. Community detection on large graphs requires graph partitioning techniques that partition large-scale graphs into several subgraphs for processing carried out in parallel, so that computational loads can be distributed across machines in the cluster. The research was conducted on graph-parallel distributed framework, GraphX, which is a graph processing component in Spark. The strategies of vertex-cut partitioning method such as RandomVertexCut, CanonicalRandomVertexCut, EdgePartition1D, and EdgePartition2D applied to the FastCD to perform community detection on large scale graphs in parallel. Furthermore, EdgePartition1D has the best performance for FastCD performing parallel community detection on large-scale graphs with the number of edges 7,600,595 and vertices 685,230. The results of this research can be seen that the FastCD algorithm with EdgePartition1D and EdgePartition2D is able to maintain valuable information contained in graphs by considering vertices and neighboring edges quickly.

We present a parallel hierarchical graph clustering algorithm that uses modularity as clustering criteria to effectively extract community structures in large graphs of different types. In order to process a large complex graph (whose vertex number and edge number are around 1 billion), we design our algorithm based on the Louvain method by investigating graph partitioning and distribution schemes on distributed memory architectures and conducting clustering in a divide-and-conquer manner. We study the relationship between graph structure property and clustering quality, carefully deal with ghost vertices between graph partitions, and propose a heuristic partition method suitable for the Louvain method. Compared to the existing solutions, our method can achieve nearly well-balanced workload among processors and higher accuracy of graph clustering on real-world large graph datasets.

The graph model enables a broad range of analysis, thus graph processing is an invaluable tool in data analytics. At the heart of every graph-processing system lies a concurrent graph data structure storing the graph. Such a data structure needs to be highly efficient for both graph algorithms and queries. Due to the continuous evolution, the sparsity, and the scale-free nature of real-world graphs, graph-processing systems face the challenge of providing an appropriate graph data structure that enables both fast analytical workloads and low-memory graph mutations. Existing graph structures offer a hard trade-off between read-only performance, update friendliness, and memory consumption upon updates. In this paper, we introduce CSR++, a new graph data structure that removes these trade-offs and enables both fast read-only analytics and quick and memory-friendly mutations. CSR++ combines ideas from CSR, the fastest read-only data structure, and adjacency lists to achieve the best of both worlds. We compare CSR++ to CSR, adjacency lists from the Boost Graph Library, and LLAMA, a state-of-the-art update-friendly graph structure. In our evaluation, which is based on popular graph-processing algorithms executed over real-world graphs, we show that CSR++ remains close to CSR in read-only concurrent performance (within 10% on average), while significantly outperforming CSR (by an order of magnitude) and LLAMA (by almost 2×) with frequent updates.

The need to efficiently store and query large scale graph datasets is evident in the growing number of data-intensive applications, particularly to maximize the mining of intelligence from these data (e.g., to inform decision making). However, directly releasing graph dataset for analysis may leak sensitive information of an individual even if the graph is anonymized, as demonstrated by the re-identification attacks on the DBpedia datasets. A key challenge in the design of graph sanitization methods is scalability, as existing execution models generally have significant memory requirements. In this paper, we propose a novel k-decomposition algorithm and define a new information loss matrix designed for utility measurement in massively large graph datasets. We also propose a novel privacy preserving framework that can be seamlessly integrated with graph storage, anonymization, query processing, and analysis. Our experimental studies show that the proposed solution achieves privacy-preserving, utility, and efficiency.

Effective keyword search on large scale graphs in a distributed system by Mengyu Li thesis 2013 ix, 56 p. : ill. ; 30 cm Keyword query is more user-friendly than the structured query (SQL, XQuery, SPARQL) as it requires no…Read more ›

Solving the all pairs shortest path (APSP) problem is crucial for graph theory and various practical applications, but remains challenging for large graphs with an enormous number of nodes and edges when the computing resources is limited. In this work, based on an effective graph partition method, we further propose a divisive approach to obtain the APSP for large graphs. A large graph is firstly divided into smaller subgraphs that may not be of an equal size, and then getting the APSP of each subgraph separately. By concatenating the APSPs of the subgraphs, the APSP of the whole graph can be obtained. Experimental results confirm the effectiveness and correctness of our approach for obtaining APSP on large graphs with limited computing resources.

SimRank is an important metric to measure the topological similarity between two nodes in a graph. In particular, single-source and top-k SimRank has numerous applications in recommendation systems, network analysis, and web mining, etc. Mathematically, given a vertex, the computation of single-machine and single-source SimRank mainly lies in matrix-matrix operations. However, it is almost impossible to directly compute on large graphs. Thus, existing works yield to two main operations: a series of random walks, and sparse matrix and dense vector multiplication operations. This brings about high computation cost for SimRank on large graphs. In real-world applications, there is always the query time and accuracy trade-off, which hinders the computation of high-precision SimRank on large-scale graphs. To handle this problem, this paper proposesClipSim, the first GPU-friendly parallel framework that accelerates the single-source SimRank on GPU with accuracy guarantee. We design a novel data structure and GPU-friendly parallel algorithms for efficient computation of all the operations of SimRank on GPU. Moreover, our theoretical derivation enables ClipSim to largely reduce the number of random walks required for each node, while maintaining the same theoretical accuracy as the state-of-the-art algorithm, ExactSim. We conduct extensive experiments on real-world and synthetic datasets to demonstrate the accuracy and efficiency of ClipSim. The results show that compared with ExactSim, ClipSim obtains single-source SimRank vectors with the same accuracy and up to 160× faster computation time.

GraphChi is the first reported disk-based graph engine that can handle billion-scale graphs on a single PC efficiently. GraphChi is able to execute several advanced data mining, graph mining and machine learning algorithms on very large graphs. With the novel technique of parallel sliding windows (PSW) to load subgraph from disk to memory for vertices and edges updating, it can achieve data processing performance close to and even better than those of mainstream distributed graph engines. GraphChi mentioned that its memory is not effectively utilized with large dataset, which leads to suboptimal computation performances. In this paper we are motivated by the concepts of "pin " from TurboGraph and "ghost" from GraphLab to propose a new memory utilization mode for GraphChi, which is called Part-in-memory mode, to improve the GraphChi algorithm performance. The main idea is to pin a fixed part of data inside the memory during the whole computing process. Part-in-memory mode is successfully implemented with only about 40 additional lines of code to the original GraphChi engine. Extensive experiments are performed with large real datasets (including Twitter graph with 1.4 billion edges). The preliminary results show that Part-in-memory mode memory management approach effectively reduces the GraphChi running time by up to 60% in PageRank algorithm. Interestingly it is found that a larger portion of data pinned in memory does not always lead to better performance in the case that the whole dataset cannot be fitted in memory. There exists an optimal portion of data which should be kept in the memory to achieve the best computational performance.

There is growing interest in studying large scale graphs having millions of vertices and billions of edges, up to the point that a specific benchmark, called Graph500, has been defined to measure the performance of graph algorithms on modern computing architectures. At first glance, Graphics Processing Units (GPUs) are not an ideal platform for the execution of graph algorithms that are characterized by low arithmetic intensity and irregular memory access patterns. For studying really large graphs, multiple GPUs are required to overcome the memory size limitations of a single GPU. In the present paper, we propose several techniques to minimize the communication among GPUs.

We developed a parallel algorithm to improve the cache behavior and overall performance for multiplication of sparse matrices with sparse vectors (SpMSpV), an operation used increasingly in large graph analytics, particularly dynamic graphs in social networks and homeland security applications. The proposed algorithm builds upon the two-phase approach of partitioning the multiplication into a scaling phase and an aggregation phase, to achieve more cache-friendly access patterns individually in each phase [6], [3]. However, to handle dynamic graphs and achieve better load balancing for parallel implementation, we use a combination of private and shared bins, with synchronized access to shared bins to exchange the product terms between the two phases. The new algorithm accumulates product terms in private bins for each thread. The algorithm then performs a bulk transfer between a private bin and a shared bin, when the private bin becomes full. Then results are aggregated from the shared bins. In addition, we employ heuristics to decide the best algorithm for SpMSpV based on the number of nonzeros involved in the operation. When the number of nonzeros is large, it may be better to perform the operation as SpMV (sparse matrix times dense vector) despite the added conversion cost. Also, if the number of nonzeros is low it is advantageous to use a simplified algorithm. We compared our algorithm with existing algorithms for SpMSpV, and our evaluation shows that execution time is reduced by several times when large graphs are considered.

Interactive graph applications are often generating irregular access patterns on very large graphs with trillions of edges and billions of vertices. In order to provide short response times for interactive queries, all these small data objects need to be stored in memory. DXRAM is a distributed in-memory system optimized to efficiently manage large amounts of small data objects. In this paper, we present DXGraph, an extension to allow graph processing on DXRAM storage nodes. For a natural graph representation, each vertex is stored as an object. We describe DXGraph's implementation of a breadth-first search (BFS) algorithm, as specified by the Graph500 benchmark. The preliminary evaluation of the BFS algorithm shows that DXGraph's implementation is up to five times faster than Grappa's and GraphLab's with a peak throughput of over 323 million traversed edges per second.

In this paper, we study a new graph learning problem: learning to count subgraph isomorphisms. Different from other traditional graph learning problems such as node classification and link prediction, subgraph isomorphism counting is NP-complete and requires more global inference to oversee the whole graph. To make it scalable for large-scale graphs and patterns, we propose a learning framework that augments different representation learning architectures and iteratively attends pattern and target data graphs to memorize intermediate states of subgraph isomorphism searching for global counting. We develop both small graphs (<= 1,024 subgraph isomorphisms in each) and large graphs (<= 4,096 subgraph isomorphisms in each) sets to evaluate different representation and interaction modules. A mutagenic compound dataset, MUTAG, is also used to evaluate neural models and demonstrate the success of transfer learning. While the learning based approach is inexact, we are able to generalize to count large patterns and data graphs in linear time compared to the exponential time of the original NP-complete problem. Experimental results show that learning based subgraph isomorphism counting can speed up the traditional algorithm, VF2, 10-1,000 times with acceptable errors. Domain adaptation based on fine-tuning also shows the usefulness of our approach in real-world applications.

There is an increasing interest in executing complex analyses over large graphs, many of which require processing a large number of multi-hop neighborhoods or subgraphs. Examples include ego network analysis, motif counting, finding social circles, personalized recommendations, link prediction, anomaly detection, analyzing influence cascades, and others. These tasks are not well served by existing vertex-centric graph processing frameworks, where user programs are only able to directly access the state of a single vertex at a time, resulting in high communication, scheduling, and memory overheads in executing such tasks. Further, most existing graph processing frameworks ignore the challenges in extracting the relevant portions of the graph that an analysis task is interested in, and loading those onto distributed memory. This paper introduces NScale, a novel end-to-end graph processing framework that enables the distributed execution of complex subgraph-centric analytics over large-scale graphs in the cloud. NScale enables users to write programs at the level of subgraphs rather than at the level of vertices. Unlike most previous graph processing frameworks, which apply the user program to the entire graph, NScale allows users to declaratively specify subgraphs of interest. Our framework includes a novel graph extraction and packing (GEP) module that utilizes a cost-based optimizer to partition and pack the subgraphs of interest into memory on as few machines as possible. The distributed execution engine then takes over and runs the user program in parallel on those subgraphs, restricting the scope of the execution appropriately, and utilizes novel techniques to minimize memory consumption by exploiting overlaps among the subgraphs. We present a comprehensive empirical evaluation comparing against three state-of-the-art systems, namely Giraph, GraphLab, and GraphX, on several real-world datasets and a variety of analysis tasks. Our experimental results show orders-of-magnitude improvements in performance and drastic reductions in the cost of analytics compared to vertex-centric approaches.