Efficient triangle listing for billion-scale graphs

In this paper, a learning model for prediction is introduced by coupling a static neural network with an external stack memory, creating a new type of recurrent system. We analyze the differences between this external memory recurrent network and recurrent neural network, which possesses internal memory. Internal memory remembers the last state while external memory remembers past useful contents. For a specific automaton, the internal memory is needed if the last state is a variable of the state transition function and the external memory is needed if the past content is a variable of state transition function. Our arguments are verified by comparing the prediction accuracy of different models with internal memory, external memory and the combination of them on counting and reversing tasks. The results shows that: network with an external stack works best for counting tasks since the variables of state transition function is composed of the current input and one past input, while network with the combination of internal and external memory works best for the reversing task since the variables of state transition function is composed of the current input, current state and the past inputs.

Chapter 8 - Memory and Sound

Due to the limited capacity of GPU memory, the majority of prior work on graph applications on GPUs has been restricted to graphs of modest sizes that fit in memory. Recent hardware and software advances make it possible to address much larger host memory transparently as a part of a feature known as unified virtual memory. While accessing host memory over an interconnect is understandably slower, the problem space has not been sufficiently explored in the context of a challenging workload with low computational intensity and an irregular data access pattern such as graph traversal. We analyse the performance of breadth first search (BFS) for several large graphs in the context of unified memory and identify the key factors that contribute to slowdowns. Next, we propose a lightweight offline graph reordering algorithm, HALO (Harmonic Locality Ordering), that can be used as a pre-processing step for static graphs. HALO yields speedups of 1.5x-1.9x over baseline in subsequent traversals. Our method specifically aims to cover large directed real world graphs in addition to undirected graphs whereas prior methods only account for the latter. Additionally, we demonstrate ties between the locality ordering problem and graph compression and show that prior methods from graph compression such as recursive graph bisection can be suitably adapted to this problem.

This thesis delves into critical challenges associated with subgraph analysis in large-scale graphs. The proliferation of data in domains such as social networks, Internet applications, and biological networks necessitates a deep understanding of relationships modeled as graphs. The complexity and scale of these graphs pose challenges for efficient extraction of meaningful information. The thesis focuses on three interconnected research problems: subgraph counting in fully-dynamic graph streams, k-clique listing in large general graphs, and maximal clique enumeration in large general graphs. The first research problem addresses subgraph counting in fully-dynamic graph streams, where graphs evolve dynamically with edge insertions and deletions. Traditional ap- proaches sample edges uniformly, overlooking the varying importance of edges. The thesis introduces a novel weighted sampling framework, WSD, for fully-dynamic graph streams, which samples the edges based on their weights that indicate their importance and reflect their properties. It determines the weights of edges in a data-driven fashion, using a novel method based on reinforcement learning. We conduct extensive experi- ments to verify that our technique can produce estimates with smaller errors while often running faster compared with existing algorithms. The second research problem focuses on k-clique listing in large general graphs, a critical task for applications such as community detection and network analysis. The state-of-the-art algorithms all adopt a branch-and-bound (BB) framework with a vertex-oriented branching strategy (called VBBkC), which forms a sub-branch by expanding a partial k-clique with a vertex. These algorithms have the time complexity of O(km(\delta/2)^(k-2)), where m is the number of edges in the graph and \delta is the degeneracy of the graph. In this thesis, we propose a BB framework with a new edge-oriented branching (called EBBkC), which forms a sub-branch by expanding a partial k-clique with two vertices that connect each other (which correspond to an edge). We explore various edge orderings for EBBkC such that it achieves a time complexity of O(m\delta + km(\tau/2)^(k-2), where \tau is an integer related to the maximum truss number of the graph and we have \tau<\delta. The time complexity of EBBkC is better than that of VBBkC algorithms for k>3 since both O(m\delta) and O(km(\tau/2)^(k-2) are bounded by O(km(\delta/2)^(k-2)). Furthermore, we develop specialized algorithms for sub-branches on dense graphs so that we can early-terminate them and apply the specialized algorithms. We conduct extensive experiments on 19 real graphs, and the results show that our newly developed EBBkC based algorithms with the early termination technique consistently and largely outperform the state-of-the- art (VBBkC based) algorithms. The third research problem centers on maximal clique enumeration (MCE) in large general graphs, crucial for tasks like community detection and biological network analysis. Existing algorithms typically adopt the branch-and-bound framework with the vertex-oriented Bron-Kerbosch (BK) branching strategy, which forms the sub-branches by expanding the partial clique with a vertex. The state-of-the-arts have a time complexity no better than O(n\delta 3^(\delta/3)), where n is the number of vertices in the graph and \delta is the degeneracy of the graph. We introduce a novel approach, HBBMC, a hybrid framework combining vertex-oriented and edge-oriented BK branching, where the latter adopts a new branch-and-bound framework which forms the sub-branches by expanding the partial clique with a edge. This hybrid strategy enables more efficient pruning of sub-branches, resulting in a worst-case time complexity of O(m\delta + m\tau 3^(\tau/3)), where \tau is related to the maximum truss number. This theoretical result is better than that of the state-of-the-arts when \delta > \tau + (3/ln3) \rho, a condition satisfied by many real-world graphs, where \rho is the edge density of the graph defined as \rho = m/n. To further enhance efficiency, we introduce an early termination technique applicable to all branch-and-bound frameworks, which leverages the topological information of the graphs within the branches. This technique accelerates the enumeration process without affecting worst-case time complexity. Extensive experiments on real and synthetic graphs demonstrate the superior performance of our approach. The proposed HBBMC algorithm with the early termination technique consistently outperforms existing vertex-oriented algorithms. In summary, this thesis makes significant contributions by introducing novel algo- rithms, frameworks, and methodologies to address subgraph-related problems. The pro- posed approaches show superior performance across various real-world and synthetic datasets, providing efficient solutions for graph analysis in large-scale, dynamic, and complex scenarios.

In this chapter we begin reporting the results of our comprehensive survey study on the interplay of internal and external memory in everyday life, and how that is changing in the early twenty-first century (N = 476 Mechanical Turk participants). After first describing data cleaning and scoring, we report the descriptive statistics for questions about attitudes and beliefs regarding internal memory, external memory, and their interplay. The overall pattern, based on participants’ self-reports, is that most think external memory enhances their everyday functioning, but they also think they would do fine without it, and they do not feel uneasy about their reliance on it. External memory is seen as augmenting human capability, not supplanting it. Furthermore, participants generally put considerable value in their external memories.

The paper implements a shift based diamond search architecture, for HDTV videos. High resolution frames like 720p need large memory to store the pixel data. This large pixel data is stored on an external memory and an internal local memory is updated with the pixel data on the reference frame and current frame required for particular diamond search iterations. This is done as internal memory is expensive. However, accessing pixel data from external memory incurs high timing latency. This paper implements a shift based DS architecture to shift data in internal local memory and update from external memory only the pixels which are new to search area. This reduces the redundant operation of accessing same pixel data for successive searches. Even when considering equal latency for internal and external memory access, when simulation was performed on the same sample 720p frame, architectures with no shift block took 173.170ms vs with shift block took 90.564ms. When considering the latency of external memory access this difference in timing becomes largely significant.

We introduce new dynamic set intersection data structures, which we call 2-3 cuckoo filters and hash tables. These structures differ from the standard cuckoo hash tables and cuckoo filters in that they choose two out of three locations to store each item, instead of one out of two, ensuring that any item in an intersection of two structures will have at least one common location in both structures. We demonstrate the utility of these structures by using them in improved algorithms for listing triangles and answering set intersection queries in internal or external memory. For a graph G of n vertices and m edges, our internal-memory triangle listing algorithm runs in O(m⌈(α(G)log w)/w⌉ + k) expected time, where α(G) is the arboricity of G, w is the number of bits in a machine word, and k is the number of output triangles. Our external-memory algorithm uses O(sort(n,α(G))+ sort(m⌈(α(G)log w)/w⌉) + sort(k)) expected number of I/Os.

Graph data structures model relations between entities in various domains. Graph processing systems enable scalable distributed computations over large graphs, but are limited to static scenarios in which the structure of the graph does not change. However, many applications are dynamic in nature, and this reflects to graphs that continuously evolve over time. In these contexts, understanding the evolution of graphs is key to enable timely reactions when necessary. We address this problem by proposing a new model to express temporal patterns over graph data structures. The model seamlessly integrates computations over graphs to extract relevant values, and temporal operators that define patterns of interest in the evolution of the graph. We present the syntax and semantics of our model and discuss its concrete implementation in FlowGraph, a middleware for temporal pattern recognition in large scale graphs. FlowGraph presents a level of performance that is comparable to state-of-the-art graph processing tools when processing static graphs. In the presence of temporal patterns, it can further optimize processing by avoiding complex graph computations until strictly necessary for pattern evaluation.

Distributed triangle enumerating has attracted a lot of interest for the capability to process large graphs quickly. However, existing solutions usually suffer from low speed and/or poor graph-size scalability due to the massive short messages passed via the network or the huge amount of intermediate data. To address the issues, we propose HOSA, a distributed out-of-core triangle enumerating algorithm. HOSA first introduces an I/O-efficient graph placement strategy to divide and place a graph across a cluster in a load-balanced manner to reduce intermediate data and exploit the aggregated I/O bandwidth of the cluster. To enumerate triangles, HOSA proposes an efficient algorithm which not only eliminates short messages, the main cause of the low speed of most previous works, but also effectively uses the aggregated network and I/O bandwidths to further improve performance, and achieves high speed and graph-size scalability. Extensive evaluations show that HOSA can process much larger graphs with better or at least comparable speed than state-of-the-art algorithms, including the best distributed-memory algorithm LiteTE.

We present a parallel hierarchical graph clustering algorithm that uses modularity as clustering criteria to effectively extract community structures in large graphs of different types. In order to process a large complex graph (whose vertex number and edge number are around 1 billion), we design our algorithm based on the Louvain method by investigating graph partitioning and distribution schemes on distributed memory architectures and conducting clustering in a divide-and-conquer manner. We study the relationship between graph structure property and clustering quality, carefully deal with ghost vertices between graph partitions, and propose a heuristic partition method suitable for the Louvain method. Compared to the existing solutions, our method can achieve nearly well-balanced workload among processors and higher accuracy of graph clustering on real-world large graph datasets.

This paper investigates the power, energy, and performance characteristics of large-scale graph processing on hybrid (i.e., CPU and GPU) single-node systems. Graph processing can be accelerated on hybrid systems by properly mapping the graph-layout to processing units, such that the algorithmic tasks exercise each of the units where they perform best. However, the GPUs have much higher Thermal Design Power (TDP), thus their impact on the overall energy consumption is unclear. Our evaluation using large real-world graphs and synthetic graphs as large as 1 billion vertices and 16 billion edges shows that a hybrid system is efficient in terms of both time-to-solution and energy.

In this chapter we introduce the topic of internal memory versus external memory. Internal memory is information stored in one’s brain, and external memory is information stored outside of one’s brain (either socially or technologically). Humans have a long history of using external memory, and we interact extensively with it in our everyday lives. However, research on this topic has been limited and fragmentary. Here we describe the method of our comprehensive survey study on the interplay of internal and external memory in everyday life, and how that is changing in the early twenty-first century (N = 476 Mechanical Turk participants). The survey included quantitative and qualitative questions about attitudes, beliefs, behaviors, and experiences.

Large-scale graph processing has become a popular research topic in domain graph partitioning and community detection. FastCD is a community-based detection algorithm based on modularity optimization capable of detecting communities on large-scale graphs. Community detection on large graphs requires graph partitioning techniques that partition large-scale graphs into several subgraphs for processing carried out in parallel, so that computational loads can be distributed across machines in the cluster. The research was conducted on graph-parallel distributed framework, GraphX, which is a graph processing component in Spark. The strategies of vertex-cut partitioning method such as RandomVertexCut, CanonicalRandomVertexCut, EdgePartition1D, and EdgePartition2D applied to the FastCD to perform community detection on large scale graphs in parallel. Furthermore, EdgePartition1D has the best performance for FastCD performing parallel community detection on large-scale graphs with the number of edges 7,600,595 and vertices 685,230. The results of this research can be seen that the FastCD algorithm with EdgePartition1D and EdgePartition2D is able to maintain valuable information contained in graphs by considering vertices and neighboring edges quickly.

Effective keyword search on large scale graphs in a distributed system by Mengyu Li thesis 2013 ix, 56 p. : ill. ; 30 cm Keyword query is more user-friendly than the structured query (SQL, XQuery, SPARQL) as it requires no…Read more ›

Breadth first search (BFS) traversal on massive graphs in external memory was considered non-viable until recently, because of the large number of I/Os it incurs. Ajwani et al. [3] showed that the randomized variant of the o(n) I/O algorithm of Mehlhorn and Meyer [24] (MM_BFS) can compute the BFS level decomposition for large graphs (around a billion edges) in a few hours for small diameter graphs and a few days for large diameter graphs. We improve upon their implementation of this algorithm by reducing the overhead associated with each BFS level, thereby improving the results for large diameter graphs which are more difficult for BFS traversal in external memory. Also, we present the implementation of the deterministic variant of MM_BFS and show that in most cases, it outperforms the randomized variant. The running time for BFS traversal is further improved with a heuristic that preserves the worst case guarantees of MM_BFS. Together, they reduce the time for BFS on large diameter graphs from days shown in [3] to hours. In particular, on line graphs with random layout on disks, our implementation of the deterministic variant of MM_BFS with the proposed heuristic is more than 75 times faster than the previous best result for the randomized variant of MM_BFS in [3].