Distributed Approaches to Butterfly Analysis on Large Dynamic Bipartite Graphs

For any positive integer k, the total k-domatic partition problem is to partition the vertices of a graph G into k pairwise disjoint total dominating sets. In this paper, we study the problem for planar graphs, chordal bipartite graphs, convex bipartite graphs, and bipartite permutation graphs. We show that the total 3-domatic partition problem on planar graphs is NP-complete. Moreover, we give an alternative algorithm to solve the total k-domatic partition problem for chordal bipartite graphs with weak elimination orderings, and adapt it to solve the problem in linear time for bipartite permutation graphs and convex bipartite graphs even if Gamma-free forms of the adjacency matrices of the considered graphs are not given.

This paper deals with embedded index coding problem (EICP), introduced by A. Porter and M. Wootters, which is a decentralized communication problem among users with side information. An alternate definition of the parameter minrank of an EICP, which has reduced computational complexity compared to the existing definition, is presented. A graphical representation for an EICP is given using directed bipartite graphs, called bipartite problem graph, and the side information alone is represented using an undirected bipartite graph called the side information bipartite graph. Inspired by the well-studied single unicast index coding problem, graphical structures, similar to cycles and cliques, are identified in the side information bipartite graph of a single unicast embedded index coding problem (SUEICP). Transmission schemes based on these graphical structures, called tree cover scheme and bi-clique cover scheme are also presented. For a class of SUEICPs, scalar linear optimal solution is given using bi-clique cover. A relation between connectedness of the side information bipartite graph and the number of transmissions required in a scalar linear solution of an EICP is established.

Given a bipartite graph, the maximum balanced biclique (MBB) problem, discovering a mutually connected while disjoint sets of equal size with the maximum cardinality, plays a significant role for mining the bipartite graph and has numerous applications. Despite the NP-hardness of the MBB problem, in this paper, we show that an exact MBB can be discovered extremely fast in bipartite graphs for real applications. We propose two exact algorithms dedicated for small dense and large sparse bipartite graphs respectively. For dense bipartite graphs, an O*(1.3803n) algorithm is proposed. This algorithm in fact can find an MBB very fast for small dense bipartite graphs that are common for applications such as VLSI design. This is because, using our proposed novel techniques, the search can fast converge to sufficiently dense bipartite graphs which we prove to be polynomial-time solvable. For large sparse bipartite graphs typical for applications such as biological data analysis, an O*(1.3803 δ) algorithm is proposed, where δ is only a few hundred for large sparse bipartite graphs with millions of vertices. The indispensible optimization that leads to this time complexity is: we transform a large sparse bipartite graph into a limited number of dense subgraphs such that each of the dense subgraphs has up to δ vertices and then apply our proposed algorithm for dense bipartite graphs on each of the subgraphs. To further speed up this algorithm, tighter upper bounds, faster heuristics and more effective reductions are proposed, allowing an MBB to be discovered within a few seconds for bipartite graphs with millions of vertices. Extensive experiments are conducted on synthetic and real large bipartite graphs to demonstrate the efficiency and effectiveness of our proposed algorithms and techniques.

Next generation analytics will be all about graphs, though performance has been a fundamental challenge for large scale graph processing. In this paper, we present an industrial graph processing engine for exploring various large scale linked data, which exhibits superior performance due to the several innovations. This engine organizes a graph as a set of edge-sets, compatible with the traditional edge-centric sharding for graphs, but becomes more amenable for large scale processing. Each time only a portion of the sets are needed for computation and the data access patterns can be highly predictable for prefetch for many graph computing algorithms. Due to the sparsity of large scale graph structure, this engine differentiates logical edge-sets from the edge-sets physically stored on the disk, where multiple logical edge-sets can be organized into a same physical edge-set to increase the data locality. Besides, in contrast to existing solution, the data structures utilized for the physical edge-sets can vary from one to another. Such heterogeneous edge-set representation explores the best graph processing performance according to local data access patterns. We conduct experiments on a representative set of property graphs on multiple platforms, where the proposed system outperform the baseline systems consistently.

Let MAJ_n denote the Boolean majority function of n input variables. In this paper, we study the construction of depth two circuits computing MAJ_n where each gate in a circuit computes MAJ_m for m < n. We first give an explicit construction of depth two MAJ_{floor[n/2]+2} o MAJ_{ = 7 such that n congruent 3 (mod 4) where MAJ_m and MAJ_{<= m} denote the majority gates that take m and at most m distinct inputs, respectively. A graph theoretic argument developed by Kulikov and Podolskii (STACS '17, Article No. 49) shows that there is no MAJ_{<= n-2} o MAJ_{n-2} circuit computing MAJ_n. Hence, our construction reveals that the use of a smaller fan-in gates at the bottom level is essential for the existence of such a circuit. Some computational results are also provided. We then show that the construction of depth two MAJ_m o MAJ_m circuits computing MAJ_n for m<n can be translated into the construction of a newly introduced version of bipartite expander graphs which we call a list expander. Intuitively, a list expander is a c-leftregular bipartite graph such that for a given d < c, every d-leftregular subgraph of the original graph has a certain expansion property. We formalize this connection and verify that, with high probability, a random bipartite graph is a list expander of certain parameters. However, the parameters obtained are not sufficient to give us a MAJ_{n-c} o MAJ_{n-c} circuit computing MAJ_n for a large constant c.

Bipartite graphs widely exist in real-world scenarios and model binary relations like host-website, author-paper, and user-product. In bipartite graphs, a butterfly (i.e., 2×2 bi-clique) is the smallest non-trivial cohesive structure and plays an important role in applications such as anomaly detection. Considerable efforts focus on counting butterflies in static bipartite graphs. However, they suffer from high time and space complexity when the bipartite graph of interest is given as a stream of edges. Although there are methods for approximately counting butterflies from bipartite graph streams, they suffer from either low accuracy or high time complexity. Therefore, it is still a challenge to accurately estimate butterfly counts from bipartite graph streams in a short time. To address this issue, we develop novel algorithms by exploiting the bipartite nature, which subtly integrates sampling and sketching techniques. We provide accurate estimators for butterfly counts and derive simple yet exact formulas for bounding their errors. We also conduct extensive experiments on a variety of real-world large bipartite graphs. Experimental results demonstrate that our algorithms are up to 20.0 times more accurate and up to 286.3 times faster than state-of-the-art methods under the same memory usage.

Reachability queries are a fundamental type of queries on graphs that find important applications in numerous domains. Although a plethora of techniques have been proposed for reachability queries, most of them require that the input graph is static, i.e., they are inapplicable to the {\em dynamic} graphs (e.g., social networks and the Semantic Web) commonly encountered in practice. There exist a few techniques that can handle dynamic graphs, but none of them can scale to sizable graphs without significant loss of efficiency. To address this deficiency, this paper presents a novel study on reachability indices for large dynamic graphs. We first introduce a general indexing framework that summarizes a family of reachability indices with the best performance among the existing techniques for static graphs. Then, we propose general and efficient algorithms for handling vertex insertions and deletions under the proposed framework. In addition, we show that our update algorithms can be used to improve the existing reachability techniques on static graphs, and we also propose a new approach for constructing a reachability index from scratch under our framework. We experimentally evaluate our solution on a large set of benchmark datasets, and we demonstrate that our solution not only supports efficient updates on dynamic graphs, but also provides even better query performance than the state-of-the-art techniques for static graphs.

Distributed graph processing has become a very popular research topic recently, particularly in domains such as the analysis of social networks, web graphs and spatial networks. In this context, graph partitioning is an important task. Several partitioning algorithms have been proposed, such as DFEP, JABEJA and POWERGRAPH, but they are limited to static graphs only. In fact, they do not consider dynamic graphs in which vertices and edges are added and/or removed. In this paper, we propose a graph partitioning method for large dynamic graphs. We present an implementation of the proposed approach on top of the AKKA framework, and we experimentally show that our approach is efficient in the case of large dynamic graphs.

Incremental maximal clique enumeration (IMCE), which maintains maximal cliques in dynamic graphs, is a fundamental problem in graph analysis. A maximal clique has a solid descriptive power of dense structures in graphs. Real-world graph data is often large and dynamic. Studies on IMCE face significant challenges in the efficiency of incremental batch computation and hybrid edge changes. Moreover, with growing graph sizes, new requirements occur on indexing global maximal cliques and obtaining maximal cliques under specific vertex scope constraints. This work presents a new data structure SOMEi to maintain intermediate maximal cliques during construction. SOMEi serves as a space-efficient index to retrieve scope-constrained maximal cliques on the fly. Based on SOMEi, we design a procedure-oriented IMCE algorithm to deal with hybrid edge changes within a unified algorithm framework. In particular, the algorithm is able to process a large batch of edge changes and significantly improve the average processing time of a single edge change through an efficient pruning strategy. Experimental results on real and synthetic graph data demonstrate that the proposed algorithm outperforms all the baselines and achieves good efficiency through pruning.

For any class C of bipartite graphs, we define quasi-C to be the class of all graphs G such that every bipartition of G belongs to C. This definition is motivated by a generalisation of the switch Markov chain on perfect matchings from bipartite graphs to nonbipartite graphs. The monotone graphs, also known as bipartite permutation graphs and proper interval bigraphs, are such a class of bipartite graphs. We investigate the structure of quasi-monotone graphs and hence construct a polynomial time recognition algorithm for graphs in this class.

This thesis delves into critical challenges associated with subgraph analysis in large-scale graphs. The proliferation of data in domains such as social networks, Internet applications, and biological networks necessitates a deep understanding of relationships modeled as graphs. The complexity and scale of these graphs pose challenges for efficient extraction of meaningful information. The thesis focuses on three interconnected research problems: subgraph counting in fully-dynamic graph streams, k-clique listing in large general graphs, and maximal clique enumeration in large general graphs. The first research problem addresses subgraph counting in fully-dynamic graph streams, where graphs evolve dynamically with edge insertions and deletions. Traditional ap- proaches sample edges uniformly, overlooking the varying importance of edges. The thesis introduces a novel weighted sampling framework, WSD, for fully-dynamic graph streams, which samples the edges based on their weights that indicate their importance and reflect their properties. It determines the weights of edges in a data-driven fashion, using a novel method based on reinforcement learning. We conduct extensive experi- ments to verify that our technique can produce estimates with smaller errors while often running faster compared with existing algorithms. The second research problem focuses on k-clique listing in large general graphs, a critical task for applications such as community detection and network analysis. The state-of-the-art algorithms all adopt a branch-and-bound (BB) framework with a vertex-oriented branching strategy (called VBBkC), which forms a sub-branch by expanding a partial k-clique with a vertex. These algorithms have the time complexity of O(km(\delta/2)^(k-2)), where m is the number of edges in the graph and \delta is the degeneracy of the graph. In this thesis, we propose a BB framework with a new edge-oriented branching (called EBBkC), which forms a sub-branch by expanding a partial k-clique with two vertices that connect each other (which correspond to an edge). We explore various edge orderings for EBBkC such that it achieves a time complexity of O(m\delta + km(\tau/2)^(k-2), where \tau is an integer related to the maximum truss number of the graph and we have \tau<\delta. The time complexity of EBBkC is better than that of VBBkC algorithms for k>3 since both O(m\delta) and O(km(\tau/2)^(k-2) are bounded by O(km(\delta/2)^(k-2)). Furthermore, we develop specialized algorithms for sub-branches on dense graphs so that we can early-terminate them and apply the specialized algorithms. We conduct extensive experiments on 19 real graphs, and the results show that our newly developed EBBkC based algorithms with the early termination technique consistently and largely outperform the state-of-the- art (VBBkC based) algorithms. The third research problem centers on maximal clique enumeration (MCE) in large general graphs, crucial for tasks like community detection and biological network analysis. Existing algorithms typically adopt the branch-and-bound framework with the vertex-oriented Bron-Kerbosch (BK) branching strategy, which forms the sub-branches by expanding the partial clique with a vertex. The state-of-the-arts have a time complexity no better than O(n\delta 3^(\delta/3)), where n is the number of vertices in the graph and \delta is the degeneracy of the graph. We introduce a novel approach, HBBMC, a hybrid framework combining vertex-oriented and edge-oriented BK branching, where the latter adopts a new branch-and-bound framework which forms the sub-branches by expanding the partial clique with a edge. This hybrid strategy enables more efficient pruning of sub-branches, resulting in a worst-case time complexity of O(m\delta + m\tau 3^(\tau/3)), where \tau is related to the maximum truss number. This theoretical result is better than that of the state-of-the-arts when \delta > \tau + (3/ln3) \rho, a condition satisfied by many real-world graphs, where \rho is the edge density of the graph defined as \rho = m/n. To further enhance efficiency, we introduce an early termination technique applicable to all branch-and-bound frameworks, which leverages the topological information of the graphs within the branches. This technique accelerates the enumeration process without affecting worst-case time complexity. Extensive experiments on real and synthetic graphs demonstrate the superior performance of our approach. The proposed HBBMC algorithm with the early termination technique consistently outperforms existing vertex-oriented algorithms. In summary, this thesis makes significant contributions by introducing novel algo- rithms, frameworks, and methodologies to address subgraph-related problems. The pro- posed approaches show superior performance across various real-world and synthetic datasets, providing efficient solutions for graph analysis in large-scale, dynamic, and complex scenarios.

On the complexity of the flow coloring problem

This article considers problems that can be characterized by large dynamic graphs. Communication networks provide the prototypical example of such problems where nodes in the graph are network IDs and the edges represent communication between pairs of network IDs. In such graphs, nodes and edges appear and disappear through time so that methods that apply to static graphs are not sufficient. Our definition of a dynamic graph is procedural. We introduce a data structure and an updating scheme that captures, in an approximate sense, the graph and its evolution through time. The data structure arises from a bottom-up representation of the large graph as the union of small subgraphs centered on every node. These subgraphs are interesting in their own right and can be enhanced to form what we call communities of interest (COI). We discuss an application in the area of telecommunications fraud detection to help motivate the ideas.

Continuous pattern detection plays an important role in monitoring-related applications. The large size and dynamic update of graphs, along with the massive search space, pose huge challenges in developing an efficient continuous pattern detection system. In this paper, we leverage a distributed graph processing framework to approximately detect a given pattern over a large dynamic graph. We aim to improve the scalability and precision, and reduce the response time and message cost in the detection. We convert a given query pattern into a Single-Sink DAG (Directed Acyclic Graph), and propose an evaluation plan with message transitions on the DAG, which is shorten by SSD plan, to detect the pattern in a large dynamic graph. SSD plan can guide the data graph exploration via messages, and the messages will converge at data sink vertices, which then detect existences of the query pattern. We also conduct join operations over partial vertices during the graph exploration to improve the precision of pattern detection. In addition, we show that SSD plan can support the continuous query over dynamic graphs with slight extensions. We further design various sink vertex selection strategies and neighborhood based transition rule attachment to lower the evaluation cost. The experiments on billion-edge real-life graphs using Giraph, an open source implementation of Pregel, illustrate the efficiency and effectiveness of our method.

Fast processing graph algorithms for large-scale graphs becomes increasingly important. Besides, there have been many attempts to process graph applications by exploiting the massive amount of parallelism of GPUs. However, most of the existing methods fail to process large-scale graphs that do not fit in GPU device memory. We propose a fast and scalable parallel processing method GStream that fully exploits the computational power of GPUs for processing large-scale graphs (e.g., billions vertices) very efficiently. It exploits the concept of nested-loop theta-join and multiple asynchronous GPU streams. Extensive experimental results show that GStream consistently and significantly outperforms the state-of-the art method.