DistR: A Distributed Method for the Reachability Query over Large Uncertain Graphs

Retrieving graphs containing a query graph from a large graph database is a key task in many graph-based applications, including chemical compounds discovery, protein complex prediction, and structural pattern recognition. However, graph data handled by these applications is often noisy, incomplete, and inaccurate because of the way the data is produced. In this paper, we study subgraph queries over uncertain graphs. Specifically, we consider the problem of answering threshold-based probabilistic queries over a large uncertain graph database with the possible world semantics. We prove that problem is #P-complete, therefore, we adopt a filtering-and-verification strategy to speed up the search. In the filtering phase, we use a probabilistic inverted index, PIndex, based on subgraph features obtained by an optimal feature selection process. During the verification phase, we develop exact and bound algorithms to validate the remaining candidates. Extensive experimental results demonstrate the effectiveness of the proposed algorithms.

Many studies have been conducted on seeking an efficient solution for graph similarity search over certain (deterministic) graphs due to its wide application in many fields, including bioinformatics, social network analysis, and Resource Description Framework data management. All prior work assumes that the underlying data is deterministic. However, in reality, graphs are often noisy and uncertain due to various factors, such as errors in data extraction, inconsistencies in data integration, and for privacy-preserving purposes. Therefore, in this paper, we study similarity graph containment search on large uncertain graph databases. Similarity graph containment search consists of subgraph similarity search and supergraph similarity search. Different from previous works assuming that edges in an uncertain graph are independent of each other, we study uncertain graphs where edges' occurrences are correlated. We formally prove that subgraph or supergraph similarity search over uncertain graphs is $$\#$&#P-hard; thus, we employ a filter-and-verify framework to speed up these two queries. For the subgraph similarity query, in the filtering phase, we develop tight lower and upper bounds of subgraph similarity probability based on a probabilistic matrix index (PMI). PMI is composed of discriminative subgraph features associated with tight lower and upper bounds of subgraph isomorphism probability. Based on PMI, we can filter out a large number of uncertain graphs and maximize the pruning capability. During the verification phase, we develop an efficient sampling algorithm to validate the remaining candidates. For the supergraph similarity query, in the filtering phase, we propose two pruning algorithms, one lightweight and the other strong, based on maximal common subgraphs of query graph and data graph. We run the two pruning algorithms against a probabilistic index that consists of powerful graph features. In the verification, we design an approximate algorithm based on the Horvitz---Thompson estimator to fast validate the remaining candidates. The efficiencies of our proposed solutions to the subgraph and supergraph similarity search have been verified through extensive experiments on real uncertain graph datasets.

A reachability query is a fundamental graph operation in real graph applications, which answers whether a node can reach another node through a path in a graph. However, the increasingly large amounts of real graph data make it more challenging for query efficiency and scalability. In this paper, we propose a Fruited-Forest (FF) approach to accelerate reachability queries in large graphs by constructing four kinds of fruited-forests from a reduced DAG in different traversal orders. We build different binary-label schemes for the four kinds of fruited-forests to cover reachability between nodes as much as possible, and create a corresponding index for the deleted edges which are deleted during the construction of fruited-forests. Our experimental results on 18 large real graph datasets show that our FF approach requires less index construct time and a smaller index size, which is more scalable to answer reachability queries compared with other existing works.

In this paper, we study a new graph learning problem: learning to count subgraph isomorphisms. Different from other traditional graph learning problems such as node classification and link prediction, subgraph isomorphism counting is NP-complete and requires more global inference to oversee the whole graph. To make it scalable for large-scale graphs and patterns, we propose a learning framework that augments different representation learning architectures and iteratively attends pattern and target data graphs to memorize intermediate states of subgraph isomorphism searching for global counting. We develop both small graphs (<= 1,024 subgraph isomorphisms in each) and large graphs (<= 4,096 subgraph isomorphisms in each) sets to evaluate different representation and interaction modules. A mutagenic compound dataset, MUTAG, is also used to evaluate neural models and demonstrate the success of transfer learning. While the learning based approach is inexact, we are able to generalize to count large patterns and data graphs in linear time compared to the exponential time of the original NP-complete problem. Experimental results show that learning based subgraph isomorphism counting can speed up the traditional algorithm, VF2, 10-1,000 times with acceptable errors. Domain adaptation based on fine-tuning also shows the usefulness of our approach in real-world applications.

Reachability queries are a fundamental type of queries on graphs that find important applications in numerous domains. Although a plethora of techniques have been proposed for reachability queries, most of them require that the input graph is static, i.e., they are inapplicable to the {\em dynamic} graphs (e.g., social networks and the Semantic Web) commonly encountered in practice. There exist a few techniques that can handle dynamic graphs, but none of them can scale to sizable graphs without significant loss of efficiency. To address this deficiency, this paper presents a novel study on reachability indices for large dynamic graphs. We first introduce a general indexing framework that summarizes a family of reachability indices with the best performance among the existing techniques for static graphs. Then, we propose general and efficient algorithms for handling vertex insertions and deletions under the proposed framework. In addition, we show that our update algorithms can be used to improve the existing reachability techniques on static graphs, and we also propose a new approach for constructing a reachability index from scratch under our framework. We experimentally evaluate our solution on a large set of benchmark datasets, and we demonstrate that our solution not only supports efficient updates on dynamic graphs, but also provides even better query performance than the state-of-the-art techniques for static graphs.

It is increasingly common to find graphs in which edges bear different types, indicating a variety of relationships. For such graphs we propose a class of reachability queries and a class of graph patterns, in which an edge is specified with a regular expression of a certain form, expressing the connectivity in a data graph via edges of various types. In addition, we define graph pattern matching based on a revised notion of graph simulation. On graphs in emerging applications such as social networks, we show that these queries are capable of finding more sensible information than their traditional counterparts. Better still, their increased expressive power does not come with extra complexity. Indeed, (1) we investigate their containment and minimization problems, and show that these fundamental problems are in quadratic time for reachability queries and are in cubic time for pattern queries. (2) We develop an algorithm for answering reachability queries, in quadratic time as for their traditional counterpart. (3) We provide two cubic-time algorithms for evaluating graph pattern queries based on extended graph simulation, as opposed to the NP-completeness of graph pattern matching via subgraph isomorphism. (4) The effectiveness, efficiency and scalability of these algorithms are experimentally verified using real-life data and synthetic data.

Mining Top-k Maximal Cliques from Large Uncertain Graphs

An important component of current research in big data is graph analytics on very large graphs. Of the many problems of interest in this domain, graph pattern matching is both challenging and practically important. The problem is, given a relatively small query graph, finding matching patterns in a large data graph. Algorithms to address this problem are used in large social networks and graph databases. Though fast querying is highly desirable, the scalability of pattern matching algorithms is hindered by the NP-completeness of the subgraph isomorphism problem. This paper presents a conceptually simple, memory-efficient, pruning-based algorithm for the subgraph isomorphism problem that outperforms commonly used algorithms on large graphs. The high performance is due in large part to the effectiveness of the pruning algorithm, which in many cases removes a large percentage of the vertices not found in isomorphic matches. In this paper, the runtime of the algorithm is tested alongside other algorithms on graphs of up to 10 million vertices and 250 million edges.

Today, many applications such as social network and biological network develop rapidly,the graph data will be expanded constantly on a large scale. Some classic methods can not effectively solve this scale of the graph data. In the reachability query, many technologies such as N-Hop, tree, interval labels, uncertain graph processing are emerging, they also solve a lot of questions about reachability query of graph. But, these methods have not put forward the effective solution for the new issues of the multiattribute constraints reachability on directed graph. In this paper, TCRQDG algorithm effectively solves this new problem. Firstly it optimizes the multiattribute constraints with decision making technology; secondly the algorithm achieves fast and accurate query by integrating with the Create virtual vertex expand, conditions filtering, cycles contraction, interval label and other technology. TCRQDG algorithm can not only effectively solve the new problem, but also provide technical support for multiple constraints optimization decisions of network transmission, transport and logistics, software testing and other applications.

Continuously Evolving Graphs (CEGs) are graphs whose connectivity constantly changes over time is uniquely important for many domains such as social networks, evolutionary genomics, communication networks etc. In many of these it is often important to keep track of connectivity among the nodes of interest as the underlying structure changes over time. While interval-based indexing has been a popular strategy for testing reach ability in static graphs, it cannot be directly applied in the context of evolving graphs. In this paper, we propose CoUPE (Continuous qUery Processing Engine), which, to our best knowledge, is the first time-efficient framework for answering continuous reach ability queries in evolving graphs. The main idea here is to maintain the indices of the evolving graph and recalculate only a subset of reach ability queries using a novel heuristic to determine the change in the state of queries because for the most recent change in the graph. We make three novel contributions while designing CoUPE. First, we introduce a generic indexing and querying framework for answering continuous queries in large time-evolving graphs. Second, we design a highly efficient, scalable and provably correct algorithm for updating the indices of graph by analyzing the changes happening on the graph. Third, we present a novel heuristic-based technique for identifying which subset of existing queries might get effected because of the most recent edit. This paper also presents a detailed experimental study demonstrating the scalability and efficiency of the processing engine.

A novel framework for the efficient evaluation of hybrid tree-pattern queries on large data graphs

It is common to find that many real world data graphs are edge-labeled, i.e., each edge attaches a label to indicate the relationship of the two vertices connected by the edge. A basic research issue on these graphs is how to make a query to test if the vertex u can reach the vertex v through the path constrained by a series of labels. In this paper, we propose a novel index framework to retain all the path label information of a data graph and answer the regular expression based reachability queries efficiently. Our index is a generalized-list-structured index and constructed based on the method of bread first search and depth first search. To improve the query efficiency, we also put forward a labeling mechanism for our index. A detailed experimental evaluation on both real and synthetic datasets shows that the performance of our method is improved in term of both index size and query time.

Graph reachability is fundamental to a wide range of applications, including XML indexing, geographic navigation, Internet routing, ontology queries based on RDF/OWL, etc. Many applications involve huge graphs and require fast answering of reachability queries. Several reachability labeling methods have been proposed for this purpose. They assign labels to the vertices, such that the reachability between any two vertices may be decided using their labels only. For sparse graphs, 2-hop based reachability labeling schemes answer reachability queries efficiently using relatively small label space. However, the labeling process itself is often too time consuming to be practical for large graphs. In this paper, we propose a novel labeling scheme for sparse graphs. Our scheme ensures that graph reachability queries can be answered in constant time. Furthermore, for sparse graphs, the complexity of the labeling process is almost linear, which makes our algorithm applicable to massive datasets. Analytical and experimental results show that our approach is much more efficient than stateof- the-art approaches. Furthermore, our labeling method also provides an alternative scheme to tradeoff query time for label space, which further benefits applications that use tree-like graphs.

Greatest fixed points of probabilistic min/max polynomial equations, and reachability for branching Markov decision processes

The importance of graphs as the fundamental structure underpinning many real world applications is no longer to be proved. Large graphs have emerged in various fields such as biological, social and transportation networks. The sheer volume of these networks poses challenges to traditional techniques for storage and analysis of graph data. In particular, OLAP analysis requires access to large portions of data to extract key information and to feed strategic decision making. OLAP provides multilevel, multiperspective views of the data. Most of the current techniques are optimized for centralized graph processing. A distributed approach providing horizontal scalability is required in order to handle the analysis workload. In this paper, we focus on applying OLAP analysis on large, distributed graph data. We describe Distributed Graph Cube, our distributed framework for graph-based OLAP cubes computation and aggregation. Experimental results on large, real-world datasets demonstrate that our method significantly outperforms its centralized counterparts. We also evaluate the performance of both Hadoop and Spark for distributed cubes computations.