Truss decomposition of uncertain graphs

Many studies have been conducted on seeking an efficient solution for graph similarity search over certain (deterministic) graphs due to its wide application in many fields, including bioinformatics, social network analysis, and Resource Description Framework data management. All prior work assumes that the underlying data is deterministic. However, in reality, graphs are often noisy and uncertain due to various factors, such as errors in data extraction, inconsistencies in data integration, and for privacy-preserving purposes. Therefore, in this paper, we study similarity graph containment search on large uncertain graph databases. Similarity graph containment search consists of subgraph similarity search and supergraph similarity search. Different from previous works assuming that edges in an uncertain graph are independent of each other, we study uncertain graphs where edges' occurrences are correlated. We formally prove that subgraph or supergraph similarity search over uncertain graphs is $$\#$&#P-hard; thus, we employ a filter-and-verify framework to speed up these two queries. For the subgraph similarity query, in the filtering phase, we develop tight lower and upper bounds of subgraph similarity probability based on a probabilistic matrix index (PMI). PMI is composed of discriminative subgraph features associated with tight lower and upper bounds of subgraph isomorphism probability. Based on PMI, we can filter out a large number of uncertain graphs and maximize the pruning capability. During the verification phase, we develop an efficient sampling algorithm to validate the remaining candidates. For the supergraph similarity query, in the filtering phase, we propose two pruning algorithms, one lightweight and the other strong, based on maximal common subgraphs of query graph and data graph. We run the two pruning algorithms against a probabilistic index that consists of powerful graph features. In the verification, we design an approximate algorithm based on the Horvitz---Thompson estimator to fast validate the remaining candidates. The efficiencies of our proposed solutions to the subgraph and supergraph similarity search have been verified through extensive experiments on real uncertain graph datasets.

We present PKT, a new shared-memory parallel algorithm and OpenMP implementation for the truss decomposition of large sparse graphs. A k-truss is a dense subgraph definition that can be considered a relaxation of a clique. Truss decomposition refers to a partitioning of all the edges in the graph based on their k-truss membership. The truss decomposition of a graph has many applications. We show that our new approach PKT consistently outperforms other truss decomposition approaches for a collection of large sparse graphs and on a 24-core shared-memory server. PKT is based on a recently proposed algorithm for k-core decomposition.

Uncertain graph models are widely used in real-world applications such as knowledge graphs and social networks. To capture the uncertainty, each edge in an uncertain graph is associated with an existential probability that signifies the likelihood of the existence of the edge. One notable issue of querying uncertain graphs is that the results are sometimes uninformative because of the edge uncertainty. In this paper, we consider probabilistic reachability queries, which are one of the fundamental classes of graph queries. To make the results more informative, we adopt a crowdsourcing-based approach to clean the uncertain edges. However, considering the time and monetary cost of crowdsourcing, it is a problem to efficiently select a limited set of edges for cleaning that maximizes the quality improvement. We prove that the edge selection problem is #P-hard. In light of the hardness of the problem, we propose a series of edge selection algorithms, followed by a number of optimization techniques and pruning heuristics for reducing the computation time. Our experimental results demonstrate that our proposed techniques outperform a random selection by up to 27 times in terms of the result quality improvement and the brute-force solution by up to 60 times in terms of the elapsed time.

The k -truss is a type of cohesive subgraphs proposed recently for the study of networks. While the problem of computing most cohesive subgraphs is NP-hard, there exists a polynomial time algorithm for computing k -truss. Compared with k -core which is also efficient to compute, k -truss represents the "core" of a k -core that keeps the key information of, while filtering out less important information from, the k -core. However, existing algorithms for computing k -truss are inefficient for handling today's massive networks. We first improve the existing in-memory algorithm for computing k -truss in networks of moderate size. Then, we propose two I/O-efficient algorithms to handle massive networks that cannot fit in main memory. Our experiments on real datasets verify the efficiency of our algorithms and the value of k -truss.

Graphs play an important role in modern world, due to their widespread use for modeling, representing and organizing linked data. Taking into consideration that most of the “killer” applications require a graph-based representation (e.g., the Web, social network management, protein-protein interaction networks), efficient query processing and analysis techniques are required, not only because these graphs are massive but also because the operations that must be supported are complex, requiring significant computational resources. In many cases, each graph edge e is annotated by a probability value p(e), expressing its existential uncertainty. This means that with probability p(e) the edge will be present in the graph and with probability \(1-p(e)\) the edge will be absent. This gives rise to the concept of probabilistic graphs (also known as uncertain graphs). Formally, a probabilistic graph \(\mathcal{G}\) is a triplet (V, E, p) where V is the set of nodes, E is the set of edges and \(p: E \rightarrow (0,1]\). The main challenge posed by this formulation is that problems that are relatively easy to solve in exact graphs become very difficult (or even intractable) in probabilistic graphs. In this paper, we perform an overview of the algorithmic techniques proposed in the literature for uncertain graph analysis. In particular, we center our focus on the following graph mining tasks: clustering, maximal cliques, k-nearest neighbors and core decomposition. We conclude the paper with a short discussion related to distributed mining of uncertain graphs which is expected to achieve significant performance improvements.

Cohesive subgraph mining is a fundamental problem in the field of graph data analysis. Many existing cohesive graph mining algorithms are mainly tailored to deterministic graphs. Real-world graphs, however, are often not deterministic, but uncertain in nature. Applications of such uncertain graphs include protein-protein interactions networks with experimentally inferred links and sensor networks with uncertain connectivity links. In this article, we study the problem of mining cohesive subgraphs from an uncertain graph. Specifically, we introduce a new <inline-formula><tex-math notation="LaTeX">$(\alpha,\gamma)$</tex-math></inline-formula> -quasi-clique model to model the cohesive subgraphs in an uncertain graph, and propose a basic enumeration algorithm to find all maximal <inline-formula><tex-math notation="LaTeX">$(\alpha,\gamma)$</tex-math></inline-formula> -quasi-cliques. We also develop an advanced enumeration algorithm based on several novel pruning rules, including early termination and candidate set reduction. To further improve the efficiency, we propose several optimization techniques. Extensive experiments on five real-world datasets demonstrate that our solutions are almost three times faster than the baseline approach.

This paper studies a novel approach to processing massive uncertain graph data. In this approach, we propose a new framework to simultaneously process a query on a set of randomly sampled possible worlds of an uncertain graph. Based on this framework, we develop a series of algorithms to analyze massive uncertain graphs, including breadth-first search, shortest distance queries, triangle counting, and core decomposition. We implement this approach based on GraphLab, one of the stateof-the-art graph processing frameworks. By sharing fine-grained internal processing steps on common substructures of sampled possible worlds, the new approach achieves tens to hundreds of times speedup in execution time on a cluster of 20 servers.

With the exponential growth of graph‐structured data, single‐machine efficient analysis has become increasingly impractical, making high‐performance distributed graph computing systems indispensable. The efficacy of these systems hinges critically on high‐quality graph partitioning. The edges of many graphs stemmed from real applications are uncertain, but many existing graph partitioning algorithms are only for deterministic graphs without considering uncertainty. This paper presents a novel partitioning algorithm, PAUG (Partitioning Algorithm for Uncertain Graphs), tailored for uncertain graphs. First, it formalizes the partitioning problem as an optimization task to minimize the cut‐edge ratio while balancing load. Second, it introduces probabilistic similarity to quantify vertex relationships under uncertainty. Finally, it details the PAUG algorithm which consists of initial partition phase and score‐function‐guided refinement strategy. Experimental results shows that PAUG achieves an average 23.2% reduction in cut‐edge ratio and a 26.2% improvement in load balance over state‐of‐the‐art algorithms.

Community detection that discovers all densely connected communities in a network has been studied a lot. In this paper, we study online community search for query-dependent communities, which is a different but practically useful task. Given a query vertex in a graph, the problem is to find meaningful communities that the vertex belongs to in an online manner. We propose a community model based on the k-truss concept, which brings nice structural and computational properties. We design a compact and elegant index structure which supports the efficient search of k-truss communities with a linear cost with respect to the community size. We also investigate the k-truss community search problem in a dynamic graph setting with frequent insertions and deletions of graph vertices and edges. In addition, to support k-truss community search over massive graphs which cannot entirely fit in main memory, we propose I/O-efficient algorithms for query processing under the semi-external model. Extensive experiments on massive real-world networks demonstrate the effectiveness of our k-truss community model, the efficiency, and the scalability of our in-memory and semi-external community search algorithms.

Social networks are constantly changing as new members join, existing members leave, and ‘followers’ or ‘friends’ are formed and disappear. The model that captures this constantly changing graph is the streaming graph model. Given a massive graph data stream wherein the number of nodes is in the order of millions and the number of edges is the tens of millions, we propose a simple algorithm to compress this graph without having read in the entire graph into the main memory. Our algorithm uses the quadtree data structure that is implicitly constructed to produce the compressed graph output. As a result of this implicit construction, our algorithm allows for node and edge additions/deletions that directly modifies the output compressed graph. We further develop algorithms to solve edge queries (is there any between two nodes?) and node queries (for a given node, list all its neighbors) that directly operates on the compressed graph. We have performed extensive empirical evaluations of our algorithms using publicly available, large social networks such as LiveJournal, Pokec, Twitter, and others. Our empirical evaluation is based on several parameters including time to compress, memory required by the compression algorithm, size of compressed graph, and time and memory size required to execute queries. We have also presented extensions to the compression algorithm that we have developed.

The k-core of a graph is the largest subgraph in which every vertex is connected to at least k other vertices within the subgraph. Core decomposition finds the k-core of the graph for every possible k. Past studies have shown important applications of core decomposition such as in the study of the properties of large networks (e.g., sustainability, connectivity, centrality, etc.), for solving NP-hard problems efficiently in real networks (e.g., maximum clique finding, densest subgraph approximation, etc.), and for large-scale network fingerprinting and visualization. The k-core is a well accepted concept partly because there exists a simple and efficient algorithm for core decomposition, by recursively removing the lowest degree vertices and their incident edges. However, this algorithm requires random access to the graph and hence assumes the entire graph can be kept in main memory. Nevertheless, real-world networks such as online social networks have become exceedingly large in recent years and still keep growing at a steady rate. In this paper, we propose the first external-memory algorithm for core decomposition in massive graphs. When the memory is large enough to hold the graph, our algorithm achieves comparable performance as the in-memory algorithm. When the graph is too large to be kept in the memory, our algorithm requires only O(k max ) scans of the graph, where k max is the largest core number of the graph. We demonstrate the efficiency of our algorithm on real networks with up to 52.9 million vertices and 1.65 billion edges.

This paper solves a longstanding open problem in fully dynamic algorithms: We present the first fully dynamic algorithms that maintain connectivity, bipartiteness, and approximate minimum spanning trees in polylogarithmic time per edge insertion or deletion. The algorithms are designed using a new dynamic technique that combines a novel graph decomposition with randomization. They are Las-Vegas type randomized algorithms which use simple data structures and have a small constant factor. Let n denote the number of nodes in the graph. For a sequence of Ω( m 0 ) operations, where m 0 is the number of edges in the initial graph, the expected time for p updates is O ( p log 3 n ) (througout the paper the logarithms are based 2) for connectivity and bipartiteness. The worst-case time for one query is O (log n /log log n ). For the k -edge witness problem (“Does the removal of k given edges disconnect the graph?”) the expected time for p updates is O ( p log 3 n ) and the expected time for q queries is O ( qk log 3 n ). Given a graph with k different weights, the minimum spanning tree can be maintained during a sequence of p updates in expected time O ( pk log 3 n ). This implies an algorithm to maintain a 1 + ε-approximation of the minimum spanning tree in expected time O (( p log 3 n log U )/ε) for p updates, where the weights of the edges are between 1 and U .

The data intensive analytical workload becomes heavy burden for OLAP engine with increasing data volume, user population and query complexity. Large capacity random access memory, multi-level cache and multi-core hardware are main streams of computer. We propose a hardware-aware OLAP model named MOSS-DB which optimizes storage model according to data access features of dimensional tables and fact tables. A hard disk & main memory two-level storage model is employed to support directly dimensional tuple accessing join operator(DDTA-JOIN), DDTA-JOIN simplifies OLAP query processing by replacing traditional join operation with directly accessing dimensional tuple with memory address. So the star schema can be seen as virtual de-normalized table, OLAP query is also simplified to table scan, select and project operations. Query processing on sequence data structure is more suitable for multi-core parallel processing. Our proposal allows massive data DRDB(Disk Resident Database) storage technique to cooperate with MMDB(Main-Memory Database) processing technique, which breaks the main memory capacity limitation. The DDTA-JOIN operation can save cost for index, hash table, etc. For multi-core era, MOSS-DB can flexibly use parallel processing capability of CPU by dynamically dividing fact table into multiple scan partitions and gain maximum cache profit for shared dimensional data. In experiments, we measure that MOSS-DB outperforms conventional DRDB system, and it also outperforms MMDB in SSB testing.

Core discovery in hidden networks

Recent progress in remote sensing has made massive amounts of high resolution terrain data readily available. Often the data is distributed as regular grid terrain models where each grid cell is associated with a height. When terrain analysis applications process such massive terrain models, data movement between main memory and slow disk (I/O), rather than CPU time, often becomes the performance bottleneck. Thus it is important to consider I/O-efficient algorithms for fundamental terrain problems. One such problem is the hierarchical decomposition of a grid terrain model into watersheds—regions where all water flows towards a single common outlet. Several different hierarchical watershed decompositions schemes have been described in the hydrology literature. One important such scheme is the Pfafstetter label method where each watershed is assigned a unique label and each grid cell is assigned a sequence of labels corresponding to the (nested) watersheds to which it belongs.In this paper we present an I/O-efficient algorithm for computing the Pfafstetter label of each cell of a grid terrain model. The algorithm uses O(s rt(T)) I/Os, the number of I/Os needed to sort T elements, where T is the total length of the cell labels. To our knowledge, our algorithm is the first efficient algorithm for the problem. We also present the results of a experimental study using massive real life terrain data that shows our algorithm is practically as well as theoretically efficient.