Efficient Range Query Processing in Peer-to-Peer Systems

Multi-dimensional data structures are applied in many real index applications, i.e. data mining, indexing multimedia data, indexing of text documents and so on. Many index structures and algorithms have been proposed. There are two major approaches to multi-dimensional indexing: data structures to indexing metric and vector spaces. R-trees, R*-trees and (B)UB-trees are representatives of the vector data structures. These data structures provide efficient processing of many types of queries, i.e. point queries, range queries and so on. As far as the vector data structures are concerned, the range query retrieves all points in defined hyper box in an n-dimensional space. The narrow range query is an important type of the range query. Its processing is inefficient in vector data structures. Moreover, the efficiency decreases as the dimension of the indexed space increases. We depict an application of the signature for more efficient processing of narrow range queries. The approach puts the signature into the multi-dimensional data structures like R-tree or UB-tree but original functionalities are preserved, i.e. the range query algorithm for general range query. The novel data structure is called the signature data structure, e.g., Signature R-tree or Signature UB-tree.

Distributed Hash Tables (DHTs) are scalable, self‐organizing, and adaptive to underlying topology changes, thus being a promising infrastructure for hosting large‐scale distributed applications. The ever‐wider use of DHT infrastructures has found more and more applications that require support for range queries. Recently, a number of DHT‐based range query schemes have been proposed. However, most of them suffer from high query delay or imbalanced load distribution. To address these problems, in this paper we first present an efficient indexing structure called Balanced Kautz (BK) tree that uniformly maps the m‐dimensional data space onto DHT nodes, and then propose a BK tree‐based range query scheme called ERQ that processes range queries in a parallel fashion and guarantees to return the results in a bounded delay. In a DHT with N nodes, ERQ can answer any range of query in less than rmlog N(2loglog N + 1) hops in a load‐balanced manner, irrespective of the queried range, the whole space size, or the number of queried attributes. The effectiveness of our proposals is demonstrated through experiments. Copyright © 2010 John Wiley & Sons, Ltd.

Range queries, retrieving all keys within a given range, is an important add-on for Distributed Hash Tables (DHTs), as they rely only on exact key matching lookup. In this paper we support range queries through a balanced tree algorithm, Decentralized Balanced Tree, that runs over any DHT system. Our algorithm is based on the B+-tree design that efficiently stores clustered data while maintaining a balanced load on hosts. The internal structure of the balanced tree is suited for range queries operations over many data distributions since it easily handles clustered data without losing performance. We analyzed, and evaluated our algorithm under a simulated environment, to show it's operation scalability for both insertions and queries. We will show that the system design imposes a fixed penalty over the DHT access cost, and thus inherits the scalability properties of the chosen underlying DHT.

We focus on range query processing on large-scale, typically distributed infrastructures, such as clouds of thousands of nodes of shared-datacenters, of p2p distributed overlays, etc. In such distributed environments, efficient range query processing is the key for managing the distributed data sets per se, and for monitoring the infrastructure’s resources. We wish to develop an architecture that can support range queries in such large-scale decentralized environments and can scale in terms of the number of nodes as well as in terms of the data items stored. Of course, in the last few years there have been a number of solutions (mostly from researchers in the p2p domain) for designing such large-scale systems. However, these are inadequate for our purposes, since at the envisaged scales the classic logarithmic complexity (for point queries) is still too expensive while for range queries it is even more disappointing. In this paper we go one step further and achieve a sub-logarithmic complexity. We contribute the ART (Autonomous Range Tree) structure, which outperforms the most popular decentralized structures, including Chord (and some of its successors), BATON (and its successor) and Skip-Graphs. We contribute theoretical analysis, backed up by detailed experimental results, showing that the communication cost of query and update operations is \(O(\log_{b}^{2} \log N)\) hops, where the base b is a double-exponentially power of two and N is the total number of nodes. Moreover, ART is a fully dynamic and fault-tolerant structure, which supports the join/leave node operations in O(loglogN) expected w.h.p. number of hops. Our experimental performance studies include a detailed performance comparison which showcases the improved performance, scalability, and robustness of ART.

Adverse Event Reporting

Distributed Hash Tables (DHTs) provide the substrate to build scalable and efficient Peer-to-Peer (P2P) networks: distributed systems with the potential to handle massive amounts of data on a very large scale. However, traditional DHTs provide very poor support for range queries. In this article we present a search mechanism that efficiently supports range queries over a ring-like DHT structure using a prefix tree index. Load balancing is improved by delegating the routing of queries to the nodes that store data, and by updating neighbor information through an optimistic approach. Our solution reduces latency and message traffic in environments where queries are more frequent than data insertion operations. We evaluate the performance of the system through simulations and show that our solution in not affected by data skewness.

Uncertain data has emerged as a key data type in many applications. New and efficient query processing techniques need to be developed due to the inherent complexity of this new type of data. In this paper, we investigate a special type of query, range queries, on uncertain data. We propose a threshold interval indexing structure that aims to balance different time consuming factors to achieve an optimal overall query performance. We also present a more efficient version of our structure which loads its primary tree into memory for faster processing. Experimental results are presented to justify the efficiency of the proposed query processing technique.

Geographic Information Systems (GIS) are increasingly managing very large sets of data and hence a centralized data repository may not always provide the most scalable solution. Here we introduce a novel approach to manage spatial data by leveraging structured Peer-to-Peer (P2P) systems based on Distributed Hash Tables (DHTs). DHT algorithms provide efficient exact-match object search capabilities without requiring global indexing and as a result they are extremely scalable. Furthermore, the adoption of uniform hash functions ensures excellent load balancing. However, range queries -- which are very common with spatial data -- cannot be executed efficiently because the hash functions unfortunately destroy any existing data locality. Here we report on the design of an Adaptive Spatial Peer-to-pEer Network (ASPEN) that extends Content Addressable Networks (CAN) to preserve spatial locality information while also retaining many of the load balancing properties of DHT systems. We introduce the concept of scatter regions, which are spatial data distribution units that optimize both load balancing and spatial range query processing at the same time. We present a data object key generation function and algorithms for spatial range queries. We rigorously evaluate our technique with both synthetic and real world data sets and the results demonstrate the efficient execution of spatial range queries in the ASPEN system.

Range queries are a very powerful tool in a wide range of data management systems and are vital to a multitude of applications. The hierarchy of structured overlay systems can be utilized in order to provide efficient techniques for processing them, resulting in the support of applications and techniques based on range queries in large-scale distributed information systems. On the other hand, due to the rapid development of the Web, applications based on the P2P paradigm gain more and more interest, having such systems started to evolve towards adopting standard database functionalities in terms of complex query processing support. This goes far beyond simple key lookups, as provided by standard distributed hashtables (DHTs) systems, which makes estimating the completeness of query answers a crucial challenge. Unfortunately, due to the limited knowledge and the usually best-effort characteristics, deciding about the completeness of query results, e.g., getting an idea when a query is finished or what amount of results is still missing, is very challenging. There is not only an urgent need to provide this information to the user issuing queries, but also for implementing sophisticated and efficient processing techniques based on them. In this chapter, the authors propose a method for solving this task. They discuss the applicability and quality of the estimations, present an implementation and evaluation for the P-Grid system, and show how to adapt the technique to other overlays. The authors also discuss the semantics of completeness for complex queries in P2P database systems and propose methods based on the notion of routing graphs for estimating the number of expected query answers. Finally, they discuss probabilistic guarantees for the estimated values and evaluate the proposed methods through an implemented system.

Supporting complex and efficient lookup queries in peer-to-peer networks is challenging, though simple keyword based lookup queries are well supported by most deployed systems. This paper presents a two-level indexing structure built on distributed hash table (DHT) aiming to support range queries on high-dimensional feature space in peer-to-peer network. Unlike most existing systems, where every node is responsible for a data partition, our design only utilizes a small part of the nodes to manage partitions. These partition nodes form the first level index. The second level index consists of one or more server nodes, which maintains links to each partition node. Additionally, a merge and split mechanism is designed to dynamically adjust the workload among nodes. Experimental results indicate that our system offers promising performance in terms of workload balance in churn networks. The flexibility to work with any DHT and the capability to support multiple feature spaces further make our proposed approach a feasible extension for file sharing networks.

P2P systems can be used to provide a wide spectrum of traditional and novel applications. These systems are built around distributed hash tables (DHTs) that allow them to achieve efficient storage and retrieval of resources. A DHT is a data structure whose storage is distributed among network peers. Chord and Kademlia are two relevant examples of such DHTs. A limiting factor for the proper operation of P2P networks is the aggregate effect of peer arrivals and departures. This phenomenon is called churn and it negatively affects the quality of the services provided by DHTs. The lookup service, for instance, is a function that receives a resource ID as the input and determines the pointer to the holder of such resource. This service may fail if the nodes that store parts of a DHT leave the network. In this work we study the performance of the lookup service of Chord and Kademlia in simulation scenarios with different peer online times. We found that, by making use of long-lived peers in the network structure, Kademlia achieves better performance than Chord. Our results suggest that this benefit is obtained even with a low fraction of long-lived peers.

Until now, the analysis of fault tolerance of peer-to-peer (p2p) systems usually only covers random faults of some kind. Contrary to traditional algorithmic research, faults as well as joins and leaves occurring in a worst-case manner in p2p systems are hardly considered. Most fault tolerance analyzes are static in the sense that it is shown that a system tolerates a certain number of simultaneous faults. The much more realistic dynamic case where faults steadily occur has not found much attention. The goal of this thesis is to work towards a general understanding of fault tolerance especially for the case of dynamic and worst-case failures. The thesis introduces a simple dynamic model where a malicious adversary — controlling the arrivals and departures of the peers — and a repairing algorithm take turns. The insights gained from this model are then used to study the more realistic scenario where a repairing algorithm maintains certain properties of the topology against a concurrent adversary. Besides the comparison of different fault-tolerance models, the thesis presents a distributed hash table which maintains an efficient search structure as well as a low peer degree in spite of the worst-case failures. By a dynamic analysis we prove that no data item is lost by the system.

A range query applies an aggregation operation over all selected cells of an OLAP data cube where selection is specified by the range of contiguous values for each dimension. Many works have focused on efficiently computing range sum or range max queries. Most of these algorithms use a uniformly partitioning scheme for the data cube. In this paper, we improve on query costs of some of these existing algorithms by noting two key areas. First, end-user range queries usually involve repetitive query patterns, which provide a variable sized partitioning scheme that can be used to partition the data cubes. Query costs are reduced because pre-computation is retrieved for entire partitions, rather than computed for a partial region in many partitions, which requires large amounts of cell accesses to the data cube. Second, data in the data cube can be arranged such that each partition is stored in as few physical storage blocks as possible, thus reducing the I/O costs for answering range queries.

Range queries (RQ) are crucial for analytical workloads, with indexing support being essential to minimize storage accesses. However, indexing support for RQ faces several challenges. Existing tree-based indexes have suboptimal RQ performance and memory consumption when long-running RQs and short-lived updates coexist. Bitmap indexes show promise in overcoming these challenges because of their small size and their succinct and readily available query result; however, they have inherent limitations: they primarily target read-only, low-cardinality attributes. In this paper, we propose Ra nge Queries with Bit map Indexing (RABIT), a solution that addresses these shortcomings. Our design relies on three principles. First, we propose Group Encoding (GE), a novel encoding scheme that provides fast RQs and real-time updates while maintaining high compressibility. Second, we propose an efficient bitvector merging mechanism for GE. Depending on the bit density of each bitvector, we merge it in either its compressed or decompressed form, leveraging SIMD instructions when beneficial. Third, we propose a multi-layer update framework that enables lightweight multi-versioning and native index-only scans, while retaining single-versioned bitvectors, significantly reducing memory usage. Putting everything together, RABIT provides efficient point and range queries on attributes with any cardinality in tables ranging from read-only to frequently updated, unlocking the use of bitmap indexing as a general-purpose secondary index. We demonstrate that RABIT accelerates key DBMS operators (Scan, Join, and Aggregation), achieving substantial performance gains. In a row-store DBMS under HTAP workloads, RABIT offers up to 2.2x faster RQs, 530x faster updates, and 118x smaller footprint than tree indexes. In columnar DuckDB, RABIT accelerates TPC-H queries by up to 14.8x.

Range queries are important in many diverse applications. In its simplest one-dimensional form, a range query is expressed by an interval [a, b] on the real line, whereas the answer consists of all elements e in [a, b]. In this work, we focus on efficient range query processing techniques in the Apache Spark engine, which is the state-of-the-art solution for big data management and analytics. We aim at developing a Spark-based indexing scheme that supports range queries in such large-scale decentralized environments and scale well w.r.t. the number of nodes and the data items stored. Towards this goal, there have been solutions in the last few years, which however turn out to be inadequate at the envisaged scale, since the classic linear or even the logarithmic complexity (for point queries) is still too expensive, whereas range query processing is even more demanding. In this paper, we go one step further and present a solution with sub-logarithmic complexity. In particular, we present SPIS (SPark-based Interpolation Search), a tree structure that outperforms the existing Spark built-in lookup techniques. We carry out an experimental evaluation by using synthetic data sets. Our experimental results demonstrate the efficiency and scalability of the proposed approach.