Pampoo: An Efficient Skip-Trie Based Query Processing Framework for P2P Systems

Within the last decade, the continuous development of mobile technology has motivated an intense research in mobile data services. Among these services, location-based services and location-dependant queries have attracted a lot of attention from the software industry as well as from researchers. Although a number of locations based service methods have been proposed, the performance is not satisfactory in the case of dynamic data management problems that introduce high overhead in update and query operations. In this paper, we introduce a proxy based distributed dynamic data management architecture for location based services that reduces network cost with required quality of service (QoS). The system proposes a distributed hierarchical database structure with collocated proxy for efficient service delivery to location-dependant queries especially Range Queries (RQ). The system that we propose have the following advantages. i) It performs efficient processing of location dependant queries, especially Range Queries spanning over multiple network cells ii) It reduces wireless communication between mobile devices and servers. iii) The system is scalable where location data is distributed and arranged hierarchically. We use proxy objects to carry out processing task for each mobile user. Thus proxies play an important role and in charge of routing the queries to different (distributed) databases and present the results to queries efficiently. Simulation study shows the effectiveness of our proposed architecture.

This paper looks at the processing of skyline queries on peer-to-peer (P2P) networks. We propose Skyframe, a framework for efficient skyline query processing in P2P systems, which addresses the challenges of quick response time, low network communication cost and query load balancing among peers. Skyframe consists of two querying methods: one is optimized for network communication while the other focuses on query response time. These methods are different in the way in which the query search space is defined. In particular, the first method uses a high dominating point that has a large dominating region to prune the search space to achieve a low cost in network communication. On the other hand, the second method relaxes the search space in order to allow parallel query processing to speed up query response. Skyframe achieves query load balancing by both query load conscious data space splitting/merging during the join/departure of nodes and dynamic load migration. We further show how to apply Skyframe to both the P2P systems supporting multi-dimensional indexing and the P2P systems supporting single-dimensional indexing. Finally, we have conducted extensive experiments on both real and synthetic data sets over two existing P2P systems: CAN (Ratnasamy in A scalable content-addressable network. In: Proceedings of SIGCOMM Conference, pp. 161---172, 2001) and BATON (Jagadish et al. in A balanced tree structure for peer-to-peer networks. In: Proceedings of VLDB Conference, pp. 661---672, 2005) to evaluate the effectiveness and scalability of Skyframe.

With the increasing popularity of the peer-to-peer (P2P) computing paradigm, many general range query schemes for distributed hash table (DHT)-based P2P systems have been proposed in recent years. Although those schemes can provide range query capability without modifying the underlying DHTs, they have the query delay depending on both the scale of the system and the size of the query space or the specific query, and thus cannot guarantee to return the query results in a bounded delay. In this paper, we propose Armada, an efficient range query processing scheme to support delay-bounded single-attribute and multiple-attribute range queries. It is the first delay-bounded general range query scheme on constant-degree DHTs, and can return the results for any range query within 2logN hops in a P2P system with N peers. Results of analysis and simulations show that the average delay in Armada is less than logN, and the average message cost of single-attribute range queries is about logN+2n 2 (n is the number of peers that intersect with the query). These results are very close to the lower bounds on delay and message cost of range queries over constant-degree DHTs.

We focus on range query processing on large-scale, typically distributed infrastructures, such as clouds of thousands of nodes of shared-datacenters, of p2p distributed overlays, etc. In such distributed environments, efficient range query processing is the key for managing the distributed data sets per se, and for monitoring the infrastructure’s resources. We wish to develop an architecture that can support range queries in such large-scale decentralized environments and can scale in terms of the number of nodes as well as in terms of the data items stored. Of course, in the last few years there have been a number of solutions (mostly from researchers in the p2p domain) for designing such large-scale systems. However, these are inadequate for our purposes, since at the envisaged scales the classic logarithmic complexity (for point queries) is still too expensive while for range queries it is even more disappointing. In this paper we go one step further and achieve a sub-logarithmic complexity. We contribute the ART (Autonomous Range Tree) structure, which outperforms the most popular decentralized structures, including Chord (and some of its successors), BATON (and its successor) and Skip-Graphs. We contribute theoretical analysis, backed up by detailed experimental results, showing that the communication cost of query and update operations is \(O(\log_{b}^{2} \log N)\) hops, where the base b is a double-exponentially power of two and N is the total number of nodes. Moreover, ART is a fully dynamic and fault-tolerant structure, which supports the join/leave node operations in O(loglogN) expected w.h.p. number of hops. Our experimental performance studies include a detailed performance comparison which showcases the improved performance, scalability, and robustness of ART.

A crowd enabled approach for processing nearest neighbor and range queries in incomplete databases with accuracy guarantee

The great advances in deep learning enable human beings to extract the semantic information in ubiquitous unstructured data into high-dimensional vectors, leading to an urgent priority of managing vector data. Nearest neighbor (NN) query is a pivotal question in vector data management. Unfortunately, the NN query has been proved to be intrinsically hard. Researchers, thus, turn to approximate nearest neighbor (ANN) query which targets better efficiency by slightly compromising accuracy. Despite this, existing ANN methods still incur high time and space costs, which are mainly due to the high dimensionality. Specifically, in most in-memory ANN algorithms, the time cost is dominated by the time for comparing the distances among vectors. Each distance comparison operation (DCO) involves hundreds of arithmetic operations as each vector has hundreds of dimensions. Similarly, the space cost is dominated by the space for storing the vectors. Towards the goal of reducing the time and space costs, we present three methods in this thesis. First, we observe that, for a DCO, if an approximate distance can confirm that a vector is not a NN, then it can be discarded without computing exact distances. Based on this observation, we propose a method named ADSampling. For a DCO, it iteratively and incrementally samples a few dimensions and computes an approximate distance. When this distance is accurate enough to decide the result of the DCO, it terminates sampling. Second, to further improve the efficiency of DCOs, we design an efficient distance estimator. We propose a new quantization method named RaBitQ, which maps $D$-dimensional vectors into $D$-bit strings. RaBitQ provides an unbiased and asymptotically optimal estimator with good empirical accuracy. In addition, we introduce efficient implementations of RaBitQ based on bitwise operations or SIMD-based operations. Third, by extending the RaBitQ method, we develop an algorithm which compresses a floating-point vector into a $B$-bit unsigned integer vector. We prove that our algorithm is asymptotically optimal in terms of the space-accuracy trade-off.

Case Based Reasoning (CBR) is a method that aims to resolve a new case by adapting the solutions contained in previous cases that are similar to the new case. The system built in this study is the CBR system to make recommendations on the topic of student thesis concentration. This study used data from undergraduate students of Informatics Engineering IST AKPRIND Yogyakarta with a total of 115 data consisting of 80 training data and 35 test data. This study aims to design and build a Case Based Reasoning system using the Nearest Neighbor and Manhattan Distance Similarity Methods, and to compare the results of the accuracy value using the Nearest Neighbor Similarity and Manhattan Distance Similarity methods. The recommendation process is carried out by calculating the value of closeness or similarity between new cases and old cases stored on a case basis using the Nearest Neighbor Method and Manhattan Distance. The features used in this study consisted of GPA and course grades. The case taken is the case with the highest similarity value. If a case doesnt get a topic recommendation or is less than the trashold value of 0.8, a case revision will be carried out by an expert. Successfully revised cases are stored in the system to be made new knowledge. The test results using the Nearest Neighbor Method get an accuracy value of 97.14% and Manhattan Distance Method 94.29%.

In this paper we introduce the notion of constrained nearest neighbor queries (CNN) and propose a series of methods to answer them. This class of queries can be thought of as nearest neighbor queries with range constraints. Although both nearest neighbor and range queries have been analyzed extensively in previous literature, the implications of constrained nearest neighbor queries have not been discussed. Due to their versatility, CNN queries are suitable to a wide range of applications from GIS systems to reverse nearest neighbor queries and multimedia applications. We develop methods for answering CNN queries with different properties and advantages. We prove the optimality (with respect to I/O cost) of one of the techniques proposed in this paper. The superiority of the proposed technique is shown by a performance analysis.KeywordsNear NeighborConvex PolygonRange QueryQuery ResultQuery PointThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In this paper, we study the problem of continuous monitoring of reverse k nearest neighbors queries in Euclidean space as well as in spatial networks. Existing techniques are sensitive toward objects and queries movement. For example, the results of a query are to be recomputed whenever the query changes its location. We present a framework for continuous reverse k nearest neighbor (RkNN) queries by assigning each object and query with a safe region such that the expensive recomputation is not required as long as the query and objects remain in their respective safe regions. This significantly improves the computation cost. As a byproduct, our framework also reduces the communication cost in client---server architectures because an object does not report its location to the server unless it leaves its safe region or the server sends a location update request. We also conduct a rigid cost analysis for our Euclidean space RkNN algorithm. We show that our techniques can also be applied to answer bichromatic RkNN queries in Euclidean space as well as in spatial networks. Furthermore, we show that our techniques can be extended for the spatial networks that are represented by directed graphs. The extensive experiments demonstrate that our techniques outperform the existing techniques by an order of magnitude in terms of computation cost and communication cost.

Nearest neighbor search is a fundamental problem in computer vision and machine learning. The straightforward solution, linear scan, is both computationally and memory intensive in large scale high-dimensional cases, hence is not preferable in practice. Therefore, there have been a lot of interests in algorithms that perform approximate nearest neighbor (ANN) search. In this paper, we propose a novel addition-based vector quantization algorithm, Asymmetric Mapping Quantization (AMQ), to efficiently conduct ANN search. Unlike existing addition-based quantization methods that suffer from handling the problem caused by the norm of database vector, we map the query vector and database vector using different mapping functions to transform the computation of L-2 distance to inner product similarity, thus do not need to evaluate the norm of database vector. Moreover, we further propose Distributed Asymmetric Mapping Quantization (DAMQ) to enable AMQ to work on very large dataset by distributed learning. Extensive experiments on approximate nearest neighbor search and image retrieval validate the merits of the proposed AMQ and DAMQ.

Broadcasting data with an index is an effective way to disseminate public information to a large clients. For a server, using multiple channels to provide services (e.g., location-based services) makes the broadcast cycle shorter than using one channel. Among location-based services, the k nearest neighbors (k-NN) search is an important one and finds the fc closest objects to a query point in a multi-dimensional space. This paper considers k nearest neighbors search on a broadcast R-tree in a multi-channel environment. We assume that a mobile client can only tune into a specified channel at one time instance. We study how a server generates the broadcast schedules on multiple channels and explore how a client executes the k-NN search on the broadcast. Different broadcast schedules with the client k-NN search processing makes different k-NN search protocols. The objectives of the protocols is to minimize the latency (i.e., the time elapsed between issuing and termination of the query), tuning time (i.e., the amount of time spent on listening to the channel), and the memory usage for k-NN search processing. Last, we present our experiments and the experiment results validate that our mechanisms achieve the objectives.

Developed from Nearest Neighbor (NN) queries, Aggregate Nearest Neighbor (ANN) queries return the object that minimizes an aggregate distance function with respect to a set of query points. Because of the multiple query points, ANN queries are much more complex than NN queries. For optimizing the query processing and improving the query efficiency, many ANN queries algorithms utilizes pruning strategies, with or without an index structure. Obviously, the pruning effect highly depends on the tightness of the bound estimation. In this paper, we figure out a property in vector space and develop some efficient bound estimations for two most popular types of ANN queries. Based on these bounds, we design the indexed and non-index ANN algorithms, and conduct experimental studies. Our algorithms show good performance, especially for high dimensional queries, for both real dataset and synthetic datasets.

A range nearest-neighbor (RNN) query retrieves the nearest neighbor (NN) for every point in a range. It is a natural generalization of point and continuous nearest-neighbor queries and has many applications. In this paper, we consider the ranges as (hyper)rectangles and propose efficient in-memory processing and secondary memory pruning techniques for RNN queries in both 2D and high-dimensional spaces. These techniques are generalized for kRNN queries, which return the k nearest neighbors for every point in the range. In addition, we devise an auxiliary solution-based index EXO-tree to speed up any type of NN query. EXO-tree is orthogonal to any existing NN processing algorithm and, thus, can be transparently integrated. An extensive empirical study was conducted to evaluate the CPU and I/O performance of these techniques, and the study showed that they are efficient and robust under various data sets, query ranges, numbers of nearest neighbors, dimensions, and cache sizes.

The k nearest neighbor (KNN) query, an essential query for information processing in sensor networks, has not received sufficient attention in the research community of sensor networks. In this paper, we examine in-network processing of KNN queries by proposing two alternative algorithms, namely the GeoRouting Tree (GRT) and the KNN Boundary Tree (KBT). The former is based on a distributed spatial index structure and prunes off the irrelevant nodes during query propagation. The latter is based upon ad-hoc geographic routing and first obtains a region within which at least k nearest sensor nodes are enclosed and then decides the k nearest nodes to the query point. We provide an extensive performance evaluation to study the impact of various system factors and protocol parameters. Our results show that GRT yields a good tradeoff between energy consumption and query accuracy in static scenarios. On the other hand, KBT achieves better energy efficiency while being more tolerant to network dynamics.

Spatial interpolations are commonly used in geometric modeling for life science applications. In large-scale spatial interpolations, it is always needed to find a local set of data points for each interpolated point using the k Nearest Neighbor (kNN) search procedure. To improve the computational efficiency of kNN search, spatial decomposition structures such as grids and trees are employed to fastly locate the nearest neighbors. Among those spatial decomposition structures, the uniform grid is the simplest one, and the size of the grid cell could strongly affect the efficiency of kNN search. In this paper, we evaluate the effect of the size of uniform grid cell on the efficiency of kNN search. Our objective is to find the relatively optimal size of grid cell by considering the distribution of scattered points (i.e., the data points and the interpolated points). We employ the Standard Deviation of points’ coordinates to measure the spatial distribution of scattered points. For the irregularly distributed scattered points, we perform several series of kNN search procedures in two dimensions. Benchmark results indicate that: in two dimensions, with the increase of the Standard Deviation of points’ coordinates, the relatively optimal size of the grid cell decreases and eventually converges. The relationships between the Standard Deviation of scattered points’ coordinates and the relatively optimal size of grid cell are also fitted. The fitted relationships could be applied to determine the relatively optimal grid cell in kNN search, and further, improve the computational efficiency of spatial interpolations.