The DR-tree: A Main Memory Data Structure for Complex Multi-dimensional Objects

Analysis of two-step index structure for complex spatial objects

Data warehouse systems store and process huge amounts of data. The main application of data warehouses is to support decision making process and to provide reports of various kinds. Those tasks require calculating aggregates, averages and other values. The calculations must be performed fast because it is very common that user wants to work interactively with a data warehouse system. There are two ways to increase the data warehouse performance. The first is to increase computing power by improving hardware infrastructure, and the second, to apply advanced, dedicated solutions like materialization, indexing etc. In this paper we present an indexing structure designed for spatial telemetric objects and suitable for processing streams of data. The presented solution is a result of integration of a few mechanisms. For spatial indexing we use CHR-Tree index which idea is based on the R-Tree index. To process data streams we use Materialized Aggregate List, a solution which allows processing data streams of any length and supports optional stream materialization. In the paper we present how those solutions are integrated into one indexing and caching structure. We also include efficiency test results showing the solution efficiency and scalability.

With the widespread application of Geographic Information System (GIS), three-dimensional spatial data, as the reflection of the real world entity, has an increasing amount of data, and the phenomenon of uneven data distribution appears. If a single spatial index structure is used to store and manage these data, there will be a waste of storage space and low query efficiency. A hybrid index structure based on 3D multi-level adaptive grid and R+ tree was proposed to solve these problems. The index structure was mainly composed of two structures, multi-level grid and R+ tree. Firstly, the data set was processed by the multi-level automatic grid algorithm based on normal distribution, and the length, width and height of the grid were obtained. Secondly, a multi-level adaptive grid structure was used to partition the data space quickly and effectively, and the advantage of zero overlap of the intermediate nodes of the R+ tree was used for efficient indexing. Finally, the maintenance and query algorithms of the index structure were given in detail, which solved the problem of low index establishment and retrieval efficiency under the condition of uneven distribution of massive data sets. In this paper, a data set subject to Gauss distribution was used to simulate the distribution of three-dimensional data. Through a large number of experimental comparison tests, it was proved that the hybrid index structure based on 3D multi-level adaptive grid-R+ tree proposed in this paper had good performance in both index structure construction and query in the case of massive data sets or uneven data distribution.

In main-memory resident index structures, secondary cache misses have a considerable effect on the performance of index structures. Recently, several main-memory resident index structures that consider cache have been proposed to reduce the impact of secondary cache misses. However, they still suffer from full secondary cache misses whenever visiting each level of index trees. In this paper, we propose a new index structure that minimizes the total amount of cache miss latency. The proposed index structure prefetches the grandchildren of a current node. The basic structure of the proposed index structure is from CSB+-Tree that uses the concept of the node group to increase fan-out. However the insert algorithm of the proposed index structure reduces the cost of a split significantly. Also, we show the superiority of our algorithm through performance evaluation.

In contrast to regular queries that are evaluated only once, a continuous query remains active over a period of time and has to be continuously evaluated to provide up to date results. We propose a method for continuous range query processing for different types of queries, characterized by mobility of objects and/or queries which follow paths in an underlying spatial network. The method assumes an available 2D indexing scheme for indexing spatial network data. An appropriately extended R∗-tree provides matching of queries and objects according to their locations on the network or their network routes. The method introduces an additional prerefinement step which generates main memory data structures to support efficient, incremental reevaluation of continuous range queries in periodically performed refinement steps.

The importance of multidimensional index structures to numerous emerging database applications is well established. However, before these index structures can be supported as access methods (AMs) in a “commercial-strength” database management system (DBMS), efficient techniques to provide transactional access to data via the index structure must be developed. Concurrent accesses to data via index structures introduce the problem of protecting ranges specified in the retrieval from phantom insertions and deletions (the phantom problem ). This paper presents a dynamic granular locking approach to phantom protection in Generalized Search Trees(GiSTs), an index structure supporting an extensible set of queries and data types. The granular locking technique offers a high degree of concurrency and has a low lock overhead. Our experiments show that the granular locking technique (1) scales well under various system loads and (2) similar to the B-tree case, provides a significantly more efficient implementation compared to predicate locking for multidimensional AMs as well. Since a wide variety of multidimensional index structures can be implemented using GiST, the developed algorithms provide a general solution to concurrency control in multidimensional AMs. To the best of our knowledge, this paper provides the first such solution based on granular locking.

A flexible indexing structure supporting multi-attribute database applications: MAIN

Chapter 7 - Maintaining timers

The shielded computation of hardware-based trusted execution environments such as Intel Software Guard Extensions (SGX) can provide secure cloud computing on remote systems under untrusted privileged system software. However, hardware overheads for securing protected memory restrict its capacity to a modest size of several tens of megabytes, and more demands for protected memory beyond the limit cause costly demand paging. Although one of the widely used applications benefiting from the enhanced security of SGX, is the in-memory key-value store, its memory requirements are far larger than the protected memory limit. Furthermore, the main data structures commonly use fine-grained data items such as pointers and keys, which do not match well with the coarse-grained paging of the SGX memory extension technique. To overcome the memory restriction, this paper proposes a new in-memory key-value store designed for SGX with application-specific data security management. The proposed key-value store, called ShieldStore, maintains the main data structures in unprotected memory with each key-value pair individually encrypted and integrity-protected by its secure component running inside an enclave. Based on the enclave protection by SGX, ShieldStore provides secure data operations much more efficiently than the baseline SGX key-value store, achieving 8--11 times higher throughput with 1 thread, and 24--30 times higher throughput with 4 threads.

An increasing number of emerging web database applications deal with large georeferenced data sets. However, exploring these large data sets through spatial queries can be very time and resource intensive. The need for interactive spatial queries has arisen in many applications such as Geographic Information Systems (GIS) for efficient decision-support. In this paper, we propose a new interactive spatial query processing technique for GIS. We present a family of the Incremental Refining Spatial Join (IRSJ) algorithms that can be used to report incrementally refined running estimates for aggregate queries while simultaneously displaying the actual query result tuples of the data sets sampled so far. Our goal is to minimize the time until an acceptably accurate estimate of the query result is available (to users) measured by a confidence interval. Our approach enables more interactive data exploration and analysis. While similar work has been done in relational databases, to the best of our knowledge, this is the first work using this approach in GIS. We investigate and evaluate different sampling methodologies through extensive experimental performance comparisons. Experiments on both real and synthetic data show an order of magnitude response time improvement relative to the final answer obtained when using a full R-tree join. We also show the impact of different index structures on the performance of our algorithms using three known sampling methods.

Spatial indexing technology is widely used in Geographic Information Systems (GIS) and spatial databases. As a data retrieval technology, spatial indexing is becoming increasingly important in the big-data age. The purpose of this study is to propose a unified indexing strategy for the mixed data of a future marine GIS. First, data organisation of the system is described. Second, the display condition of each type of data is introduced. These conditions are the basis for the construction of a unified indexing structure. Third, a unified indexing structure for mixed data is presented. The construction process and the search method of the indexing structure are described. Finally, we implement the indexing strategy in our system “Automotive Intelligent Chart Three-dimensional Electronic Chart Display and Information Systems” (AIC 3D ECDIS). Our strategy can provide fast and integrated data retrieval. The spatial indexing strategy we propose breaks through the limitation of data types in our system. It can also be applied in other GIS systems. With the advent of the big-data age, mixed data indexing will become more and more important.

Given a very large multimedia database, how to process k-nearest-neighbor queries efficiently? While the sequential scan is one of the most obvious solutions for small-to-moderate multimedia databases, it becomes practically infeasible when the database size grows. Concomitant with the volume and velocity of data, multimedia databases are frequently endowed with a complex distance-based similarity model that supports content-based data access in an adjustable and adaptive manner. Typical for many state-of-the-art distance-based similarity models is an at least quadratic computation time complexity for a single distance evaluation between two multimedia objects. Thus the search for the most query-like multimedia objects is still one of the major challenges.In this paper, we address the problem of optimal k-nearest-neighbor query processing via multiple lower bound approximations in very large multimedia databases. To this end, we propose the concepts of filter-optimality and refinement-optimality and present the Cascading Multi-Step Algorithm and the Interleaved Multi-Step Algorithm for fast query processing. Besides the algorithms’ properties, we study their query processing performance with respect to the number of CPU and I/O operations on large-scale benchmark multimedia databases. Our performance analysis shows how to process k-nearest-neighbor queries in multimedia databases efficiently and provides a guide for further research.

It is frequently the case that spatial queries require a result set of objects whose - however this may be more precisely defined - is the same as that of the query window. In this paper we present an approach which considerably improves query performance in such cases. By adding a dimension to the schema we make the index structure explicitly aware of the of a spatial object. The additional dimension causes the index structure to cluster objects not only by geographic location but also by scale. By matching scales of the query window and the objects, the query then automatically considers only relevant objects. Thus, for example, a query window encompassing an entire world map of political boundaries might return only national borders. Note that scale is not necessarily synonymous with size. This approach improves performance by both narrowing the initial selection criteria and by eliminating the need for subsequent filtering of the query result. In our performance measurements on databases with up to 40 million spatial objects, the introduction of a dimension decreased I/O by up to 4 orders of magnitude. The performance gain largely depends on the object distribution.We investigate a broad set of parameters that affect performance and show that many typical applications could benefit considerably from this technique. Its scalability is demonstrated by showing that the benefit increases with the size of the query and/or of the database. The technique is simple to apply and can be used with any multidimensional index structure that can index spatial extents and can be efficiently generalized to three or more dimensions. In our tests we have used the BANG index structure.

Based on measles case data from pediatricians in Lower Saxony, a simulation study was conducted in order to reproduce the current vaccination-disease relationship and afrerwards predict the impact of different vaccination strategies. The programs were written in S-Plus and make use of the language's array processing facilities. The process of infection was modeled by means of an age-structured compartmental model. The main data structure is a two dimensional array, representing the complete population, partitioned according to age cohorts and the compartments, which represent the stages of infection. The flow of individuals through the compartments is modeled weekly and performed by means of array operations. The number of new infections in each age cohort is simulated as a Poisson random variable, depending on the number of infectious individuals of any age and the transmission probability between ages. The transmission coefficients are organized in a matrix enabling the transmission process to be computed by matrix multiplication.

Content Centric Networking (CCN) is a novel network paradigm in which the communication process focusses on content rather than the host. In the current network architecture, routers forward the incoming packets based on the routing information that already had. Whereas, forwarding decisions in CCN are taken depending on networks situation. The contents in CCN are retrieved depending on their names which are URL like names that are composing of a number of string parts separated by’/’. These names are stored inside a specific data structure, called Forwarding Information Base (FIB), inside CCN routers and they are used to forward any incoming packet. Therefore, the main two challenges that face the design of FIB table are: search speed and storage utilization. Consequently, in this paper, we propose a new name matching technique (named BF-QF FIB) to design and implement a FIB table in CCN routers in order to decrease the storage utilization and to increase the lookup speed. This technique utilizes two kinds of query data structures: Bloom filter (BF) and Quotient filter (QF) as its main data structures. The utilization of these two data structures will ensure low memory usage and high lookup speed. The results of our evaluation show that BF-QF FIB can guarantee high search rate and offer perfect scalability to large FIB tables.