LocationSpark: In-memory Distributed Spatial Query Processing and Optimization

Direction is an important spatial concept that is used in many fields such as geographic information systems(GIS) and image interpretation. It is also frequently used as a selection condition in spatial queries. Previous work has modeled direction as a relational predicate between spatial objects. Conversely, in this paper, we model direction as a new kind of spatial object using the concepts of vectors, points and angles. The basic approach is to model direction as a unit vector. This novel view of direction has several obvious advantages: Being modeled as a spatial object, a direction object can have its own attributes and operation set. Secondly, new spatial data types such as oriented spatial objects and open spatial objects can be defined at the abstract object level. Finally, the object view of direction makes direction reasoning easy and also reduces the need for a large number of inference rules. These features are important in spatial query processing and optimization. The applicability of the direction model is demonstrated by geographic query examples.

In recent years, spatial applications have become more and more important in both scientific research and industry. Spatial query processing is the fundamental functioning component to support spatial applications. However, the state-of-the-art techniques of spatial query processing are facing significant challenges as the data expand and user accesses increase. In this paper we propose and implement a novel scheme (named VegaGiStore) to provide efficient spatial query processing over big spatial data and numerous concurrent user queries. Firstly, a geography-aware approach is proposed to organize spatial data in terms of geographic proximity, and this approach can achieve high aggregate I/O throughput. Secondly, in order to improve data retrieval efficiency, we design a two-tier distributed spatial index for efficient pruning of the search space. Thirdly, we propose an indexing + MapReduce'' data processing architecture to improve the computation capability of spatial query. Performance evaluations of the real-deployed VegaGiStore system confirm its effectiveness.

Several applications of Wireless Sensor Networks (WSN) demand information only from sub-regions of the monitored area. The user establishes the region of interest (RoI) and requires the WSN to collect data only inside this region. This kind of query is called spatial query. The state of the art of spatial query processing considers, in general, that nodes do not fail. However, nodes can be destroyed, interference can deny the communication and nodes can go to sleep mode (turn off the radio in duty cycles) in order to save energy. This work proposes a fault-tolerant energy efficient in-network spatial query processing mechanism for WSN. The proposed mechanism employs failure detection algorithms in order to avoid failed neighbors when forwarding and processing spatial queries. Our contributions are lightweight and resilient because they operate on demand and avoid the use of routing tables.

Although a number of methods have been proposed to process exact match or range queries in P2P network, very few attention has been paid on spatial query process. In this paper, we propose a triangular topology of P2P network and spatial query processing methods on this topology. Each node maintains a set of pointers to neighbor nodes determined by delaunay triangulation. This triangular network topology provides us with an efficient way to find a path from any source node to the destination node or area, and to process spatial queries. We compare two spatial query processing methods based on the triangular network by experiments.

The emerging new services for GPS and mobile users have developed applications that access and exploit spatial objects with new components (e.g. text and social network). Web objects, including blogs, tweets, photos and videos, are embedded into a map by the APIs of map service providers, where textual messages are associated with geographic information. Location-based social networking services, arising from Facebook and Foursquare, allow users to browse and share their traces of locations among the social networks. Among this class of applications, the highlight is that the retrieved spatial objects (e.g. points of interest and moving users) are featured with new components. Integrating such new components into the spatial query processing has produced large amounts of promising results. However, handling new components along with the retrieval of spatial objects increases the complexity of such joint query processing significantly. Thus, management over data from such multiple domains has been received considerable attention from database research community.\n\nIn this thesis, we introduce three interesting problems and study their sophisticated solutions for processing spatial objects with new components: (i) category-aware optimal route query (CORQ), (ii) social and spatial ranking query (SSRQ), and (iii) efficient notification of meeting point (ENMP) query. Our results for (i) and (ii) facilitate the retrieval of spatial objects from multiple domains while our solutions for (iii) provide effective tools for synchronous management of multiple moving users from a social network. Category-aware optimal route queries (CORQ) are generalized from the traveling salesman problem and enable users to retrieve shortest routes covering selected categories. Social and spatial ranking queries (SSRQ) are relevant to spatial object recommendations using social information and allow users to obtain the spatial objects that not only are near their locations but also impress them with high social influence. Efficient notification of meeting point (ENMP) queries are variants of aggregate nearest neighbor queries and provide real-time rearrangement for multiple moving users according to their locations.\n\nQuery processing in such multiple domains is complicated due to the mixture of domain information and their integration within one search. Naïve algorithms for these problems incur either numerous expensive evaluations or massive communication cost, which render them inapplicable to large datasets. Our main research purpose is to design efficient and effective solutions for the proposed problems, that avoid the aforementioned shortcomings of naïve algorithms.

Spatial information sharing and integration has now become an important issue of Geographical Information Science (GIS). Web Service technologies provide a easy and standard way to share spatial resources over network, and grid technologies which aim at sharing resources such as data, storage, and computational powers can help the sharing go deeper. However, the dynamic characteristic of grid brings complexity to spatial query optimization which is more stressed in GIS domain because spatial operations are both CPU intensive and data intensive. To address this problem, a new grid framework is employed to provide standard spatial services which can also manage and report their state information to the coordinator which is responsible for distributed spatial query optimization.

Modern location-based applications rely extensively on the efficient processing of spatial data and queries. Spatial query engines are commonly engineered as an extension to a relational database or a cluster-computing framework. Large parts of the spatial processing runtime is spent on evaluating spatial predicates and traversing spatial indexing structures. Typical high-level implementations of these spatial structures incur significant interpretive overhead, which increases latency and lowers throughput. A promising idea to improve the performance of spatial workloads is to leverage native code generation techniques that have become popular in relational query engines. However, architecting a spatial query compiler is challenging since spatial processing has fundamentally different execution characteristics from relational workloads in terms of data dimensionality, indexing structures, and predicate evaluation. In this paper, we discuss the underlying reasons why standard query compilation techniques are not fully effective when applied to spatial workloads, and we demonstrate how a particular style of query compilation based on techniques borrowed from partial evaluation and generative programming manages to avoid most of these difficulties by extending the scope of custom code generation into the data structures layer. We extend the LB2 main-memory query compiler, a relational engine developed in this style, with spatial data types, predicates, indexing structures, and operators. We show that the spatial extension matches the performance of specialized library code and outperforms relational and map-reduce extensions.

A partial index for distributed broadcasting in wireless mobile networks

The broadcasting of spatial data together with an index structure is an effective way of disseminating data in a wireless mobile environment. Mobile clients requesting data tune into a continuous broadcast only when spatial data of interest and relevance is available on the channel and thus minimize their power consumption. A mobile client experiences latency (time elapsed from requesting to receiving data) and tuning time (the amount of time spent listening to the channel). This paper studies the execution of spatial queries on broadcasted tree-based spatial index structures. The focus is on queries that require a partial traversal of the spatial index, not only a single-path root-to-leaf search. We present techniques for processing spatial queries while mobile clients are listening to a broadcast of the tree. Our algorithms can handle clients with limited memory, trees broadcast with a certain degree of replication of index nodes, and algorithms executed at the clients may employ different data structures. Experimental work on R*-trees shows that these techniques lead to different tuning times and different latencies. Our solutions also lead to efficient methods for starting the execution of a query in the middle of a broadcast cycle. Spatial query processing in a multiple channel environment is also addressed.

Machine learning (ML) and deep learning (DL) techniques are increasingly applied to produce efficient query optimizers, in particular in regards to big data systems. The optimization of spatial operations is even more challenging due to the inherent complexity of such kind of operations, like spatial join or range query, and the peculiarities of spatial data. Although a few ML-based spatial query optimizers have been proposed in literature, their design limits their use, since each one is tailored for a specific collection of datasets, a specific operation, or a specific hardware setting. Changes to any of these will require building and training a completely new model which entails collecting a new very large training dataset to obtain a good model. This article proposes a different approach which exploits the use of the novel notion of spatial embedding to overcome these limitations. In particular, a preliminary model is defined which captures the relevant features of spatial datasets, independently from the operation to be optimized and in an unsupervised manner. This model is trained with a large amount of both synthetic and real-world data, with the aim to produce meaningful spatial embeddings. The construction of an embedding model could be intended as a preliminary step for the optimization of many different spatial operations, so the cost of its building can be compensated during the subsequent construction of specific models. Indeed, for each considered spatial operation, a specific tailored model will be trained but by using spatial embeddings as input, so a very little amount of training data points is required for them. Three peculiar operations are considered as proof of concept in this article: range query, self-join, and binary spatial join. Finally, a comparison with an alternative technique, known as transfer learning, is provided and the advantages of the proposed technique over it are highlighted.

Location-based services (LBSs) have entered into people’s daily lives with the evolution of telecommunication technologies. LBSs are the services that consider the locations of users to provide tailored information. We study a variety of location-based queries in urban environments. Mainly, we focus on processing such queries in the indoor space motivated by the abundance of applications of indoor LBSs and lack of existing studies supporting indoor spatial queries. Although the spatial query processing has been extensively studied for outdoor space, these techniques are not efficient when applied to indoor venues due to inherent differences in the topology of indoor space.

Spatial data are particularly useful in mobile environments. However, due to the low bandwidth of most wireless networks, developing large spatial database applications becomes a challenging process. In this paper, we provide the first attempt to combine two important techniques, multiresolution spatial data structure and semantic caching, towards efficient spatial query processing in mobile environments. Based on the study of the characteristics of multiresolution spatial data (MSD) and multiresolution spatial query, we propose a new semantic caching model called Multiresolution Semantic Caching (MSC) for caching MSD in mobile environments. MSC enriches the traditional three-category query processing in semantic cache to five categories, thus improving the performance in three ways: 1) a reduction in the amount and complexity of the remainder queries; 2) the redundant transmission of spatial data already residing in a cache is avoided; 3) a provision for satisfactory answers before 100% query results have been transmitted to the client side. Our extensive experiments on a very large and complex real spatial database show that MSC outperforms the traditional semantic caching models significantly.KeywordsQuery ProcessingQuery ResultMobile EnvironmentSpatial ObjectMobile ClientThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Recent advances in low-power sensing devices coupled with the widespread availability of wireless ad-hoc networks have fueled the development of sensor networks. These are typically deployed over wide areas to gather data in the environment and monitor events of interest. The ability to run spatial queries is extremely useful for sensor networks. Spatial query execution has been extensively studied in the context of centralized spatial databases; however because of the energy and bandwidth limitation of sensor nodes these solutions are not directly applicable to the sensor network. In this paper we propose a scalable and distributed way of spatial query execution in sensor networks. We develop a distributed spatial index over the sensor nodes that is used in processing spatial queries in a distributed fashion. We evaluate the behavior of our approach and show that our mechanism provides an efficient and scalable way to run spatial queries over sparse and dense sensor networks.

Relational database systems internally construct a physical query execution plan (QEP) that specifies exactly how to compute a desired result. However, choosing a QEP involves determining a specific join order, deciding how to access base relations, specifying concrete physical implementations to compute the algebraic operations defined by the given SQL query, and much more. In general, choosing the optimal QEP w.r.t. a predefined cost model is a hard optimization task, referred to as query optimization problem (QOP), that requires super-exponential time in the worst-case. Even though query optimization is a fundamental problem that has been studied for decades now, related work often focuses only on a specific subtask like join ordering. Furthermore, by inspecting open-source database systems, fundamentally different query optimization strategies can be observed. These strategies exhibit vastly different optimization times while having a major impact on the resulting QEP qualities. In this work, we revisit two conceptually different approaches to solve query optimization, namely split and holistic. We discuss their advantages and disadvantages and present a detailed experimental evaluation in our research database system mu t able. Additionally, we propose a hybrid strategy called top-k that is able to rediscover the holistically optimal QEPs while being significantly closer to the optimization time of split.

The management of complex spatial objects in applications, such as geography and cartography, imposes stringent new requirements on spatial database systems, in particular on efficient query processing. As shown before, the performance of spatial query processing can be improved by decomposing complex spatial objects into simple components. Up to now, only decomposition techniques generating a linear number of very simple components, e.g. triangles or trapezoids, have been considered. In this paper, we will investigate the natural trade-off between the complexity of the components and the redundancy, i.e. the number of components, with respect to its effect on efficient query processing. In particular, we present two new decomposition methods generating a better balance between the complexity and the number of components than previously known techniques. We compare these new decomposition methods to the traditional undecomposed representation as well as to the well-known decomposition into convex polygons with respect to their performance in spatial query processing. This comparison points out that for a wide range of query selectivity the new decomposition techniques clearly outperform both the undecomposed representation and the convex decomposition method. More important than the absolute gain in performance by a factor of up to an order of magnitude is the robust performance of our new decomposition techniques over the whole range of query selectivity.