Classic distance join queries using compact data structures

We describe a powerful framework for designing efficient batch algorithms for certain large-scale dynamic problems that must be solved using external memory. The class of problems we consider, which we call colorable eztemaldecomposable problems, include rectangle intersection, orthogonal line segment intersection, range searching, and point location. We are particularly interested in these problems in two and higher dimensions. They have numerous applications in geographic information systems (GIS), spatial databases, and VLSI and CAD design. We present simplied algorithms for problems previously solved by more complicated approaches (such as rectangle intersection), and we present efficient algorithms for problems not previously solved in an efficient way (such as point location and higherdimensional versions of range searching and rectangle intersection). communication is the bottleneck in many large-scale applications such as those arising in VLSI and CAD design, spatial databases, and geographic information systems (GIS). In this paper we consider I/O-efficient algorithms for batched searching problems. We consider both batched static and batched dynamic problems, and use the correspondence which often exists between a ddimensional static problem and a (dl)-dimensional dynamic problem to obtain a number of new d-dimensional algorithms. We give experimental results concerning the running time for our approach applied to the red-blue rectangle intersection problem, which is a key component of the extremely important database operation spatial join. Our algorithm scales well with the problem size, and for large problems sizes it greatly outperforms the well-known sweepline approach. One prominent example of the problems we consider is the rectangle intersection problem, which is a key component in VLSI design rule checking [31] and in the extremely important database operation spatial join [34]. We illustrate the practical significance of our algorithms by comparing the empirical performance of our algorithm for this problem with the well-known sweepline algorithm developed for internal memory. 1.1 Problem definition and memory model

Scalable and queryable compressed storage structure for raster data

Improving Distance-Join Query processing with Voronoi-Diagram based partitioning in SpatialHadoop

Retrieval of surrounding data objects has many potential applications in spatial databases including nearby point-of-interest retrieval surrounding a user and digital gaming. This paper presents a novel query, called direction-based spatial skyline (DSS), for retrieving surrounding objects in spatial databases. A DSS query retrieves all surrounding objects from a spatial database that are not directionally dominated by other data objects w.r.t. a given query object. The proposed DSS query is rotationally invariant, fair and stable. The paper also presents efficient algorithms for processing DSS queries in spatial databases by designing novel data pruning techniques using R-Tree data indexing scheme. We conduct extensive experiments using both real and synthetic datasets to demonstrate the effectiveness and efficiency of the proposed algorithms.

Retrieval of arbitrary-shaped surrounding data objects has many potential applications in spatial databases including nearby arbitrary-shaped object-of-interests retrieval surrounding a user. In this paper, we propose directional zone concept to determine directional similarity among spatial data objects. Then, we propose a novel query, called direction-based spatial skyline (DSS), which retrieves non-dominated arbitrary-shaped surrounding data objects in spatial databases for a user. The proposed DSS query is rotationally invariant as well as fair. We develop efficient algorithms for processing DSS queries in spatial databases by designing novel data pruning techniques using R-Tree data indexing scheme. Finally, we demonstrate the effectiveness and efficiency of our approach by conducting extensive experiments with real datasets.

A graph is called hidden if the edges are not explicitly given and edge probe tests are required to detect the presence of edges. This paper studies the k most connected vertices (kMCV) problem on hidden bipartite graphs, which has applications in spatial databases, graph databases, and bioinformatics. There is a prior work on the kMCV problem, which is based on the “2-vertex testing” model, i.e., an edge probe test can only reveal the existence of an edge between two individual vertices. We study the kMCV problem, in the context of a more general edge probe test model called “group testing.” A group test can reveal whether there exists some edge between a vertex and a group of vertices. If group testing is used properly, a single invocation of a group test can reveal as much information as multiple invocations of 2-vertex tests. We discuss the cases and applications where group testing could be used, and present an algorithm, namely, GMCV, that adaptively leverages group testing to solve the kMCV problem.

In the study of sequence stratigraphy and litho-paleogeography, quantitative analysis, precise calculation and detailed comparison of tremendous geological data, such as field profiles, logging records and seismic curves from different areas, are the basic requirements. In order to obtain a more reliable and precise result, this paper presents a novel method that combines spatial database analysis with the single-factor mapping technology to establish sequence stratigraphical succession and to map the Ordovician litho-paleogeography of the Ordos Basin, one of the largest oil-gas bearing basins in North China Platform. By using this method, all of the related basic geological data can be quantitatively analyzed and effectively managed. Various attributes of the basic stratigraphic units and their characters, such as sequence thickness, penecontemporaneous dolostone content, shallow water parget content, and terrigenous material content, can be fully utilized statistically in facies analysis and in mapping. Based on this analysis, this paper has be exerted single-factor isopachous mapping quantitatively for each of the Ordovician sequences in the basin, and finally synthesized multiple factors to reconstruct the litho-paleogeography for each of the sequence intervals. The study shows that the proposed method is quite effective and has a much higher resolution in recognizing litho-paleogeographic subunits compared with traditional ways. For example, in one of the Middle Ordovician sequence intervals (SQ19 in the Lower Majiagou Formation) of the Ordos Basin, the authors have successfully developed a mathematical formula to divide the distribution of various facies units substantially, such as old lands, submarine uplifts, supratidal zones, intertidal zones and subtidal zones.

In precision agriculture, the integration of spatial information through Geographic Information System (GIS) plays a crucial role in optimizing agricultural operations. However, the increasing scale of spatial data poses computational challenges to traditional processor-centric solutions used in GIS due to memory limitations: limited memory capacity and low bandwidth. An architecture shift from processor-centric to memory-centric is necessary for modern data-intensive applications. To overcome the memory bottleneck issues, this paper introduces a novel, callable, pipelined solution in Processing Near Memory (PNM) configuration with large memory capacity and high inner bandwidth tailored for memory-intensive precision agriculture applications. The solution leverages a scalable Computational Memory Module (CMM) with a Geolocation Query Engine (GQE), which addresses the k-Nearest Neighbors (kNN) search problem, a fundamental computation used in GIS for various geolocation queries. Our experimental evaluation, conducted on three distinct datasets, showcased the effectiveness of our proposed CMM. These datasets include a substantial Wheat big data set consisting of 17,847 daily trajectories, as well as smaller Paddy and Corn datasets containing 1,634 and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{1, 2 9 0}$</tex> daily trajectories, respectively. In these experiments, our CMM consistently achieved an impressive <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{9 5 \%}$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{9 9 \%}$</tex> average reduction in query times and energy consumption compared to conventional processor-centric approaches, respectively. By efficiently managing large-scale spatial data, the proposed solution empowers precision agriculture, contributing to increased productivity, sustainability, and profitability in modern agricultural practices.

In this paper we propose the use of the compact data structure k2-treap to process data cubes of Data Warehouses (DWs) into main memory. Compact data structures are data structures that allow compacting the data without losing the capacity of querying them in their compact form. A DW is a data repository to store historical data for decision support, and consists of dimensions and facts. The former are an abstract concept that groups data with a similar meaning, they are modelled as hierarchies of levels, which contain elements. The latter are quantitative data associated to dimensions. A data cube is a typical way to retrieve facts at different levels of granularity (through navigation on dimensions hierarchies). A DW can store terabytes of data, thus the efficient processing of data cubes is key in OLAP (On-line Analytical Processing). We show that by using a compact representation of data cubes and bitmaps to represent dimensions we are able to improve the use of space in main memory, and achieve better performance for query processing.

This paper deals with uncertainty modeling in spatial object-relational databases by the use of Structured Query Language (SQL). The fundamental principles of uncertainty modeling by fuzzy sets are applied in the area of geographic information systems (GIS) and spatial databases. A spatial database system includes types of spatial data and implements the spatial extension of SQL. The implementation of the principles of fuzzy logic to spatial databases brings an opportunity for the efficient processing of uncertain data, which is important, especially when using various data sources (e.g., multi-criteria decision making (MCDM) on the basis of heterogeneous spatial data resources). The modeling and data processing of uncertainties are presented in relation to the applicable International Organization for Standardization (ISO) standards (standards of the series 19100 Geographic information) and the relevant specifications of the Open Geospatial Consortium (OGC). The fuzzy spatial query approach is applied and tested on a case study with a fundamental database for GIS in Slovakia.

The efficient management of interval sequences represents a core requirement for many temporal and spatial database applications. With the Relational Interval Tree (RI-tree), an efficient access method has been proposed to process intersection queries of spatial objects encoded by interval sequences on top of existing object-relational database systems. This paper complements that approach by effective and efficient models to estimate the selectivity and the I/O cost of interval sequence intersection queries in order to guide the cost-based optimizer whether and how to include the RI-tree into the execution plan. By design, the models immediately fit to common extensible indexing/ optimization frameworks, and their implementations exploit the built-in statistics facilities of the database server. According to our experimental evaluation on an Oracle database, the average relative error of the estimated query results and costs lies in the range of 0% to 32%, depending on the size and the structural complexity of the query objects.

The classic string indexing problem is to preprocess a string S into a compact data structure that supports efficient pattern matching queries. Typical queries include existential queries (decide if the pattern occurs in S), reporting queries (return all positions where the pattern occurs), and counting queries (return the number of occurrences of the pattern). In this paper we consider a variant of string indexing, where the goal is to compactly represent the string such that given two patterns \(P_1\) and \(P_2\) and a gap range \({[}\alpha , \beta ]\) we can quickly find the consecutive occurrences of \(P_1\) and \(P_2\) with distance in \({[}\alpha , \beta ]\), i.e., pairs of subsequent occurrences with distance within the range. We present data structures that use linear space and query time \({\widetilde{O}}(|P_1|+|P_2|+n^{2/3})\) for existence and counting and \({\widetilde{O}}(|P_1|+|P_2|+n^{2/3}\hbox {occ}^{1/3})\) for reporting. We complement this with a conditional lower bound based on the set intersection problem showing that any solution using \({\widetilde{O}}(n)\) space must use \({\widetilde{\Omega }}(|P_1| + |P_2| + \sqrt{n})\) query time. To obtain our results we develop new techniques and ideas of independent interest including a new suffix tree decomposition and hardness of a variant of the set intersection problem.

Because of the characteristics of web spatial databases, restrictions on the applications of web spatial databases focus on accessing speed. The main solution for improving accessing speed is caching and prefetching. The existing prefetching schemes prefetch future spatial objects based on the distances of spatial locations rather than accessing contents. For the improvement of the accuracy of prefetching, we propose a clustering algorithm-clust Voronoi, which based on the web user logs. And then, we apply clust Voronoi to the prefetching scheme, named clust Pref. This prefetching scheme can automatically determine the number of clusters and give the real case of spatial databases accessing. Through a simulation environment, using a real data set, we draw a conclusion that this prefetch scheme can effectively improve the spatial databases accessing performance.

The plane sweep algorithm is a foundational algorithm for many geometric and spatial computations; thus, improvements in the algorithm have far reaching effects in many applications. In this paper, we examine the performance of the serial plane sweep algorithm, and introduce a parallelization technique for the algorithm that is suitable to multi-core computers. The parallelization technique is described in detail and shown to be correct. Finally, experiments are performed using multiple data sets on computers with varying numbers of processing cores. We show that our algorithm achieves significant speedups over the serial plane sweep algorithm using a wide range of input parameters; thus, our algorithm achieves good performance without the need to tune the input parameters for specific input cases.

Combined with the actual requirements of gas pipe network management, based on the analysis of significance and effect on establishing geographic information system in Daqing nature-gas company, overall design of geographic information system in Daqing oil field gas pipe network, network structure and the analysis of spatial data and the attribute data are accomplished, what's more, graphic and attribute data is unified to store, process and analyse, using geographic information system as technical support, at the same time, the spatial database is established, which is for the maintenance and planning of gas pipe network, piping design, pipeline construction, the completion of information, the repairing decision, as well as, inquiring the relative attribute data. This paper introduces the data and network structure, in addition to, security of geographic information system in Daqing oil field gas pipe network specifically, the system chooses MS Windows 2000 (Advanced) Server or UNIX as operating system, furthermore, using other geographic information system softwares to build geographic information development platform, giving priority to the ArcGIS of ESRI company. The result shows that the development of project has reached the expected goal, laying the solid foundation for system subsequent development and further researching work.