Approximate query answering in sensor networks with hierarchically distributed caching

As database users adopt a query language to obtain information from a database, a more intelligent query answering system is increasingly needed that cooperates with the users to provide informative responses by understanding the intent behind a query. The effectiveness of decision support would improve significantly if the query answering system returned approximate answers rather than a null information response when there is no matching data available. Even when exact answers are found, neighboring information is still useful to users if the query is intended to explore some hypothetical information or abstract general fact. This chapter proposes an abstraction hierarchy as a framework to practically derive such approximate answers from ordinary everyday databases. It provides a knowledge abstraction database to facilitate the approximate query answering. The knowledge abstraction database specifically adopts an abstraction approach to extract semantic data relationships from the underlying database, and uses a multi-level hierarchy for coupling multiple levels of abstraction knowledge and data values. In cooperation with the underlying database, the knowledge abstraction database allows the relaxation of query conditions so that the original query scope can be broadened and thus information approximate to exact answers can be obtained. Conceptually abstract queries can also be posed to provide a less rigid query interface. A prototype system has been implemented at KAIST and is being tested with a personnel database system to demonstrate the usefulness and practicality of the knowledge abstraction database in ordinary database systems.

As more and more data become accessible, ranking query semantics such as ranked retrieval, possibly combined with Boolean query conditions, has gained a lot of attention lately. As the formulation of such queries is known to be difficult, we aim at providing quick approximate answers as cues for interactive query refinements. Toward the goal, we study approximate answering techniques for Boolean+ranking queries. While approximate query answering has been studied for Boolean-only queries, we observed that a straightforward extension of this work for advanced queries incurs prohibitive overheads. We thus propose a systematic framework which significantly outperforms such a naive extension. We also empirically validate the effectiveness and efficiency of our framework.

In large data warehousing environments, it is often advantageous to provide fast, approximate answers to complex aggregate queries based on statistical summaries of the full data. In this paper, we demonstrate the difficulty of providing good approximate answers for join-queries using only statistics (in particular, samples) from the base relations. We propose join synopses as an effective solution for this problem and show how precomputing just one join synopsis for each relation suffices to significantly improve the quality of approximate answers for arbitrary queries with foreign key joins. We present optimal strategies for allocating the available space among the various join synopses when the query work load is known and identify heuristics for the common case when the work load is not known. We also present efficient algorithms for incrementally maintaining join synopses in the presence of updates to the base relations. Our extensive set of experiments on the TPC-D benchmark database show the effectiveness of join synopses and various other techniques proposed in this paper.

In decision support applications, the ability to provide fast approximate answers to aggregation queries is desirable. One commonly-used technique for approximate query answering is sampling. For many aggregation queries, appropriately constructed biased (non-uniform) samples can provide more accurate approximations than a uniform sample. The optimal type of bias, however, varies from query to query. In this paper, we describe an approximate query processing technique that dynamically constructs an appropriately biased sample for each query by combining samples selected from a family of non-uniform samples that are constructed during a pre-processing phase. We show that dynamic selection of appropriate portions of previously constructed samples can provide more accurate approximate answers than static, non-adaptive usage of uniform or non-uniform samples.

Aqua is a system for providing fast, approximate answers to aggregate queries, which are very common in OLAP applications. It has been designed to run on top of any commercial relational DBMS. Aqua precomputes synopses (special statistical summaries) of the original data and stores them in the DBMS. It provides approximate answers along with quality guarantees by rewriting the queries to run on these synopses. Finally, Aqua keeps the synopses up-to-date as the database changes, using fast incremental maintenance techniques.

OnLine Analytical Processing (OLAP) research issues (Gray, Chaudhuri, Bosworth, Layman, Reichart & Venkatrao, 1997) such as data cube modeling, representation, indexing and management have traditionally attracted a lot of attention from the Data Warehousing research community. In this respect, as a fundamental issue, the problem of efficiently compressing the data cube plays a leading role, and it has involved a vibrant research activity in the academic as well as industrial world during the last fifteen years. Basically, this problem consists in dealing with massive-in-size data cubes that make access and query evaluation costs prohibitive. The widely accepted solution consists in generating a compressed representation of the target data cube, with the goal of reducing its size (given an input space bound) while admitting loss of information and approximation that are considered irrelevant for OLAP analysis goals (e.g., see (Cuzzocrea, 2005a)). Compressed representations are also referred-in-literature under the term "synopsis data structures", i.e. succinct representations of original data cubes introducing a limited loss of information. Benefits deriving from the data cube compression approach can be summarized in a relevant reduction of computational overheads needed to both represent the data cube and evaluate resource-intensive OLAP queries, which constitute a wide query class allowing us to extract useful knowledge from huge amounts of multidimensional data repositories in the vest of summarized information (e.g., aggregate information based on popular SQL aggregate operators such as SUM, COUNT, AVG etc), otherwise infeasible by means of traditional OLTP approaches. Among such queries, we recall: (i) range-queries, which extract a sub-cube bounded by a given range; (ii) top-k queries, which extract the k data cells (such that k is an input parameter) having the highest aggregate values; (iii) iceberg queries, which extract the data cells having aggregate values above a given threshold. This evidence has given raise to the proliferation of a number of Approximate Query Answering (AQA) techniques, which, based on data cube compression, aim at providing approximate answers to resource-intensive OLAP queries instead of computing exact answers, as decimal precision is usually negligible in OLAP query and report activities (e.g., see (Cuzzocrea, 2005a)). Starting from these considerations, in this article we first provide a comprehensive, rigorous survey of main data cube compression techniques, i.e. histograms, wavelets, and sampling. Then, we complete our analytical contribution with a detailed theoretical review focused on spatio-temporal complexities of these techniques. Request access from your librarian to read this chapter's full text.

AMID: Approximation of MultI-measured Data using SVD

In this paper an approach for dynamic sensor selection in large video-based sensor networks for the purpose of multi-camera object tracking is presented. The sensor selection approach is based on computational geometry algorithms and is able to determine task-relevant cameras (camera cluster) by evaluation of geometrical attributes, given the last observed object position, the sensor configurations and the environment model. Hereby, a special goal of this algorithm is to determine the minimum number of sensors needed to relocate an object, even if the object is temporarily out of sight (e.g., by non-overlapping sensor coverage). It will be shown that the algorithm enables self-organizing tracking approaches to perform optimal camera selection in a highly efficient way. In particular, the approach is applicable to very large camera networks and leads to a highly reduced network and processor load for multi-camera tracking.

REHIST: Relative Error Histogram Construction Algorithms

Estimating selectivities remains a critical task in query processing. Optimizers rely on the accuracy of selectivities when generating execution plans and, in approximate query answering, estimated selectivities affect the quality of the result. Many systems maintain synopses, e.g., histograms, and, in addition, provide sampling facilities. In this paper, we present a novel approach to combine knowledge from synopses and sampling for the purpose of selectivity estimation for conjunctive queries. We first show how to extract information from synopses and sampling such that they are mutually consistent. In a second step, we show how to combine them and decide on an admissible selectivity estimate. We compare our approach to state-of-the-art methods and evaluate the strengths and limitations of each approach.

Fast estimates for aggregate queries are useful in database query optimization, approximate query answering and online query processing. Hence, there has been a lot of focus on “selectivity estimation”, that is, computing summary statistics on the underlying data and using that to answer aggregate queries fast and to a reasonable approximation. We present two sets of results for range aggregate queries, which are amongst the most common queries.First, we focus on a histogram as summary statistics and present algorithms for constructing histograms that are provably optimal (or provably approximate) for range queries; these algorithms take (pseudo-) polynomial time. These are the first known optimality or approximation results for arbitrary range queries; previously known results were optimal only for restricted range queries (such as equality queries, hierarchical or prefix range queries).Second, we focus on wavelet-based representations as summary statistics and present fast algorithms for picking wavelet statistics that are provably optimal for range queries. No previously-known wavelet-based methods have this property.We perform an experimental study of the various summary representations show the benefits of our algorithms over the known methods.

Histograms are a concise and flexible way to construct summary structures for large data sets. They have attracted a lot of attention in database research due to their utility in many areas, including query optimization, and approximate query answering. They are also a basic tool for data visualization and analysis.In this paper, we present a formal study of dynamic multidimensional histogram structures over continuous data streams. At the heart of our proposal is the use of a dynamic summary data structure (vastly different from a histogram) maintaining a succinct approximation of the data distribution of the underlying continuous stream. On demand, an accurate histogram is derived from this dynamic data structure. We propose algorithms for extracting such an accurate histogram and we analyze their behavior and tradeoffs. The proposed algorithms are able to provide approximate guarantees about the quality of the estimation of the histograms they extract.We complement our analytical results with a thorough experimental evaluation using real data sets.

Synopses construction algorithms have been found to be of interest in query optimization, approximate query answering and mining, and over the last few years several good synopsis construction algorithms have been proposed. These algorithms have mostly focused on the running time of the synopsis construction vis-a-vis the synopsis quality. However the space complexity of synopsis construction algorithms has not been investigated as thoroughly. Many of the optimum synopsis construction algorithms are expensive in space. For some of these algorithms the space required to construct the synopsis is significantly larger than the space required to store the input. These algorithms rely on the fact that they require a smaller "working space" and most of the data can be resident on disc. The large space complexity of synopsis construction algorithms is a handicap in several scenarios. In the case of streaming algorithms, space is a fundamental constraint. In case of offline optimal or approximate algorithms, a better space complexity often makes these algorithms much more attractive by allowing them to run in main memory and not use disc, or alternately allows us to scale to significantly larger problems without running out of space. In this paper, we propose a simple and general technique that reduces space complexity of synopsis construction algorithms. As a consequence we show that the notion of "working space" proposed in these contexts is redundant. This technique can be easily applied to many existing algorithms for synopsis construction problems. We demonstrate the performance benefits of our proposal through experiments on real-life and synthetic data. We believe that our algorithm also generalizes to a broader range of dynamic programs beyond synopsis construction.

In this paper we focus on a pictorial query language, referred to as Geographical Pictorial Query Language (GeoPQL), and we revise its formal semantics by considering the polygon-polyline, polyline-polyline, and polygon-polygon topological relationships. This work proposes the Approximate Answering Engine (AAE) within a Distributed System, referred to as GeoPQLJSON (GeoPQLJ). The AAE provides approximate answers to query with empty results by following two directions: the Operator Conceptual Neighborhood (OCN) graph, and the OpenStreetMap (OSM) attribute hierarchy, giving maximum flexibility to the user choices. According to the former, the geo-operators of the queries can be replaced with the ones labeling the adjacent nodes of the OCN graph. By following the latter, the system evaluates the OSM attribute semantic similarity according to the information content approach, and proposes possible attribute replacements to the user. Note that the presence of OSM attributes allows the quick and direct access to large amount of geographical data, without requiring in our case the use of the topological elements. The functionalities of the Distributed GeoPQLJ System are illustrated by several query examples.

The Sybil attack is a particularly harmful threat to sensor networks where a single sensor node illegitimately claims multiple identities. A malicious node may generate an arbitrary number of additional node identities using only one physical device. The Sybil attack can disrupt normal functioning of the sensor network, such as the multipath routing, used to explore the multiple disjoint paths between source-destination pairs. But the Sybil attack can disrupt it when a single adversary presents multiple identities, which appear on the multiple paths. Digital certificates are a way to prove identities, but they are not suitable for the sensor network because of the large computational overheads. Researchers have proposed a light-weight identity certificate method to defeat Sybil attacks, but it is not suitable for a large scale sensor network because of the huge memory usage required at each node. In this paper, we propose a light-weight Sybil attack detection method based on a hierarchical architecture in sensor networks. The proposed method can be used in a large scale sensor network, and it only uses the symmetric cryptography avoiding the use of the public key cryptography. Simulation results show that the proposed method only needs a small memory (below 140 KB) for each node (even in a large scale sensor network). In addition, the energy consumption by new identity generations within each group is low (below 60 mJ), which is much lower than the available energy at each sensor node.