On High Dimensional Skylines

Skyline queries help users make intelligent decisions over complex data. The main shortcoming of skyline queries is that the cardinality of the result set is not known a-priori. To overcome this limitation, the representative skyline query has been proposed, which retrieves a fixed set of k skyline points that best describe all skyline points. Even though the representative skyline has been studied before in centralized environments, this is the first paper that addresses efficient computation of the representative skyline in distributed systems. The distributed nature of the environment makes the task of discovering truly representative skyline points even more challenging. In this paper, we propose a novel framework for discovering the representative skyline over distributed data sources. Our experimental study demonstrates the efficiency and effectiveness of our framework.KeywordsRepresentative PointSkyline QuerySkyline PointCommunication PhaseLocal SkylineThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Recently, skyline queries have attracted much attention in the database research community. Space partitioning techniques, such as recursive division of the data space, have been used for skyline query processing in centralized, parallel and distributed settings. Unfortunately, such grid-based partitioning is not suitable in the case of a parallel skyline query, where allpartitions are examined at the same time, since many data partitions do not contribute to the overall skyline set, resulting in a lot of redundant processing.In this paper we propose a novel angle-based space partitioning scheme using the hyperspherical coordinates of the data points. We demonstrate both formally as well as through an exhaustive set of experiments that this new scheme is very suitable for skyline query processing in a parallel share-nothing architecture. The intuition of our partitioning technique is that the skyline points are equally spread to all partitions. We also show that partitioning the data according to the hyperspherical coordinates manages to increase the average pruning power of points within a partition. Our novel partitioning scheme alleviates most of the problems of traditional grid partitioning techniques, thus managing to reduce the response time and share the computational workload more fairly. As demonstrated by our experimental study, our technique outperforms grid partitioning in all cases, thus becoming an efficient and scalable solution for skyline query processing in parallel environments.

MSSQ: Manhattan Spatial Skyline Queries

Skyline computation has many applications including multi-criteria decision making. In this paper, we study the problem of selecting k skyline points so that the number of points, which are dominated by at least one of these k skyline points, is maximized. We first present an efficient dynamic programming based exact algorithm in a 2d-space. Then, we show that the problem is NP-hard when the dimensionality is 3 or more and it can be approximately solved by a polynomial time algorithm with the guaranteed approximation ratio 1-1/e. To speed-up the computation, an efficient, scalable, index-based randomized algorithm is developed by applying the FM probabilistic counting technique. A comprehensive performance evaluation demonstrates that our randomized technique is very efficient, highly accurate, and scalable.

Skyline queries have wide-ranging applications in fields that involve multi-criteria decision making, including tourism, retail industry, and human resources. By automatically removing incompetent candidates, skyline queries allow users to focus on a subset of superior data items (i.e., the skyline), thus reducing the decision-making overhead. However, users are still required to interpret and compare these superior items manually before making a successful choice. This task is challenging because of two issues. First, people usually have fuzzy, unstable, and inconsistent preferences when presented with multiple candidates. Second, skyline queries do not reveal the reasons for the superiority of certain skyline points in a multi-dimensional space. To address these issues, we propose SkyLens, a visual analytic system aiming at revealing the superiority of skyline points from different perspectives and at different scales to aid users in their decision making. Two scenarios demonstrate the usefulness of SkyLens on two datasets with a dozen of attributes. A qualitative study is also conducted to show that users can efficiently accomplish skyline understanding and comparison tasks with SkyLens.

Skyline computation has been widely used in many research areas such as database visualization, data mining and multi-criteria decision making. In order to process massive data, in this paper, we propose a two phase method to calculate Skyline points. In the first phase, all the supported non-dominated skyline points are calculated with Quickhull algorithm. Then in the second phase, all the unsupported non-dominated points are calculated with Block Nested Loop (BNL) algorithm. We test the algorithm on three different types of synthetic data sets and compare it with BNL algorithm and Sort Filter Skyline (SFS) algorithm. Experimental results show that our proposed algorithm performs better than BNL and SFS on most of the test datasets.

Skyline queries have attracted considerable attention to assist multicriteria analysis of large-scale datasets. In this paper, we focus on multidimensional subspace skyline computation that has been actively studied for two approaches. First, to narrow down a full-space skyline, users may consider multiple subspace skylines reflecting their interest. For this purpose, we tackle the concept of a skycube, which consists of all possible non-empty subspace skylines in a given full space. Second, to understand diverse semantics of subspace skylines, we address skyline groups in which a skyline point (or a set of skyline points) is annotated with decisive subspaces. Our primary contributions are to identify common building blocks of the two approaches and to develop orthogonal optimization principles that benefit both approaches. Our experimental results show the efficiency of proposed algorithms by comparing them with state-of-the-art algorithms in both synthetic and real-life datasets.

Hyperspectral remote sensing image (HSI) consists of hundreds of bands that contain rich space, radiation and spectral information. The high-dimensional data can also lead to the curse of dimensionality problem making it difficult to be used effectively. In this paper, we proposed a manifold learning algorithm to reduce the dimensionality for HSI data. For high dimensional datasets with continuous variables, it is often the case that the data points are arranged along with low dimensional structures, named manifolds, in the high dimensional space. Manifold learning aims to identifying those special low dimensional structures for subsequent usage such as classification or regression. However, many manifold learning algorithms perform an eigenvector analysis on a data similarity matrix whose size is N×N, where N is the number of data points. The memory complexity of the analysis is at least O(N <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) that is not feasible for a regular computer to compute or storage for very large datasets. To solve this problem, we used statistical sampling methods to sample a subset of data points as landmarks. A skeleton of the manifold was then identified based on the landmarks. The remaining data points were then inserted into the skeleton by Locally Linear Embedding (LLE). We tested our algorithm on AVIRIS Salinas-A data set. The experimental results showed that the HSI dataset could be reduced to a lower-dimensional space for land use classification with good performance, and the main structure was preserved well.

Density estimation is an important tool for data analysis. Non-parametric approaches have a reputation for offering state-of-the-art density estimates limited to few dimensions. Despite providing less accurate density estimates, histogram-based approaches remain the only alternative for datasets in high-dimensional spaces. In this paper, we present a multivariate histogram approach to estimate the density of a dataset without restrictions on the number of dimensions, containing both numerical and categorical variables (without numerical encoding) and allowing missing data (without the need to preprocess them). Results from the empirical evaluation show that it is possible to estimate the density of datasets without restrictions on dimensionality, and the method is robust to missing values and categorical variables.KeywordsDensity estimateMultivariate histogramMissing data

Skyline queries are useful in decision making applications. Skyline queries in highly mobile distributed environments have attracted many attentions recently due to the development of mobile internet device. The properties of distributed computing make skyline queries more complicated especially in any subspace. Conventional skyline algorithms do not support subspace skyline queries in distributed environment. In this paper, we focus on how to perform distributed skyline queries in any subspace according to user preference. So we propose a system model in a mobile and distributed environment. An efficient parallel algorithm for processing the Subspace Skyline Query (SSQ) using MapReduce is applied to the system model. This algorithm can report skyline points in any subspace. Meanwhile, a pruning strategy is also proposed in order to reduce the network communication and minimize the response time. We conduct experiments on real and synthetic data. Experimental results indicate that our SSQ algorithm is much more efficient. Furthermore, the pruning strategy can further improve the performance of the algorithm.

The skyline query is an efficient data analysis tool for multi-criteria decision making that has received significant attention in the database community. As multi-core architectures have gone mainstream, we present a new parallel skyline query algorithm that can be applied to multi-core and multiprocessor systems, to progressively return skyline points as they are identified efficiently. In this paper, we proposed a parallel skyline algorithm which can eliminate redundant computations and improve parallelism of the skyline query. Experimental results show that our algorithm successfully exploits the features of multiple cores to improve the performance of skyline computation for large high-dimensional datasets.

The skyline query operation (also called the “maximum vector problem”) is used to identify potentially interesting or useful data points in large sets of multi-dimensional data. When the data change over time (through addition and subtraction of points), this is called the “continuous skyline” query. The 2015 MEMOCODE Design Contest problem is to implement a system to efficiently compute the continuous skyline of dynamic data. Contestants were given one month to develop a system to perform the skyline query, aiming to maximize performance or cost-adjusted performance. Teams were encouraged to consider a variety of computational targets including CPUs, FPGAs, and GPGPUs. The two winning teams, which have been invited to contribute papers describing their techniques, combined careful algorithmic and implementation optimizations; both implemented the system on multicore CPUs.

Selecting representative objects from a large-scale database is an essential task to understand the database. A skyline query is one of the popular methods for selecting representative objects. It retrieves a set of non-dominated objects. In this paper, we consider a distributed algorithm for computing skyline, which is efficient enough to handle “big data”. We have noticed the importance of “big data” and want to use it. On the other hand, we must take care of its privacy. In conventional distributed algorithms for computing a skyline query, we must disclose the sensitive values of each object of a private database to another for comparison. Therefore, the privacy of the objects is not preserved. However, such disclosures of sensitive information in conventional distributed database systems are not allowed in the modern privacy-aware computing environment. Recently several privacy-preserving skyline computation frameworks have been introduced. However, most of them use computationally expensive secure comparison protocol for comparing homomorphically encrypted data. In this work, we propose a novel and efficient approach for computing the skyline in a secure multi-party computing environment without disclosing the individual attributes’ value of the objects. We use a secure multi-party sorting protocol that uses the homomorphic encryption in the semi-honest adversary model for transforming each attribute value of the objects without changing their order on each attribute. To compute skyline we use the order of the objects on each attribute for comparing the dominance relationship among the objects. The security analysis confirms that the proposed framework can achieve multi-party skyline computation without leaking the sensitive attribute value to others. Besides that, our experimental results also validate the effectiveness and scalability of the proposed privacy-preserving skyline computation framework.

Skyline queries play an important role in applications such as multicriteria decision making and user preference systems. Recently, more attention has been paid to the problem of efficient skyline computation in the P2P systems. Due to the high distribution of the P2P networks, the skyline computation incurs too many intermediate results transferred between peers, which consumes mass of the network bandwidth. Additionally, a large number of peers are involved in the skyline computation, which introduces both heavy communication cost and computational overhead. In this paper, we propose a novel Distributed Caching Mechanism (DCM) to efficiently improve the performance of the skyline calculation in the structured P2P networks, using a Distributed Caching Index (DCI) scheme and an advanced cache utilization strategy. The DO scheme is employed to efficiently locate the cache that can properly answer a future skyline query. Exploring the property of emended skyline, we can optimize the utilization of the cached results for answering future skyline queries. Extensive evaluations on both synthetic and real dataset show that our approach can significantly reduce both bandwidth consumption and communication cost, and greatly shorten the response time.

The skyline queries are often used in several recommendation applications. Most existing related works have focused on skyline computation in many multidimensional data. However, these works do not consider an interesting query generated from non-skyline point. In this paper, we propose a new query, called skyline minimum vector which finds the minimum vector for making a non-skyline point into a skyline. The skyline minimum vector means the minimum cost for becoming a skyline. We use the Manhattan distance between skyline and query point in order to evaluate the cost. Also, we propose basic algorithm and optimized algorithm for getting skyline minimum vector. The proposed query will be very useful in many decision-making applications.