AGiDS: A Grid-Based Strategy for Distributed Skyline Query Processing

With the rapid development of network data communication technology, the complexity of the network environment makes the data in the data stream have uncertain characteristics, and a large number of data streams are generated in many fields. Therefore, it is necessary to find an efficient and accurate processing method to process a large amount of data in a computer network. This paper takes the skyline probability calculation and the actual efficiency of the unknown object index structure in the skyline query processing technology on the uncertain data stream as the research object. Based on the skyline query technology, we study an efficient skyline query processing method on the unknown network traffic based on the model SUMG. The method includes two algorithms: dynamic modeling algorithm DMG and skyline query algorithm GST. The DMG algorithm samples the data in the sliding window of the uncertain data stream and establishes a model to convert the data stream into the parameter stream in the uncertain object probability density function, the GST algorithm establishes the R-tree index structure, which is in order to reduce the amount of calculation, it uses the parameter flow of the model. In both methods, the set of local skyline results is first obtained at the distributed node, and the skyline query is performed again on the union of the local skyline results to obtain the global skyline result set. The experimental results show that compared with the skyline query method BNL for unknown network traffic without an index structure, the SUMG method can not only effectively model the link-type unknown object to assist the skyline query, but also effectively prune the uncertain data object and improve the skyline. The distributed skyline query method can cope with the skyline query task on the distributed data stream.

This paper looks at the processing of skyline queries on peer-to-peer (P2P) networks. We propose Skyframe, a framework for efficient skyline query processing in P2P systems, which addresses the challenges of quick response time, low network communication cost and query load balancing among peers. Skyframe consists of two querying methods: one is optimized for network communication while the other focuses on query response time. These methods are different in the way in which the query search space is defined. In particular, the first method uses a high dominating point that has a large dominating region to prune the search space to achieve a low cost in network communication. On the other hand, the second method relaxes the search space in order to allow parallel query processing to speed up query response. Skyframe achieves query load balancing by both query load conscious data space splitting/merging during the join/departure of nodes and dynamic load migration. We further show how to apply Skyframe to both the P2P systems supporting multi-dimensional indexing and the P2P systems supporting single-dimensional indexing. Finally, we have conducted extensive experiments on both real and synthetic data sets over two existing P2P systems: CAN (Ratnasamy in A scalable content-addressable network. In: Proceedings of SIGCOMM Conference, pp. 161---172, 2001) and BATON (Jagadish et al. in A balanced tree structure for peer-to-peer networks. In: Proceedings of VLDB Conference, pp. 661---672, 2005) to evaluate the effectiveness and scalability of Skyframe.

Effective and efficient skyline query processing over attribute-order-preserving-free encrypted data in cloud-enabled databases

Efficient processing of skyline queries has been an area of growing interest over both static and stream environments. Most existing static and streaming techniques assume that the skyline query is applied to a single data source. Unfortunately, this is not true in many applications in which, due to the complexity of the schema, the skyline query may involve attributes belonging to multiple data sources. Recently, in the context of static environments, various hybrid skyline-join algorithms have been proposed. However, these algorithms suffer from several drawbacks: they often need to scan the data sources exhaustively in order to obtain the set of skyline-join results; moreover, the pruning techniques employed to eliminate the tuples are largely based on expensive pairwise tuple-to-tuple comparisons. On the other hand, most existing streaming methods focus on single stream skyline analysis, thus rendering these techniques unsuitable for applications that require a real-time "join" operation to be carried out before the skyline query can be answered. Based on these observations, we introduce and propose to demonstrate SkySuite: a framework of skyline-join operators that can be leveraged to efficiently process skyline-join queries over both static and stream environments. Among others, SkySuite includes (1) a novel Skyline-Sensitive Join (SSJ) operator that effectively processes skyline-join queries in static environments, and (2) a Layered Skyline-window-Join (LSJ) operator that incrementally maintains skyline-join results over stream environments.

The skyline query has been received much attention as an important operator in database systems for multi-preference analysis and decision making. Most of the previous works have focused on processing skyline queries on centralized data sets. However, the related data of real applications are practically scattered at several different servers. The skyline query computation in distributed environment is needed to gather a large number of data from the connected servers. The existing methods for a skyline query in distributed environment have two problems: (i) They have slow processing time for a skyline query. (ii) Most of the transferred data among servers in the network are unnecessary. In this paper, we propose a multi-layer grid method for efficiently processing skyline queries in distributed environments (MGDS). The proposed method minimizes the unnecessary transferred data using the grid mechanism. Experiments based on various data sets show that our proposed method outperforms the existing methods.KeywordsSkyline querydistributed skyline querygrid methoddistributed data

With the growth of information technology and computer networks, there is a vital need for optimal design of distributed databases with the aim of performance improvement in terms of minimizing the round-trip response time and query transmission and processing costs. To address this issue, new fragmentation, data allocation, and replication techniques are required. In this paper, we propose enhanced vertical fragmentation, allocation, and replication schemes to improve the performance of distributed database systems. The proposed fragmentation scheme clusters highly-bonded attributes (i.e., normally accessed together) into a single fragment in order to minimize the query processing cost. The allocation scheme is proposed to find an optimized allocation to minimize the round-trip response time. The replication scheme partially replicates the fragments to increase the local execution of queries in a way that minimizes the cost of transmitting replicas to the sites. Experimental results show that, on average, the proposed schemes reduce the round-trip response time of queries by 23% and query processing cost by 15%, as compared to the related work.

(2. Zhejiang Fashion Institute of Technology, Ningbo 315021, China) Abstract — Skyline query has been applied widely in sensor networks. We propose a connected key node set- based skyline Efficient skyline query processing (EffiSky) algorithm to minimize communication traffic for resources- limited sensor networks. In the EffiSky algorithm, we dis- cover a Connected key node set (CKNS) used to trans- mit and collect queries and results among the sensor nodes, which can reduce the average communication cost of the networks significantly. We set up a two-level fil- tering scheme that prunes many useless dominated tu- ples. Both the theoretical analysis and experiment results demonstrate that EffiSky excels the existing work in terms of network traffic, scalability in network expansion, node density, and dimension change.

Data centric storages for sensor networks have been proposed to efficiently process multi-dimensional range queries as well as exact matches. Usually, a sensor network does not process only one type of the query, but processes various types of queries such as range queries, exact matches and skyline queries. Therefore, a sensor network based on a data centric storage for range queries and exact matches should process skyline queries efficiently. However, existing algorithms for skyline queries have not considered the features of data centric storages. Some of the data centric storages store similar data in sensor nodes that are placed on geographically similar locations. Consequently, all data are ordered in a sensor network. In this paper, we propose a new skyline query processing algorithm that exploits the above features of data centric storages.

Skyline queries are useful in decision making applications. Skyline queries in highly mobile distributed environments have attracted many attentions recently due to the development of mobile internet device. The properties of distributed computing make skyline queries more complicated especially in any subspace. Conventional skyline algorithms do not support subspace skyline queries in distributed environment. In this paper, we focus on how to perform distributed skyline queries in any subspace according to user preference. So we propose a system model in a mobile and distributed environment. An efficient parallel algorithm for processing the Subspace Skyline Query (SSQ) using MapReduce is applied to the system model. This algorithm can report skyline points in any subspace. Meanwhile, a pruning strategy is also proposed in order to reduce the network communication and minimize the response time. We conduct experiments on real and synthetic data. Experimental results indicate that our SSQ algorithm is much more efficient. Furthermore, the pruning strategy can further improve the performance of the algorithm.

For speeding up query processing on Big Data, frequent sub-queries or views may be materialized such that the query processing cost is minimized with optimum cost of maintaining the materialized views and/or queries. Materializing frequent sub-queries and views means that resultant data set of the views reside in the memory of one or more nodes in the cluster, so that it reduces the MapReduce cost, submission and scheduling cost of Distributed File System jobs for query processing. We have defined materialized views as resultant data of frequent sub-queries and aggregation functions of a set of Big Data warehousing queries that are saved for enhancing query performance. The problem is defined as a multi-objective optimization problem for minimizing the total query processing MapReduce cost, MapReduce cost for maintaining the materialized views and the number of views selected for materializing with maximized total size of the views selected. We applied Differential Evolution algorithm and NSGA-II to study their performances for developing a recommendation system for selecting views for materializing in Big Data warehousing.

An efficient skyline query algorithm in the distributed environment

The Survey on Skyline Query Processing for Data-Specific Applications

This research proposes an advanced approach that improves the efficiency of skyline query processing by significantly reducing the computational cost on object comparisons, i.e., dominance tests between objects. Our solutions are based on two novel concepts. The skyline query over Minimum Bounding Rectangles (MBRs) receives a set of MBRs and returns the MBRs that are not dominated by other MBRs. In the dominance test for MBRs, the detailed attribute values of objects in the MBRs are not accessed. Moreover, the dependent group of MBRs reduces the search space for dominance tests. Objects in an MBR are only compared with the ones in the corresponding dependent groups of the MBR rather than with the entire dataset. Our solutions apply the two concepts to the R-tree in order to use its hierarchical structure in which every node is a natural abstraction of an MBR. Specifically, given the R-tree index of an input dataset, we first eliminate unqualified objects by utilizing the skyline query over MBRs (i.e., intermediate nodes in the R-tree). Subsequently, we generate dependent groups for the skyline MBRs. Two dependent group generation methods that rely on either the sorting technique or the R-tree index are developed. Further, we apply an existing skyline algorithm to every dependent group, and the results of the original skyline query are the union of skyline objects in the dependent groups. In addition, we also analyze the cardinality of the two new concepts based on a probabilistic model, which enables us to analyze the computational complexity of the proposed solutions. Our experimental results show that the proposed solutions are clearly more efficient than the state-of-the-art approaches.

Efficient storage and handling of data stored in a peer-to-peer (P2P) network, proves vital for various applications such as query processing and data mining. This paper presents a distributed, scalable and robust layered overlay (L-overlay) to index and manage multidimensional data in a dynamic P2P network. The proposed method distinguishes between the data and peer layers, with efficient mapping between the two. The data is organized such that semantically similar data objects are accessed hastily. Grid and tree structures are proposed for the peer layer. As application examples of L-overlay in query processing and data mining, k-nearest neighbors query processing and distributed Naive Bayes classification algorithms, are proposed. We show the effectiveness of our scheme in static and dynamic environments using simulation. L-overlay is shown to be more efficient than SSW, an available semantic overly, in terms of maintenance and query processing costs.

The skyline query processing over uncertain data streams has received considerable attention, due to its importance in helping users make intelligent decisions over complex data. Nevertheless, existing studies only focus on retrieving the skylines over data streams in a centralized environment typically with one processor, which limits the scalability of algorithms and cannot meet the requirement for massive data analysis. The emerging cloud computing environment provides much more reliable and stable environments than the traditional distributed environments, which can be well adapted to the massive data management and complex queries. Unfortunately, existing parallel frameworks in clouds such as MapReduce and its variants are not suitable for the skyline queries over uncertain data streams. In this paper, we propose a general framework for parallelizing the probabilistic streaming skyline operator with the sliding window partitioning. Particularly, we propose four items mapping strategies CMS, AMS, DMS and APS to optimize the queries based on the proposed parallel framework. Extensive experiments with real deployment are conducted to demonstrate the effectiveness and efficiency of the proposals.