Multiple query scheduling for distributed semantic caches

In distributed query processing systems where caching infrastructure is distributed and scales with the number of servers, it is becoming more important to orchestrate and leverage a large number of cached objects in distributed caching systems seamlessly as the present trend is to build large scalable distributed systems by connecting small heterogeneous machines. With a large scale distributed caching system, a scheduling policy must consider both cache hit ratio and system load balance to optimize multiple queries. A scheduling policy that considers system load but not cache hit ratio often fails to reuse cached data by not assigning a query to the sever that has data objects the query needs. On the contrary, a scheduling policy that considers cache hit ratio but not system load balance may suffer from system load imbalance. To maximize the overall system throughput and to reduce query response time, a multiple query scheduling policy must balance system load and also leverage cached objects. In this paper, we present a distributed query processing framework that exhibits high cache hit ratio while achieving good system load balance. In order to seamlessly manage our distributed scalable caching system, our framework performs autonomic cached data migrations to improve cache hit ratio. Our experiments show that our proposed query scheduling policy and data migration policy significantly improve system throughput by achieving high cache hit ratio while avoiding system load imbalance.

MQO is a distributed multiple query processing middleware that can use resources available on the Grid to optimize query processing for data analysis and visualization applications. It does so by introducing one or more proxies that act as front-ends to a collection of backend servers. The basic idea behind this architecture is active semantic caching, whereby queries can leverage available cached results in the proxy either directly or through transformations. While this approach has been shown to speed up query evaluation under multi-client workloads, the caching infrastructure in the backend servers is not used well for query processing. Because this collective caching infrastructure scales with the number of servers, it is an important asset. In this paper, we describe a distributed multidimensional indexing scheme that enables the proxy to directly consider the cache contents available at the backend servers for query planning and scheduling. This approach is shown to produce better query plans and faster query response times as we experimentally demonstrate.

Query Processing is a key determinant in the overall performance of distributed databases. It requires processing of data at their respective sites and transmission of the same between them. These together constitute a distributed query processing strategy (DQP). DQP aims to arrive at an efficient query processing strategy for a given query. This strategy involves generation of efficient query plans for a distributed query. In case of distributed relational queries, the number of possible query plans grows exponentially with an increase in the number of relations accessed by the query. This number increases further when the relations, accessed by the query, have replicas at different sites. Such a large search space renders it infeasible to find optimal query plans. This paper presents a query plan generation algorithm that attempts to generate optimal query plans, for a given query, using genetic algorithm. The query plans so generated involve fewer sites, thus leading to efficient query processing. Further, experimental results show that the proposed algorithm converges quickly towards optimal query plans for an observed crossover and mutation probability.

The query processing in a mobile computing environment involves join processing among different sites which include static servers and mobile computers. Because of the presence of asymmetric features in a mobile computing environment, the conventional query processing for a distributed database cannot be directly applied to a mobile computing system. In this paper, we first explore some unique features of a mobile environment and then, in light of these features, devise query processing methods for both join and query processing. Remote mobile joins are said to be effectual if they are, when being interleaved into a join sequence, able to reduce the amount of data transmission cost required for distributed mobile query processing. Since mobile relations are employed as reducers in our proposed query processing cost model, more mobile joins in the query processing lead to less data transmitted through the network. With proper scheduling, interleaving effectual remote mobile joins into a query scheduling can significantly reduce the total amount of data transmission among different sites. A simulator is developed to evaluate the performance of algorithms devised. Our results show that the approach of interleaving the processing of distributed mobile queries with effectual remote mobile joins is not only efficient, but also effective in reducing the total amount of data transmission cost required to process distributed mobile queries.

Fault-tolerance has long been a feature of database systems, with transactions supporting the structuring of applications so as to ensure continuation of updating applications in spite of machine failures. For read-only queries the perceived wisdom has been that support for fault-tolerance is too expensive to be worthwhile. Distributed query processing is coming to be seen as a promising way of implementing applications that combine structured data and analysis operations in dynamic distributed settings such as computational grids. Such a query may be long-running and having to redo the whole query after a failure may cause problems (e.g. if the result may trigger business or safety critical activities). This work describes and evaluates a new scheme for adding fault-tolerance to distributed query processing through a rollback-recovery mechanism. The high level expression of user requests in a physical algebra offers opportunities for tuning the fault-tolerance provision so as to reduce the cost, and give better performance than employment of generic fault-tolerance mechanisms at the lowest level of query processing. This paper outlines how the publicly-available OGSA-DQP computational grid-based distributed query processing system can be modified to include support for fault-tolerance and presents a performance evaluation which includes measurements of the cost of both protocol overheads and rollback-recovery, for a set of example distributed queries.

A study on the influence of leadership orchestration on e-readiness accession in higher education in the Republic of Kenya concluded in 2018 showed that there is a correlation between the leadership performance on selected leadership indices and the institution’s digital age readiness. Does this correlation also apply to adoption of evidence-based decision-making and, thus, application of data analytics, visualization and business intelligence? Cross-sectional survey design was employed. Quantitative data was collected through simple random sampling using an online questionnaire in nine purposively selected universities. Seven key questions formulated around process and data were posed to the participants. Qualitative data was obtained using focus group discussion and interviews. Pearson Moment Co-efficient of Correlation and bivariate regression were done. Content analysis was applied on the qualitative data. Significant correlation was observed with six out of seven items. It is concluded that institutional perspective on process performance and evidence-based decision-making is correlated with leadership orchestration. Effective adoption and application of data analytics, visualization and business intelligence in the digital age is dependent on institutional leadership. Recommendation is made to re-orient leadership in higher education to evidence-based decision-making and data-driven culture in order to harness the power of data analytics, visualization and business intelligence to improve the quality and productivity of higher education in Kenya.

Approximate Query Processing has become increasingly popular as larger data sizes have increased query latency in distributed query processing systems. To provide such approximate results, systems return intermediate results and iteratively update these approximations as they process more data. In shared clusters, however, these systems waste resources by directing resources to queries that are no longer improving the results given to users. We describe ReLAQS, a cluster scheduling system for online aggregation queries that aims to reduce latency by assigning resources to queries with the most potential for improvement. ReLAQS utilizes the approximate results each query returns to periodically estimate how much progress each concurrent query is currently making. It then uses this information to predict how much progress each query is expected to make in the near future and redistributes resources in real-time to maximize the overall quality of the answers returned across the cluster. Experiments show that ReLAQS achieves a reduction in latency of up to 47% compared to traditional fair schedulers.

A Reverse Top-k Geo-Social Keyword Query (RkGSKQ) aims to find all the users who have a given geo-social object in their top-k geo-social keyword query results. This query is practical in detecting prospective customers for online business in social networks. Existing work on RkGSKQ only explored efficient approaches in answering a single query per time, which could not be efficient in processing multiple queries in a query batch. In many real-life applications, multiple RkGSKQs for multiple query objects can be issued at the same time. To this end, in this paper, we focus on the efficient batch processing algorithm for multiple RkGSKQs. To reduce the overall cost and find concurrently results of multiple queries, we present a group processing framework based on the current state-of-the-art indexing and group pruning strategies to answer multiple RkGSKQs by sharing common CPU and I/O costs. Extensive experiments on three data sets demonstrate the effectiveness and efficiency of our proposed methods.

Peer-to-Peer infrastructures are emerging as one of the important data management infrastructures in the World Wide Web. So far, however, most work has focused on simple P2P networks which tackle efficient query distribution to a large set of peers but assume that each query can be answered completely at each peer. For queries which need data from more than one peer to be executed this is clearly insufficient. Unfortunately, though quite a few database techniques can be re-used in the P2P context, P2P data management infrastructures pose additional challenges caused by the dynamic nature of these networks. In P2P networks, we can assume neither global knowledge about data distribution, nor the suitableness of static topologies and static query plans for these networks. Unlike in traditional distributed database systems, we cannot assume complete information schema and allocation schema instances but rather work with distributed schema information which can only direct query processing tasks from one node to one or more neighboring nodes. In this paper we first describe briefly our super-peer based topology and schema-aware distributed routing indices extended with suitable statistics and describe how this information is extracted and updated. Second we show how these indices facilitate the distribution and dynamic expansion of query plans. Third we propose a set of transformation rules to optimize query plans and discuss different optimization strategies in detail, enabling efficient distributed query processing in a schema-based P2P network.

We propose the combination of techniques that solve multiple queries for motion planning problems with single query planners in a motion planning framework that can be efficiently parallelized. In multiple query motion planning, a data structure is built during a preprocessing phase in order to quickly respond to on-line queries. Alternatively, in single query planning, there is no preprocessing phase and all computations occur during query resolution. This paper shows how to effectively combine a powerful sample-based method primarily designed for multiple query planning (the Probabilistic Roadmap Method - PRM) with sample-based tree methods that were primarily designed for single query planning (such as Expansive Space Trees, Rapidly Exploring Random Trees, and others). Our planner, which we call the Probabilistic Roadmap of Trees (PRT), uses a tree algorithm as a subroutine for PRM. The nodes of the PRM roadmap are now trees. We take advantage of the very powerful sampling schemes of recent tree planners to populate our roadmaps. The combined sampling scheme is in the spirit of the non-uniform sampling and refinement techniques employed in earlier work on PRM. PRT not only achieves a smooth spectrum between multiple query and single query planning but it combines advantages of both. We present experiments which show that PRT is capable of solving problems that cannot be addressed efficiently with PRM or single-query planners. A key advantage of PRT is that it is significantly more decoupled than PRM and sample-based tree planners. Using this property, we designed and implemented a parallel version of PRT. Our experiments show that PRT distributes well and can easily solve high dimensional problems that exhaust resources available to single machines.

Graph Analytics (GA) is a rapidly growing field that holds significant importance in numerous applications that focuses on analysing the complex relations in a network. One of the crucial concepts in GA is Query Reachability which refers to the ability to find a path connecting two nodes (u, v) in graph G. The multiple attributes in reachability query processing play a significant role to discover some interesting patterns. Existing studies did not deal with processing complex queries in attributed graphs. To fulfill some of the challenges, We contributed to this paper with three ideas for solving complex queries, finding paths, and processing multiple queries in parallel and distributed paradigms. Therefore, 1) Proposed Distributed Set Reachability Query in Attributed Networks (DSRQ_AN) algorithm using distributed computing through the Neo4j graph data platform and Cypher Query Language, 2) Finding the path to the given complex queries, and 3) Proposed a Multiple Query Processing in Parallel and Distributed (MQP_PD) algorithm to execute in a parallel environment. This paper assessed the performance of the proposed approaches through a parallel paradigm on Neo4j slaves. Furthermore, the demonstrated results of parallel execution outperform sequential execution for multiple queries on attributed graphs.

With the explosion of data in the past decade, big data is becoming a research hotspot in the information field. Many cloud-based distributed data processing platforms have been proposed to provide efficient and cost effective solutions for big data query processing, such as Hadoop, Hive, Pig, etc. However, most of the current research works are focus on improving the performance of query processing based on the view of systematics while without considering the characteristics of queries themselves, such as the query similarity, which will cause large numbers of redundant computation, effect query execution efficiency, thus having an adverse impact on promotion of the multi-queries processing performance. To solve this problem, in this paper, we propose a Multi-queries optimization framework based on MapReduce-oriented cloud environment (Multi-Q), which utilizes the dependence between multiple queries to realize query results reuse. Firstly, a cluster-based partition algorithm called CPA has been exploited to conduct the logic partition of the search range of query workload. Secondly, a multi-queries reuse dependence graph (MRDG) construction method on the basis of the cluster-based partition results has been presented to depict the dependence between the multiple queries. Finally, a Multi-Q processing algorithm based on Multi-Q Reuse Dependence Graph has been put forward to achieve the query results reuse and improve the overall query processing performance. We evaluate our approach by deploying Multi-Q based on Hadoop in a real cloud environment, called SEU-Cloud, and conducting extensive experiments based on the standard TPC-H. The result verifies that compared with Hive, the performance of improvement is approximately 39.3% by using our Multi-Q.

Query scheduling as one of the most important technologies used in query processing has been widely studied recently. Unfortunately, to the best of our knowledge, no previous work focuses on the Minimum Latency Multi-Regional Query Scheduling (ML-MRQS) problem. In this paper, we investigate the ML-MRQS problem in Wireless Sensor Networks (WSNs), which aims to generate a scheduling plan with minimum latency for a more practical query model called Multi-Regional Query (MRQ). A MRQ targets at user interested data from multiple region-sofa WSN, where each region is a subarea of the WSN. We claim that the ML-MRQS problem is NP-hard. Therefore, we propose a heuristic scheduling algorithm Multi-Regional Query Scheduling Algorithm (MRQSA) to solve this problem. Theoretical analysis shows that the latency of MRQSA is upper bounded by 23A + B + C for a MRQ with m query regions R 1 , R 2 …, R m , where A = max i=1 m D i left, B = max i=1 m {(23D i +5Δ+21)k i }, C = Σ i=1 m H i +5Δ−m+17, m is the number of regions, Δ is the maximum node degree in the WSN, A is the diameter of R i , k i is the maximum overlapped degree of sensor nodes in TU, Hi represents the distance of R i with respect to the sink, and D i left is the diameter of the non-overlapped part of R i . Extensive simulations are conducted to verify the performance of our algorithm, which show that MRQSA significantly reduces the query latency when compared with the most recently published multi-query scheduling algorithm.

Query scheduling plays an important role when systems are faced with limited resources and high workloads. It becomes even more relevant for servers applying multiple query optimization techniques to batches of queries, in which portions of datasets as well as intermediate results are maintained in memory to speed up query evaluation. We present a dynamic query scheduling model based on a priority queue implementation using a directed graph and a strategy for ranking queries. We examine the relative performance of several ranking strategies on a shared-memory machine using two different versions of an application, called the Virtual Microscope, for browsing digitized microscopy images.

With the great advancement in wireless technology, number of wireless sensor network applications have increased in which different sensor nodes communicates with each other via sending data among themselves. Query for communication among sensor nodes can be framed in different forms leading into different computation cost. So, the generation and selection of query plan of minimum cost becomes combinatorial in nature which cannot be solved in polynomial time to achieve global optimal cost of data communication. One of the solution to address this problem is nature inspired algorithms. These algorithms have served to number of real life intricate problems. Amidst of all algorithms, bio-inspired algorithms have largely accepted to assist such problems. Artificial immune system (AIS), one of bio-inspired algorithm is inspired from natural human immune system has been explored here. Clonal selection process, one of AIS approach has been discussed in this chapter to generate optimal distributed query plans in distributed wireless sensor network.