A multi-agent based query processing system using RETSINA with intelligent agents in cloud environment

The skyline query processing over uncertain data streams has received considerable attention, due to its importance in helping users make intelligent decisions over complex data. Nevertheless, existing studies only focus on retrieving the skylines over data streams in a centralized environment typically with one processor, which limits the scalability of algorithms and cannot meet the requirement for massive data analysis. The emerging cloud computing environment provides much more reliable and stable environments than the traditional distributed environments, which can be well adapted to the massive data management and complex queries. Unfortunately, existing parallel frameworks in clouds such as MapReduce and its variants are not suitable for the skyline queries over uncertain data streams. In this paper, we propose a general framework for parallelizing the probabilistic streaming skyline operator with the sliding window partitioning. Particularly, we propose four items mapping strategies CMS, AMS, DMS and APS to optimize the queries based on the proposed parallel framework. Extensive experiments with real deployment are conducted to demonstrate the effectiveness and efficiency of the proposals.

Lou, H., 2019. Massive ship fault data retrieval algorithm supporting complex query in cloud computing. In: Guido-Aldana, P.A. and Mulahasan, S. (eds.), Advances in Water Resources and Exploration. Journal of Coastal Research, Special Issue No. 93, pp. 1013–1018. Coconut Creek (Florida), ISSN 0749-0208.Aiming at the problems of the current retrieval algorithm for massive ship fault data retrieval in cloud computing, which does not support complex query, large retrieval cost and low precision, a massive ship fault data retrieval algorithm supporting complex query in cloud computing based on query performance prediction is proposed. Introducing the Lagrangian algorithm to preprocess the ship fault data information; combining the attribute values of the ship fault data and the context to calculate the similarity between different attributes, and identifying the same attribute according to the similarity; The attribute recognition result is queried by using multiple retrieval models, and the model complex query request is converted into a one-dimensional query key value, and the query key values obtained by the plurality of models are predicted, and the search result with the optimal prediction performance of the complex query mode is used as the final result. The experimental results show that the proposed algorithm has good retrieval accuracy in complex query mode, and it has less overhead than the current search algorithm.

Query optimization is a sophisticated process whose resource consumption and quality of the best execution plan depends on the query complexity, available resources of the RDBMS server, and the current instance of the database. For a self-managing RDBMS such as SAP SQL Anywhere, the query optimizer must adapt each optimization process to the current resources as these servers are run in highly diverse environments with no DBA for tuning, running mixed workloads with little correlation between query complexity, its execution time, or database size. The main goal of the work presented in this paper is to be able to dynamically choose a join enumeration algorithm among the algorithms available to the query optimizer based on the known characteristics of the algorithms, the query complexity and its expensiveness, and the available resources for the optimization process. First, the paper discusses the experimental results of the optimization time breakdown and the memory consumption for a set of join enumeration algorithms ranging from highly heuristics algorithms to dynamic programming algorithms with exhaustive bushy trees enumeration. Next, we present a novel technique to predict the optimization time and memory consumption for a current query by using similarity distances between its query graph and a set of etalon queries with known performance. The new technique can be used by any query optimizer for choosing among join enumeration algorithms, or optimization levels, based on such characteristics as query complexity and available resources.

Analytics on modern data analytic and data warehouse systems often need to run large complex queries on increasingly complex database schemas. A lot of progress has been made on executing such complex queries using techniques like scale out query processing, hardware accelerators like GPUs and code generation techniques. However, optimization of such queries remains a challenge. Existing optimal solutions either cannot be effectively parallelized, or are inefficient while doing a lot of unnecessary work. In this demonstration, we present our system, GPU-QO, which aims to demonstrate query optimization techniques for large analytical queries using GPUs. We first demonstrate Massively Parallel Dynamic Programming (MPDP) – a novel query optimization technique that can run on GPUs to generate optimal plans in a (massively) parallel and efficient manner. We then showcase IDP2-MPDP and UnionDP – two heuristic techniques, again using GPUs, that can even optimize queries containing 1000s of joins. Furthermore, we compare our techniques with current state-of-the-art solutions, and demonstrate how our techniques can reduce optimization time for optimal solutions by nearly two orders of magnitude and produce much better query plans for heuristics (up to 7x).

Efficient resource scheduling is a foundational challenge in cloud computing environments, where dynamic workloads, heterogeneous resources, and stringent service-level agreements (SLAs) converge to complicate optimal task allocation. Traditional scheduling algorithms such as Round-Robin, Min-Min, and other heuristic/metaheuristic approaches often struggle to adapt in real time to fluctuations in demand, resource availability, and cost-energy trade-offs [1],[2]. In contrast, artificial intelligence (AI) techniques including supervised learning, reinforcement learning (RL), and hybrid optimisation models offer adaptive and predictive capabilities that significantly enhance scheduling performance by learning from past workload patterns and system feedback [3],[4]. This paper investigates the application of AI-driven resource scheduling in cloud environments, proposing a framework that integrates workload forecasting, dynamic resource allocation, and SLA compliance optimisation. Experimental evaluation, based on simulated and real-trace datasets, demonstrates that the proposed AI-based scheduling approach can improve resource utilisation, reduce energy consumption, and lower SLA violation rates compared to baseline methods. The findings highlight the potential of AI to reshape scheduling in modern cloud infrastructures and underscore the need for further research into scalability, transparency of AI decisions, and hybrid edge-cloud deployment scenarios.

With the explosive growth of big data, data-intensive analysis services need a large data storage and high efficient parallel query processing techniques in cloud using lower-end machines, because most organizations adopt inexpensive low-end clusters. A key challenge is to optimize query processing in such a cloud environment. In this paper, we present a new hybrid data access architecture, called HyDB, for providing data-intensive analysis services, which is featured with a distributed data storage, parallel data access, and query optimization methods. First, we propose a new data partitioning method based on both workloads and co-located resources. The data partitioning method achieves higher consolidation and outperforms the existing approaches. Second, we provide a new parallel access method which includes parallel query processing, optimal query plan generation, and optimal path selection by using a plan tree pruning technique. We have implemented HyDB. Finally, we conduct extensive experimental studies and confirm the efficiency of our HyDB.

Massive XML data are increasingly generated for the representation, storage and exchange of web information. Twig query processing over massive XML data has become a research focus. However, most traditional algorithms cannot be directly implemented in a distributed manner. Some of the existing distributed algorithms generate a lot of useless intermediate results and execute many join operations of partial results in most cases; others require the priori knowledge of query pattern before XML partition, storage and query processing, which is impractical in the cases of large-scale data or frequent incoming new queries. To improve efficiency and scalability, in this paper, we propose a 3-phase distributed algorithm DisT3 based on node distribution mechanism to avoid unnecessary intermediate results. Furthermore, we propose a lightweight local index ReP with an enhanced XML partitioning approach using arbitrary partitioning strategy, and based on ReP we propose an improved 2-phase distributed algorithm DisT2ReP to further reduce the communication cost. After the performance guarantees are analyzed, extensive experiments are conducted to verify the efficiency and scalability of our proposed algorithms in distributed twig query applications.

The Industrial Internet of Things (IIoT) integrated with cloud computing resources offers effective advancements in the field of industrial automation with open connectivity and emergent computing. With such advancements, the supplementary services evolves in order to augment the automation process in manufacturing and production industries. However, the integration and transformation of cloud and IIoT environment poses serious limitation on reliability and computational performance. It is very necessary to develop a framework that handles and provides a bridge between the rapid input acquisitions from an IIoT device and rapid data processing or storage by the cloud. In this paper, model based engineering (MBE) model is used to control the workflow in cloud environment, which is integrated with a deep learning algorithm named recurrent neural network (RNN) algorithm. The RNN is utilized to control the future workload prediction and supply necessary resources to balance the workflow in cloud computing environment. The entire predictions are carried out based on the input data acquisition rate from the IIoT devices. The storage or processing of the input data is carried out by the Virtual Machines presents in cloud environment. The resources allocation and scheduling of workload is carried out in the control plane using RNN that optimizes the workflow. The RNN is resumed to solve iteratively the task of optimization. The experiments are conducted to test the efficacy of the model and the results are verified in terms of total energy consumption, task response time, and VM Utilization. The experiments are conducted with same/different types of IIoT devices. The simulation results reveal the improved efficacy of the MBE–RNN model than existing deep learning and MBE schedulers. The results show that the MBE–RNN acquires improved processing ability with reduced task response time and energy utilization.

Overlay analysis is a common task in geographic computing that is widely used in geographic information systems, computer graphics, and computer science. With the breakthroughs in Earth observation technologies, particularly the emergence of high-resolution satellite remote-sensing technology, geographic data have demonstrated explosive growth. The overlay analysis of massive and complex geographic data has become a computationally intensive task. Distributed parallel processing in a cloud environment provides an efficient solution to this problem. The cloud computing paradigm represented by Spark has become the standard for massive data processing in the industry and academia due to its large-scale and low-latency characteristics. The cloud computing paradigm has attracted further attention for the purpose of solving the overlay analysis of massive data. These studies mainly focus on how to implement parallel overlay analysis in a cloud computing paradigm but pay less attention to the impact of spatial data graphics complexity on parallel computing efficiency, especially the data skew caused by the difference in the graphic complexity. Geographic polygons often have complex graphical structures, such as many vertices, composite structures including holes and islands. When the Spark paradigm is used to solve the overlay analysis of massive geographic polygons, its calculation efficiency is closely related to factors such as data organization and algorithm design. Considering the influence of the shape complexity of polygons on the performance of overlay analysis, we design and implement a parallel processing algorithm based on the Spark paradigm in this paper. Based on the analysis of the shape complexity of polygons, the overlay analysis speed is improved via reasonable data partition, distributed spatial index, a minimum boundary rectangular filter and other optimization processes, and the high speed and parallel efficiency are maintained.

Multi-route query processing and optimization

This study implements a semi-join approach with the SDD-1 algorithm to obtain a fast query process and find out the benefits and costs of the semi-join method. The subject of this study is a distributed banking database with 8,910 data sets. Optimization is done with three (3) query processes that obtain a 600% speed increase, for the first query process it resulted in an increase of 300,4, the second query process resulted in an increase of 144,9, and in the third query process resulted in an increase of 154,9. After obtaining the optimization of the query through the semi-join method, the benefits and costs obtained from the three processes of query optimization, that is, the first query process benefits 2,711,168 at a cost of 806, the second The query process obtained 3,304 benefits at 238 and in the third query process it benefited 3,069,888 at a cost of 615. The process data shows that the application of a semi-join method with the SDD-1 algorithm for the optimization of distributed database queries effectively increases the query speed and obtains great benefits and low costs.

Query processing is that method or technique in area of mobile atmosphere which deal using mobile environment. This research in that way user can think advancement using query processing and it takes more use for speedily accessing global query processing and it is joint venture of query processing between dissimilar sites includes fixed server movable computer. The necessity saving of energy and also the existence of lopsided features in a moveable computing atmosphere, the predictable query processing for a scattered database cannot be straightly genuine to a moveable computing organization. The mobile environment is a collection of mobile diverse hosts, which are enabled to communicate using wireless links. These wireless links are change giving to the natures of mobile networks. Moreover, nodes in the ad-hoc network have to link without any centralized or help. The usability for the user also changes using query processing tool. Thus, this mechanism that allows the sharing of functionality among different devices in same environment change the way of user communication for searching fast query time in mobile computing environment. The aim of this research is to make innovation in query optimization technique in mobile computing environment. This research focuses on improvement of various query optimization techniques for highly effective and trustworthy query optimization methods.

The query processing in a mobile computing environment involves join processing among different sites which include static servers and mobile computers. Because of the presence of asymmetric features in a mobile computing environment, the conventional query processing for a distributed database cannot be directly applied to a mobile computing system. In this paper, we first explore some unique features of a mobile environment and then, in light of these features, devise query processing methods for both join and query processing. Remote mobile joins are said to be effectual if they are, when being interleaved into a join sequence, able to reduce the amount of data transmission cost required for distributed mobile query processing. Since mobile relations are employed as reducers in our proposed query processing cost model, more mobile joins in the query processing lead to less data transmitted through the network. With proper scheduling, interleaving effectual remote mobile joins into a query scheduling can significantly reduce the total amount of data transmission among different sites. A simulator is developed to evaluate the performance of algorithms devised. Our results show that the approach of interleaving the processing of distributed mobile queries with effectual remote mobile joins is not only efficient, but also effective in reducing the total amount of data transmission cost required to process distributed mobile queries.

The production of huge amount of data and the emergence of cloud computing have introduced new requirements for data management. Many applications need to interact with several heterogeneous data stores depending on the type of data they have to manage: relational and NoSQL (i.e., document, graph, key-value, and column) data stores. Interacting with heterogeneous data models via different APIs and query languages imposes challenging tasks to the developers of multiple data stores applications. Indeed, the execution of complex queries over heterogeneous data models cannot, currently, be achieved in a declarative way as it is used to be with single data store application, and therefore requires extra implementation efforts. In this paper we propose a mediation based component to optimize and execute complex queries over multiple data stores in Cloud environments. This component is referred to as virtual data store (VDS). The key ingredients of our solution are (1) a simple global schema describing the different data sources and their relationships, (2) a cost model to evaluate the cost of the operations, (3) an inter data stores parallelism execution model, and (4) a dynamic programming based approach to generate optimal execution plan. Quantitative and qualitative experiments are conducted to validate our approach.

Cloud computing is an inventive innovation that has acquired a progressive changes in computing administrations. It has empowered to drive the concentration from neighborhood/individual calculation to datacenter-driven algorithm by giving resources powerfully in a virtualized way by means of Internet. The aim of load adjusting is to clearly comprehend the client prerequisites, the information and data which can be sent and obtained without taking additional time. In this paper, we have focused on cloud computing, load balancing techniques, cloud resource scheduling process. Review on load balancing techniques and resource scheduling, sidechannel attack is also presented in this paper.KeywordsCloud computingLoad balancingSidechannel attacksResource scheduling