Performance analysis of randomized algorithm for optimal query plan generation in distributed environment

The technique of finding the optimal processing method to answer a query is called Query optimization, whereas a collection of various sites, distributed over a computer network is called Distributed Database. In Distributed Database, the site communicates with each other through networks. There are various issues arise during evaluation of query cost, among which the processing cost and a transmission cost are important. There are several algorithms developed to find the best possible solution for a particular query, but they all have their certain limitations. The optimizer is mainly concern on search space, search strategy, and the cost model. It primarily focuses on these three factors. The mining cost of a query depends on the order of evaluation of the operators, for the same query we can have different cost if the order is changed. Hence, to find the optimal cost for a particular query is emerging as an open challenge for many researchers. Therefore, the cost-based query optimization technique has emerged as an important concept for dealing with the query optimization. This paper explores the issues and challenges of query optimization in mining of distributed data.

Logical query optimization for Cloudera Impala system

A combined strategy of an improved simulated annealing (SA) algorithm and genetic algorithm is presented, with the goal of reducing the computational expenses. The improvements made on the SA algorithm include two parts, i.e., the adaptive regulating for the step vector, and the dynamic testing for the equilibrium of the Metropolis process. The proposed strategy has both the advantage of SA algorithm, the ability to avoid being trapped in a local optimum, and that of genetic algorithm, the ability to use the information about the searched states for the next iteration. A practical application on geometry optimization of pole shoes in large salient pole synchronous generators is effectively implemented using the strategy. The numerical results show that the number of iterations used by executing the combined strategy are only about 75% of those by executing basic SA algorithm.

LSIR

Summary The Simulated Annealing (SA) algorithm has been widely used in geophysical data processing. However, the seri al SA algorithm is limited by low computing efficiency. In this paper, the annealing schedule has been improved in the SA algorithm, and a constrained inversion of magnetotelluric (MT) data using the parallel SA algorithm has been applied. The test of synthetic data shows that the improved Very Fast Simulated Annealing (VFSA) method is significantly better than the traditional method. Additionally, the computing efficiency has been improved using the parallel SA inversion algorithm. This method has also been applied in MT data processing for geothermal exploration and good geological results have been achieved.

This paper addressed the modeling and optimization of loading path in T‐shape hydroforming of tubes using Simulated Annealing (SA) algorithm. Analysis of variance shows that some of pre‐selected parameters in loading paths have not significant effect on the deformed tube. Hence, some of optimized parameters found initially, are replaced with their own fixed optimum values in order to seek for the other parameters in more detail by the Simulated Annealing (SA) algorithm. According to the intensity of effectiveness on the deformation, six more important parameters are chosen and their minimum and maximum limitation values are determined. In this case, sixty four different tests for different loading paths are designed by Design of Experiment (DOE) and full factorial method. By using mathematical modeling all required loading parameters are obtained. Proposed models of formability embedded into Simulated Annealing algorithm and optimum value for loading parameters and optimal load paths are found. The obtained results show that more accurate loading path may be found for T‐shape of tube hydroforming.

A hybrid imperialist competitive-simulated annealing algorithm for a multisource multi-product location-routing-inventory problem

Since the very beginning of query processing in database systems, cost-based query optimization has been the essential strategy for effectively answering complex queries on large documents. XML documents can be efficiently stored and processed using native XML database management systems. Even though such systems can choose from a huge repertoire of join operators (e. g., Structural Joins and Holistic Twig Joins) and various index access operators to efficiently evaluate queries on XML documents, the development of full-fledged XML query optimizers is still in its infancy. Especially the evaluation of complex XQuery expressions using these operators is not well understood and needs further research. The extensible, rule-based, and cost-based XML query optimization framework proposed in this chapter, serves as a testbed for exploring how and whether well-known concepts from relational query optimization (e. g., join reordering) can be reused and which new techniques can make a significant contribution to speed-up query execution. Using the best practices and an appropriate cost model that will be developed using this framework, it can be turned into a robust cost-based XML query optimizer in the future.

Query optimizer is at the heart of the database systems. Cost-based optimizer studied in this paper is adopted in almost all current database systems. A cost-based optimizer introduces a plan enumeration algorithm to find a (sub)plan, and then uses a cost model to obtain the cost of that plan, and selects the plan with the lowest cost. In the cost model, cardinality, the number of tuples through an operator, plays a crucial role. Due to the inaccuracy in cardinality estimation, errors in cost model, and the huge plan space, the optimizer cannot find the optimal execution plan for a complex query in a reasonable time. In this paper, we first deeply study the causes behind the limitations above. Next, we review the techniques used to improve the quality of the three key components in the cost-based optimizer, cardinality estimation, cost model, and plan enumeration. We also provide our insights on the future directions for each of the above aspects.

In this paper, simulated annealing (SA) algorithm is employed to solve the flowshop scheduling problem with the objective of minimising the completion time variance (CTV) of jobs. Four variants of the two-phase SA algorithm (SA-I to SA-IV) are proposed to solve flowshop scheduling problem with the objective of minimising the CTV of jobs without considering the right shifting of completion time of jobs on the last machine. The proposed SA algorithms have been tested on 90 benchmark flowshop scheduling problems. The solutions yielded by the proposed SA variants are compared with the best CTV of jobs reported in the literature, and the proposed SA-III is found to perform well in minimising the chosen performance measure (CTV) particularly in medium and large size problems.

A modification of the simulated annealing (SA) algorithm for solving discrete stochastic optimization problems where the objective function is stochastic and can be evaluated only through Monte Carlo simulation is proposed. In this modification, the Metropolis criterion depends on whether the objective function values indicate a statistically significant difference at each iteration. The differences between objective function values are considered to be statistically significant based on confidence intervals associated with these values. Unlike the original SA, the proposed method uses a constant temperature. It is shown that the configuration that has been visited most often in the first k iterations converges almost surely to a global optimizer. Computational results and comparisons with other SA algorithms are presented to demonstrate the performance of the proposed SA algorithm.

Query Processing is a key determinant in the overall performance of distributed databases. It requires processing of data at their respective sites and transmission of the same between them. These together constitute a distributed query processing strategy (DQP). DQP aims to arrive at an efficient query processing strategy for a given query. This strategy involves generation of efficient query plans for a distributed query. In case of distributed relational queries, the number of possible query plans grows exponentially with an increase in the number of relations accessed by the query. This number increases further when the relations, accessed by the query, have replicas at different sites. Such a large search space renders it infeasible to find optimal query plans. This paper presents a query plan generation algorithm that attempts to generate optimal query plans, for a given query, using genetic algorithm. The query plans so generated involve fewer sites, thus leading to efficient query processing. Further, experimental results show that the proposed algorithm converges quickly towards optimal query plans for an observed crossover and mutation probability.

Maintenance planning aims to improve the reliability of assets, prevent the occurrence of asset failures, and reduce maintenance costs associated with downtime of assets and maintenance resources (such as spare parts and workforce). Thus, effective maintenance planning is instrumental in ensuring high asset availability with the minimum cost. Nevertheless, to find such optimal planning is a nontrivial task due to the (i) complex and usually nonlinear inter-relationship between different planning decisions (e.g., inventory level and workforce capacity), and (ii) stochastic nature of the system (e.g., random failures of parts installed in assets). To alleviate these challenges, we study a joint maintenance planning problem by considering several decisions simultaneously, including workforce planning, workforce training, and spare parts inventory management. We develop a hybrid solution algorithm (DRLSA\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {DRLSA}$$\\end{document}) that is a combination of Double Deep Q-Network based Deep Reinforcement Learning (DRL) and Simulated Annealing (SA) algorithms. In each episode of the proposed algorithm, the best solution found by DRL is delivered to SA to be used as an initial solution, and the best solution of SA is delivered to DRL to be used as the initial state. Different from the traditional SA algorithms where neighborhood structures are selected only randomly, the DRL part of DRLSA\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {DRLSA}$$\\end{document} learns to choose the best neighborhood structure to use based on experience gained from previous episodes. We compare the performance of the proposed solution algorithm with several well-known meta-heuristic algorithms, including Simulated Annealing, Genetic Algorithm (GA), and Variable Neighborhood Search (VNS). Further, we also develop a Machine Learning (ML) algorithm (i.e., K-Median) as another benchmark in which different properties of spare parts (e.g., failure rates, holding costs, and repair rates) are used as clustering features for the ML algorithm. Our study reveals that the DRLSA\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {DRLSA}$$\\end{document} finds the optimal solutions for relatively small-size instances, and it has the potential to outperform traditional meta-heuristic and ML algorithms.

As demonstrated in Chap. 3, different evolutionary algorithms, including genetic algorithm (GA), simulated annealing (SA) algorithm, hybrid GASA algorithm, particle swarm optimization (PSO) algorithm, continuous ant colony optimization (CACO) algorithm, artificial bee colony (ABC) algorithm, and immune algorithm (IA), are employed to determine suitable for parameter combination of an SVR-based electric load forecasting model. These forecasting results indicate that almost all SVR-based models with different evolutionary algorithms are superior to other competitive forecasting models (including ARIMA, HW, GRNN, and BPNN models); however, these algorithms almost lack knowledge memory or storage mechanisms which would be neither time-consuming nor inefficient in searching the suitable parameters, that is, premature convergence (being trapped in local optimum). For example, for SVRGA model, in GA processing, new individuals are generated by the following operators, selection, crossover, and mutation. For all types of objective functions, the generation begins with a binary coding for the parameter set. Based on this special binary coding process, GA is able to solve some specified problems which are not easily to be solved by traditional algorithms. GA can empirically provide a few best fitted offsprings from the whole population; however, after some generations, due to low diversity of the population, it might lead to a premature convergence [1]. For SVRSA model, SA algorithm is a generic probabilistic search technique that simulates the material physical process of heating and controlled cooling. Each step of SA attempts to replace the current state by a random move. The new state may then be accepted with a probability that depends both on the difference between the corresponding function values and also on a global parameter, temperature. Thus, SA has some institution to reach more ideal solutions. However, SA algorithm requires the subtle and skillful adjustment in the annealing schedule; therefore, the settings of the size of the temperature steps during annealing, the temperature range, the number of restarts, and the redirection of the search will be taken into account carefully [1]; in addition, its Monte Carlo scheme and lack of knowledge memory mechanism will also lead to time-consuming and inefficient searching in annealing process.

In this work we propose a highly optimized version of a simulated annealing (SA) algorithm adapted to the more recently developed graphic processor units (GPUs). The programming has been carried out with compute unified device architecture (CUDA) toolkit, specially designed for Nvidia GPUs. For this purpose, efficient versions of SA have been first analyzed and adapted to GPUs. Thus, an appropriate sequential SA algorithm has been developed as starting point. Next, a straightforward asynchronous parallel version has been implemented and then a specific and more efficient synchronous version has been developed. A wide appropriate benchmark to illustrate the performance properties of the implementation has been considered. Among all tests, a classical sample problem provided by the minimization of the normalized Schwefel function has been selected to compare the behavior of the sequential, asynchronous and synchronous versions, the last one being more advantageous in terms of balance between convergence, accuracy and computational cost. Also the implementation of a hybrid method combining SA with a local minimizer method has been developed. Note that the generic feature of the SA algorithm allows its application in a wide set of real problems arising in a large variety of fields, such as biology, physics, engineering, finance and industrial processes.