A Novel Hybrid OptimizationAquila Flamingo SearchOptimizer and Genetic Algorithm for Efficient ETL Processes, Materialized View Selection, and Optimal Query Recommendation in Cloud-Based Data Warehousing

A Data warehouse is a structure that stores big amount of data. This data is exploited in the best possible ways in order to improve the efficiency of decision-making. The huge volume of data makes answering queries complex and time-consuming. Therefore, materialized views are used in order to reduce the query processing time. Since materializing all views is not possible, due to space and maintenance constraints, materialized view selection became one of the crucial decisions in designing a data warehouse for optimal efficiency. In this paper, we propose a discrete - quantum based - version of Differential Evolution DE algorithm named QDE algorithm to solve the materialized view selection (MVS) problem with space constraint. This algorithm is a merging of the original DE with Quantum Evolutionary (QEA) algorithm. The experimental results show the efficiency of the proposed algorithm compared to well known algorithms used to solve MVS problem with space constraint such as HRUA and GEA.

A data warehouse is designed for answering analytical queries. These queries are usually long, complex and ad hoc in nature and their response time is high when processed against an exponentially growing data warehouse. Materialising views has been found to be an effective tool for reducing this response time. All views cannot be materialised on account of resource constraints. Further, optimal view selection is shown to be an NP-complete problem. This necessitates selection of an appropriate set of views, from amongst all possible views that reduce the query response time. Most of the view selection algorithms are greedy or evolutionary. In this paper, a differential evolution view selection algorithm (DEVSA) that selects the Top-K views from a multi-dimensional lattice is proposed. Further, it is shown that DEVSA when compared to the greedy, genetic and memetic-based view selection algorithms selects comparatively better quality views for higher dimensional datasets.

The traditional single intelligent algorithm has slow convergence speed in the frequency assignment, and the effect cannot meet the increasing frequency equipment in the naval battlefield. Based on the single intelligent frequency assignment algorithm and the frequency conflict analysis model between systems, this paper proposes three heuristic frequency assignment algorithms based on hybrid genetic algorithms, namely, Genetic Algorithm and Tabu Search (GATS), Hybrid Genetic Simulated Annealing Algorithm (HGSAA) and Genetic Algorithm-ant Colony Algorithm (GAA). The simulation results show that the hybrid algorithm can quickly converge to a better allocation result than the single intelligent algorithm in the frequency assignment problem, and the optimal value is better than the single intelligent algorithm. Among them, the convergence speed and optimal value of the GAA are the best of the three algorithms. Therefore, this algorithm can be applied to frequency assignment with more frequency-using equipment within a fixed range.

A hybrid genetic local search algorithm for the permutation flowshop scheduling problem

A data warehouse is a central repository of historical data designed primarily to support analytical processing. These analytical queries are exploratory, long and complex in nature. Further, the rapid and continuous growth in the size of data warehouse increases the response times of such queries. Query response times need to be reduced in order to speedup decision making. This problem, being an NP-Complete problem, can be appropriately dealt with by using swarm intelligence techniques. One such technique, i.e. the set-based particle swarm optimization (SPSO), has been proposed to address this problem. Accordingly, a SPSO based view selection algorithm (SPSOVSA), which selects the Top-K views from a multidimensional lattice, is proposed. Experimental based comparison of SPSOVSA with the most fundamental view selection algorithm shows that SPSOVSA is able to select comparatively better quality Top-K views for materialization. The materialization of these selected views would improve the performance of analytical queries and lead to efficient decision making.

For speeding up query processing on Big Data, frequent sub-queries or views may be materialized such that the query processing cost is minimized with optimum cost of maintaining the materialized views and/or queries. Materializing frequent sub-queries and views means that resultant data set of the views reside in the memory of one or more nodes in the cluster, so that it reduces the MapReduce cost, submission and scheduling cost of Distributed File System jobs for query processing. We have defined materialized views as resultant data of frequent sub-queries and aggregation functions of a set of Big Data warehousing queries that are saved for enhancing query performance. The problem is defined as a multi-objective optimization problem for minimizing the total query processing MapReduce cost, MapReduce cost for maintaining the materialized views and the number of views selected for materializing with maximized total size of the views selected. We applied Differential Evolution algorithm and NSGA-II to study their performances for developing a recommendation system for selecting views for materializing in Big Data warehousing.

Materialized view selection under the maintenance time constraint

This paper proposed and implemented an genetic and weed invasion hybrid algorithm in Sparse Array Synthesis. The classical weed invasion algorithm has high requirements on the convergence parameters in the process of generating weeds from seeds. Inappropriate parameters can easily lead to local convergence of evolution. The mutation and crossover operations similar to the genetic algorithm were introduced into the classical weed invasion algorithm. The method improves convergence speed, avoids local convergence, and enhances the stability of the algorithm. The algorithm is used in practical application in the synthesis of linear sparsely distributed array pattern, and the predetermined index is obtained through calculation. At the same time, compared with the genetic algorithm and the classical weed invasion algorithm, the results show that the hybrid algorithm has certain advantages.


 One of the most significant aspects of the working world is the concept of a picket schedule. It is difficult for the scheduler to make an archive since there are frequently many issues with the picket schedule. These issues include schedule clashes, requests for leave, and trading schedules. Evolutionary algorithms have been successful in solving a wide variety of scheduling issues. Evolutionary algorithms are very susceptible to data convergence. But no one has discussed where to start from, where the data converges from making schedules using evolutionary algorithms. The best algorithms among evolutionary algorithms for scheduling are genetic algorithms and memetics algorithms. When it comes to the two algorithms, using genetic algorithms or memetics algorithms may not always offer the optimum outcomes in every situation. Therefore, it is necessary to compare the genetic algorithm and the algorithm's memetic algorithm to determine which one is suitable for the nurse picket schedule. From the results of this study, the memetic algorithm is better than the genetic algorithm in making picket schedules. The memetic algorithm with a population of 10000 and a generation of 5000 does not produce convergent data. While for the genetic algorithm, when the population is 5000 and the generation is 50, the data convergence starts. For accuracy, the memetic algorithm violates only 24 of the 124 existing constraints (80,645%). The genetic algorithm violates 27 of the 124 constraints (78,225%). The average runtime used to generate optimal data using the memetic algorithm takes 20.935592 seconds. For the genetic algorithm, it takes longer, as much as 53.951508 seconds.

Data Warehousing is an approach in which data from multiple heterogeneous and distributed operational systems (OLTP) are extracted, transformed and loaded into a central repository for the purpose of decision making. Since such database stores huge amounts of historical data, it is necessary to devise methods by which complex OLAP queries can be answered as fast as possible. OLAP is an approach which facilitates analytical queries accessing multidimensional databases. Using materialized views as pre-computed results for time-consuming queries is a common method for speeding up analytical queries. However, some constraints do not allow the systems to save all possible views. Therefore, one of the crucial decisions that data warehouse designers need to make is in the selection of the right set of views to be materialized. This paper focuses on solving the multi-objective view selection problem using a Vector Evaluated Genetic Algorithm (VEGA) approach subject to disk space constraint.

Many applications use a combination of a local optimum search (LOS) and a genetic algorithm (GA), called a hybrid genetic algorithm (HGA), to solve problems. This hybrid can improve the performance of finding optimum solutions, but the HGA may produce redundancy when applying an LOS to inappropriate populations. This redundancy is the cause of high computation time, and it generates premature convergence and decreases HGA performance. This research therefore aims to reduce redundant LOS in HGAs. We propose a new technique called diversity selection (DS) and measure redundancy when applying an LOS in HGA. In this work, the DS selects appropriate populations to which to apply an LOS. The experiment then compares DS with other HGAs on numerical optimization problems. In addition, the HGAs were tested on two LOSs, a Nelder-Mead method and a quasi-Newton method to compare the speed of finding the optimum point in an LOS. The experimental results show that DS was able to quickly find the optimum point and give fewer redundant LOSs than other HGAs.

Materialized View Selection in Data Warehousing: A Survey

Materialized views have been found to be very effective at speeding up queries, and are increasingly being supported by commercial databases and data warehouse systems. However, whereas the amount of data entering a warehouse and the number of materialized views are rapidly increasing, the time window available for maintaining materialized views is shrinking. These trends necessitate efficient techniques for the maintenance of materialized views.In this paper, we show how to find an efficient plan for the maintenance of a set of materialized views, by exploiting common subexpressions between different view maintenance expressions. In particular, we show how to efficiently select (a) expressions and indices that can be effectively shared, by transient materialization; (b) additional expressions and indices for permanent materialization; and (c) the best maintenance plan — incremental or recomputation — for each view. These three decisions are highly interdependent, and the choice of one affects the choice of the others. We develop a framework that cleanly integrates the various choices in a systematic and efficient manner. Our evaluations show that many-fold improvement in view maintenance time can be achieved using our techniques. Our algorithms can also be used to efficiently select materialized views to speed up workloads containing queries and updates.

Container terminals help countries to sustain their economic development. Improving the operational efficiency in a container terminal is important. In past research, genetic algorithms (GAs) have been widely used to cope with seaside operational problems, including the berth allocation problem (BAP) and quay crane assignment problem (QCAP) individually or simultaneously. However, most GA approaches in past studies were dedicated to generate time-invariant QC assignment that does not adjust QCs assigned to a ship. This may underutilize available QC capacity. In this research, three hybrid GAs (HGAs) have been proposed to deal with the dynamic and discrete BAP (DDBAP) and the dynamic QCAP (DQCAP) simultaneously. The three HGAs supports variable QC assignment in which QCs assigned to a ship can be further adjusted. The three HGAs employ the same crossover operator but a different mutation operator and a two-stage procedure is used. In the first stage, these HGAs can generate a BAP solution and a QCAP solution that is time-invariant. The time-invariant QC assignment solution is then further transformed into a variable one in the second stage. Experiments have been conducted to investigate the effects of the three HGA and the results showed that these HGAs outperformed traditional GAs in terms of fitness value. In particular, the HGA3 with Thoros mutation operator had the best performance.

The job shop scheduling problem (JSSP) is a well-known difficult combinatorial optimization problem, as it is classified as NP-hard problem and therefore no deterministic algorithms can solve them in a reasonable amount of time. The main objective of solving this problem is to find suitable job sequences on machines to optimize the performance criteria. In this paper, a meta-heuristic approach for solving the job-shop scheduling problem (JSSP) is presented. This approach uses a hybrid genetic algorithm that is suggested by previous stuody, to generate the best solutions and then a simulated annealing algorithm to improve the quality and performance of the best solutions to produce the optimal/near-optimal solution. Ten benchmark problems adopted from the previous study are used to evaluate the performance of the proposed algorithm. The computational results validate the quality of the proposed algorithm; this is done by calculating the completion time Cmax