Design Optimization of Large Scale Products Based on Hierarchical Partitioning Technique of Genetic Algorithm Processes.

Recently, the large-scale optimization problems have become a common research topic in the field of evolutionary computation. It is hard to find optimal solutions when solving large-scale multi-objective optimization problems (LSMOPs), due to the ineffectiveness of existing operators. In the other word, the search ability of most existing MOEAs on solving LSMOPs is still weak. To address this issue, an efficient competitive swarm optimizer with a strong exploration ability, denoted as E-CSO, is presented in this paper, which designs a novel three-particle-based particle updating strategy to improve the search efficiency. The experimental results validate the high efficiency and effectiveness of our proposed approach when solving various LSMOPs.

We propose singular value decomposition (SVD) algorithms for very large-scale matrices based on a low-rank tensor decomposition technique called the tensor train (TT) format. By using the proposed algorithms, we can compute several dominant singular values and corresponding singular vectors of large-scale structured matrices given in a low-rank TT format. We propose a large-scale trace optimization problem, and in the proposed methods, the large-scale optimization problem is reduced to sequential small-scale optimization problems. We show that the computational complexity of the proposed algorithms scales logarithmically with the matrix size if the TT-ranks are bounded. Numerical simulations based on very large-scale Hilbert matrix demonstrate the effectiveness of the proposed methods.

Today, because of the exponential growth of the volume of data in big data applications, large-scale optimization problems (i.e., optimization problems with a large number of decision variables) have become ubiquitous in the real world [219, 220]. In this chapter, building on the algorithms and observations presented for solving both continuous and combinatorial optimization problems, we further present a novel multi-space evolutionary search framework based on EMT for solving large-scale single-objective optimization problem.

Whale optimization algorithm (WOA) is a relatively novel swarm-based optimization algorithm that has the advantage of less control parameters, strong global optimization ability and easy of implementation. However, there is still an insufficiency of low accuracy, premature, and slow convergence speed in tackling large-scale global optimization (LSGO) problems. To address the issue, an enhanced whale optimization algorithm (EWOA), which combines the original WOA with opposition-based learning, chaotic map and dynamic inertia weight, is proposed for solving LSGO problems. Chaotic opposition-based learning initialization strategy by comparing the fitness of an individual to its opposite and retaining well-diversified solutions is employed to accelerate convergence speed and improve the exploration. In addition, in order to make full use of and balance the exploration and exploitation of the original WOA, a novel chaotic cooperative updating strategy of nonlinear control parameter and dynamic inertia weight is given. Moreover, optimal individual chaotic searching strategy is integrated with evolutionary population updating to avoid the possibility of being trapped into local optima. The experimental results on four classical benchmark functions with dimensions ranging from 100 to 1000 demonstrate the effectiveness and efficiency of the chaotic opposition-based learning initialization strategy, chaotic nonlinear control parameter updating strategy and optimal individual chaotic searching strategy. The comparisons show that the proposed EWOA significantly improves the performance of WOA and provides competitive results for LSGO problems compared with state-of-the-art algorithms.

This thesis aims at developing efficient mathematical methods for solving hierarchical dynamic optimization problems. The main motivation is to model processes in nature, for which there is evidence to assume that they run optimally. We describe models of such processes by optimal control problems (called optimal control models (OCMs)). However, an OCM typically includes unknown parameters that cannot be derived entirely on a theoretical basis, which is in particular the case for the cost function. Therefore, we develop parameter estimation techniques to estimate the unknowns in an OCM from observation data of the process. Mathematically, this leads to a hierarchical dynamic optimization problem with a parameter estimation problem on the upper level and an optimal control problem on the lower level. We focus on multi-stage equality and inequality constrained optimal control problems based on nonlinear ordinary differential equations. The main goal of this thesis is to derive numerically efficient mathematical methods for solving hierarchical dynamic optimization problems, and to use these methods to estimate parameters in high-dimensional OCMs from real-world measurement data. We develop parameter-dependent OCMs for the gait of cerebral palsy patients and able-bodied subjects. The unknown parameters in the OCMs are then estimated from real-world motion capture data provided by the Heidelberg MotionLab of the Orthopedic University Clinic Heidelberg by using the mathematical methods developed within this work. The main novelties and contributions of this thesis to the field of hierarchical dynamic optimization are summarized herein. - We establish a novel mathematical method, a so-called direct all-at-once approach, for solving hierarchical dynamic optimization problems based on the direct multiple shooting method and first-order optimality conditions. - Furthermore, we propose an efficient numerical algorithm for large-scale hierarchical dynamic optimization problems, which fully exploits the structures inherited from both the hierarchical setting and the discretization. - Pontryagin's maximum principle is used to analyze solution properties of hierarchical dynamic optimization problems like second-order optimality conditions of the lower-level problem. - In addition, we propose and discuss alternative methods for hierarchical dynamic optimization that are based on derivative-free optimization and a bundle approach. These methods keep the hierarchical problem setting and do not reformulate the lower-level problem using first-order optimality conditions. - We establish a novel lifting method for regularizing mathematical programs with complementarity constraints, which is discussed and numerically investigated by means of a well-known collection of benchmark problems. - Proofs of regularity and convergence results for sequential quadratic programming methods applied to lifted mathematical programs with complementarity constraints are provided. - Efficient state-of-the-art implementations of all mathematical methods derived in this thesis, as well as a benchmark collection of hierarchical dynamic optimization problems are presented. - High-dimensional optimal control gait models for cerebral palsy patients and able-bodied subjects are developed. The mathematical methods derived in this thesis are used to estimate the unknown model parameters from real-world motion capture data provided by the Heidelberg MotionLab of the Orthopedic University Clinic Heidelberg. The theoretical and practical results presented in this thesis can be considered an initial motivating step towards answering open questions in current medical research in fields like treatment planning, classification of gaits, or the evaluation of surgeries by means of hierarchical dynamic optimization.

The minimization of the molecular potential energy function is one of the most important real-life problems which can help to predict the 3D structure of the protein by knowing the steady (ground) state of the molecules of the protein. In this paper, we propose a new hybrid algorithm between the social spider algorithm and the genetic algorithm in order to minimize a simplified model of the energy function of the molecule. We call the proposed algorithm by hybrid social spider optimization and genetic algorithm (HSSOGA). The HSSOGA comprises of three main steps. In the first step, we apply the social spider optimization algorithm to balance between the exploration and the exploitation processes in the proposed algorithm. In the second step, we use the dimensionality reduction process and the population partitioning process by dividing the population into subpopulations and applying the arithmetical crossover operator for each subpopulation order to increase the diversity of the search in the algorithm. In the last steps, we use the genetic mutation operator in the whole population to avoid the premature convergence and avoid trapping in local minima. The combination of three steps helps the proposed algorithm to solve the molecular potential energy function with different molecules size, especially when the problem dimension $$D>100$$D>100 with powerful performance. We test it on 13 large-scale unconstrained global optimization problems to investigate its performance on these functions. In order to investigate the efficiency of the proposed HSSOGA, we compare HSSOGA against eight benchmark algorithms when we minimize the potential energy function problem. The numerical experiment results show that the proposed algorithm is a promising and efficient algorithm and can obtain the global minimum or near global minimum of the large-scale optimization problems up to 1000 dimension and the molecular potential energy function of the simplified model with up to 200 degrees of freedom faster than the other comparative algorithms.

Genetic algorithms for large-scale optimization in chemometrics: An application

Although there are many studies on large-scale unconstrained optimization (e.g., with 100 to 1000 variables) and small-scale constrained optimization (e.g., with 10 to 30 variables) using nature-inspired algorithms (e.g., evolutionary algorithms and swarm intelligence algorithms), no publicly available nature-inspired algorithm is developed for large-scale constrained optimization. In this paper, we combine a cooperative coevolutionary particle swarm optimization (CCPSO) algorithm with the e constrained method to solve large-scale real-valued constrained optimization problems. The eCCPSO framework is proposed, and three different algorithms based on the framework, i.e., eCCPSOd, eCCPSOw and eCCPSOw2, are developed. The proposed algorithms compare favorably to the state-of-the-art constrained optimization algorithm eDEag on large-scale problems. The experimental results further suggest that eCCPSOw2 with adaptive improvement detection technique is highly competitive compared with the other algorithms considered in this work for solving large-scale real-valued constrained optimization problems.

This paper presents enhanced adaptive differential evolution (EADE) algorithm for solving high-dimensional optimization problems over continuous space. To utilize the information of good and bad vectors in the DE population, the proposed algorithm introduces a new mutation rule. It uses two random chosen vectors of the top and bottom 100p% individuals in the current population of size NP, while the third vector is selected randomly from the middle [NP-2(100p%)] individuals. The mutation rule is combined with the basic mutation strategy DE/rand/1/bin, where the only one of the two mutation rules is applied with the probability of 0.5. This new mutation scheme helps to maintain effectively the balance between the global exploration and local exploitation abilities for searching process of the DE. Furthermore, we propose a novel self-adaptive scheme for gradual change of the values of the crossover rate that can excellently benefit from the past experience of the individuals in the search space during evolution process which, in turn, can considerably balance the common trade-off between the population diversity and convergence speed. The proposed algorithm has been evaluated on the 7 and 20 standard high-dimensional benchmark numerical optimization problems for both the IEEE CEC-2008 and the IEEE CEC-2010 Special Session and Competition on Large-Scale Global Optimization. The comparison results between EADE and its version and the other state-of-art algorithms that were all tested on these test suites indicate that the proposed algorithm and its version are highly competitive algorithms for solving large-scale global optimization problems.

Cooperative coevolution (CC) algorithms based on variable decomposition methods are efficient in solving large-scale optimization problems (LSOPs). However, many decomposition methods, such as the differential grouping (DG) method and its variants, are based on the theorem of function additively separable, which may not work well on problems that are not additively separable and will result in a bottleneck for CC to solve various LSOPs. This deficiency motivates us to study how the decomposition method can decompose more kinds of separable functions, such as the multiplicatively separable function, to improve the general problem-solving ability of CC on LSOPs. With this concern, this article makes the first attempt to decompose multiplicatively separable functions and proposes a novel method called dual DG (DDG) for better LSOP decomposition and optimization. The novelty and advantage of DDG are that it can be suitable for not only additively separable functions but also multiplicatively separable functions, which can considerably expand the application scope of CC. In this article, we will first define the multiplicatively separable function, and then mathematically show its relationship to the additively separable function and how they can be transformed into each other. Based on this, the DDG can use two kinds of differences to detect the separable structure of both additively and multiplicatively separable functions. In addition, the time complexity of DDG is analyzed and a DDG-based CC algorithm framework is developed for solving LSOPs. To verify the superiority of DDG, experiments and comparisons with some state-of-the-art and champion algorithms are conducted not only on 30 LSOPs based on the test suite of the IEEE CEC large-scale global optimization competition, but also on a case study of the parameter optimization for a neural network-based application.

Meta-heuristic algorithms are widely viewed as feasible techniques to solve continuous large-scale numerical optimization problems. Grey wolf optimizer (GWO) is a relatively new stochastic algorithm with only a few parameters to adjust that can be easily used for global optimization. This paper presents an efficient and robust GWO (ERGWO) variant to solve large-scale numerical optimization problems. Inspired by particle swarm optimization, a nonlinearly adjustment strategy for parameter control is designed to balance exploration and exploitation. Additionally, a modified position-updating equation is presented to improve convergence speed. The performance of ERGWO is verified on 18 benchmark large-scale numerical optimization problems with dimensions ranging from 30 to 10,000, 30 benchmarks from CEC 2014, 30 functions in CEC 2017, respectively. Numerical experiments are performed to compare ERGWO to the basic GWO algorithm, other GWO variants, and other well-known meta-heuristic search techniques. Simulations demonstrate that the proposed ERGWO algorithm can find high quality solutions with low computational cost and very fast convergence.

This paper addresses large-scale urban transportation optimization problems with time-dependent continuous decision variables, a stochastic simulation-based objective function, and general analytical differentiable constraints. We propose a metamodel approach to address, in a computationally efficient way, these large-scale dynamic simulation-based optimization problems. We formulate an analytical dynamic network model that is used as part of the metamodel. The network model formulation combines ideas from transient queueing theory and traffic flow theory. The model is formulated as a system of equations. The model complexity is linear in the number of road links and is independent of the link space capacities. This makes it a scalable model suitable for the analysis of large-scale problems. The proposed dynamic metamodel approach is used to address a time-dependent large-scale traffic signal control problem for the city of Lausanne. Its performance is compared to that of a stationary metamodel approach. The proposed approach outperforms the stationary approach. This comparison illustrates the added value of providing the algorithm with analytical dynamic problem-specific structural information. The performance of a signal plan derived by the proposed approach is also compared to that of an existing signal plan for the city of Lausanne, and to that of a signal plan derived by a mainstream commercial signal control software. The proposed method can systematically identify signal plans with good performance. The online appendix is available at https://doi.org/10.1287/trsc.2016.0717 .

A novel approach using transient chaotic neural network (TCNN) and self-organizing feature map (SOFM) process to solve large-scale combinatorial optimization problems has been proposed. With the clustering function of self-organizing feature map, the computational cost of a large-scale combinatorial optimization problem solved by TCNN is reduced. Numerical simulation of TSP shows that the proposed method is effective to solve large-scale optimization problems.

Genetic algorithms are able to search very large, variable complex spaces and locate the global optimum. However, there exist many difficulties in applying GA to large-scale nonlinear system optimization or "GA hard" problems. This paper presents an improved GA based on domain evolution model and fuzzy environment factor. Simulation study shows that it is a powerful search technique which can avoid premature convergence and locate the real global optimum. An example is given to show how this new algorithm can be successfully applied to solve large-scale industrial chemical separation process optimization problem.

Column Generation is a very powerful class of combinatorial optimization algorithms that has been used successfully to solve a variety of large scale optimization problems. Its application has helped many companies in various industries increase revenue and reduce costs significantly, particularly in transportation, energy, manufacturing, and telecommunication companies. In this chapter, we will first discuss the motivations for column generation, then we will provide an intuitive but rigorous treatment of the mechanisms of column generation – how it works, why it works. We will then give descriptions on the branch and price algorithm and several examples of column generation’s successful applications in one of the world’s largest airlines. We will discuss monthly airline crew schedule optimization for bidlines, crew pairing optimization, and integrated modeling of fleet and routing in the optimization of aircraft scheduling. Part of the focus is on business requirements and priorities in these areas and how the column generation models are built to effectively meet these challenges. Some airline industry domain-specific details are provided to allow the readers to better appreciate the scheduling problems’ complexities that made the master-subproblem approach in column generation essential. We will also discuss the significant run-time speedups for these large scale scheduling problems due to various practical model enhancements, as well as progress in the large scale optimization space made possible by technologies such as parallel processing, big data, and better chips. At last, we will briefly discuss several example variants of column generation and their applications in various industries. We will also review recent applications of optimization techniques to machine learning as well as the future potentials of large scale optimization in this field. This chapter can be used as a primer on the fundamentals of column generation techniques since it clearly addresses essential theoretical concepts that are sometimes elusive to researchers and graduate students who are new to this area. The chapter should also be helpful to practitioners who would like to gain insights into how to build effective column generation models to solve real world large scale optimization problems.