Performance analysis of algorithms for large-scale nonlinear optimization

A large-scale continuous optimization benchmark suite with versatile coupled heterogeneous modules

Meta-heuristic algorithms are widely viewed as feasible techniques to solve continuous large-scale numerical optimization problems. Grey wolf optimizer (GWO) is a relatively new stochastic algorithm with only a few parameters to adjust that can be easily used for global optimization. This paper presents an efficient and robust GWO (ERGWO) variant to solve large-scale numerical optimization problems. Inspired by particle swarm optimization, a nonlinearly adjustment strategy for parameter control is designed to balance exploration and exploitation. Additionally, a modified position-updating equation is presented to improve convergence speed. The performance of ERGWO is verified on 18 benchmark large-scale numerical optimization problems with dimensions ranging from 30 to 10,000, 30 benchmarks from CEC 2014, 30 functions in CEC 2017, respectively. Numerical experiments are performed to compare ERGWO to the basic GWO algorithm, other GWO variants, and other well-known meta-heuristic search techniques. Simulations demonstrate that the proposed ERGWO algorithm can find high quality solutions with low computational cost and very fast convergence.

Although there are many studies on large-scale unconstrained optimization (e.g., with 100 to 1000 variables) and small-scale constrained optimization (e.g., with 10 to 30 variables) using nature-inspired algorithms (e.g., evolutionary algorithms and swarm intelligence algorithms), no publicly available nature-inspired algorithm is developed for large-scale constrained optimization. In this paper, we combine a cooperative coevolutionary particle swarm optimization (CCPSO) algorithm with the e constrained method to solve large-scale real-valued constrained optimization problems. The eCCPSO framework is proposed, and three different algorithms based on the framework, i.e., eCCPSOd, eCCPSOw and eCCPSOw2, are developed. The proposed algorithms compare favorably to the state-of-the-art constrained optimization algorithm eDEag on large-scale problems. The experimental results further suggest that eCCPSOw2 with adaptive improvement detection technique is highly competitive compared with the other algorithms considered in this work for solving large-scale real-valued constrained optimization problems.

In this paper, we propose a framework to accelerate the computational efficiency of evolutionary algorithms on large-scale multiobjective optimization. The main idea is to track the Pareto optimal set (PS) directly via problem reformulation. To begin with, the algorithm obtains a set of reference directions in the decision space and associates them with a set of weight variables for locating the PS. Afterwards, the original large-scale multiobjective optimization problem is reformulated into a low-dimensional single-objective optimization problem. In the reformulated problem, the decision space is reconstructed by the weight variables and the objective space is reduced by an indicator function. Thanks to the low dimensionality of the weight variables and reduced objective space, a set of quasi-optimal solutions can be obtained efficiently. Finally, a multiobjective evolutionary algorithm is used to spread the quasi-optimal solutions over the approximate Pareto optimal front evenly. Experiments have been conducted on a variety of large-scale multiobjective problems with up to 5000 decision variables. Four different types of representative algorithms are embedded into the proposed framework and compared with their original versions, respectively. Furthermore, the proposed framework has been compared with two state-of-the-art algorithms for large-scale multiobjective optimization. The experimental results have demonstrated the significant improvement benefited from the framework in terms of its performance and computational efficiency in large-scale multiobjective optimization.

In recent years, much work has been done on implementing a variety of algorithms in nonlinear programming software. In this paper, we analyze the performance of several state-of-the-art optimization codes on large-scale nonlinear optimization problems. Extensive numerical results are presented on different classes of problems, and features of each code that make it efficient or inefficient for each class are examined.Keywordsinterior-point methodslarge-scale optimizationnonlinear programming

Ant-colony optimization (ACO) is a popular swarm intelligence metaheuristic scheme that can be applied to almost any optimization problem. In this paper, we address a performance evaluation of an ACO-based algorithm for solving large-scale global optimization problems with continuous variables, labeled Differential Ant-Stigmergy Algorithm (DASA). The DASA transforms a real-parameter optimization problem into a graph-search problem. The parameters' differences assigned to the graph vertices are used to navigate through the search space. The performance of the DASA is evaluated on the set of benchmark problems provided for CEC'2010 Special Session and Competition on Large-Scale Global Optimization.

The scalability performance of the traditional evolutionary algorithms (EAs) deteriorates rapidly as the dimensionality of the optimization problems increases. Therefore, cooperative coevolutionary (CC) framework is proposed to overcome the defect. Different from existing CC algorithms, a novel self-adaptive based cooperative coevolutionary (SaCC) algorithm is presented in this paper. The SaCC employs three algorithms which with self-adaptive mechanism as sub-algorithms. The focus of this paper is on investigating two different cooperative coevolutionary manners. In the first manner, SaCC executes its sub-algorithms in parallel during evolve process and the corresponding algorithm is termed as SaCC-M1. In the second manner, SaCC executes its sub-algorithms in serial and the corresponding algorithm is termed as SaCC-M2. 26 test functions with 1000 dimensionalities are employed to verify the validity of SaCC-M1 and SaCC-M2. Experiment results demonstrate that SaCC-M2 outperforms its sub-algorithms and SaCC-M1. Besides, the results indicate that serial manner is another simple yet efficient manner for CC algorithms to solve large-scale global optimization problems.

Many engineering optimization problems are typically multi-objective in their natures and multidisciplinary with a large number of decision variables. Furthermore, Pareto dominance loses its effectiveness in such situations. Thus, developing a robust optimization algorithm undoubtedly becomes a true challenge. This paper proposes a multi-objective orthogonal opposition-based crow search algorithm (M2O-CSA) for solving large-scale multi-objective optimization problems (LSMOPs). In the M2O-CSA, a multi-orthogonal opposition strategy is employed to mitigate the conflicts among the convergence and distribution of solutions. First, two individuals are randomly chosen to undergo the crossover stage and then orthogonal array is presented to obtain nine individuals. Then individuals are used in the opposition stage to improve the diversity of solutions. The effectiveness of the proposed M2O-CSA is investigated by implementing it on different dimensions of multi-objective optimization problems (MOPs). The Pareto front solutions of these MOPs have various characteristics such as convex, non-convex and discrete. It is also applied to solve multi-objective design applications with distinctive features such as four bar truss (FBT) design, welded beam (WB) deign, disk brake (DB) design, and speed reduced (SR) design, where they involve different characteristics. In this context, a new decision making tool based on multi-objective optimization on the basis of ratio analysis (MOORA) technique is employed to help the designer for extracting the operating point as the best compromise or satisfactory solution to execute the candidate engineering design. Simulation results affirm that the proposed M2O-CSA works efficiently and effectively.

ABSTRACTThis paper proposes a nonmonotone scaled conjugate gradient algorithm for solving large-scale unconstrained optimization problems, which combines the idea of scaled memoryless Broyden–Fletcher–Goldfarb–Shanno preconditioned conjugate gradient method with the nonmonotone technique. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under appropriate assumptions, the method is proven to possess global convergence for nonconvex smooth functions, and R-linear convergence for strongly convex functions. Preliminary numerical results and related comparisons show the efficiency of the proposed method in practical computation.

In this paper, we propose the application of a Cooperative Co-evolutionary LSHADE (CCLSHADE) algorithm for Large-Scale Global Optimization (LSGO). We illustrate that by tuning two simple parameters of the CC framework, one can obtain very competitive results. The results are achieved without the need of incorporating local search modules, a re-initialization step, or adaptively configuring the CC framework budget allocation. The two parameters studied in this work are the number of iterations for which to run each sub-optimizer in a single cycle and the maximum size of the component containing the separable problem variables. The performance of CCLSHADE is compared against six state-of-the-art algorithms developed for LSGO using the CEC10 benchmarks. Experimental results and statistical tests confirm the competitiveness of the proposed algorithm.

Recently the authors have proposed a homogeneous and self-dual algorithm for solving the monotone complementarity problem (MCP) [5]. The algorithm is a single phase interior-point type method; nevertheless, it yields either an approximate optimal solution or detects a possible infeasibility of the problem. In this paper we specialize the algorithm to the solution of general smooth convex optimization problems, which also possess nonlinear inequality constraints and free variables. We discuss an implementation of the algorithm for large-scale sparse convex optimization. Moreover, we present computational results for solving quadratically constrained quadratic programming and geometric programming problems, where some of the problems contain more than 100,000 constraints and variables. The results indicate that the proposed algorithm is also practically efficient.

A view of the limitations, opportunities, and challenges in parallel nonlinear optimization

The spectral projected gradient method SPG is an algorithm for large-scale bound-constrained optimization introduced recently by Birgin, Martinez, and Raydan. It is based on the Raydan unconstrained generalization of the Barzilai-Borwein method for quadratics. The SPG algorithm turned out to be surprisingly effective for solving many large-scale minimization problems with box constraints. Therefore, it is natural to test its perfomance for solving the sub-problems that appear in nonlinear programming methods based on augmented Lagrangians. In this work, augmented Lagrangian methods which use SPG as the underlying convex-constraint solver are introduced (ALSPG) and the methods are tested in two sets of problems. First, a meaningful subset of large-scale nonlinearly constrained problems of the CUTE collection is solved and compared with the perfomance of LANCELOT. Second, a family of location problems in the minimax formulation is solved against the package FFSQP.

In large-scale multi-objective problems, the traditional offspring generation operators is directionless and blind, which leads to the low searching capability in a huge search space. For this, we propose a two-stage multi-objective evolutionary algorithm, named MOEA-BTS to solve large-scale multi-objective problems(LSMOPs). In MOEA-BTS, the offspring generation process is divided into two stages. In the early stage, a new hybrid of local and global search direction construction method is proposed, aiming to balance the exploitation and exploration of the search. In the late stage, a series of weight vectors divide the decision space into subspaces, where the competitive swarm optimization algorithm is performed for further precise optimizations. Experiments are conducted on the LSMOPs with 500 and 1000 decision variables and results demonstrate that our proposed algorithm can perform better than several state-of-the-art evolutionary algorithms.

Nonsmooth differential-algebraic equations in chemical engineering