Application of a Derivative-Free Method with Projection Skill to Solve an Optimization Problem

Application of a Derivative-Free Method with Projection Skill to Solve an Optimization Problem

Abstract. The approach of conditional nonlinear optimal perturbation (CNOP) was previously proposed to find the optimal initial perturbation (CNOP-I) in a given constraint. In this paper, we extend the CNOP approach to search for the optimal combined mode of initial perturbations and model parameter perturbations. This optimal combined mode, also named CNOP, has two special cases: one is CNOP-I that only links with initial perturbations and has the largest nonlinear evolution at a prediction time; while the other is merely related to the parameter perturbations and is called CNOP-P, which causes the largest departure from a given reference state at a prediction time. The CNOP approach allows us to explore not only the first kind of predictability related to initial errors, but also the second kind of predictability associated with model parameter errors, moreover, the predictability problems of the coexistence of initial errors and parameter errors. With the CNOP approach, we study the ENSO predictability by a theoretical ENSO model. The results demonstrate that the prediction errors caused by the CNOP errors are only slightly larger than those yielded by the CNOP-I errors and then the model parameter errors may play a minor role in producing significant uncertainties for ENSO predictions. Thus, it is clear that the CNOP errors and their resultant prediction errors illustrate the combined effect on predictability of initial errors and model parameter errors and can be used to explore the relative importance of initial errors and parameter errors in yielding considerable prediction errors, which helps identify the dominant source of the errors that cause prediction uncertainties. It is finally expected that more realistic models will be adopted to investigate this use of CNOP.

To reduce the cost of ocean observations and improve prediction accuracy of the Kuroshio region temperature, this study investigates the related targeted observation by using the conditional nonlinear optimal perturbation (CNOP) approach. Results show that the scheme of vertical-integrated energy is more suitable for the identification of sensitive area in the related targeted observation. By conducting a set of observation system simulation experiments (OSSEs), we discovered that the sensitive areas identified by the CNOP exert substantial influence on temperature predictions within the target area. The dynamic diagnosis further indicated that the pressure gradient and Coriolis force in the momentum equations greatly contribute the development of the prediction biases. These findings implied that the implement of CNOP-based targeted observation represents a cost-effective strategy for enhancing temperature predictions in the Kuroshio region.

In this paper, a nonlinear optimization method is used to explore the finite-time instability of the atmospheric circulation with a three-level quasigeostrophic model under the framework of the conditional nonlinear optimal perturbation (CNOP). As a natural generalization of linear singular vector (SV), CNOP is defined as an initial perturbation that makes the cost function the maximum at a prescribed forecast time under certain physical constraint conditions. Special attentions are paid to the different structures and energy evolutions of the optimal perturbations. The results show that the most instable region of the global atmospheric circulation lies in the midlatitude Eurasian continent. More specially, SV and CNOP in the total energy norm with an optimization time of 2 days both present localness: they are mainly located in the midlatitude Asian continent and its east coast. With extension of the optimization time, SVs are more upstream and less localized in the zonal direction, and CNOPs differ essentially from SVs with broader zonal and meridional coverages; as a result, CNOPs acquire larger kinetic and available potential energy amplifications than SVs in the nonlinear model at the corresponding optimization time. For the climatological wintertime flow, it is seen that the baroclinic terms remain small over the entire time evolution, and the energy production comes essentially from the eddy kinetic energy, which is induced by the horizontal shear of the basic flow. In addition, the effects of SVs and CNOPs on the Eurasian atmospheric circulation are explored. The results show that the weather systems over the Eurasian continent in the perturbed fields by CNOPs are stronger than those by SVs at the optimization time. This reveals that the CNOP method is better in evaluating the instability of the atmospheric circulation while the SV method underestimates the possibility of extreme weather events.

In this work, a method to control the parameters of Differential Evolution (DE) algorithm is proposed. Here the control parameters of DE are co-evolved by Particle Swarm Optimization (PSO) algorithm. The classical DE algorithm has two main control parameters: Scale Factor (F) and Cross-over Rate (CR). These are selected on trial-and-error basis for solving optimization problems. Several optimization problems lead to optimal or sub-optimal solution by proper selection of control parameters of the DE algorithm. In this proposed method, PSO algorithm is used to tune the scale factor and cross-over rate in DE algorithm. Basically PSO algorithm is used as a meta-optimizer for DE algorithm. The proposed method is termed as mPSO-DE in this paper. The mPSO-DE algorithm is applied on 12 benchmark unconstrained optimization problems. The obtained results are compared with that of classical DE algorithm. From the experimental studies, it has been found that the proposed mPSO-DE algorithm performed better than DE algorithm.

In recent years, Differential Evolution (DE) algorithm that is one of the most important delegate of the evolutionary algorithms have been successfully used for solving different optimization problems with its both serial and parallel variants. In this study, the performance of DE algorithm powered with cooperative emigrant creation strategy was investigated over the different types of numerical problems and obtained results were compared with the results of the serial DE and parallel DE algorithm supported with conventional emigrant creation strategy. Experimental studies showed that the convergence speed of the parallel DE algorithm is significantly improved with the cooperative emigrant creation strategy and the proposed parallel DE algorithm produces better results compared to the serial DE algorithm and parallel DE algorithm with the conventional emigrant creation strategy.

A self-competitive mutation strategy for Differential Evolution algorithms with applications to Proportional–Integral–Derivative controllers and Automatic Voltage Regulator systems

Abstract. The El Niño–Southern Oscillation (ENSO) is a significant climate phenomenon that appears periodically in the tropical Pacific. The intermediate coupled ocean–atmosphere Zebiak–Cane (ZC) model is the first and classical one designed to numerically forecast the ENSO events. Traditionally, the conditional nonlinear optimal perturbation (CNOP) approach has been used to capture optimal precursors in practice. In this paper, based on state-of-the-art statistical machine learning techniques1, we investigate the sampling algorithm proposed in Shi and Sun (2023) to obtain optimal precursors via the CNOP approach in the ZC model. For the ZC model, or more generally, the numerical models with a large number O(104−105) of degrees of freedom, the numerical performance, regardless of the statically spatial patterns and the dynamical nonlinear time evolution behaviors as well as the corresponding quantities and indices, shows the high efficiency of the sampling method compared to the traditional adjoint method. The sampling algorithm does not only reduce the gradient (first-order information) to the objective function value (zeroth-order information) but also avoids the use of the adjoint model, which is hard to develop in the coupled ocean–atmosphere models and the parameterization models. In addition, based on the key characteristic that the samples are independently and identically distributed, we can implement the sampling algorithm by parallel computation to shorten the computation time. Meanwhile, we also show in the numerical experiments that the important features of optimal precursors can still be captured even when the number of samples is reduced sharply.

The Differential Evolution (DE) algorithm, under the family of Evolutionary Algorithms (EAs), is one of the powerful algorithms used for solving continuous parameter optimization challenges. The simplistic nature and robustness of the classical DE algorithm have drawn researchers' attention towards its progressive enhancement. This work reports on an investigation of the behavioral changes of the classical DE algorithm, evoked when its mutation and crossover components are fine tuned for enhancement of DE's performance. The scope of this study covers the implementation of a mutation level enhancement and a crossover level enhancement, followed by their integration. The mutation and the crossover components are augmented by incorporation of Centroid DE and Superior-Superior & Superior-Inferior DE logics, respectively. The algorithms appraised in this inquiry were classical DE, Centroid based DE(cDE), Superior-Superior based DE (ssDE), Superior-Inferior DE (siDE), Centroid Superior-Superior DE (cssDE) and Centroid Superior-Inferior DE (csiDE). These algorithms were evaluated by comparison of the values of their mean objective function (MOV), and their speed, at solving the global optimization problems in a simple benchmarking function suite with 4 functions of different categories. The study concludes that the DE algorithm shows enhancement performance with modified mutation and crossover components. However, with a trend for inconsistency for varying values of its control parameters and benchmarking problems.

Parameters setting is an important problem of evolution algorithms, include differential evolution algorithm. It has an effect on the performance of evolution algorithms. Although there is only three control parameters in differential evolution (DE) algorithm, the parameters setting is also a difficult problem. Self-adaptation is highly beneficial for adjusting the control parameters, especially when done without any user interaction. This paper presents differential evolution algorithm, called (AdaptDE), which use self-adaptive mechanisms applied to the control parameters. It can get the optimal control parameters for different optimization problem without user interaction. Experimental results indicate that AdaptDE algorithm is efficient and feasible. It is superior to jDE algorithm and FADE algorithm on the quality of solution.

Regulation-based formulas and improved Differential Evolution(DE) algorithm is used to optimize PID parameters. In order to improve the global search ability and the convergence rate of the common DE algorithm, self-adaptive method is introduced to obtain DE parameters. On the other hand, initial population quality of DE algorithm has important influence on convergence of the algorithm. So the regulation-based formulas are used to guide the production of initial population, which is good to improve the convergence rate and realize obtaining of PID parameters completely adaptive without any personal experience. The simulation is developed on steam temperature system of cycle fluidized bed boiler with serious parameter uncertainties and many disturbance and long-time delay. The results show that the improved DE algorithm has higher optimal speed, small amount of calculation and effective optimization of parameters. The proposed method has better control quality and system robustness.

The paper considers dimensional synthesis of a path generator four-bar linkage by applying the modified Differential Evolution (DE) algorithm. During the search of the space, the DE algorithm may skip the right solution and lead to premature convergence. If the step is reduced, the search space is reduced as well, so that convergence becomes slower. Search efficiency may be achieved by including Lévy Flight in the DE algorithm. Namely, when the DE algorithm identifies the space of best solution, then a Levý Flight step is used for local search in that region, which significantly increases the exploitation capability and allows obtaining a very efficient solution of the optimization problem. The algorithm was tested with three examples of dimensional synthesis of a path generator four-bar linkage. The proposed algorithm can be very efficiently applied to solving very complex optimization problems.

This parameters selection is an important issue in the research of ¿-support vector regression machine (¿-SVRM), whose nature is an optimization selection process. Motivated by the effectiveness of differential evolution (DE) algorithm on optimization problem, a new automatic searching method based on DE algorithm was proposed. Experimental results demonstrate that ¿-SVRM model optimization based on DE algorithm has better prediction capability compared with the methods based on genetic algorithm (GA), ant colony optimization (ACO) and particle swarm optimization (PSO).

A novel modified bat algorithm hybridizing by differential evolution algorithm

Information utilization based modified Differential evolution algorithm (IUMDE) is an improved version of DE algorithm. In IUMDE, Mutation and Selection operation of basic Differential Evolution (DE) algorithm are modified. In proposed IUMDE initially modified mutation operation provides a selection criterion of base vector before performing the basic DE mutation operation. Further the modified selection operation utilizes the neglected trial vector and hence helps to improve the performance of DE algorithm. In this paper IUMDE algorithm is employed on 6 constrained benchmark optimization problems, selected from the literature. The numerical results expose that the IUMDE algorithm perform more accurately and significantly better than other algorithms with these constrained optimization problems.