Deterministic global derivative-free optimization of black-box problems with bounded Hessian

Hammerstein–Wiener models constitute a significant class of block-structured dynamic models, as they approximate process nonlinearities on the basis of input–output data without requiring identification of a full nonlinear process model. Optimization problems with Hammerstein–Wiener models embedded are nonconvex, and thus local optimization methods may obtain suboptimal solutions. In this work, we develop a deterministic global optimization strategy that exploits the specific structure of Hammerstein–Wiener models to extend existing theory on global optimization of systems with linear dynamics. At first, we discuss alternative formulations of the dynamic optimization problem with Hammerstein–Wiener models embedded, demonstrating that careful selection of the optimization variables of the problem can offer significant numerical advantages to the solution approach. Then, we develop convex relaxations for the proposed optimization problem and discuss implementation aspects to obtain the global solution focusing on a control parametrization technique. Finally, we apply our optimization strategy to case studies comprising both offline and online dynamic optimization problems. The results confirm an improved computational performance of the proposed solution approach over alternative options not exploiting the linear dynamics for all considered examples. They also underline the tractability of deterministic global dynamic optimization when using few control intervals in online applications like nonlinear model predictive control.

Gaussian processes (Kriging) are interpolating data-driven models that are frequently applied in various disciplines. Often, Gaussian processes are trained on datasets and are subsequently embedded as surrogate models in optimization problems. These optimization problems are nonconvex and global optimization is desired. However, previous literature observed computational burdens limiting deterministic global optimization to Gaussian processes trained on few data points. We propose a reduced-space formulation for deterministic global optimization with trained Gaussian processes embedded. For optimization, the branch-and-bound solver branches only on the free variables and McCormick relaxations are propagated through explicit Gaussian process models. The approach also leads to significantly smaller and computationally cheaper subproblems for lower and upper bounding. To further accelerate convergence, we derive envelopes of common covariance functions for GPs and tight relaxations of acquisition functions used in Bayesian optimization including expected improvement, probability of improvement, and lower confidence bound. In total, we reduce computational time by orders of magnitude compared to state-of-the-art methods, thus overcoming previous computational burdens. We demonstrate the performance and scaling of the proposed method and apply it to Bayesian optimization with global optimization of the acquisition function and chance-constrained programming. The Gaussian process models, acquisition functions, and training scripts are available open-source within the “MeLOn—MachineLearning Models for Optimization” toolbox (https://git.rwth-aachen.de/avt.svt/public/MeLOn).

Global optimization problems widely exist in the fields of economic model, finance, engineering design and control. Since it is easy to fall into multiple local optimal solutions that are different from the global optimal solution, how to obtain the global optimal solution is a very important subject. Inspired by the recently proposed deterministic global optimization method – Granular Sieving (GrS) algorithm, this paper proposes a parallel method for global optimization – P-GrS. Supported by the mathematical theory of GrS, P-GrS can theoretically guarantee to find the global optimum and the complete set of global optimal solutions through the parallel design of GrS. The method has better performance than the traditional GrS in most bench mark functions, and the results show the feasibility and effectiveness of the algorithm.

Guidance algorithms for the return flight of suborbital spaceplanes must generate a variety of guidance trajectories that satisfy terminal conditions even in unexpected abort operations. To tackle this issue, a trajectory optimization method that combines convex quadratic programming and a global derivative-free optimization technique in a nested structure has been recently studied by the authors. This hybrid method efficiently explores the three-dimensional Bezier trajectories and associated guidance commands that exactly fulfill the equality terminal conditions and command continuity. In this paper, Monte-Carlo simulations are performed to investigate the applicability of this guidance method to the realistic scenario of unpowered return flight. Six stochastic evolutionary algorithms, a Bayesian optimization method, and three deterministic search algorithms are implemented and tested as global optimizers. They are compared in terms of computational and implementational complexities, robustness, and diversity of solutions obtained. The results show that reliable and real-time trajectory generation is possible, when an optimizer and its settings are properly chosen. It also reveals that diverse trajectories between initial and terminal conditions are successfully generated.KeywordsGuidanceTrajectory optimizationGlobal optimizationReal-time optimizationSuborbital spaceplane

A deterministic global optimization algorithm based on augmented Lagrangian is proposed for solving the nonlinear programming with cone constraints, which is often encountered in technical design and operational research. At each outer iteration, the method requires the -global minimization of the augmented Lagrangian of problems with cone constraints, where. The convergence to an-global minimizer of the original problem is proved. Key words: Global optimization, augmented Lagrangians, cone optimization, algorithms.

The rising temporal mismatch between energy supply and demand increases the need for large-scale energy storage. Carnot batteries offer a location-independent energy storage option and rely on readily available components. However, the low round-trip efficiency is an issue. Also, Carnot batteries have many degrees of freedom, making their optimal design and operation challenging. Additionally, the process is highly dependent on the working fluids used for the charging and discharging process. We optimize the Carnot battery design and nominal operation, including the working fluid selection. We perform deterministic global optimization to maximize the round-trip efficiency. Extending our previous work, we formulate a hybrid mechanistic/data-driven model in reduced space. We propose a model formulation for the thermal energy storage that is tailored to our problem and demonstrate that it results in substantial computational savings. We also extend our previously used surrogate model training procedure, by choosing surrogate models based on their relaxation tightness, strongly improving worst-case computational performance. These model improvements allow us to screen working fluids for the charging and discharging process by enumeration, globally optimizing each flowsheet with MAiNGO v0.7.2. Optimal round-trip efficiencies vary between 30% and 60% and are typically found in the root node (multistart) but for some cases during branch-and-bound. However, the top working fluid combinations include fluids with a strong environmental impact, indicating the need to include environmental objectives.

This paper deals with a deterministic and rational way to design piezoelectric transformers in radial mode. The proposed approach is based on the study of the inverse problem of design and on its reformulation as a mixed constrained global optimization problem. The methodology relies on the association of the analytical models for describing the corresponding optimization problem and on an exact global optimization software, named IBBA and developed by the second author to solve it. Numerical experiments are presented and compared in order to validate the proposed approach.

During the last three decades, the research area of systems engineering has emerged as a domain of fundamental importance and major impact within chemical engineering, as well as a cornerstone area in interdisciplinary research initiatives with computer science, operations research, applied mathematics, materials and life sciences. This is attributed to the unique characteristics of systems engineering which are the combination of analysis and synthesis for the design, optimization, and operation of processes and products. The product and process discovery research efforts are founded on fundamental advances in mathematical modeling, optimization theory and algorithms, and insights derived either from existing operations and/or from biology, chemistry, and physics. The fundamental advances are epitomized through new theoretical, algorithmic, and modeling frameworks for (a) mixed-integer linear and nonlinear optimization, (b) deterministic global optimization, and (c) dynamic simulation and optimization. The proposed modeling and optimization approaches have multiscale applications ranging from macroscopic to mesoscopic to microscopic systems, and which provide a natural and fundamental link between systems engineering, computational chemistry, computational biology and systems biology. Approaches based on mixed-integer linear and nonlinear optimization which were typically identified with process synthesis, scheduling and planning applications, have entered the domains of gene regulatory networks, metabolic networks, signal transduction networks, beta-sheet topology prediction in proteins, de novo peptide and protein design, DNA recombination, phase problem in X-ray crystallography, side chain optimization in protein prediction, peptide identification via tandem mass spectroscopy. Approaches based on deterministic global optimization associated with process design, synthesis, scheduling, and pooling/blending applications, are now in the main stream of product design, structure prediction in protein folding, dynamics of protein folding, NMR protein structure refinement, and de novo protein design. Approaches based on dynamic models and large scale optimization which were identified with process models and their applications, are suited for metabolic and signal transduction networks. As a result, a synergism between systems engineering, computational biology, and systems biology has evolved gradually, opened new research avenues and has reached the stage where fascinating research contributions address important questions in computational biology with methods and tools from systems engineering which combine mathematical rigor with key biological insights. This article provides a perspective on the challenges and opportunities that emerge from the fundamental developments in the research fields of systems engineering and computational biology. The advances in the areas of deterministic global optimization and process scheduling are introduced first, followed by their respective research opportunities. The revolution of genomics is discussed next with a particular focus on the advances and challenges in the areas of structure prediction in protein folding, de novo peptide and protein design, and peptide and protein identification via tandem mass spectroscopy. This article is based on material presented as an invited talk at the session on “The Future of Chemical Engineering Research III” during the 2004 annual AIChE meeting.

This thesis presents new algorithms for the deterministic global optimization of general non-linear programming problems (NLPs). It is proven that the αBB general underestimator may provide exact lower bounds on a function only if rigorous conditions are satisfied. These conditions are derived and the μ-subenergy methodology is proposed to achieve tighter αBB underestimation when they are violated. An interval lower bounding test is proposed to improve αBB lower bounds and avoid expensive algorithmic steps. Piecewise-linear relaxations (PLR) are proposed for the underestimation of general functions. Calculation of these relaxations is accelerated using parallel computing. Quality bounds tightening (QBT) is proposed to reduce the cost of bounds tightening algorithms by avoiding unnecessary calculations. Violation branching is proposed to improve the performance of branching strategies by considering constraint violation when selecting a branching variable. Furthermore, a novel bounds tightening method, PLR bounds tightening (PLR-BT), is proposed. Variable-based convexity (VBC) is proposed as a general reformulation algorithm, to build tighter relaxations by exploiting underlying convexity. This work also introduces algorithms for parallel branching for the global solution of NLPs. A parallel node subdivision strategy, Multisection Branching, is proposed to achieve tighter bounds, and a parallel node selection strategy, Future Branching, is proposed to accelerate the investigation of the branch-and-bound tree. A parallel solver is presented, where MPI is used to distribute node calculations and an asynchronous bounds tightening algorithm is proposed to reduce bounds tightening wall times. This algorithm is implemented using multithreading for asynchronous feasibility-based bounds tightening (AF-BBT). All algorithms are implemented in a global solver, and its parallel architecture and features are described. This solver is used to perform numerical studies on the benefits of using the new algorithms in tandem. The new solver is benchmarked against the BARON 15.9.22 global solver for a set of problems, in which it achieves comparable performance.

For the performance of global optimization algorithms, the rate of convergence of convex relaxations to the objective and constraint functions is critical. We extend results from Bompadre and Mitsos (J Glob Optim 52(1):1–28, 2012) to characterize the convergence rate of parametric bounds and relaxations of the solutions of ordinary differential equations (ODEs). Such bounds and relaxations are used for global dynamic optimization and are computed using auxiliary ODE systems that use interval arithmetic and McCormick relaxations. Two ODE relaxation methods (Scott et al. in Optim Control Appl Methods 34(2):145–163, 2013; Scott and Barton in J Glob Optim 57:143–176, 2013) are shown to give second-order convergence, yet they can behave very differently from each other in practice. As time progresses, the prefactor in the convergence-order bound tends to grow much more slowly for one of these methods, and can even decrease over time, yielding global optimization procedures that require significantly less computation time.

In this paper, we consider the solution of global optimization problems involving hidden constraints. We present a novel deterministic derivative-free global optimization algorithm based on our recently introduced DIRECT-GL (Stripinis et al. in Optim Lett. 12(7):1699–1712, 2018). The new algorithm (DIRECT-GLh) incorporates two additional techniques for faster feasibility detection and solution improvement, especially when the solution lies on the border of the a priori unknown feasible region. The algorithm's potential is demonstrated on 67 problems from the current release of DIRECTLib and 100 Emmental-type test problems. The numerical comparison reveals the proposed algorithm's superiority to the considered DIRECT-type approaches for such problems.

We consider here the average deviation as the most important objective when designing numerical techniques and algorithms. We call that a Bayesian approach. We start by describing the Bayesian approach to the continu­ ous global optimization. Then we show how to apply the results to the adaptation of parameters of randomized techniques of optimiza­ tion. We assume that there exists a simple function which roughly predicts the consequences of decisions. We call it heuristics. We define the probability of a decision by a randomized decision func­ tion depending on heuristics. We fix this decision function, except for some parameters that we call the decision parameters. We repeat the randomized decision procedure several times given the decision parameters and regard the best outcome as a result. We optimize the decision parameters to make the search more efficient. Thus we replace the original optimization problem by an auxiliary problem of continuous stochastic optimization. We solve the auxiliary problem by the Bayesian methods of global op­ timization. Therefore we call the approach as the Bayesian one. We discuss the advantages and disadvantages of the Bayesian approach. We describe the applications to some of discrete pro-

A novel technique that addresses the solution of the general nonlinear bilevel programming problem to global optimality is presented. Global optimality is guaranteed for problems that involve twice differentiable nonlinear functions as long as the linear independence constraint qualification condition holds for the inner problem constraints. The approach is based on the relaxation of the feasible region by convex underestimation, embedded in a branch and bound framework utilizing the basic principles of the deterministic global optimization algorithm, αBB [2, 4, 5, 11]. Epsilon global optimality in a finite number of iterations is theoretically guaranteed. Computational studies on several literature problems are reported.

Recent advances in the theory of deterministic global optimization have resulted in the development of very efficient algorithmic procedures for identifying the global minimum of certain classes of nonconvex optimization problems. The advent of powerful multiprocessor machines combined with such developments make it possible to tackle with substantial efficiency otherwise intractable global optimization problems. In this paper, we will discuss implementation issues and computational results associated with the distributed implementation of the decomposition-based global optimization algorithm, GOP, [5], [6]. The NP-complete character of the global optimization problem, translated into extremely high computational requirements, had made it difficult to address problems of large size.The parallel implementation made it possible to successfully tackle the increased computational requirements in in order to identify the global minimum in computationally realistic times. The key computational bottlnecks are identified and properly addressed. Finaly, results on an Intel-Paragon machine are presented for large scale Indefinite Quadratic Programming problems, with up to 350 quadratic variables, and Blending-Pooling Problems, with up to 12 components and 30 qualities.

Industrial energy systems are increasingly required to reduce operating costs and CO2 emissions while integrating variable renewable energy sources. Managing these objectives under uncertainty requires advanced optimization strategies capable of delivering reliable and real-time decisions. To address these challenges, this study focuses on the short-term operational planning of an industrial energy supply system using the rolling horizon approach (RHA). The RHA offers an effective framework to handle uncertainties by repeatedly updating forecasts and re-optimizing over a moving time window, thereby enabling adaptive and responsive energy management. To solve the resulting nonlinear and constrained optimization problem at each RHA iteration, we propose a novel hybrid algorithm that combines Bayesian optimization (BO) with the Interior Point OPTimizer (IPOPT). While global deterministic and stochastic optimization methods are frequently used in practice, they often suffer from high computational costs and slow convergence, particularly when applied to large-scale, nonlinear problems with complex constraints. To overcome these limitations, we employ the BO–IPOPT, integrating the global search capabilities of BO with the efficient local convergence and constraint fulfillment of the IPOPT. Applied to a large-scale real-world case study of a food and cosmetic industry in Germany, the proposed BO–IPOPT method outperformed state-of-the-art solvers in both solution quality and robustness, achieving up to 97.25%-better objective function values at the same CPU time. Additionally, the influence of key parameters, such as forecast uncertainty, optimization horizon length, and computational effort per RHA iteration, was analyzed to assess their impact on system performance and decision quality.