An Embarrassingly Parallel Method for Large-Scale Stochastic Programs

A Scalable Bounding Method for Multi-Stage Stochastic Integer Programs

Representing the uncertainties with a set of scenarios, the optimization problem resulting from a robust nonlinear model predictive control (NMPC) strategy at each sampling instance can be viewed as a large-scale stochastic program. This paper solves these optimization problems using the parallel Schur complement method developed to solve stochastic programs on distributed and shared memory machines. The control strategy is illustrated with a case study of a multidimensional unseeded batch crystallization process. For this application, a robust NMPC based on min–max optimization guarantees satisfaction of all state and input constraints for a set of uncertainty realizations, and also provides better robust performance compared with open-loop optimal control, nominal NMPC, and robust NMPC minimizing the expected performance at each sampling instance. The performance of robust NMPC can be improved by generating optimization scenarios using Bayesian inference. With the efficient parallel solver, the solution time of one optimization problem is reduced from 6.7 min to 0.5 min, allowing for real-time application.

In this paper we extend the well-known L-Shaped method to solve two-stage stochastic programming problems with decision-dependent uncertainty. The method is based on a novel, unifying, formulation and on distribution-specific optimality and feasibility cuts for both linear and integer stochastic programs. Extensive tests on three production planning problems illustrate that the method is extremely effective on large-scale instances.

We describe computational procedures for solving a wide-ranging class of stochastic programs with chance constraints where the random components of the problem are discretely distributed. Our procedures are based on a combination of Lagrangian relaxation and scenario decomposition, which we solve using a novel variant of Rockafellar and Wets' progressive hedging algorithm [Rockafellar, R. T., R. J.-B. Wets. 1991. Scenarios and policy aggregation in optimization under uncertainty. Math. Oper. Res. 16(1) 119–147]. Experiments demonstrate the ability of the proposed algorithm to quickly find near-optimal solutions—where verifiable—to both difficult and very large chance-constrained stochastic programs, both with and without integer decision variables. The algorithm exhibits strong scalability in terms of both run time required and final solution quality on large-scale instances. There is a Video Overview associated with this paper. Click here to view the Video Overview . To save the file, right click and choose “Save Link As” from the menu.

Many planning problems involve choosing a set of optimal decisions for a system in the face of uncertainty of elements that may play a central role in the way the system is analyzed and operated. During the past decade, there has been a renewed interest in the modelling, analysis, and solution of such problems due to a remarkable development of both new theoretical results and novel computational techniques in stochastic optimization. A prominent approach is to develop upper and lower bounding approximations to the problem along with procedures to sharpen bounds until an acceptable tolerance is satisfied. The contributions of this dissertation are concerned with the latter approach. The thesis first studies the stochastic linear programming problem with randomness in both the objective coefficients and the constraints. A convex concave saddle property of the value function is utilized to derive new bounding techniques which generalize previously known results. These approximations require discretizing bounded domains of the random variables in such a way that tight upper and lower bounds result. Such techniques will prove attractive with the recent advances in large scale linear programming. The above results are also extended to obtain new upper and lower bounds when the domains of random variables are unbounded. While these bounds are tight, the approximating models are large-scale deterministic linear programs. In particular, with a proposed order-cone decomposition for the domains, these linear programs are well-structured, thus enabling one to use efficient techniques for solution, such as parallel computation. The thesis next considers convex stochastic programs. Using aggregation concepts from the deterministic literature, new bounds are developed for the problem which are computable using standard convex programming algorithms. Finally, the discussion is focused on a stochastic convex program arising in a certain resource allocation problem. Exploiting the problem structure, bounds are developed via the Karush-Kuhn-Tucker conditions. Rather than discretizing domains, these approximations advocate replacing difficult multidimensional integrals by a series of simple univariate integrals. Such practice allows one to preserve differentiability properties so that smooth convex programming methods can be applied for solution.

Stochastic programming involves large-scale optimization with exponentially many scenarios. This paper proposes an optimization-based scenario reduction approach to generate high-quality solutions and tight lower bounds by only solving small-scale instances, with a limited number of scenarios. First, we formulate a scenario subset selection model that optimizes the recourse approximation over a pool of solutions. We provide a theoretical justification of our formulation, and a tailored heuristic to solve it. Second, we propose a scenario assortment optimization approach to compute a lower bound—hence, an optimality gap—by relaxing nonanticipativity constraints across scenario “bundles.” To solve it, we design a new column-evaluation-and-generation algorithm, which provides a generalizable method for optimization problems featuring many decision variables and hard-to-estimate objective parameters. We test our approach on stochastic programs with continuous and mixed-integer recourse. Results show that (i) our scenario reduction method dominates scenario reduction benchmarks, (ii) our scenario assortment optimization, combined with column-evaluation-and-generation, yields tight lower bounds, and (iii) our overall approach results in stronger solutions, tighter lower bounds, and faster computational times than state-of-the-art stochastic programming algorithms.History: Accepted by Andrea Lodi, Area Editor for Design and Analysis of Algorithms–Discrete.Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoc.2023.1295 .

We present StochasticPrograms.jl, a user-friendly and powerful open-source framework for stochastic programming written in the Julia language. The framework includes both modeling tools and structure-exploiting optimization algorithms. Stochastic programming models can be efficiently formulated using an expressive syntax, and models can be instantiated, inspected, and analyzed interactively. The framework scales seamlessly to distributed environments. Small instances of a model can be run locally to ensure correctness, whereas larger instances are automatically distributed in a memory-efficient way onto supercomputers or clouds and solved using parallel optimization algorithms. These structure-exploiting solvers are based on variations of the classical L-shaped, progressive-hedging, and quasi-gradient algorithms. We provide a concise mathematical background for the various tools and constructs available in the framework along with code listings exemplifying their usage. Both software innovations related to the implementation of the framework and algorithmic innovations related to the structured solvers are highlighted. We conclude by demonstrating strong scaling properties of the distributed algorithms on numerical benchmarks in a multinode setup. Summary of Contribution: This paper presents StochasticPrograms.jl, an open-source framework for stochastic programming implemented in the Julia programming language. The framework includes an expressive syntax for formulating stochastic programming models as well as versatile analysis tools and parallel optimization algorithms. The framework will prove useful to researchers, educators, and industrial users alike. Researchers will benefit from the readily extensible open-source framework, in which they can formulate complex stochastic models or quickly typeset and test novel optimization algorithms. Educators of stochastic programming will benefit from the clean and expressive syntax. Moreover, the framework supports analysis tools and stochastic programming constructs from classical theory and leading textbooks. We strongly believe that the StochasticPrograms.jl framework can reduce the barrier to entry for incoming practitioners of stochastic programming. Industrial practitioners can make use of StochasticPrograms.jl to rapidly formulate complex models, analyze small instances locally, and then run large-scale instances in production. In doing so, they get distributed capabilities for free without changing the code and access to well-tested state-of-the-art implementations of parallel structure-exploiting solvers. As the framework is open-source, anyone from these target audiences can contribute with new functionality to the framework. In conclusion, by providing both an intuitive interface for new users and an extensive development environment for expert users, StochasticPrograms.jl has strong potential to further the field of stochastic programming.

Stochastic programming models can lead to very large-scale optimization problems for which it may be impossible to enumerate all possible scenarios. In such cases, one adopts a sampling-based solution methodology in which case the reliability of the resulting decisions may be suspect. For such instances, it is advisable to adopt methodologies that promote variance reduction. One such approach goes under a framework known as “compromise decision", which requires multiple replications of the solution procedure. This paper studies the reliability of stochastic programming solutions resulting from the “compromise decision" process. This process is characterized by minimizing an aggregation of objective function approximations across replications, presumably conducted in parallel. We refer to the post-parallel-processing problem as the problem of “compromise decision". We quantify the reliability of compromise decisions by estimating the expectation and variance of the “pessimistic distance" of sampled instances from the set of true optimal decisions. Such a pessimistic distance is defined as an estimate of the largest possible distance of the solution of the sampled instance from the “true" optimal solution set. The Rademacher average of instances is used to bound the sample complexity of the compromise decision.

For many practical problems, solutions obtained from deterministic models are unsatisfactory because they fail to hedge against certain contingencies that may occur in the future. Stochastic models address this shortcoming, but up to recently seemed to be intractable due to their size. Recent advances both in solution algorithms and in computer technology now allow us to solve important and general classes of practical stochastic problems. We show how large-scale stochastic linear programs can be efficiently solved by combining classical decomposition and Monte Carlo (importance) sampling techniques. We discuss the methodology for solving two-stage stochastic linear programs with recourse, present numerical results of large problems with numerous stochastic parameters, show how to efficiently implement the methodology on a parallel multi-computer and derive the theory for solving a general class of multi-stage problems with dependency of the stochastic parameters within a stage and between different stages.

The standard method for establishing the comparative statics of risk changes in optimization problems has been confined to comparing unique interior solutions, relying on strong assumptions about payoff functions and decision variables. We propose a simple and intuitive approach that hinges on considerably weaker assumptions. Merging insights from the monotone comparative statics literature with insights from the risk apportionment literature, we show that the ranking of simple lottery pairs is all that is needed for establishing the comparative statics of risk changes. We use this approach to analyze the comparative statics of Nth-degree stochastic dominance shifts in a general setting with one and with multiple decision variables, and we show how these results can be applied to generalize the classical theories of precautionary saving, self-protection, and others. This paper was accepted by James Smith, decision analysis.

A two-phase inspection model for a single component system with three-stage degradation

This paper proposes integrated production and inventory cycle in a whole green manufacturing supply chain network which consists of multiple-raw materials suppliers named as tier-2 suppliers, multiple-parts suppliers named as tier-1 suppliers, a manufacturer, multiple distributors, multiple retailers and a third party product collector. In a whole manufacturing supply chain network, tier-2 suppliers supply raw materials to tier-1 suppliers producing parts. Parts from tier-1 suppliers are then supplied to a manufacturer which manufactures and assembles parts into finished products. Then, the finished products will be delivered to distributors distributing them to retailers. Because each player wishes to minimize total cost for itself and can supply multiple raw materials and parts, there is a need for coordination among players of the supply chain. The coordination which is used in this paper is for both centralized and decentralized decision processes. Due to environmental and ecological consciousness and responsibility as a result of increasingly rigid environmental legislations and growing environmental concerns in recent years, this paper also considers recovery option of finished products where some of the parts will be reused in new finished products. A mathematical model with multiple decision variables is developed. A numerical example is presented and results discussed.

Estimation of parameters of chaotic systems is a subject of substantial and well-developed research issue in nonlinear science. From the viewpoint of optimization, parameter estimation can be formulated as a multi-modal constrained optimization problem with multiple decision variables. This investigation makes a systematic examination of the feasibility of applying a newly proposed population-based optimization method labeled here as teaching---learning-based optimization (TLBO) to identify the unknown parameters for a class of chaotic system. The preliminary test demonstrates that despite its global fast coarse search capability, teaching---learning-based optimization often risks getting prematurely stuck in local optima. To enhance its fine (local) searching performance of TLBO, Nelder---Mead simplex algorithm-based local improvement is incorporated into TLBO so as to continually search for the global optima through the reflection, expansion, contraction, and shrink operators. Working with the well-established Lorenz system, we assess the effectiveness and efficiency of the proposed improved TLBO strategy. The empirical results indicate the success of the proposed hybrid approach in which the global exploration and the local exploitation are well balanced, providing the best solutions for all instances used over other state-of-the-art metaheuristics for chaotic identification in literature, including particle swarm optimization, genetic algorithm, and quantum-inspired evolutionary algorithm.

Digital IIR filter design problem is a crucial research issue in control and signal processing. From the viewpoint of optimization, IIR filter designing problems can be formulated as a multi-modal optimization problem with multiple decision variables. This study investigate the feasibility of applying a newly proposed optimization method labeled as water wave optimization (WWO) algorithm to identify the unknown parameters for IIR filter. To enhance its fine (local) searching performance of WWO, Nelder-Mead simplex algorithm based local improvement is incorporated into WWO so as to continually search for the global optima through the reflection, expansion, contraction, and shrink operators. By working on well-known IIR designing benchmark problems, we evaluate the effectiveness and efficacy of the proposed improved WWO algorithm. Experimental results as well as the comparisons with some state-of-the-art algorithm illustrate merit of the hybrid algorithm in which the local search and global search are well balanced.

Stirred tank bioreactors, employed in the production of a variety of biologically active chemicals, are often operated in batch, fed-batch, and continuous modes of operation. The optimal design of bioreactor is dependent on the kinetics of the biological process, as well as the performance criteria (yield, productivity, etc.) under consideration. In this paper, a general framework is proposed for addressing the two key issues related to the optimal design of a bioreactor, namely, (i) choice of the best operating mode and (ii) the corresponding flow rate trajectories. The optimal bioreactor design problem is formulated with initial conditions and inlet and outlet flow rate trajectories as decision variables to maximize more than one performance criteria (yield, productivity, etc.) as objective functions. A computational methodology based on genetic algorithm approach is developed to solve this challenging multiobjective optimization problem with multiple decision variables. The applicability of the algorithm is illustrated by solving two challenging problems from the bioreactor optimization literature.