Stochastic Dynamic rogramming for Uncertainty Handling in UAV Operations

The aim of this study is to estimate the optimal forest harvest considering economic and environmental objectives in an uneven-aged forest in the north of Iran. Required data were collected such as stumpage price, harvesting variable costs, growth, volume, basal area, sequestrated carbon, carbon dioxide storage, and price. The required input models for optimization were estimated using the collected data. A stochastic dynamic programming technique was used for optimization. The results indicated that the harvesting volume was higher when only the economic objective was considered instead of considering economic and environmental objectives. The results also indicated that the harvesting volume would occur at a higher level of volume per hectare if we only consider the environmental objective. The results showed that the expected present value was higher when economic and environmental objectives were considered simultaneously. The results indicated when interest rate increased, the harvesting time and the expected present value decreased. The results showed that the expected present value increases with increasing the risk. The results of this study can be a guideline for optimal forest management.

A stochastic dynamic programming approach for delay management of a single train line

<p style='text-indent:20px;'>In this paper, we study the robust optimal asset- problems for an ambiguity-averse investor, who does not have perfect information in the drift terms of the risky asset and liability processes. Two different kinds of objectives are considered: <inline-formula><tex-math id="M1">\begin{document}$ (i) $\end{document}</tex-math></inline-formula> Maximizing the minimal expected utility of the terminal wealth; <inline-formula><tex-math id="M2">\begin{document}$ (ii) $\end{document}</tex-math></inline-formula> Minimizing the maximal cumulative deviation. The ambiguity in both problems is described by a set of equivalent measures to the reference model. By the stochastic dynamic programming approach and Hamilton-Jacobi-Bellman (HJB) equation, we derive closed-form expressions for the value function and corresponding robust optimal investment strategy in each problem. Furthermore, some special cases are provided to investigate the effect of model uncertainty on the optimal investment strategy. Finally, the economic implication and parameter sensitivity are analyzed by some numerical examples. We also compare the robust optimal investment strategies in two different problems.</p>

This paper presents a novel formulation for integrated bidding and hydrothermal scheduling, where the market clearing price (MCP) is modeled as a Markov chain and the risks are directly formulated in the objective function in terms of MCP variances. The self-scheduling requirements and interactions between the energy and reserve markets are considered. A Lagrangian relaxation based algorithm is then developed to solve this problem, and a stochastic dynamic programming approach is developed to solve the unit subproblems. Since the bidding of pumped-storage units plays important roles in both the energy and reserve markets, and significantly affects thermal biddings, it is the focus in this paper. Effects of self-scheduling constraints, risk management and reserve market on the energy bidding strategies are analyzed. Numerical testing based on an 11-unit system in New England shows that the method can generate energy and reserve bid curves in 4-5 minutes, and risks of buying power at high market price are effectively reduced.

Joint production and maintenance optimization in flexible hybrid Manufacturing–Remanufacturing systems under age-dependent deterioration

In this paper, we consider a single item single level production planning problem with uncertain and dynamic demand. The uncertainties are modelled through the probability density function of the demand. The system is studied in a rolling-horizon framework, in which data is updated at each iteration and makes the demand non-stationary. In this context, we model a cost minimization problem using a stochastic dynamic programming approach, which allows to integrate the update of the demand density into the optimization process. We calculate the optimal production quantities in a specific case where linear regression is the forecasting method used and the production lead-time is equal to one period. We finally run a simulation to illustrate the optimality of the solution, and to show its efficiency compared to others classical production planning procedures.

ABSTRACTPerformance measures are often outlined in the section of the service‐level agreement (SLA) of the contract between a supplier and a retailer. They are monitored periodically, and penalty and/or bonus payments are imposed in each performance review period, according to the SLA clauses. Previous studies have mostly considered a static inventory policy in analyzing SLAs. However, in practice, the supplier may have an opportunity to adjust the stock level in each inventory review period, according to the observed performance. This study analyzes the dynamic stocking decision for a supplier facing an SLA where the supplier sells a single product to the retailer. The ready rate is used to measure the performance in an SLA. To this end, models for both lump‐sum and linear penalty/bonus structures are developed, and the optimal stocking decisions for a strategic supplier are calculated using the stochastic dynamic programming approach. The results are then compared with the optimal static inventory policy, and new insights are derived to efficiently design an inventory system for the suppliers that are subject to service‐level incentives. In addition, we investigate the impact of SLA parameters—such as the length of the performance review period and incentive structures—on a supplier's performance, with the probability of meeting or exceeding the target service levels and the supplier's cost. We also consider the impact of demand distribution and inventory holding costs. Results show that under lump‐sum incentives, a longer performance review period benefits both the supplier and the buyer, given that the average ready rate increases with less variability as the length of the performance review period increases, leading to decrements in the supplier's total costs. In this scenario, there is a higher chance of gaining bonuses/avoiding penalties for a strategic supplier who adopts a dynamic inventory policy. On the other hand, under linear incentives, the impact of the performance review period on the supplier's cost and the performance measure (i.e., ready rate) is complicated and depends on the magnitude of the holding cost and the bonus and/or penalty structure of the contract. Under this scheme, the performance of a static inventory policy is highly dependent on the holding cost because a high holding cost may lead to failure to meet the contract requirements in terms of the service level.

In this study, a microgrid with storage (battery, hot water tank) and solar panel is considered. We benchmark two algorithms, MPC and SDDP, that yield online policies to manage the microgrid, and compare them with a rule based policy. Model Predictive Control (MPC) is a well-known algorithm which models the future uncertainties with a deterministic forecast. By contrast, Stochastic Dual Dynamic Programming (SDDP) models the future uncertainties as stagewise independent random variables with known probability distributions. We present a scheme, based on out-of-sample validation, to fairly compare the two online policies yielded by MPC and SDDP. Our numerical studies put to light that MPC and SDDP achieve significant gains compared to the rule based policy, and that SDDP overperforms MPC not only on average but on most of the out-of-sample assessment scenarios.

Risk-averse stochastic dual dynamic programming approach for the operation of a hydro-dominated power system in the presence of wind uncertainty

The least squares Monte Carlo method is a decision evaluation method that can capture the effect of uncertainty and the value of flexibility of a process. The method is a stochastic approximate dynamic programming approach to decision making. It is based on a forward simulation coupled with a recursive algorithm which produces the near-optimal policy. It relies on the Monte Carlo simulation to produce convergent results. This incurs a significant computational requirement when using this method to evaluate decisions for reservoir engineering problems because this requires running many reservoir simulations. The objective of this study was to enhance the performance of the least squares Monte Carlo method by improving the sampling method used to generate the technical uncertainties used in obtaining the production profiles. The probabilistic collocation method has been proven to be a robust and efficient uncertainty quantification method. By using the sampling methods of the probabilistic collocation method to approximate the sampling of the technical uncertainties, it is possible to significantly reduce the computational requirement of running the decision evaluation method. Thus, we introduce the least squares probabilistic collocation method. The decision evaluation considered a number of technical and economic uncertainties. Three reservoir case studies were used: a simple homogeneous model, the PUNQ-S3 model, and a modified portion of the SPE10 model. The results show that using the sampling techniques of the probabilistic collocation method produced relatively accurate responses compared with the original method. Different possible enhancements were discussed in order to practically adapt the least squares probabilistic collocation method to more realistic and complex reservoir models. Furthermore, it is desired to perform the method to evaluate high-dimensional decision scenarios for different chemical enhanced oil recovery processes using real reservoir data.

Summary 1. Establishing biological control agents in the field is a major step in any classical biocontrol programme, yet there are few general guidelines to help the practitioner decide what factors might enhance the establishment of such agents. 2. A stochastic dynamic programming (SDP) approach, linked to a metapopulation model, was used to find optimal release strategies (number and size of releases), given constraints on time and the number of biocontrol agents available. By modelling within a decision‐making framework we derived rules of thumb that will enable biocontrol workers to choose between management options, depending on the current state of the system. 3. When there are few well‐established sites, making a few large releases is the optimal strategy. For other states of the system, the optimal strategy ranges from a few large releases, through a mixed strategy (a variety of release sizes), to many small releases, as the probability of establishment of smaller inocula increases. 4. Given that the probability of establishment is rarely a known entity, we also strongly recommend a mixed strategy in the early stages of a release programme, to accelerate learning and improve the chances of finding the optimal approach.

The determination of optimal operations policies for multi-reservoir systems under stochastic inflows is a hard problem. Dynamic Programming (DP) is widely used to solve such problems when the number of reservoirs is small. When there are many reservoirs in the system, the Stochastic Dynamic Programming (SDP) approach is impractical due to the “curse of dimensionality”. Therefore, some approximations have to be used to reduce the dimensionality of the problem. The Aggregation / Decomposition technique is one of the approximate methods used to solve multi-reservoir stochastic optimization problems. Stochastic differential equations can be used to model the dynamics of a multi-reservoir system. Under the assumption of Gaussian inputs, Ito’s stochastic calculus and the automatic formulation of moment equations may be used to derive the equivalent deterministic differential equations for the original stochastic system. These deterministic differential equations describe the evolution of the mean and the variance vector of the state variables of the system, namely, the reservoir storage volumes. Using these equations, optimal operational policies can be derived with the additional use of nonlinear optimization tools. In this paper, the advantages and disadvantages of these two stochastic methods for multi-reservoir problems will be presented.

A parallel computing implementation of a Serial Stochastic Dynamic Programming approach referred to as the S-SDP algorithm is introduced to solve large-scale multiperiod mixed 0–1 optimization problems under uncertainty. The paper presents Inner and Outer Parallelization versions of the S-SDP algorithm, referred to as Inner P-SDP and Outer P-SDP, respectively, so that the problem solving elapsed time and gap reduction is analyzed. The basic idea of Inner P-SDP consists of parallelizing the optimization of variations of the MIP subproblems attached to the sets of scenario clusters created by the modeler-defined stages to decompose the original problem. The Outer P-SDP performs simultaneous interconnected executions of the serial algorithm, so that a wider feasibility area is explored using iterative communication to redefine search directions. Strategies are presented to analyze the performance of parallel computation based on Message-Passing Interface threads to solve stage-related subproblems versus the serial version of SDP methodology. The results of using the parallelization are remarkable, as not only faster but also better solutions than the serial version are obtained. In particular, we report up to 10 times speedup for 12 threads on the Inner P-SDP algorithm. The new approach allows problems to be solved using less computing time than a state-of-the-art MIP solver. It can thus solve very large-scale problems that could not otherwise be achieved by plain use of the solver or by the S-SDP algorithm in acceptable elapsed time, if any.

In this work, we consider a point-to-point communication link where the transmitter has a hybrid supply of energy. Specifically, the hybrid energy is supplied by a constant energy source and an energy harvester, which harvests energy from its surrounding environment and stores it in a battery which suffers from energy leakage. Our goal is to minimize the power consumed by the constant energy source for transmission of a given amount of data in a given number of time intervals. Two scenarios are considered for packet arrival. In the first scenario, we assume that all data packets have arrived before transmission begins, whereas in the second scenario, we assume that data packets are arriving during the course of data transmission. For both scenarios, we propose an optimal offline transmit power allocation scheme which provides insight into how to efficiently consume the energy supplied by the constant energy source and the energy harvester. For offline power allocation, we assume that causal and non-causal information regarding the channel and the amount of harvested energy is available a priori. For optimal online power allocation, we adopt a stochastic dynamic programming (DP) approach for both considered scenarios. For online power allocation, only causal information regarding the channel and the amount of harvested energy is assumed available. Due to the inherent high complexity of DP, we propose suboptimal online algorithms which are appealing because of their low complexity. Simulation results reveal that the offline scheme performs best among all considered schemes and the suboptimal online scheme provides a good performance-complexity tradeoff.

Maintenance optimization has been extensively studied in the past decades. However, most of the existing maintenance models focus on single-component systems and are not applicable to complex systems consisting of multiple components, due to various interactions among the components. The multicomponent maintenance optimization problem, which joins the stochastic processes regarding the failures of components with the combinatorial problems regarding the grouping of maintenance activities, is challenging in both modeling and solution techniques, and has remained an open issue in the literature. In this paper, we study the multicomponent maintenance problem over a finite planning horizon and formulate the problem as a multistage stochastic integer program with decision-dependent uncertainty. There is a lack of general efficient methods to solve this type of problem. To address this challenge, we use an alternative approach to model the underlying failure process and develop a novel two-stage model without decision-dependent uncertainty. Structural properties of the two-stage problem are investigated, and a progressive-hedging-based heuristic is developed based on the structural properties. Our heuristic algorithm demonstrates a significantly improved capacity to handle large-size two-stage problems comparing to three conventional methods for stochastic integer programming, and solving the two-stage model by our heuristic in a rolling horizon provides a good approximation of the multistage problem. The heuristic is further benchmarked with a dynamic programming approach and a structural policy, which are two commonly adopted approaches in the literature. Numerical results show that our heuristic can lead to significant cost savings compared with the benchmark approaches.