Effective Streamflow Process Modeling for Optimal Reservoir Operation Using Stochastic Dual Dynamic Programming

In this study, a microgrid with storage (battery, hot water tank) and solar panel is considered. We benchmark two algorithms, MPC and SDDP, that yield online policies to manage the microgrid, and compare them with a rule based policy. Model Predictive Control (MPC) is a well-known algorithm which models the future uncertainties with a deterministic forecast. By contrast, Stochastic Dual Dynamic Programming (SDDP) models the future uncertainties as stagewise independent random variables with known probability distributions. We present a scheme, based on out-of-sample validation, to fairly compare the two online policies yielded by MPC and SDDP. Our numerical studies put to light that MPC and SDDP achieve significant gains compared to the rule based policy, and that SDDP overperforms MPC not only on average but on most of the out-of-sample assessment scenarios.

Stochastic dual dynamic programming is a cutting plane type algorithm for multi-stage stochastic optimization originated about 30 years ago. In spite of its popularity in practice, there does not exist any analysis on the convergence rates of this method. In this paper, we first establish the number of iterations, i.e., iteration complexity, required by a basic dynamic cutting plane method for solving relatively simple multi-stage optimization problems, by introducing novel mathematical tools including the saturation of search points. We then refine these basic tools and establish the iteration complexity for both deterministic and stochastic dual dynamic programming methods for solving more general multi-stage stochastic optimization problems under the standard stage-wise independence assumption. Our results indicate that the complexity of some deterministic variants of these methods mildly increases with the number of stages $T$, in fact linearly dependent on $T$ for discounted problems. Therefore, they are efficient for strategic decision making which involves a large number of stages, but with a relatively small number of decision variables in each stage. Without explicitly discretizing the state and action spaces, these methods might also be pertinent to the related reinforcement learning and stochastic control areas.

This article presents a methodology for including wind power generation in the medium-term planning ofhydrothermal systems, where Stochastic Dual Dynamic Programming (SDDP) is widely applied in the literature to solve this class of problem. To assess the impact of the intermittent generation, wind power scenarios were generated through Weibull Distribution, which were applied to reduce the load, generating several demand scenarios. Thus, the aim of this paper is to improve the computational eort of the conventional SDDP with demand scenarios, where the main contribution of the work consists of applying the Immediate Cost Function (ICF) to accelerate the SDDP convergence process. The proposed methodology was analyzed using part of the Brazilian system, considering a wind farm.

Intermittent energy sources demand temporal storages to balance generation and load, and batteries stand out as an alternative. However, the lifetime is limited, and cycling depth affects the battery degradation rate. Current stochastic multi-stage methods lack proper representation of battery degradation. This paper proposes a stochastic multi-stage model for optimizing battery operation in a microgrid considering battery degradation with a piece-wise linear cost function with uncertain wind power production and load. The model is solved using Stochastic Dual Dynamic Programming (SDDP) and is demonstrated on a 4-bus test case with limited import and export capacity to illustrate the battery degradation cost's impacts on the battery cycling strategy. The results show that the importance of a stochastic method is more pronounced when battery degradation is modelled.

Stochastic dual dynamic programming (SDDP) is a popular method for hydro-thermal planning and recently has been applied to energy optimization problems in smart grids involving wind energy resources. In this method, there is a need for incorporating dynamic models in order to take into account time correlations of uncertainties. The models used should posses certain linearity properties to conserve the convexity of the optimization problem to be solved. This is crucial for the convergence of the SDDP algorithm. In the past, especially in hydro-thermal planning, linear autoregressive (AR) models were usually used for this purpose. However, they can produce negative values which is not realistic and can lead to difficulties in SDDP computations. As a remedy, one can employ additive or multiplicative AR models which are capable of producing nonnegative time series. Despite of this fact, the works using such models in SDDP based wind energy applications are very rare and model estimation methods were not elaborated. Moreover, their accuracy in representing real wind uncertainty was not studied well. Motivated with these facts, in the present study, modeling of wind speed distribution by such AR models is investigated. The parameters of the models were estimated by solving constrained least squares optimization problems. In order to asses the accuracy of corresponding distributions, a nonparametric method for computing the K-L divergence between probability density functions was utilized. Finally, the models were compared in terms of accuracy of their distributions and features of their parameter estimation algorithms.

The long-term generation scheduling (LTGS) problem aims to build an operating policy over a multi-year planning horizon, correlating thermal generation and deficit costs with water storage. The LTGS is modeled as a linear multistage stochastic program, whose state-of-art solution is the stochastic dual dynamic programming (SDDP). One critical challenge is to represent the time-dependency of river inflows accurately. Despite recent advances, modeling simplifications are needed to allow the LTGS computational tractability via SDDP since it is necessary to increase the state space to include the time-dependent variables properly. Thus, most works simplify the hydro production function (HPF) to represent the inflows in detail to overcome this negative aspect. However, the literature lacks this trade-off, i.e., finding a balance between stochastic inflow modeling and HPF representation. Thus, this paper proposes an alternative approach to analyzing this trade-off, using the inflow aggregation and run-of-the-river energy inflow instead of individual inflow in SDDP. The proposed approach drastically reduces the number of state variables in SDDP, allowing a detailed HPF representation in the LTGS. We employ a one-sided confidence interval for expected cost estimation to show that our approach provides better performance than the individualized inflows. The analysis is performed in several large-scale computational instances using a power system with 53 geographically widespread hydro plants. The numerical results demonstrate that inflow aggregation provides, on average, a 4 % reduction on the operating cost, whereas for the run-of-the-river inflow energy, the reduction is, on average, 2 %.

We study the uncapacitated lot-sizing problem with uncertain demand and costs. We consider a multi-stage decision process and rely on a scenario tree to represent the uncertainty. We propose to solve this stochastic combinatorial optimization problem thanks to a new extension of the stochastic dual dynamic integer programming algorithm. Our results show that this approach can provide good quality solutions in a reasonable time for large-size instances.

Water uses in the Indus River Basin (IRB) – agriculture, hydropower, etc. – primarily depend on meltwater from Himalayan glaciers, seasonal snow, and monsoon rains. This dependency makes the IRB vulnerable to climate change and associated deeply uncertain shifts in seasonal melt patterns, long-term glacier recession and erratic monsoon patterns. Therefore, there's a growing emphasis on creating robust adaptation plans suitable for various future scenarios, rather than focusing on a specific uncertain future. Our research aims to develop a robustness framework to capture the multi-reservoir system with multiple contributions to streamflow from various sources with unique characteristics for the IRB. Our methodology quantifies streamflow components across multiple reservoirs for the IRB into base flow, rainfall, snowmelt, and glacial melt. This breakdown enables us to model each individual component's impact under a changing climate. We use Fourier series transformations to represent shifts in magnitude and seasonality of each component. This then enables us to explore how these shifts impact the resilience of reservoir operating policies, typically based on static hydrological conditions. To determine operating policies in complex multi-reservoir systems, we use Stochastic Dual Dynamic Programming (SDDP), due to its ability to capture the stochastic nature of streamflow amidst dynamic and uncertain hydrological inputs. Running SDDP with historical flows yields policies whose robustness to streamflow change can be tested; alternatively, running SDDP with select climate change scenarios yields altered policies whose climate “niche” can then be determined. This approach enables the design of coordinated reservoir operation policy, optimal under different streamflow conditions, as well as their evaluation over a wide range of potential future conditions, enhancing the optimisation of water use and management policies for the IRB. Our findings are expected to offer valuable insights assisting policymakers, stakeholders, and water resource managers in developing more resilient and sustainable strategies to address the impacts of climate change on water resources.

This study introduces a novel approach for optimizing residential energy systems by combining linear policy graphs with stochastic dual dynamic programming (SDDP) algorithms. Our method optimizes residential solar power generation and battery storage systems, reducing costs through strategic charging and discharging patterns. Using stylized test data, we evaluate battery storage optimization strategies by comparing various SDDP model configurations against a linear programming (LP) benchmark model. The SDDP optimization framework demonstrates robust performance in battery operation management, efficiently handling diverse pricing scenarios while maintaining computational efficiency. Our analysis reveals that the SDDP model achieves positive financial returns with small-scale battery installations, even in scenarios with limited photovoltaic generation capacity. The results confirm both the economic viability and environmental benefits of residential solar–battery systems through two key strategies: aligning battery charging with renewable energy availability and shifting energy consumption away from peak periods. The SDDP framework proves effective in managing battery operations across dynamic pricing scenarios, achieving performance comparable to LP methods while handling uncertainties in PV generation, consumption, and pricing.

Risk neutral and risk averse Stochastic Dual Dynamic Programming method

In this paper, we study multistage stochastic mixed-integer nonlinear programs (MS-MINLP). This general class of problems encompasses, as important special cases, multistage stochastic convex optimization with non-Lipschitzian value functions and multistage stochastic mixed-integer linear optimization. We develop stochastic dual dynamic programming (SDDP) type algorithms with nested decomposition, deterministic sampling, and stochastic sampling. The key ingredient is a new type of cuts based on generalized conjugacy. Several interesting classes of MS-MINLP are identified, where the new algorithms are guaranteed to obtain the global optimum without the assumption of complete recourse. This significantly generalizes the classic SDDP algorithms. We also characterize the iteration complexity of the proposed algorithms. In particular, for a (T+1)-stage stochastic MINLP satisfying L-exact Lipschitz regularization with d-dimensional state spaces, to obtain an varepsilon -optimal root node solution, we prove that the number of iterations of the proposed deterministic sampling algorithm is upper bounded by {mathcal {O}}((frac{2LT}{varepsilon })^d), and is lower bounded by {mathcal {O}}((frac{LT}{4varepsilon })^d) for the general case or by {mathcal {O}}((frac{LT}{8varepsilon })^{d/2-1}) for the convex case. This shows that the obtained complexity bounds are rather sharp. It also reveals that the iteration complexity depends polynomially on the number of stages. We further show that the iteration complexity depends linearly on T, if all the state spaces are finite sets, or if we seek a (Tvarepsilon )-optimal solution when the state spaces are infinite sets, i.e. allowing the optimality gap to scale with T. To the best of our knowledge, this is the first work that reports global optimization algorithms as well as iteration complexity results for solving such a large class of multistage stochastic programs. The iteration complexity study resolves a conjecture by the late Prof. Shabbir Ahmed in the general setting of multistage stochastic mixed-integer optimization.

ABSTRACTIn bid-based markets, energy producers seek bidding strategies that maximize their revenue. In this article, we seek the maximum-revenue bidding schedule for a single price-maker hydroelectric producer. We assume the producer sells energy in the day-ahead electricity market and has the ability to impact the market-clearing price with its bids. To obtain the price-maker hydroelectric producer’s bidding schedule, we use a combination of Stochastic Dual Dynamic Programming and Lagrangian relaxation. In this framework, we dualize the water balance equations, allowing an exact representation of the non-concave immediate revenue function, while preserving the concave shape of the future revenue function. We model inflow uncertainty and its stagewise dependence by a periodic autoregressive model. To demonstrate our approaches’ utility, we model Honduras’ electricity market assuming that the thermal producers act as price-takers and that one price-maker hydro producer operates all of the hydroelectric plants.

Risk-averse stochastic dual dynamic programming approach for the operation of a hydro-dominated power system in the presence of wind uncertainty

This paper describes the development and application of the Stochastic Dual Dynamic Programming (SDDP) algorithm for solving the optimal medium-term hydrothermal scheduling in the Greek power system. The method aims at providing a more elaborate optimization tool compared to the traditional approaches used for medium-term hydrothermal scheduling in Greece; the latter have been developed on deterministic assumptions, while the proposed tool recognizes the impact of inflow uncertainty and provides well-hedged management strategies for each reservoir. The described method is developed using GAMS/CPLEX and tested for a power system similar to the Greek Interconnected Power System. The method proposed can be a stepping stone towards the calculation of the variable cost of each hydroplant on a monthly basis, a goal that has been officialy set forth by the Greek Regulatory Authority for Energy.

To address the uncertain renewable energy in the day-ahead optimal dispatch of energy and reserve, a multi-stage stochastic programming model is established in this paper to minimize the expected total costs. The uncertainties over the multiple stages are characterized by a scenario tree and the optimal dispatch scheme is cast as a decision tree which guarantees the flexibility to decide the reasonable outputs of generation and the adequate reserves accounting for different realizations of renewable energy. Most importantly, to deal with the “Curse of Dimensionality” of stochastic programming, stochastic dual dynamic programming (SDDP) is employed, which decomposes the original problem into several sub-problems according to the stages. Specifically, the SDDP algorithm performs forward pass and backward pass repeatedly until the convergence criterion is satisfied. At each iteration, the original problem is approximated by creating a linear piecewise function. Besides, an improved convergence criterion is adopted to narrow the optimization gaps. The results on the IEEE 118-bus system and real-life provincial power grid show the effectiveness of the proposed model and method.