Stochastic dynamic programming applied to multiple reservoir systems

ABSTRACT: This paper describes two methods that are introduced to improve the computational effort of stochastic dynamic programming (SDP) as applicable to the operation of multiple urban water supply reservoir systems. The stochastic nature of streamflow is incorporated explicitly by considering it in the form of a multivariate probability distribution. The computationally efficient Gaussian Legendre quadrature method is employed to compute the conditional probabilities of streamflow, which accounts for the serial correlation of streamflow into each storage and the cross correlation between the streamflow into various storages. A realistic assumption of cross correlation of streamflow is introduced to eliminate the need to consider the streamflow combinations which are unlikely to occur in the SDP formulation. A “corridor” approach is devised to eliminate the need to consider the infeasible and/or inferior storage volume combinations in the preceding stage in computing the objective function in the recursive relation. These methods are verified in terms of computational efficiency and accuracy by using a hypothetical example of three interconnected urban water supply reservoirs. Therefore, it can be concluded that these methods allow SDP to be more attractive for deriving optimal operating rules for multiple urban water supply reservoir systems.

Reasonable optimal operation policy for complex multiple reservoir systems is very important for the safe and efficient utilization of water resources. The operation policy of multiple hydropower reservoirs should be optimized to maximize total hydropower generation, while ensuring flood control safety by effective and efficient storage and release policy of multiple reservoirs. To achieve this goal, a new meta-heuristic algorithm, salp swarm algorithm (SSA), is used to optimize the joint operation of multiple hydropower reservoirs for the first time. SSA is a competitive bio-inspired optimizer, which has received substantial attention from researchers in a wide variety of applications in finance, engineering, and science because of its little controlling parameters and adaptive exploratory behavior. However, it still faces few drawbacks such as lack of exploitation and local optima stagnation, leading to a slow convergence rate. In order to tackle these problems, multiple strategies combining sine cosine operator, opposition-based learning mechanism, and elitism strategy are applied to the original SSA. The sine cosine operator is applied to balance the exploration and exploitation over the course of iteration; the opposition-based learning mechanism is used to enhance the diversity of the swarm; and the elitism strategy is adopted to find global optima. Then, the improved SSA (ISSA) is compared with six well-known meta-heuristic algorithms on 23 classical benchmark functions. The results obtained demonstrate that ISSA outperforms most of the well-known algorithms. Then, ISSA is applied to optimal operation of multiple hydropower reservoirs in the real world. A multiple reservoir system, namely Xiluodu Reservoir and Xiangjiaba Rservoir, in the upper Yangtze River of China are selected as a case study. The results obtained show that the ISSA is able to solve a real-world optimization problem with complex constraints. In addition, for the typical flood with a 100 return period in 1954, the maximum hydropower generation of multiple hydropower reservoirs is about 6671 GWh in the case of completing the flood control task, increasing by 1.18% and 1.77% than SSA and Particle Swarm Optimization (PSO), respectively. Thus, ISSA can be used as an alternative effective and efficient tool for the complex optimization of multiple hydropower reservoirs. The water resources in the river basin can be further utilized by the proposed method to cope with the increasingly serious climate change.

In this research, application of some real-time operation methods on a single-reservoir system, Karoon IV, with decreasing sum of drinking, industry and agriculture deficit has been considered. These methods include standard operation policy (SOP), stochastic dynamic programming (SDP) and some types of operational rule curves with different orders of inflow and reservoir storage volume. To rank aforesaid, a multiattribute decision method, ELECTRE I, with combination of indices, objective function and reservoir performance criteria (reliability, resiliency and vulnerability) has been used. This ranking is accomplished in two different states including: (1) performances criteria and (2) objective function and performance criteria using the same weights for all criteria. Results show that rule curves were selected as the suitable policy in real-time operation and models validation has been presented by testing nominated curve for dry, normal and wet years.

Multiple reservoir operation is of paramount importance due to tradeoffs in water supply and their cost functions. Understanding this complexity is important for optimizing water supply and increasing synergies gained from the joint operation. Therefore, this study aimed to develop a conceptual framework for addressing the effects of climate change on water security under the operating rules of the multiple reservoir system in northern France. A dynamic programming approach (DP) was employed to find the cost–benefit analysis that best fit with the objectives of reservoir operation, while the space rule was applied to balance the available space in each reservoir of a parallel system. A finite-horizon optimal regulation was then adopted for determining daily reservoir storage based on probability-based rule curves. The results indicated that the predicted inflow during the drawdown–refill cycle period to the Marne and Pannecière reservoirs would be the largest and lowest, respectively. The proposed upper rule curves during high-flow conditions suggested that the release from Aube reservoir should be postponed from July to August until September. At 50- and 100-year return periods, quite a high release rate from Seine and Marne reservoirs was observed during the dry season. A decrease in future water supply from Pannecière reservoir was found during summer, while the withdrawal in November could cause excessive water in the Seine tributary and Paris City. Under low-flow conditions in all return periods, the proposed lower rule curves recommended that the reservoir storage should go below the current operating rule, with a clear difference in July (the largest in Marne and the smallest in Pannecière) and almost no difference in November. Moreover, the web-based support system IRMaRA was developed for revising operating rules of four main reservoirs located in the Seine River Basin. The novelty of this modeling framework would contribute to the practice of deriving optimal operating rules for a multi-reservoir system by the probability-based rule curve method. Based on the evaluation of the effects of applying the estimated reservoir storage capacity under different return periods, both less overflow and water shortage represented by different levels of quantity and severity can be expected compared to the existing target storage at specified control points. Finally, the obtained finding revealed that the application of dynamic programming for reservoir optimization would help in developing a robust operating policy for tackling the effects of climate change.

A proposed stochastic dynamic programming (SDP) method with uncertainties in stream flow and water demand is developed to calculate optimal reservoir operation rules. The SDP method extends the classic SDP method that considers only one uncertainty in its operation rules. Time series of reservoir inflow and agricultural water demand for the Aydoghmoush Reservoir in eastern Azarbayejan, Iran, were obtained from meteorological data, available climate parameters, hydrologic data, and crop water demand. The application of the developed SDP with two uncertainties was evaluated with operation rules corresponding to four different scenarios, and optimal reservoir releases were determined for a drought year. Reservoir operation results were evaluated with different performance indices. This study's results demonstrate the advantage of considering uncertainties in reservoir inflow and water demand with the SDP method. The developed SDP method has general applicability under a range of climatic conditions, and the calculated operation rules cover the expected ranges of streamflow and water demand during the operating years.

In this paper, we present a proportionate-type normalized least mean square algorithm which operates by choosing adaptive gains at each time step in a manner designed to maximize the joint conditional probability that the next-step coefficient estimates reach their optimal values. We compare and show that the performance of the joint maximum conditional probability density function (PDF) one-step algorithm is superior to the proportionate normalized least mean square algorithm when operating on a sparse impulse response. We also show that the new algorithm is superior to a previously introduced algorithm which assumed that the conditional PDF could be represented by the product of the marginal conditional PDFs, i.e., that the weight deviations are mutually conditionally independent.

Reservoir management is a highly complex problem, which includes the uncertainty in the inflows as well as in the objectives. Stochastic dynamic programming (SDP) has been widely used to develop the optimal operating policies of multipurpose reservoirs. In the recent years, fuzzy optimization has been used to incorporate the uncertainty caused due to the imprecise nature of objectives. In the present study, a comparison is made between a conventional SDP model and fuzzy dynamic programming models for optimal operation of a multipurpose reservoir. The two models are developed for Hirakud Reservoir on River Mahanadi in the State of Orissa in India. The performance of the reservoir is evaluated under the operating policies derived from these models through simulation. The results indicate that the fuzzy dynamic programming model performs better in achieving the flood control objective and thus in the overall performan1ce of reservoir.

Compared with the conventional energy power generation investments, the uncertainty of wind power investment will bring greater risk. For wind power investors, wind power investment strategy-makers, even for wind power equipment manufacturers, all urgently need decision-support tools to minimize the risk and obtain optimal investment opportunities. In this paper, firstly we analyze the current problems and risks of wind power industry in China. Secondly, based on stochastic theory and dynamic programming methods, the paper establishes model about wind power investment decision-making. It considers the wind power project investment uncertainties risk factors. Example analysis shows that the model can be used for wind power developers to choose the right investment opportunities and to provide decision-making reference.

Reliable long-term streamflow forecasting is invaluable for water resource planning and management which allocates water supply according to the demand of water users. It is necessary to get probabilistic forecasts to establish risk-based reservoir operation policies. Probabilistic forecasts may be useful for the users who assess and manage risks according to decision-making responding forecasting results. Probabilistic forecasting of seasonal inflow to Andong dam is performed and assessed using selected predictors from sea surface temperature and 500 hPa geopotential height data. Categorical probability forecast by Piechota's method and logistic regression analysis, and probability forecast by conditional probability density function are used to forecast seasonal inflow. Kernel density function is used in categorical probability forecast by Piechota's method and probability forecast by conditional probability density function. The results of categorical probability forecasts are assessed by Brier skill score. The assessment reveals that the categorical probability forecasts are better than the reference forecasts. The results of forecasts using conditional probability density function are assessed by qualitative approach and transformed categorical probability forecasts. The assessment of the forecasts which are transformed to categorical probability forecasts shows that the results of the forecasts by conditional probability density function are much better than those of the forecasts by Piechota's method and logistic regression analysis except for winter season data.

This paper studies a class of renewable resource allocation problems for the processing of dynamically arriving tasks with deterministic deadlines. This class of problems has many applications, however, it conforms to neither the standard resource allocation model nor the standard optimal control model. A new problem formulation has to be developed and analyzed to provide a satisfactory solution. In this paper, a new formulation is presented. By augmenting the state variabls, the problem is converted into a Markovian decision problem. It can then be treated, at least in principle, by using the stochastic dynamic programming (SDP) method. However, since the system dynamics involves the evolution of sets (of tasks and resources), the implementation of the dynamic programming equation is by no means straightforward. For a problem with infinite planning horizon, the optimal strategy is shown to be stationary under mild conditions. An SDP algorithm based on a successive approximation technique is developed to obtain the optimal stationary strategy. The implementation of the algorithm employs a special coding scheme to handle set variables, and utilizes a dominance property for computational efficiency. Effects of key system parameters on optimal decisions are investigated and analyzed through numerical examples. As the computational complexity of the algorithm is of exponential increase, practical applications of the algorithm is limited to problems of moderate size. Two heuristic rules are therefore investigated and compared to the optimal policy.

This paper proposes a method (SDP method) of off-line signature verification incorporating a sequential application of the dynamic programming matching method in its preprocessing stage. Based on the frequencies of black pixels in each signature projected on the x-axis, the SDP method determines an optimum segmentation of any signature into the same number of components as the representative reference signature. The SDP method measures the dissimilarity between the suspect signature and the representative signature, and accept the suspect signature as genuine without any further scrutiny, if the dissimilarity is below a threshold. When the suspect signature is not accepted, the SDP method subsequently applies a verification procedure proposed by Yoshimura and Yoshimura(1997) to it. This paper evaluates also the effectiveness of the SDP method through an experiment using a database of 800 Japanese signatures, half of which were obtained on the area of countersignature on sheets similar to a traveler's check, whereas the remaining were on the area of authentic signature. The experiment used the countersignatures as suspect signatures to obtain estimates of mis-verification rate, taking five signatures alternatively from the twenty authentic signatures as the reference signatures. It achieved values as low as 12.9% in an average mis-verification rate.KeywordsReference SignatureDynamic Programming AlgorithmDynamic Time WarpingInput SignatureBlack PixelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A conditional confidence interval for the scale parameter of the extrernevalue distribution following a preliminary test is investigated. The conditional confidence interval (CCI) may be shorter than the unconditional confidence interval (UCI) in contrast to the findings for the mean of a normal distribution by Meeks and D'Agostino (1983). The conditional coverage probability of the UCI is obtained by computing the coverage probability under the conditional probability density function. It is shown that the conditional coverage probability of the UCI is not uniformly greater than or less than the nominal level. The UCI is appropriate only when one does not perform any pre-test. However, if a pre-test is performed before the constructicin of a confidence interval, then the appropriate confidence interval is the CCI because the pre-test affects subsequent inference procedures.

Reservoir operation rules are logical or mathematical equations that take into account system variables to calculate water release from a reservoir based on inflow and storage volume values. In fact, previous experiences of the system are used to balance reservoir system parameters in each operational period. Commonly, reservoir operation rules have been considered to be linear decision rules (LDRs) and constant coefficients developed by using various optimization procedures. This paper addresses the application of real-time operation rules on a reservoir system whose purpose is to supply total downstream demand. Those rules include standard operation policy (SOP), stochastic dynamic programming (SDP), LDR, and nonlinear decision rule (NLDR) with various orders of inflow and reservoir storage volume. Also, a multi-attribute decision method, elimination and choice expressing reality (ELECTRE)-I, with a combination of indices, objective functions, and reservoir performance criteria (reliability, resiliency, and vulnerability) are used to rank the aforementioned rules. The ranking method employs two combinations of indices: (1) performance criteria and (2) objective function and performance criteria by using the same weights for all criteria. Results show that the NLDR gives an appropriate rule for real-time operation. Moreover, NLDR validation is presented by testing predefined curves for dry, normal, and wet years.

Despite the high potential of geothermal reservoirs in Meshkinshahr, we only see the government entering the electricity extraction of geothermal energy because the cost of the Meshkinshahr geothermal electricity is higher than the balanced price of the electricity market and the private sector is disadvantaged for entering it. So, the government has adopted a feed in tariff policy for geothermal electricity to encourage private sector investors. This research is aimed at financial appraisal of Maskinshahr geothermal power plant with the assumption of feed in tariff by the government with Equivalent Uniform Annual Worth, Equivalent Uniform Annual Worth, and Equivalent Uniform Annual Cost methods. Equivalent Uniform Annual Benefit is obtained from feed in tariff of government and Equivalent Uniform Annual Cost is determined based on technical and economic components of the geothermal power plant and macroeconomic parameters. The results of the data analysis show that the construction of the geothermal power plant is fully justified with the 14% reduction rate, but if the investor's minimum expected rate exceeds 54%, the construction of the power plant has no economic justification. Also, if the construction of Meshkinshahr geothermal plant takes more than 13 years and 5 months like the government project, the generation of geothermal power is not cost-effective.

The natural flow regime can sustain the ecological integrity of riverine ecosystems. Different reservoir operation polices differ in their effects on the degree of alteration of natural flow regimes. Dynamic programming plays an important role in developing operation policies. When using dynamic programming models to develop operation policies, the discrete number of storage states (DNSS), which is a key factor affecting the reservoirs operation policies, is always been determined based on computational efficiency and economic benefits. Little consideration has been given to the ecological disturbance caused by different DNSS‐based operation policies. To analyze the impact of DNSS, we built a deterministic dynamic programming model to explore the relationship among DNSS, the flow regime alteration (ecological disturbance), and the cumulative annual power generation (economic benefits) by setting a range of DNSS scenarios. We used three reservoirs with different storage coefficients (ratios of usable storage to annual average runoff) as examples and used the range of variability approach to assess the ecological disturbance under these scenarios. We compared the results with those of a stochastic dynamic programming (SDP) model and a Bayesian SDP (BSDP) model. We found that when DNSS is low, increasing DNSS improves economic benefits but causes a more severe ecological disturbance; when DNSS is high, increasing DNSS improves the economic benefits only slightly, without exacerbating the ecological disturbance; for a given DNSS, the BSDP model provides higher economic benefits than the SDP model and a similar disturbance of the riverine ecosystem; and larger reservoirs more often cause more severe disturbance of riverine ecosystems because monthly mean flows and annual extreme flows change more drastically. Our results will help to protect the riverine ecosystems and improve economic benefits if reservoir operation managers consider DNSS using dynamic programming models.