Research on Order Policy Based on Multi-Stage Dynamic Programming

In Short-term Cascade Reservoirs Optimal Operation (SCROO), the flow from upstream reservoir to downstream reservoir which propagates in the natural channel can be generalized as not only transposition but also attenuation. In order to get the relative accurate inflow series of downstream reservoir, the paper tries to adopt the Muskingum model to simulate the inflow of the downstream reservoir instead of the easier processing in most papers of SCROO: ignoring the flow attenuation in natural channel. Considering the flow attenuation between the cascade reservoirs, the SCROO problem do not satisfy the “Principe of Optimality” anymore, so the Dynamic Programming(DP) is no longer applicable. The paper proposes a new improved DP named Multi-Stage Dynamic Programming (MSDP) based on DP. In MSDP, multi-stage’s outflow of upstream reservoir is taken into account at the same time and then inflow of downstream reservoir can be calculated by the Muskingum model, which can include much more flow information than the easier processing with DP, and the inflow of downstream reservoir will be closer to the actual one. The paper takes the cascade hydro-power stations consisting of Jindong and Guandi in Yalong river basin as an example to solve the SCROO problem with MSDP. By comparing the operation result of MSDP and DP with the easier processing, the operation strategy of MSDP can gain further benefits than DP’s in actual operation.

This paper studies the online dispatch of distribution networks (DNs), which is formulated as a multi-stage dynamic programming (MSDP) to ensure the non-anticipativity of dispatch decisions. Existing approaches usually relegate expensive online optimization to offline learning (typically value function learning) using the given uncertainty distribution or training samples. However, practical DNs may encounter various scenarios where the distributions of uncertainty differ significantly. The optimality of these approaches may degrade substantially in out-of-distribution scenarios unless frequent re-training is conducted. To address this obstacle, this paper proposes a scenario-generalized MSDP (S-MSDP) scheme for online dispatch of DNs. Its main advantage is the ability to directly adapt to new scenarios with high optimality, without re-training or fine-tuning. S-MSDP extends MSDP by learning a universal value function that maps scenario contexts to the corresponding value functions, so that the optimal dispatch policies under different scenarios can be directly inferred by using the learned universal value function. To facilitate the computation and storage burdens brought by large scenario space, a sparse and low-rank tensor approximation is introduced for universal value function learning. Numerical studies verify the optimality, generalization, and scalability of S-MSDP.

Rate-distortion-smoothness optimized rate allocation schemes for Spectral Fine Granular Scalable video coding technique

Considering water propagation impact in short-term optimal operation of cascade reservoirs using Nested Progressive Optimality Algorithm

Case study on optimization scheme design based on load prediction for regional heating system in industrial community

This study models and explains the business situation of an organisation which has regular and emergency outsourcing sources and where decisions have to be made at the beginning of every period regarding how much to order from these sources, so as to balance between different cost components of current and future periods. Previous works in this area have tried to attack this problem with dynamic programming. In this project, neuro-dynamic programming has been applied, and the reasons for doing so have been clearly stated. This model not only derives policies in order to minimise the expected total discounted cost over a period of time, but also enables the system to learn to make such decisions, and to improve its actions by using reinforcement learning. The performance of the present work has been measured quantitatively and has been compared with the models stated in literature. This study will be very useful for the organisations where such business problems exist or are likely to exist. This study will also be of great use to researchers, who are keen to understand and model the given business situation with distribution independent demand based models.

Optimal ordering, issuance and disposal policies for inventory management of perishable products