Stochastic Task Selection and Renewable Resource Allocation

Compared with the conventional energy power generation investments, the uncertainty of wind power investment will bring greater risk. For wind power investors, wind power investment strategy-makers, even for wind power equipment manufacturers, all urgently need decision-support tools to minimize the risk and obtain optimal investment opportunities. In this paper, firstly we analyze the current problems and risks of wind power industry in China. Secondly, based on stochastic theory and dynamic programming methods, the paper establishes model about wind power investment decision-making. It considers the wind power project investment uncertainties risk factors. Example analysis shows that the model can be used for wind power developers to choose the right investment opportunities and to provide decision-making reference.

A proposed stochastic dynamic programming (SDP) method with uncertainties in stream flow and water demand is developed to calculate optimal reservoir operation rules. The SDP method extends the classic SDP method that considers only one uncertainty in its operation rules. Time series of reservoir inflow and agricultural water demand for the Aydoghmoush Reservoir in eastern Azarbayejan, Iran, were obtained from meteorological data, available climate parameters, hydrologic data, and crop water demand. The application of the developed SDP with two uncertainties was evaluated with operation rules corresponding to four different scenarios, and optimal reservoir releases were determined for a drought year. Reservoir operation results were evaluated with different performance indices. This study's results demonstrate the advantage of considering uncertainties in reservoir inflow and water demand with the SDP method. The developed SDP method has general applicability under a range of climatic conditions, and the calculated operation rules cover the expected ranges of streamflow and water demand during the operating years.

The time needed to complete a software development project is hard to estimate in practice. In addition to the uncertainty about the duration of the individual development steps, software projects often suffer from unexpected rework and delays. As a result, project scheduling remains a difficult task for the managers of software projects. In this paper, we compute optimal scheduling strategies for a set of sample software projects. The computations are based on a stochastic Markov decision model for software projects which focuses on capturing the feedback between concurrent development activities. The optimal strategies are computed using a variant of the value iteration algorithm from stochastic dynamic programming. Since the underlying process model is stochastic, the strategies are stochastically optimal, that is, they minimize the expected project duration. The ultimate goal of this research is to develop guidelines for managers how to schedule their software projects under uncertainty in the best possible way. The sample projects are similar, but differ in certain characteristics of the project or product, such as the strength of the coupling between the components or the degree of specialization of the teams on the tasks. By using a set of related projects, we can study how the project characteristics influence the optimal scheduling decisions in a project. In addition to computing the optimal scheduling policies for the sample projects, we use extensive simulations to compare the performance of the optimal policy against the so-called list policies for each given setting. List policies are a simple, but commonly used class of scheduling policies. For our sample projects, we find that the best list policy in general is not optimal. The performance gap is the larger the higher the degree of specialization of the teams is. On the other hand, the stronger the coupling between the components, the smaller is the improvement which the optimal policy achieves over the best list policy.

In this paper, the application of the dynamic programming approach to constrained stochastic control problems with expected value constraints is demonstrated. Specifically, two such problems are analyzed using this approach. The problems analyzed are the problem of minimizing a discounted cost infinite horizon expectation objective subject to an identically structured constraint, and the problem of minimizing a discounted cost infinite horizon minimax objective subject to a discounted expectation constraint. Using the dynamic programming approach, optimality equations, which are the chief contribution of this paper, are obtained for these problems. In particular, the dynamic programming operators for problems with expectation constraints differ significantly from those of standard dynamic programming and problems with worst-case constraints. For the discounted cost infinite horizon cases, existence and uniqueness of solutions to the dynamic programming equations are explicitly shown by using the Banach fixed point theorem to show that the corresponding dynamic programming operators are contractions. The theory developed is illustrated by numerically solving the constrained stochastic control dynamic programming equations derived for simple example problems. The example problems are based on a two-state Markov model that represents an error prone system that is to be maintained.

For Meyer type optimal control problems the principle of optimality states that every subarc of an optimal solution that extends to the final time furnishes again a solution to the same optimal control problem, only with different initial conditions. Through a simple counter example it is shown in this paper that this principle is not satisfied, in general, for solutions obtained from a standard collocation approach. Introduction The principle of optimality [1] is one of the most intuitive and yet most fundamental theorems in the theory of optimal control. Its basic statement is that for Meyer type optimal control problems every subarc of an optimal solution that extends to the final time furnishes again a solution to the same optimal control problem, only with different initial conditions. This theorem has significant implications. For example, it is the starting point for the derivation of the Hamilton-Jacobi-Bellman equations [2] and it is the underlying principle in Bellman's dynamic programming approach [1], [3]. The present paper shows that the principle of optimality is not satisfied, in general, for solutions obtained from a standard collocation approach[4]. For a simple example whose collocation solution can be generated analytically it is shown that further improvements are possible along an optimal collocation solution if the design parameters along a number of leading nodes are frozen and the remainder of the trajectory is re-optimized. I. Continuous Optimal Control Problem In this section of the paper we introduce a simple class of optimal control problems without state and control constraints. In Section 1.2 we present the associated necessary conditions for optimality [2], [5]. For ease of notation we assume that the right-hand side of the 'Supervising Engineer, Member AIAA, t Senior Engineer, Member AIAA, *17 Research Drive, Hampton, VA 23666. ^Graduate Student, Member AIAA, Department, 304 Town Engineering, Ames, IA 50011 equations of motion does not depend explicitly on time t. This does not represent a loss of generality, as it is always possible to transform explicit time dependence away through the introduction of an additional, timelike state variable. 1. Problem Formulation Let us consider the following class of optimal control problems: min «e(Pwc[*o,*/]), (1) subject to the conditions ±(t) = /(*(*),«(*)), tf.(z(*oMo) = 0, (2)

Optimal Encoders Maximizing Directed Information of Channels with Memory and Feedback: Stochastic Control and Dynamic Programming

Summary 1. Establishing biological control agents in the field is a major step in any classical biocontrol programme, yet there are few general guidelines to help the practitioner decide what factors might enhance the establishment of such agents. 2. A stochastic dynamic programming (SDP) approach, linked to a metapopulation model, was used to find optimal release strategies (number and size of releases), given constraints on time and the number of biocontrol agents available. By modelling within a decision‐making framework we derived rules of thumb that will enable biocontrol workers to choose between management options, depending on the current state of the system. 3. When there are few well‐established sites, making a few large releases is the optimal strategy. For other states of the system, the optimal strategy ranges from a few large releases, through a mixed strategy (a variety of release sizes), to many small releases, as the probability of establishment of smaller inocula increases. 4. Given that the probability of establishment is rarely a known entity, we also strongly recommend a mixed strategy in the early stages of a release programme, to accelerate learning and improve the chances of finding the optimal approach.

A comprehensive Genetic Algorithm (GA) model has been developed and applied to derive optimal operational strategies of a multi-purpose reservoir, namely Perunchani Reservoir, in Kodaiyar Basin in Tamil Nadu, India. Most of the water resources problem involves uncertainty, in order to see that the GA model takes care of uncertainty in the input variable, the result of the GA model is compared with the performance of a detailed Stochastic Dynamic Programming (SDP) model. The SDP models are well established and proved that it takes care of uncertainty in-terms of either implicit or explicit approach. In the present study, the objective function of the models is set to minimize the annual sum of squared deviation from desired target release and desired storage volume. In the SDP model the optimal policies are derived by varying the state variables from 3 to 9 representative class intervals, and then the cases are evaluated for their performance using a simulation model for longer length of inflow data, generated using a Thomas–Fiering model. From the performance of the SDP model policies, it is found that the system encountered irrigation deficit, whereas GA model satisfied the demand to a greater extent. The sensitivity analysis of the GA model in selecting optimal population, optimal crossover probability and the optimal number of generations showed the values of 150, 0.76 and 175 respectively. On comparing the performance of SDP model policy with GA model, it is found that GA model has resulted in a lesser irrigation deficit. Thus based on the present case study, it may be concluded that the GA model performs better than the SDP model.

In this study, a dual interval robust stochastic dynamic programming (DIRSDP) method is developed for planning water resources management systems under uncertainty. As an extension of the existing interval stochastic dynamic programming (ISDP) method, DIRSDP can deal with two-stage stochastic programming (TSP)-based planning problems associated with dynamic features, input uncertainties, and multistage concerns. Compared with other optimization methods dealing with uncertainties, the developed DIRSDP method has advantages in addressing uncertainties with complex presentations and reflecting decision makers’ risk-aversion attitudes within its optimization process. Parameters in the DIRSDP model can be represented as probability distributions as well as single and/or dual intervals. Decision makers’ risk-aversion attitudes can be reflected through restricting the deviation of the recourse costs to a tolerance level. Water-allocation plans can then be developed based on the analysis of tradeoffs between the system benefit and solution robustness. The developed method is applied to a case of water resources management planning. The solutions are reasonable, indicating applicability of the developed methodology.

Reservoir management is a highly complex problem, which includes the uncertainty in the inflows as well as in the objectives. Stochastic dynamic programming (SDP) has been widely used to develop the optimal operating policies of multipurpose reservoirs. In the recent years, fuzzy optimization has been used to incorporate the uncertainty caused due to the imprecise nature of objectives. In the present study, a comparison is made between a conventional SDP model and fuzzy dynamic programming models for optimal operation of a multipurpose reservoir. The two models are developed for Hirakud Reservoir on River Mahanadi in the State of Orissa in India. The performance of the reservoir is evaluated under the operating policies derived from these models through simulation. The results indicate that the fuzzy dynamic programming model performs better in achieving the flood control objective and thus in the overall performan1ce of reservoir.

Probabilistic approaches to scheduling reserve selection

We analyse an optimal portfolio and consumption problem with stochastic factor and delay over a finite time horizon. The financial market includes a risk-free asset, a risky asset and a stochastic factor. The price process of the risky asset is modelled as a stochastic differential delay equation whose coefficients vary according to the stochastic factor; the drift also depends on its historical performance. Employing the stochastic dynamic programming approach, we establish the associated Hamilton–Jacobi–Bellman equation. Then we solve the optimal investment and consumption strategies for the power utility function. We also consider a special case in which the price process of the stochastic factor degenerates into a Cox–Ingersoll–Ross model. Finally, the effects of the delay variable on the optimal strategies are discussed and some numerical examples are presented to illustrate the results. doi:10.1017/S1446181119000014

Although there have been numerous studies on the economics of weed control, relatively few of these studies have focused on natural ecosystems. The aims of this papers are: A) to identify the combination of control options that is optimal for blackberry (Rubus anglocandicans) in Australian natural ecosystems, B) to assess whether an integrated control strategy is superior to chemical-only strategies, C) to evaluate the economic net benefits of specific biologically oriented control methods (a rust and grazing by goats), D) to determine how changes in model parameters affect the optimal control strategy. To address these aims, a stochastic dynamic simulation model and a stochastic dynamic programming model are developed. The results indicate that, while an integrated strategy combining chemical and non-chemical control methods may be optimal in certain circumstances, it is not necessarily superior to a chemical-dominant strategy in all cases. Results show that in most cases a combination of herbicides and one physical control method (mowing) is optimal in the base-case scenario. However, if chemical efficacy is greater than in the base-case, the optimal strategy involves chemicals only. The results indicate that grazing goats for control of blackberry can be optimal despite its stochastic effectiveness and relatively low efficacy.

A new hybrid metaheuristic method for optimizing the objective function on a parallelepiped set of admissible solutions is proposed. It mimics the behavior of a school of river perch when looking for food. The algorithm uses the ideas of several methods: a frog-leaping method, migration algorithms, a cuckoo algorithm and a path-relinking procedure. As an application, a wide class of problems of finding the optimal control of deterministic discrete dynamical systems with a nonseparable performance criterion is chosen. For this class of optimization problems, it is difficult to apply the discrete maximum principle and its generalizations as a necessary optimality condition and the Bellman equation as a sufficient optimality condition. The desire to extend the class of problems to be solved to control problems of trajectory bundles and stochastic problems leads to the need to use not only classical adaptive random search procedures, but also new approaches combining the ideas of migration algorithms and swarm intelligence methods. The efficiency of this method is demonstrated and an analysis is performed by solving several optimal deterministic discrete control problems: two nonseparable problems (Luus–Tassone and LiHaimes) and five classic linear systems control problems with known exact solutions.

The increased water demand and the increased capital construction costs of water resource development projects force engineers and water planners to utilize efficiently the existing and proposed water resources systems. Hence, it is considered necessary to find the optimum real time operating rules for systems of multiple reservoirs.Stochastic dynamic programming (SDP) has been used as the tool for deriving the real time operating rules for a system of multiple reservoirs, in preference to simulation analysis and linear programming. This is because the operation of the storages is considered as a sequential decision problem and dynamic programming is well suited to handle sequential, multi-period type decision problems. Further, the stochastic nature of streamflow is considered explicitly through a multivariate distribution for streamflow. The method theoretically produces the global optimum solution.The computational difficulties in terms of computer time and memory space have prohibited the realistic application of this method to systems of multiple reservoirs in the past. In this study, two methods have been introduced to reduce the computational difficulties of the method, thus making SDP more attractive to use in the derivation of the real time operating rules for a system of multiple reservoirs.The first method, the assumption of cross correlation of streamflow, reduces the number of states in the SDP method which reduces computer time and memory space. Further, it reduces the number of conditional probability estimates of streamflow which are used in recursive relation of the SDP algorithm. The second method, the “corridor” approach, reduces computer time involved in the SDP method by eliminating the computation of infeasible and/or inferior (suboptimal) solutions. These two methods have resulted in significant savings in the computer effort of the SDP method.The conditional probabilities of streamflow are computed by considering the hypervolume under the conditional probability density function between the integration limits using the computationally efficient Gaussian Legendre quadrature method. The conditional probability density function is derived from the historical streamflow data.Real time operating rules are expressed as a function of the storage volume(s) at the beginning of the current time interval and the forecast inflow into the storage(s) during the current time interval.The Melbourne water supply system is used as the example. The derived operating rules have been tested by operating a simulation model and the conclusion is made that the SDP method is a satisfactory technique for objectively deriving real time operating rules for a system of multiple reservoirs.Finally, a combined stochastic dynamic programming – statistical disaggregation approach is introduced to determine the real time operating rules for a system of multiple reservoirs where operational data are available. Releases from individual storages are a function of the total system storage volume and inflow. The method uses SDP to determine the release rules from the equivalent single reservoir of the multiple reservoir system; these releases are then disaggregated to individual releases using the statistical release disaggregation model whose parameters are derived from the historical operational data. The approach was used to derive real time operating rules for the Melbourne water supply system and the operating rules were validated using a simulation model of the system.