Stochastic Dynamic Programming for Wind Farm Power Maximization

A stochastic control problem is formulated and we get the explicit form of the optimal control for initial-data-parameterized linear quadratic stochastic optimal control problems with random jumps. The optimal control can be proved to be unique. A stochastic Riccati equation is rigorous derived from the stochastic Hamilton system, which provides an optimal feedback control. This completes the the interrelationship between the stochastic Riccati equation and stochastic Hamilton system as two different but equivalent tools for the stochastic linear quadratic problem.

A stochastic minimax optimal time-delay state feedback control strategy for uncertain quasi-integrable Hamiltonian systems is proposed. First, a stochastic optimal state feedback control problem of uncertain quasi-integrable Hamiltonian system with time delay in feedback control subjected to Gaussian white noise is formulated. Then, the time-delayed state feedback control forces are approximated by the control forces without time delay and the original problem is converted into a stochastic optimal state feedback control problem of an uncertain system without time delay. After that, by following a procedure based on the stochastic averaging method and stochastic differential game, the worst-case disturbances and the optimal controls are obtained from solving a Hamiltonian–Jacobi–Isaacs (HJI) equation. As an example, the stochastic minimax optimal state feedback control of a Duffing oscillator with parametric disturbances and time-delayed feedback control is worked out in detail to illustrate the procedure and effectiveness of the proposed control strategy.

This paper considers optimal feedback control for a general continuous time finite-dimensional deterministic system with finite horizon cost functional. A practically feasible algorithm to calculate the numerical solution of the optimal feedback control by dynamic programming approach is developed. The highlights of this algorithm are: a) It is based on a convergent constructive algorithm for optimal feedback control law which was proposed by the authors before through an approximation for the viscosity solution of the time-space discretization scheme developed by dynamic programming method; b) The computation complexity is significantly reduced since only values of viscosity solution on some local cones around the optimal trajectory are calculated. Two numerical experiments are presented to illustrate the effectiveness and fastness of the algorithm.

Integrating product and process design with optimal control: A case study of solvent recycling

A nonlinear stochastic control problem related to ow control is considered. It is assumed that the state of a link is described by a controlled hidden Markov process with a finite state set, while the loss ow is described by a counting process with intensity depending on a current transmission rate and an unobserved link state. The control is the transmission rate, and it has to be chosen as a nonanticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function that takes into account losses of the transmitted information. Originally, the problem belongs to the class of stochastic control problems with incomplete information; however, optimal filtering equations that provide estimation of the current link state based on observations of the loss process allow one to reduce the problem to a standard stochastic control problem with full observations. Then a necessary optimality condition is derived in the form of a stochastic maximum principle, which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. Optimal and suboptimal controls are investigated and compared with the ow control schemes used in TCP/IP (Transmission Control Protocols/Internet Protocols) networks. In particular, the optimal control demonstrates a much smoother behavior than the TCP/IP congestion control currently used.

This article examines an economic growth model that expresses the interaction between production, technology stock, and research and development (R&D) investments. The goal of this study is to maximize production. Considering the presence of Gaussian white noises, this model is reformulated as a stochastic optimal control problem, where the R&D investment rate is defined as the control input. We aim to explore the efficiency of Kalman filtering approaches for solving this problem. Here, the extended Kalman filter (EKF) and unscented Kalman filter (UKF) are applied for state estimation. The state equation linearization is made in the EKF, while the unscented transform is taken in the UKF for generating a set of sigma points. These approaches aim to estimate the state dynamics from different perspectives. With these state estimates, two different computational algorithms are proposed, the EKF for state-control (EKF4SC) and UKF for state-control (UKF4SC) algorithms. The optimal control policy is designed to minimize the cost function. For illustration, the model's parameters are considered in the simulation experiment. The simulation results showed that the UKF has higher accuracy in the state estimation with the smallest mean squares of error compared with the EKF. Moreover, the optimal control policy based on the state estimate generated from the UKF could optimize the cost function of the problem. Hence, the results show the efficiency of the algorithms proposed. In conclusion, the application of the Kalman filtering algorithms to the economic growth model is presented. The significance of this study is to provide a stochastic optimal control model for the economic growth problem and to suggest an efficient computational approach for solving this stochastic optimal control problem.

Wind energy is a rapidly expanding source of renewable energy, but is highly intermittent. The performance of a wind farm, composed of a collection of wind turbines, depends not only on the placement of wind turbines in a farm, but also control actions taken by individual turbines. The wind turbine placement (layout) design problem involves adjusting turbine locations within a given area to improve a performance objective (such as maximizing annualized energy production). This layout problem has been addressed previously considering the effect of constraints such land configuration, installed capacity, and wake model choice on the performance of wind farms. All the studies, however, ignore the effects of the control system, which can have significant impact on performance. A well designed wind farm — without an optimal controller — will not achieve the full system level optimal performance, and vice-versa. In this article, we propose a novel layout co-design approach that includes optimal control considerations to exploit this synergy between farm layout and control. Layout case studies involving 8 and 12 turbines are presented. An annual energy production improvement of up to 17% is observed when accounting for coupling between control and layout design, when compared to layout-only optimization.

This paper presents a now method of optimal digital PID feedback control for a linear multivariate system by using a, parameter optimization technique. The most difficult points for the design of optimal digital PID feedback control system by the modern control theory are : (1) the control structure of it is specified in advance, (2) unless all the state variables are fed back, the optimal control cannot be obtained by dynamic programming or the maximum principle. However, the optimal digital PID feedback gains are determined by using the parameter optimization technique in this paper. When there are some states which are not fed back in the control system, the optimal feedback gains depend on the initial states. First, the optimal solution is presented in the case where all the initial Btates are known. Next, it is extended to the case where the initial states are random variables and their statistical properties are known,

This paper aims at developing near optimal traffic signal control for multi-intersections in city. As a new optimization technique, adaptive dynamic programming (ADP) combines concepts of reinforcement learning and dynamic programming. ADP could learn continually from experience to achieve a near optimal control policy under varying conditions. However, without the cooperation among adjacent intersections, the near optimal control for each individual intersection can not guarantee a larger traffic area composing several intersections to be near optimal. This paper presents a new signal control method based on a model-free action-dependent ADP (ADHDP). This method can be used for cooperative control of multiple intersections. In every intersection, an ADHDP signal controller is adopted to adjust signal time according to an integrated unity parameter. The unity parameter is designed to consider not only the control performance in local intersection but also those in the neighbor intersections. Thus the designed controllers could achieve a set of near optimal control police for multi-intersections in a long run. Simulation results show that the trained controller achieves shorter average vehicular delay.

This paper describes a theoretical framework for the design of controllers to satisfy probabilistic safety specifications for partially observable discrete time stochastic hybrid systems. We formulate the problem as a partial information stochastic optimal control problem, in which the objective is to maximize the probability that the state trajectory remains within a given safe set in the hybrid state space, using observations of the history of inputs and outputs. It is shown that this optimal control problem, which has a multiplicative payoff structure, is equivalent to a terminal payoff problem when the state space is augmented with a binary random variable capturing the safety of past state evolution. This allows us to derive a sufficient statistic for the probabilistic safety problem as a set of Bayesian filtering equations updating a conditional distribution on the augmented state space, as well as an abstract dynamic programming algorithm for computing the maximal probability of safety and an optimal control policy.

This chapter presents an optimal tracking control methodology for an offshore steel jacket platform subject to external wave force. Based on a dynamic model of an offshore steel jacket platform with an AMD mechanism and a linear exogenous system model of the external wave force on the offshore platform, an optimal tracking control scheme with feedforward compensation is proposed to attenuate wave-induced vibration of the offshore platform. A feedforward and feedback optimal tracking controller (FFOTC) can be obtained by solving an algebraic Riccati equation and a Sylvester equation, respectively. It is demonstrated that the wave-induced vibration amplitudes of the offshore platform under the FFOTC are much smaller than the ones under the feedback optimal tracking controller (FOTC) and the feedforward and feedback optimal controller (FFOC). Furthermore, the required control force under the FFOTC is smaller than the ones under the FOTC and the FFOC.

We consider a finite-capacity single-server queuing system in which the problems of capacity design and service quality control are integrated. The amount of time spent on servicing a customer is used as a measure of the quality of service provided. The expected reward from servicing is assumed to be a nondecreasing function of the service duration, while a linear holding cost is assumed for customers waiting in the system. We maximize the long-run average return per unit time by optimally controlling the quality of service to be provided as a function of the workload facing the server for different waiting-room capacities. We show that the optimal control policy for a given capacity is monotone and also examine the effect of varying the system capacity on the optimal control policy. Furthermore, we analyze the design problem of selecting an optimal capacity by assuming that an optimal service control policy will be followed for each system capacity.

High order optimal feedback control of space trajectories with bounded control

In this paper, the stochastic optimal control problems, which frequently occur in economic and finance are investigated. First, using Bellman’s dynamic programming method the stochastic optimal control problems are converted to Hamilton-Jacobi-Bellman (HJB) equation. Then, obtained HJB equation is solved through the method of separation of variables by guessing a solution via its terminal condition. Also, the non-linear optimal feedback control law is constructed. Finally, the solution procedure is illustrated for solving some examples that two of them are financial models. In fact, to highlight the applications of stochastic optimal control problems in financial mathematics, some financial models are presented.

In this paper, we study the optimal control problem for Takagi–Sugeno (T-S) fuzzy systems with disturbances. Due to the presence of disturbances in the T-S fuzzy systems, a fuzzy optimal state feedback control approach is presented by employing a two-player zero-sum game. Since the analytical optimal control solutions and the worst-case disturbance policies of T-S fuzzy systems can be boiled down to solving the algebraic Riccati equations (AREs), which are difficult to be obtained directly, a model-free policy iteration (PI) learning algorithm is proposed to obtain their approximation solutions. It is proved that the developed fuzzy optimal state feedback controller can ensure the fuzzy systems to be asymptotically stable and satisfy the disturbance attenuation condition simultaneously. Also, the designed PI learning algorithm can converge to their optimal solutions. Finally, we apply the developed fuzzy optimal state feedback control method to the truck-trailer system and the simulation results demonstrated the effectiveness of the developed scheme.