Dynamic Programming Algorithms in Global Optimization

Overview of improved dynamic programming algorithm for optimizing energy distribution of hybrid electric vehicles

The applicability of the dynamic programming method to two-dimensional slope stability analyses is studied. The critical slip surface is defined as the slip surface that yields the minimum value of an optimal function. The only assumption regarding the shape of the critical slip surface is that the surface is an assemblage of linear segments. Stresses acting along the critical slip surface are computed using a finite element stress analysis. Assumptions associated with limit equilibrium methods of slices related to the shape of the critical slip surface and the relationship between interslice forces are no longer required. A computer program named DYNPROG was developed based on the proposed analytical procedure, and numerous example problems have been analyzed. Results obtained when using DYNPROG were compared with those obtained when using several well-known limit equilibrium methods. The comparisons demonstrate that the dynamic programming method provides a superior solution when compared with conventional limit equilibrium methods. Analyses conducted also show that factors of safety computed when using the dynamic programming method are generally slightly lower than those computed using conventional limit equilibrium methods of slices; however, as Poisson's ratio approaches 0.5, the computed factors of safety from the dynamic programming method and the limit equilibrium method appear to become similar.Key words: dynamic programming, slope stability, stress analysis, optimization theory, limit equilibrium methods of slices.

Recently, as the market share of electric vehicles (EVs) has increased, how to handle the increased electricity demand for EV charging in the power grid and how to use EV batteries from a grid-operating aspect have become more important. Also, from the perspective of individual EVs, Vehicle-to-Grid (V2G) technologies that reduce the cost for each vehicle’s charging in conjunction with the power grid are significant. In this paper, the V2G control problem at the individual vehicle level is studied using a Dynamic Programming (DP) algorithm that considers EVs’ charging efficiency. The DP algorithm is developed to generate an optimized charging/discharging power profile that minimizes electricity costs, while satisfying the constraints of the initial and final battery states of charge, for given a time-of-use electricity price. To show the effectiveness of the proposed algorithm, simulation is conducted for three different charging scenarios (unidirectional charging, bidirectional charging, and unidirectional charging with cost variations based on electricity usage), and the results showed that DP can achieve significant cost savings of about 30% compared to the normal charging method. Also, the result of DP is compared with that of Linear Programming, demonstrating that DP outperforms Linear Programming in cost savings for the V2G control problem.

We address the problem of determining optimal stepsizes for estimating parameters in the context of approximate dynamic programming. The sufficient conditions for convergence of the stepsize rules have been known for 50 years, but practical computational work tends to use formulas with parameters that have to be tuned for specific applications. The problem is that in most applications in dynamic programming, observations for estimating a value function typically come from a data series that can be initially highly transient. The degree of transience affects the choice of stepsize parameters that produce the fastest convergence. In addition, the degree of initial transience can vary widely among the value function parameters for the same dynamic program. This paper reviews the literature on deterministic and stochastic stepsize rules, and derives formulas for optimal stepsizes for minimizing estimation error. This formula assumes certain parameters are known, and an approximation is proposed for the case where the parameters are unknown. Experimental work shows that the approximation provides faster convergence than other popular formulas.

Optimal load clipping with time of use rates

In this paper, a dynamic programming (DP) algorithm is proposed to compute the maximum a posteriori probability (MAP) estimate of the random location of multiple sources by passive sensor array. Based on the principle of optimality, the multivariate MAP decision problem can be transformed into a multistage one‐dimensional maximization problem. By a recursive computation and backward optimum searching technique, the MAP solution can be obtained. The computational complexity is much less than that of the direct MAP searching method, especially when the number of sensors and/or the number of sources is large. In addition, the DP computation technique is equally applicable both in the case of correlated sources and the case of uncorrelated sources.

Waste problems often occur in big cities such as Jakarta, Surabaya, Medan, Bandung, Yogyakarta and Semarang. Urban waste is waste produced by human activities in urban areas, including various types of materials such as plastic, paper, metal and organic materials. Therefore, waste needs to be collected, transported and disposed of immediately to prevent negative impacts on the environment. With the increase in population in urban areas, it is likely that the amount of waste will also increase. The solution that can be used to solve this problem is to optimize the distance using a dynamic programming model forward recursive equation algorithm. The application of dynamic programming can be used to solve various problems such as determining the shortest distance, optimal flow, task or resource scheduling, production planning, network optimization, and problems in research and development projects. Based on determining the optimal route for transporting waste in Medan Johor District using the dynamic programming algorithm forward recursive equation model, the total distance is 26,2km.

The structure of many sequential optimization problems over a finite or infinite horizon can be summarized in the mapping that defines the related dynamic programming algorithm. In this paper we take as a starting point this mapping and obtain results that are applicable to a broad class of problems. This approach has also been taken earlier by Denardo under contraction assumptions. The analysis here is carried out without contraction assumptions and thus the results obtained can be applied, for example, to the positive and negative dynamic programming models of Blackwell and Strauch. Most of the existing results for these models are generalized and several new results are obtained relating mostly to the convergence of the dynamic programming algorithm and the existence of optimal stationary policies.

In this chapter, following the theoretical fundamentals, the applications of dynamic programming (DP) on several HEV types including series, parallel and series–parallel are specifically given. ADVISOR-based simulation results prove that the DP algorithm is capable of finding optimal power which is from the battery to the electric motor and then the power ratio between them. Quadratic programming (QP) is also contained in discussion of this chapter due to its advantages of saving computational time in real-time realization. The power management strategy on basis of QP algorithm for fuel optimization of PHEV is depicted in details.

The article addresses the treatment of applying the method of dynamic linear programming to solve the task of choosing the optimal strategy for the containers dispatch, taking into account the significant unevenness of loading and shipment of containers. The optimizing container transportation dynamic planning could be treated as mathematical model of the dynamic multi-period task of the loaded containers shipment, which allows the choice of the optimal strategy for sending containers, taking into account the significant unevenness of their loading and dispatch from the railway freight station. The efficiency of considered method is proved by numerical calculation being presented to disclose the dynamic linear programming algorithm implementing to solve the problem.

The industry and academe broadly use linear programming to analyze the logistics or supply chain solutions, which is based on the hypothesis of fixed environment and restricted procedures. However, in the dynamic market, the supply chain driven by the market faces multi units such as customers, suppliers, manufactures, distributors and carriers and multi uncertain factors such as physical distribution, manufacturing support and purchase. Through a comparative analysis of application of linear programming and dynamic programming in supply chain policy decision, the paper concludes that dynamic programming is more suitable for logistics and supply chain system policy decision study under changing market environments.

The production planning in agriculture is one of the most important decision problems of the farmer. Although some decision support tools based mainly on linear programming and addressed to agriculture authorities were presented, their direct application by a farmer is not possible. This is mainly due to the local character of the models developed for particular agricultural conditions and also due to the complexness of underlying mathematical programming models.This paper aims to develop dynamic programming model for the long run crop plan optimization covering the typical conditions of Czech farms, which could serve as a platform for further enlargements and changes according to needs and conditions of particular farm. The dynamic programming algorithm is developed in detail for model case of four areas to be planted by four crops each year. The possibility of covering different constraints by generating the state space is discussed, and the generating procedure for crop rotation rules is shown. The goal function reflects the farmers objective of profit maximization and it is defined with respect to harvests’ randomness. The case study is solved for the data from South Moravian agriculture cooperative and the optimal solution is presented and discussed.

Sponge Supercapacitor rule-based energy management strategy for wireless sensor nodes optimized by using dynamic programing algorithm

ABSTRACT: In this paper, the maximum principle of optimal control theory and the method of dynamic programming are discussed in relation to the minimization of fuel consumption in ship routing. The connection between the two methods is indicated for the case in which ship routing is treated as a continuous process, meaning that the sailing paths are not restricted to arcs of a grid as in the discrete dynamic programming method, but can vary continuously in the navigation area. Practical aspects are also discussed, such as the discrete approach of dynamic programming, as well as the finite version of the continuous approach and the limited predictability of the weather. Results are presented showing least-time routes, which are obtained with computational methods based on the maximum principle and the corresponding continuous type of dynamic programming.

This paper presents a novel method to find the profit-maximizing Nash equilibriums in allocating generation amounts with consideration of ramp-rates under competitive market environment. In order to find the Nash equilibriums it is necessary to search all the feasible combinations of generators' outputs which satisfy various constraints. The procedure to eliminate the dominated strategies can be formulated using Bellman's optimality principle of dynamic programming problem and hence the backward or forward search algorithm of dynamic programming can be easily applied. Therefore, the Nash equilibriums are found using dynamic programming method and we found that there exist several Nash equilibriums in the generation allocation problem. Individual generators participate in a game to maximize its profit through competitions and play aa game with bidding strategies of its generation quantities in a spot market. The ramp-rate physically or technically limits generators to increase or decrease outputs in its range and restricts the number of bidding strategies of each generator. We suggest the dynamic programming to find the Nash equilibriums while removing the dominated strategies in each stage (or each time). In the case studies, we analyzed the generation allocation game for a 12-hour multi-stage and compared it with the results of dynamic economic dispatch. Both of the two cases were considered generator's ramp-rate effects.