DP-GP OPTIMIZATION: A GEOMETRIC PROGRAMMING APPROXIMATION TECHNIQUEFOR CURSE OF DIMENSIONALITY IN DYNAMIC PROGRAMMING AND ITS REAL-LIFE APPLICATION

We address the problem of determining optimal stepsizes for estimating parameters in the context of approximate dynamic programming. The sufficient conditions for convergence of the stepsize rules have been known for 50 years, but practical computational work tends to use formulas with parameters that have to be tuned for specific applications. The problem is that in most applications in dynamic programming, observations for estimating a value function typically come from a data series that can be initially highly transient. The degree of transience affects the choice of stepsize parameters that produce the fastest convergence. In addition, the degree of initial transience can vary widely among the value function parameters for the same dynamic program. This paper reviews the literature on deterministic and stochastic stepsize rules, and derives formulas for optimal stepsizes for minimizing estimation error. This formula assumes certain parameters are known, and an approximation is proposed for the case where the parameters are unknown. Experimental work shows that the approximation provides faster convergence than other popular formulas.

Interest in non-serial dynamic programming is deeply rooted in real life applications. Since the initial efforts in the early 1960's, however, a number of scattered treatments of this interesting field of dynamic programming has emerged. This paper traces the development of non-serial dynamic programming from the basic theory underlying dynamic programming to the latest applications of non-serial dynamic programming. The works of such major contributors as Nemhauser, Wilde, Beightler, Bertele and Brioschi are critically examined and recent relevant computational advances are discussed. The paper is concluded with a few examples of various applications of non-serial dynamic programming together with some crystal ball gazing into future applications. An extensive list of references is also included.

This paper examines long-term optimal operation using dynamic programming for a large hydropower system of 10 reservoirs in Northeast China. Besides considering flow and hydraulic head, the optimization explicitly includes time-varying electricity market prices to maximize benefit. Two techniques are used to reduce the 'curse of dimensionality' of dynamic programming with many reservoirs. Discrete differential dynamic programming (DDDP) reduces the search space and computer memory needed. Object-oriented programming (OOP) and the ability to dynamically allocate and release memory with the C++ language greatly reduces the cumulative effect of computer memory for solving multi-dimensional dynamic programming models. The case study shows that the model can reduce the 'curse of dimensionality' and achieve satisfactory results.

In this paper we have studied the Dynamic programming problem and major area of applications of this approach has been introduced. Dynamic programming provides a means for determining optimal long-term crop management plans. However, most applications and their analysis on annual time steps with fixed strategies within the year, effectively ignoring conditional responses during the year. We suggest an alternative approach that captures the strategic responses within a cropping season to random weather variables as they unfold, reflecting farmers’ ability to adapt to weather realizations. Multistage decision problems a problem of dynamic programming problem there is numerically challenging. So for the analytical results, dynamic programming is able to obtain the optimal agricultural product problem, and also decides how many it consumes and how many it saves in material and permanently store in each period economically. However, in this study, the problem is considered deterministic in which all input parameters are constant. The objective is to find a sequence of actions (a so-called policy) that minimizes the total cost over the decision making horizon the purpose of this paper has been to introduce application of dynamic programming techniques by way of example. The end result of the model formulation reveals the applicability of dynamic programming in resolving long time of the problem.

The structure of many sequential optimization problems over a finite or infinite horizon can be summarized in the mapping that defines the related dynamic programming algorithm. In this paper we take as a starting point this mapping and obtain results that are applicable to a broad class of problems. This approach has also been taken earlier by Denardo under contraction assumptions. The analysis here is carried out without contraction assumptions and thus the results obtained can be applied, for example, to the positive and negative dynamic programming models of Blackwell and Strauch. Most of the existing results for these models are generalized and several new results are obtained relating mostly to the convergence of the dynamic programming algorithm and the existence of optimal stationary policies.

Graphical models are useful for capturing interdependencies of statistical variables in various fields. Estimating parameters describing sparse graphical models of stationary multivariate data is a major task in areas as diverse as biostatistics, econometrics, social networks, and climate data analysis. Even though time series in these applications are often non stationary, revealing interdependencies through sparse graphs has not advanced as rapidly, because estimating such time varying models is challenged by the curse of dimensionality and the associated complexity which is prohibitive. The goal of this paper is to introduce novel algorithms for joint segmentation and estimation of sparse, piecewise stationary, graphical models. The crux of the proposed approach is application of dynamic programming in conjunction with cost functions regularized with terms promoting the right form of sparsity in the right application domain. As a result, complexity of the novel schemes scales gracefully with the problem dimension.

: Work accomplished over the first three years (1972-75) may be summarized as follows: (1) An extensive study of models for optimally scheduling lots of N products on a single processor under various demand process structures. (2) Solution of a problem of scheduling N jobs on parallel processors to minimize a penalty function based on job due dates when there are no precedence relations among jobs. (3) Two studies treating the efficient solution of minimum cost flow problems. (4) Research on methodologies for discrete optimization related to (a) Tree search in methods of implicit enumeration, (b) Circumventing the 'curse of dimensionality' in dynamic programming, and (c) The reduction method of integer programming for a specialized version of the generalized assignment problem. (5) Surveys of past and current research in (a) Scheduling multiple processors, (b) Flow networks, (c) Project planning networks, (d) Basic concepts used in branch-and-bound, and (e) Production planning.

We study lot-size policies in a serial, multistage manufacturing/inventory system with two key generalizations, namely (1) random yields at each production stage and (2) an autoregressive demand process. Previous research shows that the optimal policies in models with random yields (even in models with a single installation) lack the familiar order-up-to structure and are not myopic. Thus, dynamic programming algorithms are needed to compute optimal policies, and one encounters the “curse of dimensionality”; this is exacerbated here by the need to expand the size and dimension of the state space to accommodate the autoregressive demand feature. Nevertheless, although our model is more complex, we prove that there is an optimal policy with the order-up-to feature and, more importantly, that the optimal policy is myopic. This avoids the computational burden of dynamic programming. Our results depend on two assumptions concerning the stochastic yield, namely that the expected yield at a work station is proportional to the lot size, and the distribution of the deviation of the yield from its mean does not depend on the lot size. We introduce the concept of echelon-like variables, a generalization of Clark and Scarf's classical concept of echelon variables, to derive the structure of optimal policies. Furthermore, we show that the same kind of policy is optimal for several criteria: infinite-horizon discounted cost, infinite-horizon long-run average cost, and finite-horizon discounted cost (with the appropriate choice of the salvage value function).

Dynamic Programming (DP) is a useful approach to multi-stage decision problems. On the basis of Bellman’s optimality principle such a problem can be decomposed into subproblems through recursive formulae. Bellman (1957), 1957a and Dantzig (1957) introduced a DP method for integer-variable linear programs. This method yields pseudo-polynomial algorithms when the number of constraints is fixed (Papadimitriou (1981)). In the class of linear programs, problems with a staircase structure can be efficiently treated by DP methods which can be interpreted as iterative predictor-corrector processes using a pricing mechanism for subproblems at individual stages (e.g., Dantzig (1963), Ho and Manne (1974), Ho (1978), Ho and Loute (1980), Abrahamson (1981) . Various applications of DP have also been developed in production planning (Manne (1958) , Wagner and Whitin (1959), Clark and Scarf (1960), Veinott (1965), 1969), Bessler and Veinott (1966), Zangwill (1965), 1966) (1969), Konno (1973), (1988), Bitran and Yanasse (1982), Bitran et al. (1984),... In particular, polynomial algorithms have been obtained for concave cost lotsizing problems and their extensions (e.g., Wagner and Whitin (1959), Zangwill (1965), (1966), (1959), Dreyfus and Law (1977).

The industry and academe broadly use linear programming to analyze the logistics or supply chain solutions, which is based on the hypothesis of fixed environment and restricted procedures. However, in the dynamic market, the supply chain driven by the market faces multi units such as customers, suppliers, manufactures, distributors and carriers and multi uncertain factors such as physical distribution, manufacturing support and purchase. Through a comparative analysis of application of linear programming and dynamic programming in supply chain policy decision, the paper concludes that dynamic programming is more suitable for logistics and supply chain system policy decision study under changing market environments.

Generalized dynamic programming for multicriteria optimization

ABSTRACTSatellites offer many services through communication with stations, such as tracking, navigation, telecommand uplink, earth observation, etc. How to coordinate these services is referred to as the satellite range scheduling problem (SRSP). In the research, it is found that only the resources (referring to time slots of stations) requested by more than one satellite simultaneously influence scheduling results. These resources are called critical resources and selected as scheduling elements, which makes some jobs optimally served in advance and the problem be decomposable into a multi-stage decision process, so dynamic programming is suitable to be employed. For large-scale SRSPs, a route-reduction-based dynamic programming (RR-DP) is presented, wherein a multi-level route reduction strategy is adopted to alleviate ‘the curse of dimensionality’. Experimental results reveal that RR-DP can find optimal solutions for small-to-medium sized problems and outperforms state-of-the-art methods for large-scale problems.

Dynamic programming (DP) is an elegant way to solve problems related to optimization and optimal control of processes. DP, however, has one major drawback, namely the “curse of dimensionality”. To overcome this shortcoming, an approach called neuro-dynamic programming (NDP) has been developed. This approach solves the “curse of dimensionality” problem of DP. For this purpose, a neural network is used in NDP, which ignores the poor results of the utility criterion. In this way, the time for solving the specific task is significantly shortened. In this work, an NDP algorithm is presented for the optimal control of a fed-batch biotechnological process for the production of L-lysine by the strain Brevibacterium flavum 22LD. Application of the NDP algorithm ensures maximum productivity of the L-lysine.

This article reviews the history and theory of dynamic programming (DP), a recursive method of solving sequential decision problems under uncertainty. It discusses computational algorithms for the numerical solution of DP problems, and an important limitation in our ability to solve realistic large-scale dynamic programming problems, the ‘curse of dimensionality’. It also summarizes recent research in complexity theory that delineates situations where the curse can be broken (allowing us to solve DPs using fast polynomial time algorithms), and situations where it is insuperable. The literature on econometric estimation and testing of DP models is reviewed, as is another ‘scientific limit to knowledge’, namely, the identification problem.

The determination of optimal operations policies for multi-reservoir systems under stochastic inflows is a hard problem. Dynamic Programming (DP) is widely used to solve such problems when the number of reservoirs is small. When there are many reservoirs in the system, the Stochastic Dynamic Programming (SDP) approach is impractical due to the “curse of dimensionality”. Therefore, some approximations have to be used to reduce the dimensionality of the problem. The Aggregation / Decomposition technique is one of the approximate methods used to solve multi-reservoir stochastic optimization problems. Stochastic differential equations can be used to model the dynamics of a multi-reservoir system. Under the assumption of Gaussian inputs, Ito’s stochastic calculus and the automatic formulation of moment equations may be used to derive the equivalent deterministic differential equations for the original stochastic system. These deterministic differential equations describe the evolution of the mean and the variance vector of the state variables of the system, namely, the reservoir storage volumes. Using these equations, optimal operational policies can be derived with the additional use of nonlinear optimization tools. In this paper, the advantages and disadvantages of these two stochastic methods for multi-reservoir problems will be presented.