Generalized dynamic programming for multicriteria optimization

We address the problem of determining optimal stepsizes for estimating parameters in the context of approximate dynamic programming. The sufficient conditions for convergence of the stepsize rules have been known for 50 years, but practical computational work tends to use formulas with parameters that have to be tuned for specific applications. The problem is that in most applications in dynamic programming, observations for estimating a value function typically come from a data series that can be initially highly transient. The degree of transience affects the choice of stepsize parameters that produce the fastest convergence. In addition, the degree of initial transience can vary widely among the value function parameters for the same dynamic program. This paper reviews the literature on deterministic and stochastic stepsize rules, and derives formulas for optimal stepsizes for minimizing estimation error. This formula assumes certain parameters are known, and an approximation is proposed for the case where the parameters are unknown. Experimental work shows that the approximation provides faster convergence than other popular formulas.

In this paper we have studied the Dynamic programming problem and major area of applications of this approach has been introduced. Dynamic programming provides a means for determining optimal long-term crop management plans. However, most applications and their analysis on annual time steps with fixed strategies within the year, effectively ignoring conditional responses during the year. We suggest an alternative approach that captures the strategic responses within a cropping season to random weather variables as they unfold, reflecting farmers’ ability to adapt to weather realizations. Multistage decision problems a problem of dynamic programming problem there is numerically challenging. So for the analytical results, dynamic programming is able to obtain the optimal agricultural product problem, and also decides how many it consumes and how many it saves in material and permanently store in each period economically. However, in this study, the problem is considered deterministic in which all input parameters are constant. The objective is to find a sequence of actions (a so-called policy) that minimizes the total cost over the decision making horizon the purpose of this paper has been to introduce application of dynamic programming techniques by way of example. The end result of the model formulation reveals the applicability of dynamic programming in resolving long time of the problem.

<p>A potentially parallel iterative algorithm for the solution of the unconstrained N-stage decision problem of Dynamic Programming is developed. This new solution method, known as Variable Metric Dynamic Programming, is based on the use of variable metric minimisation techniques to develop quadratic approximations to the optimal cost function for each stage. The algorithm is applied to various test problems, and a comparison with an existing similar algorithm proves favourable. The Variable Metric Dynamic Programming solution method is used in the implementation of an adaptive highlevel scheduling mechanism on a multiprogrammed computer in a university environment. This demonstrates a practical application of the new algorithm. More importantly, the application of Variable Metric Dynamic Programming to a scheduling problem illustrates how Mathematical Programming may be used in complex computer scheduling problems to provide in a natural way the required dynamic feedback mechanisms.</p>

Dynamic Programming (DP) is a useful approach to multi-stage decision problems. On the basis of Bellman’s optimality principle such a problem can be decomposed into subproblems through recursive formulae. Bellman (1957), 1957a and Dantzig (1957) introduced a DP method for integer-variable linear programs. This method yields pseudo-polynomial algorithms when the number of constraints is fixed (Papadimitriou (1981)). In the class of linear programs, problems with a staircase structure can be efficiently treated by DP methods which can be interpreted as iterative predictor-corrector processes using a pricing mechanism for subproblems at individual stages (e.g., Dantzig (1963), Ho and Manne (1974), Ho (1978), Ho and Loute (1980), Abrahamson (1981) . Various applications of DP have also been developed in production planning (Manne (1958) , Wagner and Whitin (1959), Clark and Scarf (1960), Veinott (1965), 1969), Bessler and Veinott (1966), Zangwill (1965), 1966) (1969), Konno (1973), (1988), Bitran and Yanasse (1982), Bitran et al. (1984),... In particular, polynomial algorithms have been obtained for concave cost lotsizing problems and their extensions (e.g., Wagner and Whitin (1959), Zangwill (1965), (1966), (1959), Dreyfus and Law (1977).

The industry and academe broadly use linear programming to analyze the logistics or supply chain solutions, which is based on the hypothesis of fixed environment and restricted procedures. However, in the dynamic market, the supply chain driven by the market faces multi units such as customers, suppliers, manufactures, distributors and carriers and multi uncertain factors such as physical distribution, manufacturing support and purchase. Through a comparative analysis of application of linear programming and dynamic programming in supply chain policy decision, the paper concludes that dynamic programming is more suitable for logistics and supply chain system policy decision study under changing market environments.

The applicability of the dynamic programming method to two-dimensional slope stability analyses is studied. The critical slip surface is defined as the slip surface that yields the minimum value of an optimal function. The only assumption regarding the shape of the critical slip surface is that the surface is an assemblage of linear segments. Stresses acting along the critical slip surface are computed using a finite element stress analysis. Assumptions associated with limit equilibrium methods of slices related to the shape of the critical slip surface and the relationship between interslice forces are no longer required. A computer program named DYNPROG was developed based on the proposed analytical procedure, and numerous example problems have been analyzed. Results obtained when using DYNPROG were compared with those obtained when using several well-known limit equilibrium methods. The comparisons demonstrate that the dynamic programming method provides a superior solution when compared with conventional limit equilibrium methods. Analyses conducted also show that factors of safety computed when using the dynamic programming method are generally slightly lower than those computed using conventional limit equilibrium methods of slices; however, as Poisson's ratio approaches 0.5, the computed factors of safety from the dynamic programming method and the limit equilibrium method appear to become similar.Key words: dynamic programming, slope stability, stress analysis, optimization theory, limit equilibrium methods of slices.

The quality of the magnetic resonance spectroscopy (MRS) depends on the stability of magnetic resonance (MR) system performance and optimal hardware functioning, which ensure adequate levels of signal-to-noise ratios (SNR) as well as good spectral resolution and minimal artifacts in the spectral data. MRS quality control (QC) protocols and methodologies are based on phantom measurements that are repeated regularly. In this work, a signal partitioning algorithm based on a dynamic programming (DP) method for QC assessment of the spectral data is described. The proposed algorithm allows detection of the change points—the abrupt variations in the time series data. The proposed QC method was tested using the simulated and real phantom data. Simulated data were randomly generated time series distorted by white noise. The real data were taken from the phantom quality control studies of the MRS scanner collected for four and a half years and analyzed by LCModel software. Along with the proposed algorithm, performance of various literature methods was evaluated for the predefined number of change points based on the error values calculated by subtracting the mean values calculated for the periods between the change-points from the original data points. The time series were checked using external software, a set of external methods and the proposed tool, and the obtained results were comparable. The application of dynamic programming in the analysis of the phantom MRS data is a novel approach to QC. The obtained results confirm that the presented change-point-detection tool can be used either for independent analysis of MRS time series (or any other) or as a part of quality control.

Interest in non-serial dynamic programming is deeply rooted in real life applications. Since the initial efforts in the early 1960's, however, a number of scattered treatments of this interesting field of dynamic programming has emerged. This paper traces the development of non-serial dynamic programming from the basic theory underlying dynamic programming to the latest applications of non-serial dynamic programming. The works of such major contributors as Nemhauser, Wilde, Beightler, Bertele and Brioschi are critically examined and recent relevant computational advances are discussed. The paper is concluded with a few examples of various applications of non-serial dynamic programming together with some crystal ball gazing into future applications. An extensive list of references is also included.

Waste problems often occur in big cities such as Jakarta, Surabaya, Medan, Bandung, Yogyakarta and Semarang. Urban waste is waste produced by human activities in urban areas, including various types of materials such as plastic, paper, metal and organic materials. Therefore, waste needs to be collected, transported and disposed of immediately to prevent negative impacts on the environment. With the increase in population in urban areas, it is likely that the amount of waste will also increase. The solution that can be used to solve this problem is to optimize the distance using a dynamic programming model forward recursive equation algorithm. The application of dynamic programming can be used to solve various problems such as determining the shortest distance, optimal flow, task or resource scheduling, production planning, network optimization, and problems in research and development projects. Based on determining the optimal route for transporting waste in Medan Johor District using the dynamic programming algorithm forward recursive equation model, the total distance is 26,2km.

In this paper, we develop an optimization method for solving the problem of the curse of dimensionality in dynamic programming. The method has the advantage of providing a cost for decision-making and eliminates the curse of dimensionality, which restricts the application of dynamic programming to only small classes of problems and, therefore, restricts the real-life application of dynamic programming. We established some relationships between geometric programming and dynamic programming parameters. We applied the method to a problem on course allocation and obtained the cost for course allocation and the optimal decision policy to be (0.1, 0.2, 1, 3).

The structure of many sequential optimization problems over a finite or infinite horizon can be summarized in the mapping that defines the related dynamic programming algorithm. In this paper we take as a starting point this mapping and obtain results that are applicable to a broad class of problems. This approach has also been taken earlier by Denardo under contraction assumptions. The analysis here is carried out without contraction assumptions and thus the results obtained can be applied, for example, to the positive and negative dynamic programming models of Blackwell and Strauch. Most of the existing results for these models are generalized and several new results are obtained relating mostly to the convergence of the dynamic programming algorithm and the existence of optimal stationary policies.

In this chapter, following the theoretical fundamentals, the applications of dynamic programming (DP) on several HEV types including series, parallel and series–parallel are specifically given. ADVISOR-based simulation results prove that the DP algorithm is capable of finding optimal power which is from the battery to the electric motor and then the power ratio between them. Quadratic programming (QP) is also contained in discussion of this chapter due to its advantages of saving computational time in real-time realization. The power management strategy on basis of QP algorithm for fuel optimization of PHEV is depicted in details.

This paper treats a class of finite-stage dynamic programming (DP) problems whose state space is an interval in the one-dimensional Euclidean space R1. By introducing a notion of “reward space” we can define, in this class, an inverse problem (called the inverse DP) to a given DP (called the main DP), because of the monotonicity and continuity of the reward functions and state transformations. Roughly speaking the inverse DP to the main DP is defined by replacing “max”, “reward functions” and “state transformations” with “min”, “inverse (in some sense) to the state transformations” and “inverse (in some sense) to the reward functions”, respectively, by replacing “terminal reward function” with “inverse to itself”, and by alternating “reward spaces” with “state spaces”, but by letting “action spaces” remain as they are. According to the monotonicity of the reward functions and state transformations we clarify the relation between the main DP and the inverse DP separately for four cases. Our main result is stated as follows. The pair of optimal reward functions and optimal policy for the main DP characterizes, in an inverse sense, the pair of optimal reward functions and optimal policy for the inverse DP, and vice versa (Section 2). Further we show that our DP represents a mathematical programming problem which has the property “recursiveness with monotonicity” in [5] and conversely such a problem can also be represented by our DP (Section 3). The last section is devoted to the illustration of several examples.

The Option Value Model in the Retirement Literature: The Trade-Off between Computational Complexity and Predictive Validity

In this paper the Dynamic Programming technique and the HMMS rule are applied on a production control model of an assembly factory.