Some new techniques in the dynamic programming solution of variational problems

General Imbedding Theory

A technique called dynamic programming can be used to identify an optimal rebalancing schedule, which significantly reduces rebalancing and sub-optimality costs. Dynamic programming provides solutions to multi-stage decision processes in which the decisions made in prior periods affect the choices available in later periods. Dynamic programming provides the optimal year-by-year decision policy by working backwards from year 10. The results of the test of the relative efficacy of dynamic programming and the MvD heuristic with data on domestic equities, domestic fixed income, non-US equities, non-US fixed income, and emerging market equities, show that the MvD heuristic performs quite well compared to the dynamic programming solution for the two-asset case and substantially better than other heuristics. The increase in the number of assets reduces the advantage of dynamic programming over the MvD heuristic and is reversed at the level of five assets. Dynamic programming cannot be applied beyond five assets, but the MvD heuristic can be extended up to 100 assets. The MvD heuristic reduces total costs relative to all of the other heuristics by substantial amounts. The performance of the MvD heuristic improves relative to the dynamic programming solution as more assets are added but this improvement reflects a growing reliance on an approximation for the dynamic programming approach.

This article mainly focuses on exploring the current application status, concrete things implemented by dynamic programming and advantages of dynamic programming in problems (the concept and operation of dynamic programming and the role it plays in programs) in different fields, future development trends and areas for improvement. It also introduces the gaps and problems currently encountered by dynamic programming, then provides corresponding solutions to make it more helpful and efficient in the future. In this regard, by consulting different references and online resources written by previous scientists, it finds out that they all have different points of view on dynamic programming and different solutions to its shortcomings. In the current research, dynamic programming is the most widely used algorithm with few limitations that can achieve the global optimal solution at any time; however, it does not mean that it can achieve the local optimal solution at any time, which also indirectly results in the memory space it requires being larger than other optimization algorithms. After analyzing the literature and researching and testing multiple applications, it is obvious that dynamic programming can make the code run easier and faster than traditional methods, even though it will take up more space.

Dynamic Programming (DP) technique is an effective algorithm to find the global optimum. However when applying DP for finite state problems, if the state variables are discretized, it increases the cumulative errors and leads to suboptimal results. In this paper we develop and present a new DP algorithm that overcomes the above problem by eliminating the need to discretize the state space by the use of sets. We show that the proposed DP leads to a globally optimal solution for a discrete time system by minimizing a cost function at each time step. To show the efficacy of the proposed DP, we apply it to optimize the fuel economy of the series and parallel Hybrid Electric Vehicle (HEV) architectures and the case study of Chevrolet Volt 2012 and the Honda Civic 2012 for the series and parallel HEV’s respectively are considered. Simulations are performed over predefined drive cycles and the results of the proposed DP are compared to previous DP algorithm (DPdis). The proposed DP showed an average improvement of 2.45% and 21.29% over the DPdis algorithm for the series and the parallel HEV case respectively over the drive cycles considered. We also propose a real time control strategy (RTCS) for online implementation based on the concept of Preview Control. The RTCS proposed is applied for the series and parallel HEV’s over the drive cycles and the results obtained are discussed.

<p>A potentially parallel iterative algorithm for the solution of the unconstrained N-stage decision problem of Dynamic Programming is developed. This new solution method, known as Variable Metric Dynamic Programming, is based on the use of variable metric minimisation techniques to develop quadratic approximations to the optimal cost function for each stage. The algorithm is applied to various test problems, and a comparison with an existing similar algorithm proves favourable. The Variable Metric Dynamic Programming solution method is used in the implementation of an adaptive highlevel scheduling mechanism on a multiprogrammed computer in a university environment. This demonstrates a practical application of the new algorithm. More importantly, the application of Variable Metric Dynamic Programming to a scheduling problem illustrates how Mathematical Programming may be used in complex computer scheduling problems to provide in a natural way the required dynamic feedback mechanisms.</p>

In the dynamic programming paradigm the value of an optimal solution is recursively defined in terms of optimal solutions to subproblems. Such dynamic programming definitions can be very tricky and error-prone to specify. This paper presents a novel, elegant method based on tabled logic programming that simplifies the specification of such dynamic programming solutions. Our method introduces a new mode declaration for tabled predicates. The arguments of each tabled predicate are divided into indexed and non-indexed ones so that tabled predicates can be regarded as functions: indexed arguments represent input values and non-indexed arguments represent output values. The non-indexed arguments in a tabled predicate can be further declared to be aggregate, e.g., the minimum, so that while generating answers, the global table will dynamically maintain the smallest value for that argument. This mode declaration scheme, coupled with recursion, provides a considerably easy-to-use method for dynamic programming: there is no need to define the value of an optimal solution recursively, instead, defining a general solution suffices. The optimal value as well as its corresponding concrete solution can be derived implicitly and automatically using tabled logic programming systems. Experimental results are shown to indicate that the mode declaration improves both time and space performances in solving dynamic programming problems on tabled LP systems. Additionally, our mode declaration scheme provides an alternative implementation vehicle for preference logic programming.

In the dynamic programming paradigm the value of an optimal solution is recursively defined in terms of optimal solutions to subproblems. Such dynamic programming definitions can be tricky and error‐prone to specify. This paper presents an elegant method based on tabled logic programming (TLP) that simplifies the specification of such dynamic programming solutions. Our method introduces a new mode declaration for tabled predicates. The arguments of each tabled predicate are divided into indexed and non‐indexed arguments so that tabled predicates can be regarded as functions: indexed arguments represent input values and non‐indexed arguments represent output values. The non‐indexed arguments in a tabled predicate can be further declared to be aggregated, for example, the minimum, so that while generating answers, the global table will dynamically maintain the smallest value for that argument. This mode‐declaration scheme, coupled with recursion, provides an easy‐to‐use method for dynamic programming: there is no need to define the value of an optimal solution recursively, as the definition of a general solution suffices. The optimal value as well as its corresponding concrete solution can be derived implicitly and automatically using tabled logic programming systems. Our experimental results show that mode declarations improve performance in solving dynamic programming problems on TLP systems. Copyright © 2007 John Wiley & Sons, Ltd.

Shift-and-merge technique for the DP solution of the time-constrained backpacker problem

This paper studies a class of renewable resource allocation problems for the processing of dynamically arriving tasks with deterministic deadlines. This class of problems has many applications, however, it conforms to neither the standard resource allocation model nor the standard optimal control model. A new problem formulation has to be developed and analyzed to provide a satisfactory solution. In this paper, a new formulation is presented. By augmenting the state variabls, the problem is converted into a Markovian decision problem. It can then be treated, at least in principle, by using the stochastic dynamic programming (SDP) method. However, since the system dynamics involves the evolution of sets (of tasks and resources), the implementation of the dynamic programming equation is by no means straightforward. For a problem with infinite planning horizon, the optimal strategy is shown to be stationary under mild conditions. An SDP algorithm based on a successive approximation technique is developed to obtain the optimal stationary strategy. The implementation of the algorithm employs a special coding scheme to handle set variables, and utilizes a dominance property for computational efficiency. Effects of key system parameters on optimal decisions are investigated and analyzed through numerical examples. As the computational complexity of the algorithm is of exponential increase, practical applications of the algorithm is limited to problems of moderate size. Two heuristic rules are therefore investigated and compared to the optimal policy.

In this paper we propose an algorithm for the constrained two-dimensional cutting stock problem (TDC) in which a single stock sheet has to be cut into a set of small pieces, while maximizing the value of the pieces cut. The TDC problem is NP-hard in the strong sense and finds many practical applications in the cutting and packing area. The proposed algorithm is a hybrid approach in which a depth-first search using hill-climbing strategies and dynamic programming techniques are combined. The algorithm starts with an initial (feasible) lower bound computed by solving a series of single bounded knapsack problems. In order to enhance the first-level lower bound, we introduce an incremental procedure which is used within a top-down branch-and-bound procedure. We also propose some hill-climbing strategies in order to produce a good trade-off between the computational time and the solution quality. Extensive computational testing on problem instances from the literature shows the effectiveness of the proposed approach. The obtained results are compared to the results published by Alvarez-Valdes et al. (2002).

In this chapter we present a class of discrete-time structural models of labour force dynamics which are estimable using the same methodology. There have been a number of such models discussed in the literature, but only a very few have been estimated in their structural form. Although the models are categorized as either fitting into labour force participation, job search, or job matching frameworks, they all share in common the reservation wage property, namely that there is a change in state only when the random wage draw moves above or below a threshold value. They key to estimation is the numerical solution of the dynamic programming problem for the reservation wage path. In the finite horizon case considered here reservation wages are time varying and all of the economic theory is contained in the restrictions on the shape of the reservation wage path; only a few parameters fully describe the entire path, although not generally in closed form. The likelihood function based on observed labour force states is also a function of the reservation wage path. Thus, the parameters which determine the reservation wage path can be estimated by iterating between the dynamic programming solution and the likelihood function.

A hybrid threshold curve model for optimal yield management: neural networks and dynamic programming

A dynamic programming (DP) technique has been developed for the detection of subpixel-sized, low-SNR targets observed by mosaic imaging sensors. The primary advantages of DP are its sensitivity to weak targets along with its robustness to target maneuvers and sensor instabilities. Such enhancements are achieved by performing target data association and detection in a single optimization procedure. To the extent that the target motion and noise behavior can be accurately modeled, the DP algorithm is shown to be optimal for this task. A prototype IR tracking system based on the DP approach was developed and tested for a step-staring IR camera application. Performance analysis indicates a sensitivity improvement of several dB over conventional sequential detection tracking approaches.© (1990) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

This paper introduces a new approach for addressing the unit commitment problem by including solar photovoltaic (PV) integration. The uncertainties associated with photovoltaic (PV) output are classified into three distinct categories based on their varying levels of penetration. The data utilized in this study pertains to the IEEE 39-bus system, which incorporates 10-unit generators. The analysis considers various system restrictions in the unit commitment process, including power balancing, system reserve requirements, generator generation limits, and minimum up and down time constraints. The Ant Lion Optimizer (ALO) is a proposed solution for the unit commitment problem, drawing inspiration from the hunting behavior of the ant lion. In this study, the performance of the suggested method is evaluated and compared with that of the Dynamic Programming (DP) technique. The evaluation is conducted in terms of generation scheduling, total operating cost (TOC), and calculation time. Based on the findings, it can be concluded that the ALO technique demonstrates superior performance in generation scheduling, resulting in reduced total operating costs (TOC) when compared to the DP technique. The integration of photovoltaic (PV) systems into the grid infrastructure results in an annual cost savings of approximately $6,612,705.

Problems and Techniques