Dynamic Programming as a Scheduling Tool in Multiprogrammed Computing Systems

We address the problem of determining optimal stepsizes for estimating parameters in the context of approximate dynamic programming. The sufficient conditions for convergence of the stepsize rules have been known for 50 years, but practical computational work tends to use formulas with parameters that have to be tuned for specific applications. The problem is that in most applications in dynamic programming, observations for estimating a value function typically come from a data series that can be initially highly transient. The degree of transience affects the choice of stepsize parameters that produce the fastest convergence. In addition, the degree of initial transience can vary widely among the value function parameters for the same dynamic program. This paper reviews the literature on deterministic and stochastic stepsize rules, and derives formulas for optimal stepsizes for minimizing estimation error. This formula assumes certain parameters are known, and an approximation is proposed for the case where the parameters are unknown. Experimental work shows that the approximation provides faster convergence than other popular formulas.

Background:
 A new optimization algorithm is presented. The method is biased with new non monotone line search an accelerated three term conjugate gradient method damped of Quasi Newton method compared to previews method design efficiency in provident of more than one factor for different optimization problem are more dramatic due to the ability of the technique to utile existing data.
 Materials and Methods:
 New monotone line search, new monotone line search, new modification of Damped Quasi-Newton method, Motivation and New Quasi-Newton Algorithm (MQ) and Global convergence.
 Results:
 In this work, we have n tendency to compare our new algorithm with same classical strategies like [7] by exploiting of unconstrained nonlinear optimization problem the functions obtained from Andrei [5, 6] Waziri and Sabiu (2015)[10] and La couzetul (2004)[3]. The numerical experiments demonstrate the performance of the proposed method. We selected seven relatively unconstrained problems with the size varies from 10 to 100. We consider the three sizes of each problem so that the total number of problem is 21 test problems. We stop the iteration when is satisfied All codes were written in Matlab R2017a and run on a pc with Intel COREi4 with a processor with 4GB of Ram and CPU 2.3GHZ we solved test problems using two different initial starting points.
 Conclusion:
 In this research article, a project on an accelerated three-term efficient algorithm for numerical optimization has presented the method as completely a derivative-free algorithm with less NOI and NOF and CPU time computed to the existing methods .using classical assumption the global convergence was also proved. Numerical results using the three terms efficient algorithm show that the algorithm is promising.

In this paper we have studied the Dynamic programming problem and major area of applications of this approach has been introduced. Dynamic programming provides a means for determining optimal long-term crop management plans. However, most applications and their analysis on annual time steps with fixed strategies within the year, effectively ignoring conditional responses during the year. We suggest an alternative approach that captures the strategic responses within a cropping season to random weather variables as they unfold, reflecting farmers’ ability to adapt to weather realizations. Multistage decision problems a problem of dynamic programming problem there is numerically challenging. So for the analytical results, dynamic programming is able to obtain the optimal agricultural product problem, and also decides how many it consumes and how many it saves in material and permanently store in each period economically. However, in this study, the problem is considered deterministic in which all input parameters are constant. The objective is to find a sequence of actions (a so-called policy) that minimizes the total cost over the decision making horizon the purpose of this paper has been to introduce application of dynamic programming techniques by way of example. The end result of the model formulation reveals the applicability of dynamic programming in resolving long time of the problem.

Dynamic Programming (DP) is a useful approach to multi-stage decision problems. On the basis of Bellman’s optimality principle such a problem can be decomposed into subproblems through recursive formulae. Bellman (1957), 1957a and Dantzig (1957) introduced a DP method for integer-variable linear programs. This method yields pseudo-polynomial algorithms when the number of constraints is fixed (Papadimitriou (1981)). In the class of linear programs, problems with a staircase structure can be efficiently treated by DP methods which can be interpreted as iterative predictor-corrector processes using a pricing mechanism for subproblems at individual stages (e.g., Dantzig (1963), Ho and Manne (1974), Ho (1978), Ho and Loute (1980), Abrahamson (1981) . Various applications of DP have also been developed in production planning (Manne (1958) , Wagner and Whitin (1959), Clark and Scarf (1960), Veinott (1965), 1969), Bessler and Veinott (1966), Zangwill (1965), 1966) (1969), Konno (1973), (1988), Bitran and Yanasse (1982), Bitran et al. (1984),... In particular, polynomial algorithms have been obtained for concave cost lotsizing problems and their extensions (e.g., Wagner and Whitin (1959), Zangwill (1965), (1966), (1959), Dreyfus and Law (1977).

The industry and academe broadly use linear programming to analyze the logistics or supply chain solutions, which is based on the hypothesis of fixed environment and restricted procedures. However, in the dynamic market, the supply chain driven by the market faces multi units such as customers, suppliers, manufactures, distributors and carriers and multi uncertain factors such as physical distribution, manufacturing support and purchase. Through a comparative analysis of application of linear programming and dynamic programming in supply chain policy decision, the paper concludes that dynamic programming is more suitable for logistics and supply chain system policy decision study under changing market environments.

Generalized dynamic programming for multicriteria optimization

Constrained scheduling problems are quite common in project management, manufacturing, distribution, transportation, logistics, supply chain management, software engineering, and computer networks etc. The need to use integer and binary decision variables representing the allocation of different resources to many activities and numerous specific, universal and additional constraints on these decision variables are typical components of the resource-constrained scheduling problems (RCSPs) modeling. It is often necessary to model additional resources and constraints. For these reasons, models are becoming computationally demanding. This is particularly noticeable when methods of operations research (mathematical programing (MP), network programing, and dynamic programming) are used. On the other hand, most RCPSs can be easily modeled as instances of the constraint satisfaction problems (CSPs) and solved using constraint programming (CP) methods. In contrast to the MP-based environment, the CP-based environment methods deal well with binary constraints but worse in optimization. Therefore, the hybrid approach based on integration mathematical programming and constraint logic programming to optimization resource-constrained scheduling problems has been proposed. To evaluate the applicability and computational efficiency of the proposed approach and its implementation programming framework, the illustrative examples of optimization resource-constrained scheduling problems are implemented separately for mathematical programming and hybrid method.

Constrained scheduling problems are common in manufacturing, project management, transportation, supply chain management, software engineering, computer networks etc. Multiple binary and integer decision variables representing the allocation of resources to activities and numerous specific constraints on these variables are typical components of the constraint scheduling problem modeling. With their increased computational complexity, the models are more demanding, particularly when methods of operations research (mathematical programming, network programming, dynamic programming) are used. By contrast, most resource-constrained scheduling problems can be easily modeled as instances of the constraint satisfaction problems (CSPs) and solved using constraint programming (CP) or others methods. In the CP-based environment the problem definition is separated from the methods and algorithms used to solve the problem. Therefore, a constraint-based approach to resource-constrained scheduling problems that combines an OR-based approach for problem solving and a CP-based approach for problem modeling is proposed. To evaluate the applicability and efficiency of this approach and its implementation framework, illustrative examples of resource-constrained scheduling problems are implemented separately for different environments.

A technique called dynamic programming can be used to identify an optimal rebalancing schedule, which significantly reduces rebalancing and sub-optimality costs. Dynamic programming provides solutions to multi-stage decision processes in which the decisions made in prior periods affect the choices available in later periods. Dynamic programming provides the optimal year-by-year decision policy by working backwards from year 10. The results of the test of the relative efficacy of dynamic programming and the MvD heuristic with data on domestic equities, domestic fixed income, non-US equities, non-US fixed income, and emerging market equities, show that the MvD heuristic performs quite well compared to the dynamic programming solution for the two-asset case and substantially better than other heuristics. The increase in the number of assets reduces the advantage of dynamic programming over the MvD heuristic and is reversed at the level of five assets. Dynamic programming cannot be applied beyond five assets, but the MvD heuristic can be extended up to 100 assets. The MvD heuristic reduces total costs relative to all of the other heuristics by substantial amounts. The performance of the MvD heuristic improves relative to the dynamic programming solution as more assets are added but this improvement reflects a growing reliance on an approximation for the dynamic programming approach.

Interest in non-serial dynamic programming is deeply rooted in real life applications. Since the initial efforts in the early 1960's, however, a number of scattered treatments of this interesting field of dynamic programming has emerged. This paper traces the development of non-serial dynamic programming from the basic theory underlying dynamic programming to the latest applications of non-serial dynamic programming. The works of such major contributors as Nemhauser, Wilde, Beightler, Bertele and Brioschi are critically examined and recent relevant computational advances are discussed. The paper is concluded with a few examples of various applications of non-serial dynamic programming together with some crystal ball gazing into future applications. An extensive list of references is also included.

Waste problems often occur in big cities such as Jakarta, Surabaya, Medan, Bandung, Yogyakarta and Semarang. Urban waste is waste produced by human activities in urban areas, including various types of materials such as plastic, paper, metal and organic materials. Therefore, waste needs to be collected, transported and disposed of immediately to prevent negative impacts on the environment. With the increase in population in urban areas, it is likely that the amount of waste will also increase. The solution that can be used to solve this problem is to optimize the distance using a dynamic programming model forward recursive equation algorithm. The application of dynamic programming can be used to solve various problems such as determining the shortest distance, optimal flow, task or resource scheduling, production planning, network optimization, and problems in research and development projects. Based on determining the optimal route for transporting waste in Medan Johor District using the dynamic programming algorithm forward recursive equation model, the total distance is 26,2km.

The applicability of the dynamic programming method to two-dimensional slope stability analyses is studied. The critical slip surface is defined as the slip surface that yields the minimum value of an optimal function. The only assumption regarding the shape of the critical slip surface is that the surface is an assemblage of linear segments. Stresses acting along the critical slip surface are computed using a finite element stress analysis. Assumptions associated with limit equilibrium methods of slices related to the shape of the critical slip surface and the relationship between interslice forces are no longer required. A computer program named DYNPROG was developed based on the proposed analytical procedure, and numerous example problems have been analyzed. Results obtained when using DYNPROG were compared with those obtained when using several well-known limit equilibrium methods. The comparisons demonstrate that the dynamic programming method provides a superior solution when compared with conventional limit equilibrium methods. Analyses conducted also show that factors of safety computed when using the dynamic programming method are generally slightly lower than those computed using conventional limit equilibrium methods of slices; however, as Poisson's ratio approaches 0.5, the computed factors of safety from the dynamic programming method and the limit equilibrium method appear to become similar.Key words: dynamic programming, slope stability, stress analysis, optimization theory, limit equilibrium methods of slices.

In this paper, we develop an optimization method for solving the problem of the curse of dimensionality in dynamic programming. The method has the advantage of providing a cost for decision-making and eliminates the curse of dimensionality, which restricts the application of dynamic programming to only small classes of problems and, therefore, restricts the real-life application of dynamic programming. We established some relationships between geometric programming and dynamic programming parameters. We applied the method to a problem on course allocation and obtained the cost for course allocation and the optimal decision policy to be (0.1, 0.2, 1, 3).

The structure of many sequential optimization problems over a finite or infinite horizon can be summarized in the mapping that defines the related dynamic programming algorithm. In this paper we take as a starting point this mapping and obtain results that are applicable to a broad class of problems. This approach has also been taken earlier by Denardo under contraction assumptions. The analysis here is carried out without contraction assumptions and thus the results obtained can be applied, for example, to the positive and negative dynamic programming models of Blackwell and Strauch. Most of the existing results for these models are generalized and several new results are obtained relating mostly to the convergence of the dynamic programming algorithm and the existence of optimal stationary policies.

A quasi-newton differential dynamic programming algorithm for discrete-time optimal control