Dynamic programming operators for the bi-objective Traveling Thief Problem

In this paper, the application of the dynamic programming approach to constrained stochastic control problems with expected value constraints is demonstrated. Specifically, two such problems are analyzed using this approach. The problems analyzed are the problem of minimizing a discounted cost infinite horizon expectation objective subject to an identically structured constraint, and the problem of minimizing a discounted cost infinite horizon minimax objective subject to a discounted expectation constraint. Using the dynamic programming approach, optimality equations, which are the chief contribution of this paper, are obtained for these problems. In particular, the dynamic programming operators for problems with expectation constraints differ significantly from those of standard dynamic programming and problems with worst-case constraints. For the discounted cost infinite horizon cases, existence and uniqueness of solutions to the dynamic programming equations are explicitly shown by using the Banach fixed point theorem to show that the corresponding dynamic programming operators are contractions. The theory developed is illustrated by numerically solving the constrained stochastic control dynamic programming equations derived for simple example problems. The example problems are based on a two-state Markov model that represents an error prone system that is to be maintained.

Understanding the interactions between different combinatorial optimisation problems in real-world applications is a challenging task. Recently, the traveling thief problem (TTP), as a combination of the classical traveling salesperson problem and the knapsack problem, has been introduced to study these interactions in a systematic way. We investigate the underlying non-linear packing while traveling (PWT) problem of the TTP where items have to be selected along a fixed route. We give an exact dynamic programming approach for this problem and a fully polynomial time approximation scheme (FPTAS) when maximising the benefit that can be gained over the baseline travel cost. Our experimental investigations show that our new approaches outperform current state-of-the-art approaches on a wide range of benchmark instances.

We study the approximate dynamic programming approach to revenue management in the context of attended home delivery. We draw on results from dynamic programming theory for Markov decision problems, convex optimisation and discrete convex analysis to show that the underlying dynamic programming operator has a unique fixed point. Moreover, we also show that - under certain assumptions - for all time steps in the dynamic program, the value function admits a continuous extension, which is a finite-valued, concave function of its state variables. This result opens the road for achieving scalable implementations of the proposed formulation, as it allows making informed choices of basis functions in an approximate dynamic programming context. We illustrate our findings using a simple numerical example and conclude with suggestions on how our results can be exploited in future work to obtain closer approximations of the value function.

This work describes a fast method for computing dense stereo correspondences that is capable of generating results close to the state-of-the-art. We propose running a separate disparity computation process in each image pixel. The idea is to root a tree graph on the pixel whose disparity needs to be reconstructed. The tree thereby forms an individual approximation of the standard four-connected grid for this specific pixel. An exact optimum of a predefined energy function on the applied tree structure is determined via dynamic programming (DP), and the root pixel is assigned to the disparity of optimal costs. We present two simple tree structures that allow for the efficient calculation of all trees’ optima with only four scanline-based DP passes. These simple trees are designed to capture all pixels of the reference frame and incorporate horizontal and vertical smoothness edges in order to weaken the scanline streaking problem inherent in DP-based approaches. We evaluate our results using the Middlebury test set. Our algorithm currently ranks at the eighth position of approximately 30 algorithms in the Middlebury database. More importantly, it is the currently best-performing method that does not use image segmentation and is significantly faster than most competing algorithms. Our method needs less than a second to determine the disparity map for typical stereo pairs.

This article explains how the dynamic programming application presented in the August, 1968 issue of The Journal of Industrial Engineering entitled “Determination of Standard Sizes to be Manufactured Using Dynamic Programming” and authored by D. M. Jackson and D. R. Zerbe, is nonoptimal. It also considers the more realistic problem in which the number of standard sizes is not preselected, and in which the manufacturing cost of a standard size is not a constant, but interrelated with the quantity to be manufactured.

A piecewise deterministic Markov process (PDP) is a continuous time Markov process consisting of continuous, deterministic trajectories interrupted by random jumps. The trajectories may be controlled with the object of minimizing the expected costs associated with the process. A method of representing this controlled PDP as a discrete time decision process is presented, allowing the value function for the problem to be expressed as the fixed point of a dynamic programming operator. Decisions take the form of trajectory segments. The expected costs may then be minimized through a dynamic programming algorithm, rather than through the solution of the Bellman--Hamilton--Jacobi equation, assuming the trajectory segments are numerically tractable. The technique is applied to the optimal capacity expansion problem, that is, the problem of planning the construction of new production facilities to meet rising demand.

This paper compares two well-known approaches for valuing a risky investment using real options theory: contingent claims (CC) with risk neutral valuation and dynamic programming (DP) using a constant risk adjusted discount rate. Both approaches have been used in valuing forest assets. A proof is presented which shows that, except under certain restrictive assumptions, DP using a constant discount rate and CC will not yield the same answers for investment value. A few special cases are considered for which CC and DP with a constant discount rate are consistent with each other. An optimal tree harvesting example is presented to illustrate that the values obtained using the two approaches can differ when we depart from these special cases to a more realistic scenario. We conclude that for real options problems the CC approach is preferred when data exists (such as futures prices) that allow the estimation of the market price of risk or convenience yield. Even when such data do not exist we argue that the CC approach is preferred as it has the advantage of allowing the individual specification of the prices of different sources of risk.

The increase in air traffic worldwide requires improvement of flight operational efficiency. This study aims to reveal the potential benefits, namely, savings on fuel consumption and flight time, which are expected for Japanese airspace, by statistically evaluating the operational efficiency defined by average differences of fuel consumption, flight time, and flight distance between the original and the optimized flight of domestic flights in Japan. The aircraft position and time data used in this study were obtained from Collaborative Actions for Renovation of Air Traffic Systems Open Data—the radar data released by the Japan Civil Aviation Bureau. Flight information, such as air data and fuel flow, is estimated by applying meteorological data and aircraft performance model to the position information of radar data. Each reconstructed trajectory is optimized in terms of flight fuel consumption and flight time with an assumed cost index (CI). Dynamic programming is used as the trajectory optimization method. The flight fuel consumption and flight time of the optimized flight are compared with the original values to evaluate the operational efficiency. Herein, approximately one-third of 1-day data, i.e., 1087 cases of four aircraft types, are analyzed with reasonable CI settings. Our research findings suggest that flight fuel consumption and flight distance can be saved by 312 kg and 19.7 km, respectively, on average for the object flights. Following a statistical comparison between the original and the optimized flights, it was observed that two types of features, namely, flying on a detoured path and flying with nonoptimal altitude and speed in the cruise phase, are major factors which deteriorate the total operational efficiency in terms of fuel consumption, flight time, and flight distance.

Many continuous process industries are required to operate multiproduct supply chains under significant uncertainty in market demands and prices, and mathematical programming formulations have been developed for the deterministic version of the problem. However, available extensions to the stochastic case cannot be readily used to obtain an optimal operating policy in practice because the uncertainty results in an explosion of the scenarios to be considered over an extended time horizon. In this article we represent the uncertainty through Markov chains and use an approach based on stochastic dynamic programming (DP), which can generate a dynamic operating policy that incorporates information about the uncertainty in the problem at each time step. DP is computationally infeasible because of the size of the state and action space for realistic problems. We propose a novel stochastic optimization algorithmic framework called “DP in a heuristically restricted state space.” The confined state space is generated by simulating various potential scenarios under centralized dynamic inventory and production policy generated by combining static local inventory policy heuristics. The resulting DP policy responds to the time‐varying demand for products by appropriately stitching together decisions made by the local static policies. This is a new formulation for effectively combining heuristic policies. © 2006 American Institute of Chemical Engineers AIChE J, 2006

Aligning DNA and protein sequences is a core technique in molecular biology. Often, it is desirable to include partial prior knowledge and conditions in an alignment. Going beyond prior work, we aim at the integration of such side constraints in free combination into alignment algorithms. The most common and successful technique for efficient alignment algorithms is dynamic programming (DP). However, a weakness of DP is that one cannot include additional constraints without specifically tailoring a new DP algorithm. Here, we discuss a declarative approach that is based on constraint techniques and show how it can be extended by formulating additional knowledge as constraints. We take special care to obtain the efficiency of DP for sequence alignment. This is achieved by careful modeling and applying proper solving strategies. Finally, we apply our method to the scanning for RNA motifs in large sequences. This case study demonstrates how the new approach can be used in real biological problems. A prototypic implementation of the method is available at http://www.bioinf.uni-freiburg.de/Software/CTE-Alignment.

This article reviews the history and theory of dynamic programming (DP), a recursive method of solving sequential decision problems under uncertainty. It discusses computational algorithms for the numerical solution of DP problems, and an important limitation in our ability to solve realistic large-scale dynamic programming problems, the ‘curse of dimensionality’. It also summarizes recent research in complexity theory that delineates situations where the curse can be broken (allowing us to solve DPs using fast polynomial time algorithms), and situations where it is insuperable. The literature on econometric estimation and testing of DP models is reviewed, as is another ‘scientific limit to knowledge’, namely, the identification problem.

To improve the operational efficiency of automated container terminals and the coordination between multiple operations, this article studies the integrated scheduling optimization problem of automated guided vehicles (AGVs) and yard cranes. The impact of charging constraints on AGV task allocation and scheduling is considered. With the goal of minimizing the maximum completion time of all tasks, a mixed integer programming model is proposed. A solution method based on a dynamic programming algorithm is designed, where a heuristic algorithm is used to assign tasks to the yard cranes, and the dynamic programming method is used to assign tasks to the AGVs based on the task assignment results of the yard cranes. Finally, the validity of the model and algorithm is tested by numerical experiments. Furthermore, the influence of the quantity of AGVs on the terminal operational efficiency and the impact of AGV charging strategies on AGV scheduling are analysed.

Retrieval-oriented storage relocation optimization of an automated storage and retrieval system

Coal is one of the main energy sources and the carriage of coals is inevitable in China. How to choose the modes between each cities is one of the key factors that decide the quality of the coal carriage. A model for that problem was formulated to be a mixed integer programming problem at first. Then, a Dynamic Programming algorithm was proposed to obtain the optimal combination strategy of transport modes. The corresponding program was designed based on MATLAB and implemented on a personal computer. Finally, a real life problem was solved to show the feasibility and efficiency of the proposed model and algorithm.

This paper presents a novel model for resource bounded computation based on process algebras. Such model is called the $-calculus (cost calculus). Resource bounded computation attempts to find the best answer possible given operational constraints. The $-calculus provides a uniform representation for optimization in the presence of limited resources. It uses cost-optimization to find the best quality solutions while using a minimal amount of resources. A unique aspect of the approach is to propose a resource bounded process algebra as a generic problem solving paradigm targeting interactive AI applications. The goal of the $-calculus is to propose a computational model with built-in performance measure as its central element. This measure allows not only the expression of solutions, but also provides the means to incrementally construct solutions for computationally hard, real-life problems. This is a dramatic contrast with other models like Turing machines, λ-calculus, or conventional process algebras. This highly expressive model must therefore be able to express approximate solutions. This paper describes the syntax and operational cost semantics of the calculus. A standard cost function has been defined for strongly and weakly congruent cost expressions. Example optimization problems are given which take into account the incomplete knowledge and the amount of resources used by an agent. The contributions of the paper are twofold: firstly, some necessary conditions for achieving global optimization by performing local optimization in time and/or space are found. That deals with incomplete information and complexity during problem solving. Secondly, developing an algebra which expresses current practices, e.g., neural nets, cellular automata, dynamic programming, evolutionary computation, or mobile robotics as limiting cases, provides a tool for exploring the theoretical underpinnings of these methods. As the result, hybrid methods can be naturally expressed and developed using the algebra.