A recursive algorithm for bandwidth partitioning

Optimal consumption and portfolio policies when asset prices follow a diffusion process

Optimal Consumption Plans—A Dynamic Programming Approach

This research is about the development of a dynamic programming model for solving fuzzy linear programming problems. Initially, fuzzy dynamic linear programming model FDLP is developed. This research revises the established dynamic programming model for solving linear programming problems in a crisp environment. The mentioned approach is upgraded to address the problem in an uncertain environment. Dynamic programming model can either be passing forward or backward. In the proposed approach backward dynamic programming approach is adopted to address the problem. It is then followed by implementing the proposed method on the education system of Pakistan. The education system of Pakistan comprises of the Primary, Middle, Secondary, and Tertiary education stages. The problem is to maximize the efficiency of the education system while achieving the targets with minimum usage of the constrained resources. Likewise the model tries to maximize the enrollment in the Primary, Middle, Secondary and Tertiary educational categories, subject to the total available resources in a fuzzy uncertain environment. The solution proposes that the enrollment can be increased by an amount 9997130, by increasing the enrollment in the Middle and Tertiary educational categories. Thus the proposed method contributes to increase the objective function value by 30%. Moreover, the proposed solutions violate none of the constraints. In other words, the problem of resources allocation in education system is efficiently managed to increase efficiency while remaining in the available constrained resources. The motivation behind using the dynamic programming methodology is that it always possesses a numerical solution, unlike the other approaches having no solution at certain times. The proposed fuzzy model takes into account uncertainty in the linear programming modeling process and is more robust, flexible and practicable.

This study formulates and solves an advertising pulsation problem for a monopolistic firm using dynamic programming (DP). The firm aims at maximising profit through an optimal allocation of the advertising budget in terms of rectangular pulses over a finite planning horizon. Aggregate sales response to the advertising effort is assumed to be governed by a modified version of the Vidale–Wolfe model in continuous time proposed by Little. Using a numerical example in which a planning horizon of one year is divided into one, two through ten equal time periods, computing routines are developed to solve 150 DP problems. Computational results show among other findings that the performance yielded by the DP policy dominates the uniform advertising policy (constant spending) for a concave advertising response function and the advertising pulsing policy (turning advertising on and off) for a linear or convex response function.

In this paper, the application of the dynamic programming approach to constrained stochastic control problems with expected value constraints is demonstrated. Specifically, two such problems are analyzed using this approach. The problems analyzed are the problem of minimizing a discounted cost infinite horizon expectation objective subject to an identically structured constraint, and the problem of minimizing a discounted cost infinite horizon minimax objective subject to a discounted expectation constraint. Using the dynamic programming approach, optimality equations, which are the chief contribution of this paper, are obtained for these problems. In particular, the dynamic programming operators for problems with expectation constraints differ significantly from those of standard dynamic programming and problems with worst-case constraints. For the discounted cost infinite horizon cases, existence and uniqueness of solutions to the dynamic programming equations are explicitly shown by using the Banach fixed point theorem to show that the corresponding dynamic programming operators are contractions. The theory developed is illustrated by numerically solving the constrained stochastic control dynamic programming equations derived for simple example problems. The example problems are based on a two-state Markov model that represents an error prone system that is to be maintained.

Opaque pricing is a form of pricing where certain characteristics of the product or service are hidden from the consumer until after purchase. In essence, opaque selling transforms a differentiated good into a commodity. Opaque pricing has become popular in service pricing as it allows firms to sell their differentiated product at higher prices to regular brand loyal customers while simultaneously selling to non‐brand loyal customers at discounted prices. We use a nested logit model in combination with logistic regression and dynamic programming to illustrate how a service firm can optimally set prices on an opaque sales channel. The choice model allows the characterization of consumer trade‐offs when purchasing opaque products while the dynamic programming approach allows the characterization of the optimal pricing policy as a function of inventory and time remaining. We compare optimal prices and expected revenues when dynamic pricing is restricted to daily price changes. We provide an illustrative example using data from an opaque selling mechanism ( Hotwire.com ) and a Washington DC‐based hotel.

A EROSPACE engineering applications greatly stimulated the development of optimal control theory during the 1950s and 1960s, where the objective was to drive the system states in such a way that some defined cost was minimized. This turned out to have very useful applications in the design of regulators (where some steady state is to be maintained) and in tracking control strategies (where some predetermined state trajectory is to be followed). Among such applications was the problem of optimal flight trajectories for aircraft and space vehicles. Linear optimal control theory in particular has been very well documented and widely applied, where the plant that is controlled is assumed linear and the feedback controller is constrained to be linear with respect to its input. However, the availability of powerful low-cost microprocessors has spurred great advantages in the theory and applications of nonlinear control. The competitive era of rapid technological change, particularly in aerospace exploration, now demands stringent accuracy and cost requirements in nonlinear control systems. This has motivated the rapid development of nonlinear control theory for application to challenging, complex, dynamical real-world problems, particularly those that bear major practical significance in aerospace, marine, and defense industries. Infinite-time horizon nonlinear optimal control (ITHNOC) presents a viable option for synthesizing stabilizing controllers for nonlinear systems by making a state-input tradeoff, where the objective is to minimize the cost given by a performance index. The original theory of nonlinear optimal control dates from the 1960s. Various theoretical and practical aspects of the problem have been addressed in the literature over the decades since. In particular, the continuous-time nonlinear deterministic optimal control problem associated with autonomous (time-invariant) nonlinear regulator systems that are affine (linear) in the controls has been studied by many authors. The long-established theory of optimal control offers quite mature and well-documented techniques for solving this control-affine nonlinear optimization problem, based on dynamic programming or calculus of variations, but their application is generally a very tedious task. Bellman’s dynamic programming approach reduces to solving a nonlinear first-order partial differential equation (PDE), expressed by the Hamilton–Jacobi–Bellman (HJB) equation. The solution to the HJB equation gives the optimal performance/cost value (or storage) function and determines an optimal control in feedback form under some smoothness assumptions. Alternatively, in the classical calculus of variations, optimal control problems can be characterized locally in terms of the Hamiltonian dynamics arising from Pontryagin’s minimum principle. These are the characteristic equations of the HJB PDE, which result in a nonlinear, constrained two-point boundary value problem (TPBVP) that, in general, can only be solved by successive approximation of the optimal control input using iterative numerical techniques for each set of initial conditions. Numerically, even though the nonlinear TPBVP is somewhat easier to solve than the HJBPDE, control signals can only be determined offline and are thus best suited for feedforward control of plants for which the state trajectories are known a priori. Therefore, contrary to the dynamic programming approach, the resultant control law is not generally in feedback form. Open-loop control, however, is sensitive to random disturbances and requires that the initial state be on the optimal trajectory. In contrast, nonlinear optimal feedback has inherent robustness properties (inherent in the sense that it is obtained by ignoring uncertainty and disturbances). The potential difficulty with the HJB approach is that no efficient algorithm is available to solve the PDE when it is nonlinear and the problem dimension is high, making it impossible to derive exact expressions for optimal controls for most nontrivial problems of interest. The optimal can only be computed in special cases, such as linear dynamics and quadratic cost, or very low-dimensional systems. In particular, if the plant is linear time invariant (LTI) and the (infinite-time) performance index is quadratic, then the corresponding HJB equation for this infamous linear-quadratic regulator (LQR) problem reduces to an algebraic Riccati equation (ARE). Contrary to the well-developed and widely applied theory and computational tools for theRiccati equation (for example, see [1]), theHJB equation is difficult, if not impossible, to solve for most practical applications. The exact solution for the optimal control policies is very complex

Multicast is an efficient technique for delivering data to a large group of users. For some applications, offering multicast security is an important issue. In such a system, a new member should not be able to decrypt the multicast data sent before its joining and a former member should not be able to decrypt the multicast data sent after its leaving. Traditional approaches generally focus on reducing the re-key messages for a single server. However, these approaches still lead to large exchange overhead when the group is large. We consider a distributed server network in which the user pool is split into multiple groups and served by multiple servers. Given the user traffic, there is a trade-off between the amount of re-key messaging and the total data bandwidth needed. We present a simple model for the system and study how the total bandwidth (including the re-key messaging and data traffic) can be minimized by optimizing the number of servers in the network. As the underlying user traffic is dynamic, a server should be able to split and merge user groups to minimize its total bandwidth. We propose a scheme for such a purpose. We show that a distributed server network is able to reduce substantively the total bandwidth required in the system as compared to the traditional scheme.

In the next decades, many public infrastructure assets will reach the end of their life that they were originally designed for. Replacement costs are high, and therefore increasing effort is put into lifetime-extending maintenance, including major overhauls and renovations. A key question is whether the investments in lifetime-extending maintenance justify the postponement of a full replacement. This question becomes more complicated when future life cycle cash flows are non-repeatable. Differential inflation and technological change, including multiple intervention strategies to maintain a desired functionality, cause such non-repeatability. In this case, classic replacement analysis techniques will not suffice in answering this question. Literature demonstrates that case-specific modelling with dynamic or linear programming techniques is required to find economic optimisation. However, such literature primarily addresses replacement interval optimisation of new investments within relative short time horizons, whereas the current research develops a nested dynamic programming (DP) approach for typical ageing infrastructure assets over long service life periods. The model can deal with multiple and various successive intervention strategies and addresses differential inflation and age-related cost increases. Finally, it is shown in an infrastructure case study that this DP approach leads to a better decision in comparison to the application of classical replacement techniques.

This paper considers optimal feedback control for a general continuous time finite-dimensional deterministic system with finite horizon cost functional. A practically feasible algorithm to calculate the numerical solution of the optimal feedback control by dynamic programming approach is developed. The highlights of this algorithm are: a) It is based on a convergent constructive algorithm for optimal feedback control law which was proposed by the authors before through an approximation for the viscosity solution of the time-space discretization scheme developed by dynamic programming method; b) The computation complexity is significantly reduced since only values of viscosity solution on some local cones around the optimal trajectory are calculated. Two numerical experiments are presented to illustrate the effectiveness and fastness of the algorithm.

In modern production systems, it is critical to perform maintenance, calibration, installation, and upgrade tasks during planned downtime. Otherwise, the systems become unreliable and new product introductions are delayed. For reasons of safety, testing, and access, task performance often requires the vicinity of impacted equipment to be left in a specific “end state” when production halts. Therefore, planning the shutdown of a production system to balance production goals against enabling non-production tasks yields a challenging optimization problem. This article proposes a mathematical formulation of this problem and a dynamic programming approach that efficiently finds optimal shutdown policies for deterministic serial production lines. An event-triggered re-optimization procedure that is based on the proposed deterministic dynamic programming approach is also introduced for handling uncertainties in the production line for the stochastic case. It is demonstrated numerically that in these cases with random breakdowns and repairs, the re-optimization procedure is efficient and even obtains results that are optimal or nearly optimal.

A hybrid exact algorithm for complete set partitioning

This chapter is devoted to the modification of an extension of dynamic programming approach for optimization of decision rules relative to length. “Classical” dynamic programming approach allows one to obtain optimal rules, i.e., rules with the minimum length. This fact is important from the point of view of knowledge representation. The idea of the dynamic programming approach for optimization of decision rules is based on a partitioning of a decision table into subtables. The algorithm constructs a directed acyclic graph. Basing on the constructed graph, sets of rules with the minimum length, attached to each row of a decision table, can be described. Proposed modification is based on the idea that not the complete graph is constructed but its part. It allows one to obtain values of length of decision rules close to optimal ones, and the size of the graph is smaller than in case of “classical” dynamic programming approach. The chapter also contains results of experiments with decision tables from UCI Machine Learning Repository.

This paper investigates an optimal decentralized control problem for a system with B-matrices dependent on stochastic parameters. It is assumed that these parameters are independent in time and available locally to each controller. The objective is to find a decentralized state feedback control policy that minimizes a multi step quadratic cost function. We first solve this problem for the case of one time step and show that the optimal policy can be reached through iterating the best responses of each player. For the optimal multiple step problem, a dynamic programming approach is employed while using the result of the one step control at each step. Finally in the infinite horizon case, we provide sufficient conditions under which the optimal control policy is unique and stabilizes the system in a mean square sense.

This article presents a novel dynamic programming approach to determine the robust controllability of Boolean control networks (BCNs) subject to stochastic disturbances. By applying Bellman's optimality principle, we derive the recurrence relation for computing the optimal time matrix, a crucial concept characterizing robust reachability between two arbitrary states. We develop a finite-termination dynamic programming algorithm to calculate the optimal time matrix exactly and efficiently, with a rigorously certified iteration count. Sufficient and necessary conditions for robust controllability are then established based on the optimal time matrix. Furthermore, for any pair of reachable states, we construct time-optimal state feedback control laws to steer the system from the initial state to the target state, regardless of disturbances. Finally, extensive numerical experiments with biological networks validate the effectiveness of the proposed approach, showing significant improvements in computational efficiency. Additionally, we introduce a Q-learning-based algorithm and compare its performance, highlighting the advantages of our dynamic programming approach in terms of both efficiency and solution quality.