Optimal Adjustment in the Presence of Deterministic Process Drift and Random Adjustment Error

Optimal consumption and portfolio policies when asset prices follow a diffusion process

Optimal risk adjustment with adverse selection and spatial competition

Optimal transmission policies for a two-user multiple access broadcast channel with binary feedback are investigated. The system is modeled as a dynamic team. Conditioned on the channel feedback, the past history of buffer states is redundant to each user. Once this redundant data is removed, the information structure, although non-classical, satisfies the sufficient conditions of Mahajan, Nayyar, and Teneketzis, 2008 for a dynamic team problem to be tractable. Using the idea of a virtual coordinator of Mahajan, Nayyar, and Teneketzis, a dynamic programming decomposition is presented. This dynamic program is defined over a countable state space and a finite action space. When the arrival rates of both users are symmetric, an optimal policy is identified by solving the dynamic program analytically. This policy matches the optimal window protocol proposed by Hluchyj and Gallager, 1981. Thus, this paper presents the an example of a non-trivial dynamic team with non-linear dynamics where an exact analytic solution is obtained.

An uplink cognitive radio system with a single primary user (PU) and multiple secondary users (SUs) is considered. The SUs have an individual average delay constraint and an aggregate average interference constraint to the PU. If the interference channels between the SUs and the PU are statistically different due to the different physical locations of the SUs, the SUs will experience different delay performances. This is because SUs located closer to the PU transmit with lower power levels. A dynamic scheduling-and- power-allocation policy that uses the dynamic programming, is proposed. The proposed policy can provide the required average delay guarantees to all SUs irrespective of their location as well as protect the PU from harmful interference. The policy is shown to be asymptotically delay optimal in the light traffic regime. Motivated, by the high complexity of the dynamic programming algorithm in the optimal policy we exploit the structure of the problem's solution to present an alternative suboptimal policy. Through simulations we show that in both light and heavy traffic regimes, the delay performance of the suboptimal policy is within $0.3\%$ from the optimal policy and both outperforming existing methods.

Motivated by the urgent demand for the electric vehicle (EV) fast refueling technologies, battery swapping and charging stations (BSCSs) are envisioned as a promising solution to provide timely EV refueling services. However, inappropriate battery charging operation in BSCSs cannot only incur unnecessary high charging cost but also threaten the reliability of the power grid. In this paper, we aim at obtaining an optimal charging operation policy for a single BSCS to minimize its charging cost while ensuring its quality-of-service. Leveraging the novel queueing network model, we propose to formulate the charging operation problem as a constrained Markov decision process and derive the optimal policy by the standard Lagrangian method and dynamic programming. To avoid the curse of dimensionality in practical large-scale systems, we further analyze the structure of the optimal policy and transform the dynamic programming procedure into an equivalent threshold optimization problem with a discrete separable convex objective function. Numerical results validate our theoretical analysis and the computational efficiency of our proposed algorithms. This paper also shows the impact of the system parameters (e.g., numbers of batteries and chargers) on the average cost under the optimal charging policy, which gives rich insights into the infrastructure planning of future BSCS networks.

Determining the optimum replacement policy for Holstein dairy herds in Iran

Synthesis of optimal controllers for a class of maximization problems

Optimal control policies for reducing the maximum size of a closed epidemic—II. Stochastic dynamics

We consider a physical asset consisting of complex systems, where the systems may require upgrades during the lifetime of the asset. In practice, the asset owner and system supplier can make the upgrade decisions together, requiring a decision-support model that can be jointly used to optimize the total benefit for both parties. Motivated by a real-life use case including an asset owner and a system supplier, we build a continuous-time model to optimize the upgrade decisions of a system during the fixed lifetime of the asset. In our model, we capture the key critical factors that drive the upgrade decisions: increasing functionality requirements due to evolving technology, age-dependent maintenance costs, a predetermined overhaul plan of the asset, and the lifetime of the asset. A system upgrade is less costly if it is executed jointly with an asset overhaul. We first analyze the case with no additional cost of upgrading outside an overhaul. We analytically characterize the structure of the optimal upgrade policy under various realistic assumptions that lead to different types of cost functions. We then use these results as a building block to characterize the optimal policy for a generalized cost function. When there is a penalty for upgrading outside an overhaul moment, we propose a dynamic programming approach that efficiently determines the optimal upgrade policy by using our analytical results. We also prove that as this penalty increases, the optimal policy can only change to one where the number of upgrades not jointly executed with overhauls is reduced. However, the optimal number of upgrades is a nonincreasing function of this penalty. Also, surprisingly, more overhauls may lead to a smaller number of upgrades under the optimal policy. Funding: This publication is part of the project “Maritime Remote Control Tower for Service Logistics Innovation (MARCONI)” (project 439.18.309) of the research program “Integrator-Logistics as Enabler for Enhancing Society,” which is (partly) financed by the Dutch Research Council (NWO).

After the design and construction of manufacturing tools, adjustment is often necessary to reduce mean deviation from target of product dimensions. We formulate a stochastic dynamic program to find the optimal adjustment policy when the adjustment outcome is uncertain. The optimal policy minimizes the sum of adjustment cost during pre-production and quality cost during production. The formulation is developed for a single process and for two processes producing components for assembly. In the single process case, sufficient conditions are given for the optimal policy to have an upper and lower control limit form. In the two process case, we compare separate and combined policies and give an example where appreciable savings can be realized from a combined policy.

This paper deals with the problem of allocating attention among multiple tasks in a supervisory control system. The situation is modeled in the framework of a single server priority queueing system. Using Bellman's principle of optimality, a functional equation for the optimal, state dependent, nonpreemptive priority policy is obtained. The optimal policy is contrasted with an easily computable, suboptimal threshold policy, based on heavy traffic approximation. It is concluded that the utility of the optimal policy is significant at medium to high traffic intensities, and that the heavy traffic, suboptimal policy should be of considerable use in this regard. Several modifications of the single server model, as well as extensions to multiple, interacting sever models are pointed out.

We formulate and analyze a Markov decision process (dynamic programming) model for airline seat allocation (yield management) on a single-leg flight with multiple fare classes. Unlike previous models, we allow cancellation, no-shows, and overbooking. Additionally, we make no assumptions on the arrival patterns for the various fare classes. Our model is also applicable to other problems of revenue management with perishable commodities, such as arise in the hotel and cruise industries. We show how to solve the problem exactly using dynamic programming. Under realistic conditions, we demonstrate that an optimal booking policy is characterized by state- and time-dependent booking limits for each fare class. Our approach exploits the equivalence to a problem in the optimal control of admission to a queueing system, which has been well studied in the queueing-control literature. Techniques for efficient implementation of the optimal policy and numerical examples are also given. In contrast to previous models, we show that 1) the booking limits need not be monotonic in the time remaining until departure; 2) it may be optimal to accept a lower-fare class and simultaneously reject a higher-fare class because of differing cancellation refunds, so that the optimal booking limits may not always be nested according to fare class; and 3) with the possibility of cancellations, an optimal policy depends on both the total capacity and the capacity remaining. Our numerical examples show that revenue gains of up to 9% are possible with our model, compared with an equivalent model omitting the effects of cancellations and no-shows. We also demonstrate the computational feasibility of our approach using data from a real-life airline application.

Optimal replacement policy for a deteriorating system with increasing repair times

Optimal adjustment policies of a small open indebted economy : an introduction Optimal control of economies with external debt above its long run target level is studied and time-inconsistent and time-consistent optimal policies are derived successively. Every optimal policy entails initially large values of the domestic interest rate.

Optimal and Near Optimal Control of a Two-Part-Type Stochastic Manufacturing System with Dynamic Setups