Optimal and System Myopic Policies for Multi-Echelon Production/Inventory Assembly Systems

Ratchet consumption over finite and infinite planning horizons

Minimizing Risk Models in Markov Decision Processes with Policies Depending on Target Values

THE PURPOSE of the dissertation is to integrate financial intermediation into a theoretical model of real economic output and growth. The methodology involves construction of a model in which capital accumulation is based upon an interest-elastic demand for the capital stock. This model is shown to be capable of adjusting to an equilibrium capital intensity and saving share. To this growth model is added a financial intermediary sector to form a two-sector model of an economy. The production sector uses real inputs of capital and labor to produce a homogeneous good which is then allocated to consumption and real capital accumulation. The financial intermediary sector uses real inputs of capital and labor to produce financial intermediary services. Financial intermediary output is described by a production function for the financial intermediary in which real capital and labor inputs are related to the units of real capital placed by the financial intermediary with production sector firms. That is, for each unit of real capital used to produce the homogeneous good, the financial intermediary expends real resources of capital and labor to gather capital from surplus households in order to place this capital with deficit production units. It is assumed that at the firm level the financial production functions exhibit the usual decreasing average product after some point, which leads to increasing average unit costs. At the aggregate sector level the financial production function exhibits constant returns to scale and constant unit costs, as expansion is carried out by the introduction of new and similarly efficient financial intermediaries. The financial intermediary thus employs real capital and labor in profit-maximizing quantities and pays these factors the competitive factor returns. The intermediary pays deposit rates to attract deposit capital which it then lends to production sector units and earns a loan rate equal to the marginal product of capital. The financial intermediary earns unit revenues equal to the difference between the loan rate and deposit rate. This quantity, called the unit cost of finance, is a per period rate per unit of capital. Total revenues of the intermediary are then composed of these per unit charges. When competition prevails in the financial intermediary market for deposit capital the intermediary is forced to pay a high enough deposit rate so that the intermediary merely covers its real factor input costs. The intermediary thus is capable of adjustments in deposit rates and factor inputs; and in a competitive equilibrium situation it produces financial services at minimum average costs. The deposit rate in a competitive equilibrium is then equal to the production sector marginal product of capital (loan rate) less the minimum average unit cost of financial services. The solution for the variables of the system are determined by a simultaneous solution of the sectoral production functions, the capital demand function (saving function), and capital and labor allocation equations (perfect substitutability). The model is solved for the sectoral capital intensities, the distribution of labor and capital between sectors, the real rates in the system (deposit rate and marginal product of capital), as well as aggregate and per capita income, investment and consumption.

Optimal replacement policy for the case where the damage process is a one-sided Lévy process

This paper studies a periodic-review production-inventory control model under an energy buy-back program over an infinite planning horizon, in which a fixed setup cost and compensation levels corresponding to various market states are involved. The objective is to identify the manufacturer’s optimal production-inventory policy that can minimize his total discounted cost or long-run average cost. By using Veinott’s conditions, it is shown that such a state dependent optimal policy is of either an [Formula: see text], or partly an [Formula: see text] type.

Cost-effective asset management is an area of interest across several industries. Specifically, this paper develops a deep reinforcement learning (DRL) solution to automatically determine an optimal rehabilitation policy for continuously deteriorating water pipes. We approach the problem of rehabilitation planning in an online and offline DRL setting. In online DRL, the agent interacts with a simulated environment of multiple pipes with distinct lengths, materials, and failure rate characteristics. We train the agent using deep Q-learning (DQN) to learn an optimal policy with minimal average costs and reduced failure probability. In offline learning, the agent uses static data, e.g., DQN replay data, to learn an optimal policy via a conservative Q-learning algorithm without further interactions with the environment. We demonstrate that DRL-based policies improve over standard preventive, corrective, and greedy planning alternatives. Additionally, learning from the fixed DQN replay dataset in an offline setting further improves the performance. The results warrant that the existing deterioration profiles of water pipes consisting of large and diverse states and action trajectories provide a valuable avenue to learn rehabilitation policies in the offline setting, which can be further fine-tuned using the simulator.

This paper studies a class of renewable resource allocation problems for the processing of dynamically arriving tasks with deterministic deadlines. This class of problems has many applications, however, it conforms to neither the standard resource allocation model nor the standard optimal control model. A new problem formulation has to be developed and analyzed to provide a satisfactory solution. In this paper, a new formulation is presented. By augmenting the state variabls, the problem is converted into a Markovian decision problem. It can then be treated, at least in principle, by using the stochastic dynamic programming (SDP) method. However, since the system dynamics involves the evolution of sets (of tasks and resources), the implementation of the dynamic programming equation is by no means straightforward. For a problem with infinite planning horizon, the optimal strategy is shown to be stationary under mild conditions. An SDP algorithm based on a successive approximation technique is developed to obtain the optimal stationary strategy. The implementation of the algorithm employs a special coding scheme to handle set variables, and utilizes a dominance property for computational efficiency. Effects of key system parameters on optimal decisions are investigated and analyzed through numerical examples. As the computational complexity of the algorithm is of exponential increase, practical applications of the algorithm is limited to problems of moderate size. Two heuristic rules are therefore investigated and compared to the optimal policy.

We consider an assembly system with multiple stages, multiple items, and multiple customer classes. The system consists of m production facilities, each producing a different item. Items are produced in variable batch sizes, one batch at a time, with exponentially distributed batch production times. Demand from each class takes place continuously over time according to a compound Poisson process. At each decision epoch, we must determine whether or not to produce an item and, should demand from a particular class arise, whether or not to satisfy it from existing inventory, if any is available. We formulate the problem as a Markov decision process and use it to characterize the structure of the optimal policy. In contrast to systems with exogenous and deterministic production lead times, we show that the optimal production policy for each item is a state-dependent base-stock policy with the base-stock level nonincreasing in the inventory level of items that are downstream and nondecreasing in the inventory level of all other items. For inventory allocation, we show that the optimal policy is a multilevel state-dependent rationing policy with the rationing level for each demand class nonincreasing in the inventory level of all nonend items. We also show how the optimal control problem can be reformulated in terms of echelon inventory and how the essential features of the optimal policy can be reinterpreted in terms of echelon inventory.

We study the optimal behavior of an investor who is forced to withdraw funds continuously at a fixed rate per unit time (e.g., to pay for a liability, to consume, or to pay dividends). The investor is allowed to invest in any or all of a given number of risky stocks, whose prices follow geometric Brownian motion, as well as in a riskless asset which has a constant rate of return. The fact that the withdrawal is continuously enforced, regardless of the wealth level, ensures that there is a region where there is a positive probability of ruin. In the complementary region ruin can be avoided with certainty. Call the former region the danger-zone and the latter region the safe-region. We first consider the problem of maximizing the probability that the safe-region is reached before bankruptcy, which we call the survival problem. While we show, among other results, that an optimal policy does not exist for this problem. we are able to construct explicit ∊-optimal policies, for any ∊ > 0. In the safe-region, where ultimate survival is assured, we turn our attention to growth. Among other results, we find the optimal growth policy for the investor, i.e., the policy which reaches another (higher valued) goal as quickly as possible. Other variants of both the survival problem as well as the growth problem are also discussed. Our results for the latter are intimately related to the theory of Constant Proportions Portfolio Insurance.

Optimal replacement policy for a deteriorating system with increasing repair times

The part‐period balancing algorithm and its look ahead‐look back feature: A theoretical and experimental analysis of a single stage lot‐sizing procedure

We consider the economic behavior of a single-channel queuing system operating with the following cost structure: a server start-up cost, a server shut-down cost, a cost per unit time when the server is turned off, a cost per unit time when the server is turned on, and a holding cost per unit time spent in the system for each customer. For the undiscounted, infinite-horizon problem, bounds are obtained for the cost rate and optimal policy when the interarrival time distribution is a member of the class of increasing-failure-rate distributions. Some of the bounds hold for all interarrival-time distributions.

Stochastic control problems for controlled Markov processes models with an infinite planning horizon are considered, under some non-standard cost criteria. The classical discounted and average cost criteria can be viewed as complementary, in the sense that the former captures the short-time and the latter the long-time performance of the system. Thus, the authors study a cost criterion obtained as weighted combinations of these criteria, extending to a general state and control space framework several results by Feinberg and Shwartz (1992), and by Krass et al. (1992). In addition, a functional characterization is given for overtaking optimal policies, for problems with countable state spaces and compact control spaces; the authors' approach is based on qualitative properties of the optimality equation for problems with an average cost criterion. >

Joint Production and Replacement Planning for an Unreliable Manufacturing System Subject to Random Demand and Quality

In this paper we consider the optimal impulse control of a system which evolves randomly in accordance with a homogeneous diffusion process in ℜ1. Whenever the system is controlled a cost is incurred which has a fixed component and a component which increases with the magnitude of the control applied. In addition to these controlling costs there are holding or carrying costs which are a positive function of the state of the system. Our objective is to minimize the expected discounted value of all costs over an infinite planning horizon. Under general assumptions on the cost functions we show that the value function is a weak solution of a quasi-variational inequality and we deduce from this solution the existence of an optimal impulse policy. The computation of the value function is performed by means of the Finite Element Method on suitable truncated domains, whose convergence is discussed.