IPA for delay threshold violation using stochastic fluid models

Performance regulation of event-driven dynamical systems using infinitesimal perturbation analysis

We consider a class of single-stage, single-product Make-to-Stock production-inventory system (MTS system) with backorders. The system employs a continuous-review base-stock policy which strives to maintain a prescribed base-stock level of inventory. In a previous paper of Zhao and Melamed (Methodology and Computing in Applied Probability 8:191–222, 2006), the Infinitesimal Perturbation Analysis (IPA) derivatives of inventory and backorders time averages with respect to the base-stock level and a parameter of the production-rate process were computed in Stochastic Fluid Model (SFM) setting, where the demand stream at the inventory facility and its replenishment stream from the production facility are modeled by stochastic rate processes. The advantage of the SFM abstraction is that the aforementioned IPA derivatives can be shown to be unbiased. However, its disadvantages are twofold: (1) on the modeling side, the highly abstracted SFM formulation does not maintain the identity of transactions (individual demands, orders and replenishments) and has no notion of lead times, and (2) on the applications side, the aforementioned IPA derivatives are brittle in that they contain instantaneous rates at certain hitting times which are rarely known, and consequently, need to be estimated. In this paper, we remedy both disadvantages by using a discrete setting, where transaction identity is maintained, and order fulfillment from inventory following demand arrivals and inventory restocking following replenishment arrivals are modeled as discrete jumps in the inventory level. We then compute the aforementioned IPA derivatives with respect to the base-stock level and a parameter of the lead-time process in the discrete setting under any initial system state. The formulas derived are shown to be unbiased and directly computable from sample path observables, and their computation is both simple and computationally robust.

In this paper, a stochastic fluid model is used to study a manufacturing/ remanufacturing system composed by two parallel machines, a serviceable inventory, a remanufacturing inventory and customers who demand a stochastic quantity of product. Stochastic fluid model is adopted to describe the system and to take into account machine failure, stochastic demand, stochastic returned products and remanufacturing products. The goal of this paper is to evaluate the optimal serviceable inventory level which allows minimizing the sum of inventory and lost sales costs. Perturbation analysis is applied to the stochastic fluid model to optimize the considered system. The trajectories of the serviceable inventory level are studied and the perturbation analysis estimates are evaluated. The unbiasedness of these estimates is proved and then they are implemented in an optimization algorithm which determines the optimal serviceable inventory in the presence of stochastic returned products and stochastic demand.

Infinitesimal Perturbation Analysis (IPA) estimators of the response gradient for a discrete event stochastic simulation are typically developed within the framework of Generalized semi-Markov processes (GSMPs). Unfortunately, while mathematically rigorous, GSMPs are not particularly useful for modeling real systems. We describe a procedure that allows IPA gradient estimation to be easily and automatically implemented in the more general and intuitive modeling context of event graphs. The intent is to make IPA gradient estimation more easily understood and more widely accessible. The pictorial nature of Event Graphs also provides insights into the basic IPA calculations and alternative descriptions of conditions under which the IPA estimator is known to be unbiased.

Simulation of IPA gradients in hybrid network systems

In this paper we study Make-To-Stock manufacturing systems and seek on-line algorithms for determining optimal or near optimal buffer capacities (hedging points) that balance inventory against stockout costs. Using a zStochastic Fluid Model (SFM), we derive sample derivatives (sensitivities) which, under very weak structural assumptions on the defining demand and service processes, are shown to be unbiased estimators of the sensitivities of a cost function with respect to these capacities. When evaluated based on the sample path of discrete-part systems, we show that these estimators are greatly simplified. Thus, they can be easily implemented and evaluated on line. Though the implementation on discrete-part systems does not necessarily preserve the unbiasedness property, simulation results show that stochastic approximation algorithms that use such estimates do converge to optimal or near optimal hedging points.

This work extends the single product, single facility and single warehouse make-to-stock (MTS) model (Zhao and Melamed 2004), by allowing subcontracting. In addition to the inventory triggering level employed in their model, a new subcontracting triggering level is used to manage the subcontracted amount. A stochastic fluid model (SFM) is used to model the MTS system with subcontracting. In the current model, the subcontractor is assumed capable of satisfying any difference between the demand and the in-house production to keep the inventory at the subcontracting triggering level. The sensitivity of the inventory on-hand, backorders and amounts subcontracted with respect to the two employed triggering levels are found using infinitesimal perturbation analysis (IPA).

This talk considers loss-related performance measures in high-speed networks from the standpoint of gradient estimation by the Infinitesimal Perturbation Analysis (IPA) technique. The underlying model consists of an interconnected system of continuous flow models (CFMs) processing data in a fluid form, and the performance measures of interest are related to the average loss volume caused by buffer overflow over a period of time. The main results presented are: (i) an easily implementable, distributed algorithm for computing the IPA derivative, and (ii) analysis of the algorithm that proves the unbiasedness of the IPA estimator. 'Me distributed algorithm can naturally be implemented in real-time at a network's nodes, and hence potentially can be used for on-line control. The analysis highlights some of the special structure of CFMs, and suggests a general scope of 1PRs unbiasedness, far beyond the specific performance measures discussed. The talk will conclude with remarks about the potential applicability of IPA as a design and analysis tool for networks of CFMs.

In this paper we perform Infinitesimal Perturbation Analysis (IPA) for a single-stage stochastic fluid queue that is shared between two competing sources, one that employs additive loss-feedback congestion control and the other that employs no congestion-control (i.e., it is unresponsive). This scenario is applicable within the realm of computer communication networks particularly at bottleneck router queues where multiple and diverse flows compete for bandwidth. We optimize the tradeoff between total loss volume and queue workload (a measure for queueing delay). Although a sound knowledge of the system's dynamics is required to derive the IPA gradient estimators, no knowledge of the underlying probability distributions governing the system is required. What results are fairly simple counting processes, whose values can be computed directly from an ongoing live stream of traffic.

Stochastic Flow Models (SFMs) form a class of hybrid systems used as abstractions of complex Discrete Event Systems (DES) for the purpose of deriving performance sensitivity estimates through Infinitesimal Perturbation Analysis (IPA) techniques when these cannot be applied to the original DES. In this paper, we establish explicit connections between gradient estimators obtained through a SFM and those obtained in the underlying DES, thus providing analytical evidence for the effectiveness of these estimators which has so far been limited to empirical observations. We consider DES for which analytical expressions of IPA (or finite difference) estimators are available, specifically G/G/1 and G/G/1/K queueing systems. In the case of the G/G/1 system, we show that, when evaluated on the same sample path of the underlying DES, the IPA gradient estimators of states, event times, and various performance metrics derived through SFMs are, under certain conditions, the same as those of the associated DES or their expected values are asymptotically the same under large traffic rates. For G/G/1/K systems without and with feedback, we show that SFM-based derivative estimates capture basic properties of finite difference estimates evaluated on a sample path of the underlying DES.

In this paper, a manufacturing system composed of a single-product machine, a buffer and a stochastic demand is considered. Two models are presented: continuous and discrete flow models including constant delivery times, machine failures and random demands. The objective is to determine the value of the optimal buffer level, for a hedging point policy which minimises the total average cost function. The cost function is the sum of inventory, transportation and lost sales costs. Infinitesimal perturbation analysis is used for optimisation of the failure-prone manufacturing system. The trajectories of buffer level are studied for the continuous and discrete cases and the infinitesimal perturbation analysis estimators are evaluated. These estimators are shown to be unbiased and then they are implemented in an optimisation algorithm which determines the optimal buffer level in the presence of constant delivery time. Numerical results are presented for continuous and discrete flow models and then compared in order to evaluate the application of the infinitesimal perturbation analysis on the discrete flow model.

We investigate the use of a distributed asynchronous algorithm utilizing infinitesimal perturbation analysis (IPA) gradient estimators for on-line optimization of tandem networks of queues. In our scheme, each queue has a processor that updates a control parameter associated with the queue according to a stochastic gradient algorithm driven by IPA estimates of the gradient of the performance measure. The update times of the processors are not synchronized. The processors also communicate results of computations with each other, and this communication involves delay. We give conditions under which the algorithm converges with probability one. In our proof of convergence we analyze a particular subsequence of the sequence of control parameters, and show that this subsequence behaves like a sequence generated by a centralized synchronous gradient algorithm which updates before the start of certain busy periods of the network, and with gradient estimates that are asymptotically unbiased.

Steady-state sample path derivative Estimators

Alternative methods such as likelihood ratios (LRs), finite perturbation analysis (FPA), smoothed perturbation analysis (SPA), and rare perturbation analysis (RPA) have been proposed and could be used when infinitesimal perturbation analysis (IPA) does not apply directly. The authors discuss some links that exist between these methods and explain them by showing how each of them can be applied to a specific example, namely to estimate the derivative of the expected number of customers per regenerative cycle in a GI/G/1 queue, with respect to parameters of the interarrival and/or service-time distributions. Two efficient SPA estimators for estimating the derivative of the expected number of customers per regenerative cycle in a GI/G/1 queue are introduced. For the examples examined, these estimators outperformed those that had been proposed previously for the same problem. >

The authors consider an M/G/1 queue with a time-varying arrival rate. The objective is to obtain infinitesimal perturbation analysis (IPA) gradient estimates for various performance measures of interest with respect to certain system parameters. In particular, the authors consider the mean system time over n arrivals and an arrival rate alternating between two values. By choosing a convenient sample path representation of this system, they derive an unbiased IPA gradient estimator which, however, is not consistent, and investigate the nature of this problem. >