Application of RPA and the harmonic gradient estimators to a priority queueing system

Alternative methods such as likelihood ratios (LRs), finite perturbation analysis (FPA), smoothed perturbation analysis (SPA), and rare perturbation analysis (RPA) have been proposed and could be used when infinitesimal perturbation analysis (IPA) does not apply directly. The authors discuss some links that exist between these methods and explain them by showing how each of them can be applied to a specific example, namely to estimate the derivative of the expected number of customers per regenerative cycle in a GI/G/1 queue, with respect to parameters of the interarrival and/or service-time distributions. Two efficient SPA estimators for estimating the derivative of the expected number of customers per regenerative cycle in a GI/G/1 queue are introduced. For the examples examined, these estimators outperformed those that had been proposed previously for the same problem. >

We investigate the use of a distributed asynchronous algorithm utilizing infinitesimal perturbation analysis (IPA) gradient estimators for on-line optimization of tandem networks of queues. In our scheme, each queue has a processor that updates a control parameter associated with the queue according to a stochastic gradient algorithm driven by IPA estimates of the gradient of the performance measure. The update times of the processors are not synchronized. The processors also communicate results of computations with each other, and this communication involves delay. We give conditions under which the algorithm converges with probability one. In our proof of convergence we analyze a particular subsequence of the sequence of control parameters, and show that this subsequence behaves like a sequence generated by a centralized synchronous gradient algorithm which updates before the start of certain busy periods of the network, and with gradient estimates that are asymptotically unbiased.

Stochastic Flow Models (SFMs) form a class of hybrid systems used as abstractions of complex Discrete Event Systems (DES) for the purpose of deriving performance sensitivity estimates through Infinitesimal Perturbation Analysis (IPA) techniques when these cannot be applied to the original DES. In this paper, we establish explicit connections between gradient estimators obtained through a SFM and those obtained in the underlying DES, thus providing analytical evidence for the effectiveness of these estimators which has so far been limited to empirical observations. We consider DES for which analytical expressions of IPA (or finite difference) estimators are available, specifically G/G/1 and G/G/1/K queueing systems. In the case of the G/G/1 system, we show that, when evaluated on the same sample path of the underlying DES, the IPA gradient estimators of states, event times, and various performance metrics derived through SFMs are, under certain conditions, the same as those of the associated DES or their expected values are asymptotically the same under large traffic rates. For G/G/1/K systems without and with feedback, we show that SFM-based derivative estimates capture basic properties of finite difference estimates evaluated on a sample path of the underlying DES.

Using Infinitesimal Perturbation Analysis of Stochastic Flow Models to Recover Performance Sensitivity Estimates of Discrete Event Systems

Infinitesimal Perturbation Analysis (IPA) estimators of the response gradient for a discrete event stochastic simulation are typically developed within the framework of Generalized semi-Markov processes (GSMPs). Unfortunately, while mathematically rigorous, GSMPs are not particularly useful for modeling real systems. We describe a procedure that allows IPA gradient estimation to be easily and automatically implemented in the more general and intuitive modeling context of event graphs. The intent is to make IPA gradient estimation more easily understood and more widely accessible. The pictorial nature of Event Graphs also provides insights into the basic IPA calculations and alternative descriptions of conditions under which the IPA estimator is known to be unbiased.

In this paper, a manufacturing system composed of a single-product machine, a buffer and a stochastic demand is considered. Two models are presented: continuous and discrete flow models including constant delivery times, machine failures and random demands. The objective is to determine the value of the optimal buffer level, for a hedging point policy which minimises the total average cost function. The cost function is the sum of inventory, transportation and lost sales costs. Infinitesimal perturbation analysis is used for optimisation of the failure-prone manufacturing system. The trajectories of buffer level are studied for the continuous and discrete cases and the infinitesimal perturbation analysis estimators are evaluated. These estimators are shown to be unbiased and then they are implemented in an optimisation algorithm which determines the optimal buffer level in the presence of constant delivery time. Numerical results are presented for continuous and discrete flow models and then compared in order to evaluate the application of the infinitesimal perturbation analysis on the discrete flow model.

A simple example of a discrete event dynamic system is used to show that given that an interevent time has a distribution function that is a function of t/ tau , where tau is the mean interarrival time and t is the interdeparture time, one cannot conclude that a perturbation of delta tau , which is the parameter perturbation, would cause every interevent time to stretch by the factor delta tau / tau . This example is used to illustrate the fact that, to ensure that the constructed perturbed path is physically possible, when performing general perturbation analysis one must keep track of events in both the nominal and perturbed paths; but when performing infinitesimal perturbation analysis (IPA) one must employ interevent distribution functions given the event sequence observed in the nominal path. One implication of the latter observation is that IPA estimates are independent of the representation. >

This talk considers loss-related performance measures in high-speed networks from the standpoint of gradient estimation by the Infinitesimal Perturbation Analysis (IPA) technique. The underlying model consists of an interconnected system of continuous flow models (CFMs) processing data in a fluid form, and the performance measures of interest are related to the average loss volume caused by buffer overflow over a period of time. The main results presented are: (i) an easily implementable, distributed algorithm for computing the IPA derivative, and (ii) analysis of the algorithm that proves the unbiasedness of the IPA estimator. 'Me distributed algorithm can naturally be implemented in real-time at a network's nodes, and hence potentially can be used for on-line control. The analysis highlights some of the special structure of CFMs, and suggests a general scope of 1PRs unbiasedness, far beyond the specific performance measures discussed. The talk will conclude with remarks about the potential applicability of IPA as a design and analysis tool for networks of CFMs.

Simulation of IPA gradients in hybrid network systems

The authors consider an M/G/1 queue with a time-varying arrival rate. The objective is to obtain infinitesimal perturbation analysis (IPA) gradient estimates for various performance measures of interest with respect to certain system parameters. In particular, the authors consider the mean system time over n arrivals and an arrival rate alternating between two values. By choosing a convenient sample path representation of this system, they derive an unbiased IPA gradient estimator which, however, is not consistent, and investigate the nature of this problem. >

We develop an effective Monte Carlo method for estimating sensitivities or gradients of expectations of sufficiently smooth functionals of a reflected diffusion in a convex polyhedral domain with respect to its defining parameters, namely its initial condition, drift, and diffusion coefficients and directions of reflection. Our method, which falls into the class of infinitesimal perturbation analysis (IPA) methods, uses a probabilistic representation for such sensitivities as the expectation of a functional of the reflected diffusion and its associated derivative process. The latter process is the unique solution to a constrained linear stochastic differential equation with jumps whose coefficients, domain, and directions of reflection are modulated by the reflected diffusion. We propose a consistent estimator for such sensitivities using an Euler approximation of the reflected diffusion and its associated derivative process. Proving that the Euler approximation converges is challenging because the derivative process jumps whenever the reflected diffusion hits the boundary of the domain. A key step in the proof is establishing a continuity property of the related derivative map, which is of independent interest. We compare the performance of our IPA estimator with a standard likelihood ratio estimator (whenever the latter is applicable) and provide numerical evidence that the variance of the former is substantially smaller than that of the latter. We illustrate our method with an example of a rank-based interacting diffusion model of equity markets. Interestingly, we show that estimating certain sensitivities of the rank-based interacting diffusion model using our method for a reflected Brownian motion description of the model outperforms a finite difference method for a stochastic differential equation description of the model.

A service facility with multiple customer classes is considered. Each class forms an independent Poisson arrival process with admission controlled by a threshold parameter and is characterized by a general service time distribution. Gradient estimators are derived for performance measures of the system (mean system time, throughput, blocking probabilities) using perturbation analysis techniques. The approach is based on exploiting alternative sample path representations of generalized semi-Markov process models of the system, such that the resulting sample functions are continuous with respect to parameters of interest. The unbiasedness of the estimators is proved. Simulation examples are included to illustrate their properties. >

A gradient-estimation procedure for a general class of stochastic discrete-event systems is developed. In contrast to most previous work, the authors focus on performance measures whose realizations are inherently discontinuous (in fact, piecewise constant) functions of the parameter of differentiation. Two broad classes of finite-horizon discontinuous performance measures arising naturally in applications are considered. Because of their discontinuity, these important classes of performance measures are not susceptible to infinitesimal perturbation analysis (IPA). Instead, the authors apply smoothed perturbation analysis, formalizing it and generalizing it in the process. Smoothed perturbation analysis uses conditional expectations to smooth jumps. The resulting gradient estimator involves two factors: the conditional rate at which jumps occur, and the expected effect of a jump. Among the types of performance measures to which the methods can be applied are transient state probabilities, finite-horizon throughputs, distributions on arrival, and expected terminal cost.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

We consider discrete-time dynamic scheduling problems of the following three types ofG/G/1 queue withKdifferent customer classes: (i) aG/DFR/1 queue withKclasses under preemptive resume service discipline, (ii) aG/IFR/1 queue with two classes under preemptive resume service discipline, and (iii) aG/G/1 queue with two classes under non-preemptive service discipline. Interchange arguments are used to show that simple index policies of different type minimize the total holding cost of customers in a finite-horizon scheduling period for the three cases. Our results extend the result for aG/M/1 queue by Buyukkoc et al. (1985) to general queues.

We consider a queueing system with two classes of customers, two heterogeneous servers, and discriminatory random order service (DROS) discipline. The two servers may have either the same or different DROS weights for each class. Customers of each class arrive according to a Poisson process and the service times of each class of customers are assumed to be exponentially distributed with service rate depending on both the customer's class and the servers. We provide stability and instability conditions for this two-class two-server queue with DROS discipline.