The Solution of Queueing and Inventory Models by Semi-Markov Processes

Semi-Markov Processes: Applications in System Reliability and Maintenance

Many important factors affect the spread of childhood infectious disease. To better understand the fundamental drivers of infectious disease spread, several researchers have estimated seasonal transmission coefficients in discrete-time models. In this paper, we build upon this previous work and also develop a framework for efficient estimation using continuous differential equation models. We introduce nonlinear programming formulations to efficiently estimate model parameters and seasonal transmission profiles from existing case count data for the childhood disease, measles. We compare results from discrete time and continuous time models and address several shortcomings of the discrete-time method, including removing the need for the data reporting interval to match the time between successive cases in the chain of transmission or serial interval of the disease. Using a simultaneous approach for optimization of differential equation systems, we demonstrate that seasonal transmission parameters can be effectively estimated using continuous time models instead of discrete-time models.

A fundamental identity, due to Miller (1961a), (1962a, b) and Kemperman (1961), is generalized to semi-Markov processes. Thus the identity applies to processes defined on a Markov chain with discrete state space and random walks with Markov dependent steps (Section 2). Wald's identity is discussed briefly in Section 3. Section 4 is a study of the maxima of partial sums, and Section 5 of maxima in a semi-Markov process.

We present an algorithm for numerically finding the limiting distribution of the number in the system for the bulk-arrival, multiserver queueing system MX/D/c. Sample numerical results and graphs of various quantities of interest are also presented. In all cases, the proposed method is computationally efficient, accurate and reliable for both high and low values of the model parameters. The procedure is adaptable to other queueing models in discrete and continuous time, to problems in inventory control, the theory of dams, etc. Exact results found here will also be found useful by people dealing with inequalities, bounds, and approximations.

The article analyses the operation of reciprocal internal combustion engines, with marine engines used as an example. The analysis takes into account types of energy conversion in the work spaces (cylinders) of these engines, loads of their crankshaft-piston assemblies, and types of fuel combustion which can take place in these spaces during engine operation. It is highlighted that the analysed time-dependent loads of marine internal combustion engine crankshaft-piston assemblies are random processes. It is also indicated that the wear of elements of those assemblies resulting from their load should also be considered a random process. A hypothesis is formulated which explains random nature of load and the absence of the theoretically expected detonation combustion in engines supplied with such fuels as Diesel Oil, Marine Diesel Oil, and Heavy Fuel Oil. A model is proposed for fuel combustion in an arbitrary work space of a marine Diesel engine, which has the form of a stochastic four-state process, discrete in states and continuous in time. The model is based on the theory of semi-Markov processes.

It is shown that relay-races as a form of concurrency, are widespread in such fields of human activity, as industry, defence, economics, politics, etc. The analytical description of J stages relay-races, in which participate M teams, is considered. The model based on the semi-Markov process theory allows investigating a process evolution in detail but is of little use for computer calculation of forfeit, which the winner team receives from loser teams. To adapt analytical description to computer calculations, the method based on sampling of continual time densities of passing stages is proposed. Due to the method, initial semi-Markov process is converted to semi-Markov process with degenerative distributions, and the task of forfeit calculation is reduced to the task of rigid schedules effectiveness analysis, in which schedules are selected stochastically from the densities samples. Formulae for the stochastic summation of forfeits, is obtained. The results may be used for optimal planning the activity of participant teams when passing the distance.

American Option Pricing with Discrete and Continuous Time Models: An Empirical Comparison

For several specific queueing models with a vacation policy, the stationary system occupancy at the beginning of a random slot is distributed as the sum of two independent random variables. One of these variables is the stationary number of customers in an equivalent queueing system with no vacations. For models in continuous time with Poissonian arrivals, this result is well-known, and referred to as stochastic decomposition, with proof provided by Fuhrmann and Cooper. For models in discrete time, this result received less attention, with no proof available to date. In this paper, we first establish a proof of the decomposition result in discrete time. When compared to the proof in continuous time, conditions for the proof in discrete time are somewhat more general. Second, we explore four different examples: non-preemptive priority systems, slot-bound priority systems, polling systems, and fiber delay line (FDL) buffer systems. The first two examples are known results from literature that are given here as an illustration. The third is a new example, and the last one (FDL buffer systems) shows new results. It is shown that in some cases the queueing analysis can be considerably simplified using this decomposition property.

According to the characteristics of Ad Hoc networks, distributed intrusion response mobile firewall system is designed based on mobile Agent. Surveillance agent monitors neighbors communication behavior, and reports to the control agent. Control agent is in charge of judging, and activates tracing agent and blocking agent. Defensive agent is set up in system to keep the firewall mobile features. Depending on semi-Markov process (SMP) theory, we have created a dynamic behavior model of the mobile firewall system and analyze the performance of network security.

When animals have to make a number of decisions during a limited time interval, they face a fundamental problem: how much time they should spend on each decision in order to achieve the maximum possible total outcome. Deliberating more on one decision usually leads to more outcome but less time will remain for other decisions. In the framework of sequential sampling models, the question is how animals learn to set their decision threshold such that the total expected outcome achieved during a limited time is maximized. The aim of this paper is to provide a theoretical framework for answering this question. To this end, we consider an experimental design in which each trial can come from one of the several possible “conditions.” A condition specifies the difficulty of the trial, the reward, the penalty and so on. We show that to maximize the expected reward during a limited time, the subject should set a separate value of decision threshold for each condition. We propose a model of learning the optimal value of decision thresholds based on the theory of semi-Markov decision processes (SMDP). In our model, the experimental environment is modeled as an SMDP with each “condition” being a “state” and the value of decision thresholds being the “actions” taken in those states. The problem of finding the optimal decision thresholds then is cast as the stochastic optimal control problem of taking actions in each state in the corresponding SMDP such that the average reward rate is maximized. Our model utilizes a biologically plausible learning algorithm to solve this problem. The simulation results show that at the beginning of learning the model choses high values of decision threshold which lead to sub-optimal performance. With experience, however, the model learns to lower the value of decision thresholds till finally it finds the optimal values.

In this paper, we model a discrete-time renewal input queue with change over times and Bernoulli schedule vacation interruption under batch service (a, c, b) policy. The service, working vacation and change over times are geometrically distributed. At the instants of a service completion, the vacation is interrupted and the server is resumed to a regular busy period with probability 1 − q if there are c or more customers in the system, or continues the vacation with probability q. Employing the supplementary variable and recursive techniques, we have derived the steady state queue length distributions at various epochs. Some performance measures and a cost model have been presented and an optimum service rate has been obtained using geneticalgorithm. Numerical results showing the effect of the parameters of the modelon the key performance measures are presented.

This paper considers a discrete-time \((s, S)\) inventory system with positive service and \(N\)-policy. The arrival of customers constitutes a Bernoulli process. The system will be on vacation up to \(N\) customers and it starts batch service of size \(N\) with geometrically distributed service time. The subsequent customers who arrive during the batch service period are served in a single with geometrically distributed service time. The maximum storage of inventory is \(s\). Whenever the on-hand inventory level drops to prefixed level \(s\), an order for replenishment is placed. Lead time is also geometrically distributed. The system is analysed and the stability condition is derived using Matrix Analytic Method. Busy period, waiting-time distribution, reorder time distribution and inter-replenishment time are obtained. Numerical experiments are also incorporated to study system variation of system parameters.

We derive explicit closed expressions for the moment generating functions of whole collections of quantities associated with the waiting time till the occurrence of composite events in either discrete or continuous-time models. The discrete-time models are independent, or Markov-dependent, binary trials and the events of interest are collections of successes with the property that each two consecutive successes are separated by no more than a fixed number of failures. The continuous-time models are renewal processes and the relevant events are clusters of points. We provide a unifying technology for treating both the discrete and continuous-time cases. This is based on first embedding the problems into similar ones for suitably selected Markov chains or Markov renewal processes, and second, applying tools from the exponential family technology.

The operation of digital systems are based on time slotting and it finds many applications particularly in communication networks. It is important to analyze a digital network on some digital concepts. In this paper we analyze the network buffer through discrete time queueing system in which queue is discrete in nature and it is used to analyze the performance of the communication networks buffer. Discrete time models can be used to represents such type of systems in terms of slotted time units to models any physical system. More than one state changes in discrete time models from one time unit to another. Discrete time markov chains are used to describes the behavior of the network under the specific conditions and various performance measures are evaluated such as queue length, mean waiting time and mean number in the system.

An analytic model of a long used item functioning with regular period of check-out operations and unreliable checking conditions is represented. The model is based on the theory of semimarkov processes using matrix methods for mathematical operations. Consideration of the model reveals the influence of the first and the second kind check-out errors on the reliability level.