Semi-Markov Processes: Applications in System Reliability and Maintenance

A fundamental identity, due to Miller (1961a), (1962a, b) and Kemperman (1961), is generalized to semi-Markov processes. Thus the identity applies to processes defined on a Markov chain with discrete state space and random walks with Markov dependent steps (Section 2). Wald's identity is discussed briefly in Section 3. Section 4 is a study of the maxima of partial sums, and Section 5 of maxima in a semi-Markov process.

The article provides a theoretical description of the processes of changing the technical state of road-building machines to assess their reliability at the stage of operation based on semi-Markov models. Reliability is the most important property of machines and characterizes their ability to perform their functions during normal operation. Reliability covers all stages of the life cycle of equipment-its design, manufacture, and operation. The operation stage is the longest in time and the costliest to ensure reliability. This explains the relevance of developing a mathematical model of operational reliability of road-building machines. During the operation of technical objects, their technical condition changes under the influence of objective and subjective factors. As a result, quantitative indicators that characterize the technical condition of machines vary widely and are random in nature. Road-building machines may be in one of the following technical states during operation: perfect, imperfect, upstate, down state. At the same time, a perfect machine is always upstate, and an imperfect one can be either upstate or down state. The transition from one technical state to another is random and can be considered instantaneous with a sufficient degree of accuracy. The probability of any technical state in the future depends only on the state of the process in the present and does not depend on when and how the process was in this state. This makes it possible to use the theory of semi-Markov processes when formalizing the process and developing a mathematical model. Based on semi-Markov modeling, the article shows the sequence of constructing a graph of states and transitions from one state to another for a road construction machine during operation; matrices of transitions and probabilities of transitions from one state to another; systems of differential equations for determining the limiting probabilities of finding a machine in a particular technical state during operation. The developed method of semi-Markov modeling allows estimating the operational reliability of equipment with the reliability allowed by probability theory.

Passage-time computation and aggregation strategies for large semi-Markov processes

A DNN-based reliability evaluation method for multi-state series-parallel systems considering semi-Markov process

SUMMARY In this paper we set forth an approach to queueing and inventory problems which is based on the theory of semi-Markov processes. The method is applicable to a wide variety of models in continuous or discrete time. After developing two basic theorems, we illustrate the method by determining the limiting distribution of a “batch service” queue, in which K customers are always served at a time. The batch service queue is also shown to be equivalent to a classical single-server queue with gamma interarrival distribution and general service distribution, and to an inventory model of the popular “s, S” type. Waiting-time distributions and expected lengths of busy periods are shown for the queues, and the application of the method to finite queues is indicated.

IntroductionEpidemiological models benefit from incorporating detailed time-to-event data to understand how disease risk evolves. For example, decompensation risk in liver cirrhosis depends on sojourn time spent with cirrhosis. Semi-Markov and related models capture these details by modeling time-to-event distributions based on published survival data. However, implementations of semi-Markov processes rely on Monte Carlo sampling methods, which increase computational requirements and introduce stochastic variability. Explicitly calculating the evolving transition likelihood can avoid these issues and provide fast, reliable estimates.MethodsWe present the sojourn time density framework for computing semi-Markov and related models by calculating the evolving sojourn time probability density as a system of partial differential equations. The framework is parametrized by commonly used hazard and models the distribution of current disease state and sojourn time. We describe the mathematical background, a numerical method for computation, and an example model of liver disease.ResultsModels developed with the sojourn time density framework can directly incorporate time-to-event data and serial events in a deterministic system. This increases the level of potential model detail over Markov-type models, improves parameter identifiability, and reduces computational burden and stochastic uncertainty compared with Monte Carlo methods. The example model of liver disease was able to accurately reproduce targets without extensive calibration or fitting and required minimal computational burden.ConclusionsExplicitly modeling sojourn time distribution allows us to represent semi-Markov systems using detailed survival data from epidemiological studies without requiring sampling, avoiding the need for calibration, reducing computational time, and allowing for more robust probabilistic sensitivity analyses.HighlightsTime-inhomogeneous semi-Markov models and other time-to-event–based modeling approaches can capture risks that evolve over time spent with a disease.We describe an approach to computing these models that represents them as partial differential equations representing the evolution of the sojourn time probability density.This sojourn time density framework incorporates complex data sources on competing risks and serial events while minimizing computational complexity.

It is shown that relay-races as a form of concurrency, are widespread in such fields of human activity, as industry, defence, economics, politics, etc. The analytical description of J stages relay-races, in which participate M teams, is considered. The model based on the semi-Markov process theory allows investigating a process evolution in detail but is of little use for computer calculation of forfeit, which the winner team receives from loser teams. To adapt analytical description to computer calculations, the method based on sampling of continual time densities of passing stages is proposed. Due to the method, initial semi-Markov process is converted to semi-Markov process with degenerative distributions, and the task of forfeit calculation is reduced to the task of rigid schedules effectiveness analysis, in which schedules are selected stochastically from the densities samples. Formulae for the stochastic summation of forfeits, is obtained. The results may be used for optimal planning the activity of participant teams when passing the distance.

According to the characteristics of Ad Hoc networks, distributed intrusion response mobile firewall system is designed based on mobile Agent. Surveillance agent monitors neighbors communication behavior, and reports to the control agent. Control agent is in charge of judging, and activates tracing agent and blocking agent. Defensive agent is set up in system to keep the firewall mobile features. Depending on semi-Markov process (SMP) theory, we have created a dynamic behavior model of the mobile firewall system and analyze the performance of network security.

This chapter concentrates primarily on the general aspects of occupancy times when studying the reliability of systems that are described by finite-state continuous-time Markov or semi-Markov processes. Occupancy times for Markov models for system reliability are discussed together with a brief comment on reward processes. Similar problems, focused on joint distributions and cumulative distributions, are considered for semi-Markov models. Similar distribution problems for time interval omission situations are given briefly in the chapter. Some numerical examples are presented to illustrate the results obtained. The chapter also presents some general results for certain probability distributions of occupancy times in terms of Laplace-Stieltjes transforms. The advantages are that the results are elegant and moments can usually be easily obtained by differentiation. The chapter points out the similarity of some Markov and semi-Markov models for reliability and ion-channel theory. Controlled Vocabulary Terms laplace transforms

Performance characteristics and assessment of fire alarm system

6 Dependence notions in reliability theory

The article analyses the operation of reciprocal internal combustion engines, with marine engines used as an example. The analysis takes into account types of energy conversion in the work spaces (cylinders) of these engines, loads of their crankshaft-piston assemblies, and types of fuel combustion which can take place in these spaces during engine operation. It is highlighted that the analysed time-dependent loads of marine internal combustion engine crankshaft-piston assemblies are random processes. It is also indicated that the wear of elements of those assemblies resulting from their load should also be considered a random process. A hypothesis is formulated which explains random nature of load and the absence of the theoretically expected detonation combustion in engines supplied with such fuels as Diesel Oil, Marine Diesel Oil, and Heavy Fuel Oil. A model is proposed for fuel combustion in an arbitrary work space of a marine Diesel engine, which has the form of a stochastic four-state process, discrete in states and continuous in time. The model is based on the theory of semi-Markov processes.

When animals have to make a number of decisions during a limited time interval, they face a fundamental problem: how much time they should spend on each decision in order to achieve the maximum possible total outcome. Deliberating more on one decision usually leads to more outcome but less time will remain for other decisions. In the framework of sequential sampling models, the question is how animals learn to set their decision threshold such that the total expected outcome achieved during a limited time is maximized. The aim of this paper is to provide a theoretical framework for answering this question. To this end, we consider an experimental design in which each trial can come from one of the several possible “conditions.” A condition specifies the difficulty of the trial, the reward, the penalty and so on. We show that to maximize the expected reward during a limited time, the subject should set a separate value of decision threshold for each condition. We propose a model of learning the optimal value of decision thresholds based on the theory of semi-Markov decision processes (SMDP). In our model, the experimental environment is modeled as an SMDP with each “condition” being a “state” and the value of decision thresholds being the “actions” taken in those states. The problem of finding the optimal decision thresholds then is cast as the stochastic optimal control problem of taking actions in each state in the corresponding SMDP such that the average reward rate is maximized. Our model utilizes a biologically plausible learning algorithm to solve this problem. The simulation results show that at the beginning of learning the model choses high values of decision threshold which lead to sub-optimal performance. With experience, however, the model learns to lower the value of decision thresholds till finally it finds the optimal values.

This paper shows the span of results which can be obtained by modeling a system's behavior by stochastic processes and demonstrates practical rules for employing Markov and semi-Markov models. The introduction summarizes several methods for reliability analysis and gives the advantages and drawbacks of four methods: Markov processes, semi-Markov processes, supplementary variables, the method of stages. The remainder deals with reliability and availability modeling of a 2-unit redundant computer system. There are a) two types of maintenance: corrective (c. m.) and preventive (p. m.), and b) two system parameters: coverage, and an increased failure rate when one unit is under repair or inspection. Approximate expressions for reliability, mean time to failure, and asymptotic availability show the effects of the system parameters as well as of the shapes of the Cdf's of the times related to maintenance actions. For c.m., Markov modeling is a good approximation. For p.m., Markov modeling is a rough approximation; one can go to semi-Markov models or to the method of stages. Lastly, an approximate expression is given for the mean inspection interval which maximizes reliability and availability for p.m.

When combining several modules into a flight control software system, it is non-trivial to determine, prior to deployment, performance and reliability values of the software system. Moreover, once the system is deployed, it is often the case that during operation it fails to meet its performance and reliability requirement and one needs to detect what has gone wrong (i. e., performance/reliability bottlenecks). To study these issues, a Semi-Markov Process (SMP) formulation is proposed in this work for the reliability and performance of flight control software systems. By explicitly including failure states into the SMP representation of a module, we can compute both its performance and reliability using a single SMP. We can also detect its performance and reliability bottlenecks by applying the formal sensitivity analysis technique. We demonstrate our approach by choosing a representative flight control software system that is validated using experiments on real flight control computer.