A partition method for bounding continuous-time Markov chain models of general reaction network.

We present in this paper a mathematical method based on stochastic comparisons of Markov chains in order to compute in a mobile network performance measures that are important Q parameters for users: the dropping handover of voice and the blocking probability of a new voice call. The key idea of this methodology is that given a complex system represented by a multi-dimensional Markov chain which is too large to be solved, we propose to reduce the state space, and so to define a new Markov chain which is a simplified version of the original one. This reduced Markov chain is defined as an aggregated one, which represents a stochastic bound for performance measures written as increasing reward function on the stationary distribution. The main steps of the construction of the aggregated Markov chain applied into mobile networks are presented in this paper. As the number of mobile users with different kinds of applications increases, the associated model is more complex and it can be represented by multi-dimensional continuous time Markov chain with a very large size. Thus, we define an aggregated Markov chain represented by a multi-dimensional birth and death process which is very easy to solve. We have proved that there is a weak ordering between the original Markov chain and the aggregated one using increasing set formalism. We have computed upper bounds of dropping handover and blocking probability for different values of input parameters. Numerical results prove that upper bounds give good results and so the stochastic methodology is an interesting mathematical tool for the performance evaluation of complex systems. Keywords: Mobile Networks, Quality of Service, Stochastic ordering, stochastic comparisons, Continuous Time Markov Chains.

Ray–Knight compactification of birth and death processes

A reaction network is a chemical system involving multiple reactions and chemical species. The simplest stochastic models of such networks treat the system as a continuous time Markov chain with the state being the number of molecules of each species and with reactions modeled as possible transitions of the chain. This chapter is devoted to the mathematical study of such stochastic models. We begin by developing much of the mathematical machinery we need to describe the stochastic models we are most interested in. We show how one can represent counting processes of the type we need in terms of Poisson processes. This random time-change representation gives a stochastic equation for continuous-time Markov chain models. We include a discussion on the relationship between this stochastic equation and the corresponding martingale problem and Kolmogorov forward (master) equation. Next, we exploit the representation of the stochastic equation for chemical reaction networks and, under what we will refer to as the classical scaling, show how to derive the deterministic law of mass action from the Markov chain model. We also review the diffusion, or Langevin, approximation, include a discussion of first order reaction networks, and present a large class of networks, those that are weakly reversible and have a deficiency of zero, that induce product-form stationary distributions. Finally, we discuss models in which the numbers of molecules and/or the reaction rate constants of the system vary over several orders of magnitude. We show that one consequence of this wide variation in scales is that different subsystems may evolve on different time scales and this time-scale variation can be exploited to identify reduced models that capture the behavior of parts of the system. We will discuss systematic ways of identifying the different time scales and deriving the reduced models.

Stochastic effects play an important role in modeling the time evolution of chemical reaction systems in fields such as systems biology, where the concentrations of some constituent molecules can be low. The most common stochastic models for these systems are continuous time Markov chains, which track the molecular abundance of each chemical species. Often, these stochastic models are studied by computer simulations, which can quickly become computationally expensive. A common approach to reduce computational effort is to approximate the discrete valued Markov chain by a continuous valued diffusion process. However, existing diffusion approximations either do not respect the constraint that chemical concentrations are never negative (linear noise approximation) or are typically only valid until the concentration of some chemical species first becomes zero (chemical Langevin equation). In this paper, we propose (obliquely) reflected diffusions, which respect the nonnegativity of chemical concentrations, as approximations for Markov chain models of chemical reaction networks. These reflected diffusions satisfy “constrained Langevin equations,” in that they behave like solutions of chemical Langevin equations in the interior of the positive orthant and are constrained to the orthant by instantaneous oblique reflection at the boundary. To motivate their form, we first illustrate our constrained Langevin approximations for two simple examples. We then describe the general form of our proposed approximation. We illustrate the performance of our approximations through comparison of their stationary distributions for the two examples with those of the Markov chain model and through simulations of more complex examples.

We consider the susceptible-infective (SI) epidemiological model, a variant of the Kermack-McKendrick models, and let the contact rate be a function of the number of infectives, an indicator of disease spread during the course of the epidemic. We represent the resultant model as a continuous-time Markov chain. The result is a pure death (or birth) process with state-dependent rates, for which we find the probability distribution of the associated Markov chain by solving the Kolmogorov forward equations. This model is used to find the analytic solution of the SI model as well as the distribution of the epidemic duration. We use the maximum likelihood method to estimate contact rates based on observations of inter-infection time intervals. We compare the stochastic model to the corresponding deterministic models through a numerical experiment within a typical household. We also incorporate different intervention policies for vaccination, antiviral prophylaxis, isolation, and treatment considering both full and partial adherence to interventions among individuals.

We prove a limit theorem for a process in a random one-dimensional medium, which has been considered before as a model for hopping conduction in a disordered medium. To the edge between the two integersj and (j+ 1) a rate λj > 0 is attached. Theseλ j :j integral are taken as independent, identically distributed random variables, and represent the medium. For given values λj, X(t) is a Markov chain in continuous time which jumps fromj to (j + 1) and from (j + 1) toj at the same rate λj. We show that in many cases there exists normalizing constants y(t) (which tend to oo witht) such that the distribution of X(t)/γ(t), or more generally of the whole processX(st)/γ(t) S⩾0, converges to a limit as t→ ∞. The limit process is continuous and self-similar.

The presence of a pathogen among multiple competing species has important ecological implications. For example, a pathogen may change the competitive outcome, resulting in replacement of a native species by a non-native species. Alternately, if a pathogen becomes established, there may be a drastic reduction in species numbers. Stochastic variability in the birth, death and pathogen transmission processes plays an important role in determining the success of species or pathogen invasion. We investigate these phenomena while studying the dynamics of deterministic and stochastic models for n competing species with a shared pathogen. The deterministic model is a system of ordinary differential equations for n competing species in which a single shared pathogen is transmitted among the n species. There is no immunity from infection, individuals either die or recover and become immediately susceptible, an SIS disease model. Analytical results about pathogen persistence or extinction are summarized for the deterministic model for two and three species and new results about stability of the infection-free state and invasion by one species of a system of n-1 species are obtained. New stochastic models are derived in the form of continuous-time Markov chains and stochastic differential equations. Branching process theory is applied to the continuous-time Markov chain model to estimate probabilities for pathogen extinction or species invasion. Finally, numerical simulations are conducted to explore the effect of disease on two-species competition, to illustrate some of the analytical results and to highlight some of the differences in the stochastic and deterministic models.

In molecular programming, the Chemical Reaction Network model is often used to describe systems of interacting molecules. This model can describe either real systems, allowing us to analyze and determine their computational function; or describe hypothetical systems, with known computational function but perhaps no known physical example. One significant breakthrough in the field is that any Chemical Reaction Network can be approximated by a system using DNA Strand Displacement mechanisms. This allows the Chemical Reaction Network model to be treated like a programming language, where programs can be written in the abstract and then compiled into physical molecules. Given a programming language and a proof-of-concept compiler, one would want to take the compiler from the proof-of-concept stage into a more reliable, more systematic, and better understood process. This thesis is made up of my contributions to that effort. First, given a programming language and a compiler, it would be useful to formally verify that the compiler is correct. My collaborators, Qing Dong and Erik Winfree, and I defined a Chemical Reaction Network-specific form of bisimulation equivalence, which can compare two such networks and verify that one is (or is not) a correct implementation of the other. For example, the compiler-produced DNA circuit can be verified as an implementation of its abstract program, although this is not the only possible use. After defining this concept of equivalence, we show that it can be checked by algorithm; although various parts of the problem are NP-complete or PSPACE-complete, we give algorithms that meet these lower bounds. We also prove a number of interesting properties of Chemical Reaction Network bisimulation equivalence, including transitivity and modularity properties which are particularly useful for stepwise checking of large systems. We further extend this bisimulation method to linear Polymer Reaction Networks, a strictly more powerful abstraction which has been occasionally used in molecular programming. Again we prove complexity hardness results, which in this case are as expected uncomputable in the general case; however, many practical systems can still be verified, and we give one such example. Finally, we use bisimulation to identify a class of single-locus networks that are practical to implement. Thus we show a method of verification which can simplify use of the above-mentioned compiler by proving general statements of correctness about its results. Second, given a programming language and a concept of compiling it, it would be useful to optimize the result of the compilation. One particular area of optimization is the number of DNA strands per prepared complex; some experiments suggest that systems with no more than 2 strands per complex are more robust. Lulu Qian and I developed some proposed DNA Strand Displacement schemes for general Chemical Reaction Network implementations with no more than 2 strands per complex, and a number of other desirable properties. Meanwhile, having been shown to be useful for many reasons, the mechanisms of DNA Strand Displacement have recently been formalized, abstracted, and analyzed. I show that this formalization, combined with the bisimulation methods above, can prove various statements about the limits of DNA Strand Displacement systems. For example, a set of desirable conditions including the 2-strand limit cannot be achieved by any general Chemical Reaction Network implementation scheme. I also observe that two of the new schemes we discovered, each meeting all but one condition of the impossible set, were found in the process of coming up with this proof. I thus argue that through formalization of DNA Strand Displacement we can have a more systematic method of finding and designing molecular programs, and of knowing when the programs we want do not exist.

Continuous Time Markov Chains (CTMC) are a mathematical tool widely used in many modeling areas. Main drawback in using CTMCs is the difficulty in building and managing models with a high number of states. In [3] a new modeling technique based on Markovian Agents has been proposed to study Wireless Sensor Networks with a large number of interacting nodes. A Markovian Agent is a CTMC with the ability to interact with other Markovian Agents by sending and perceiving messages. Thanks to message mechanisms, it is possible to limit the growth of the overall state space. In this paper, we introduce a software to implement Markovian Agent models in a simple way. Software functionalities are provided through a C library we named MAGNET (Markovian AGent NETworks). It provides a simple interface to easily build complex interacting MA models and a numerical algorithm to study them in transient exploiting multi-threading.

Tipping points have been shown to be ubiquitous, both in models and empirically in a range of physical and biological systems. The question of how tipping points cascade through systems has been less explored and is an important one. A study of noise-induced tipping, in particular, could provide key insights into tipping cascades. Here, we consider a specific example of a simple model system that could have cascading tipping points. This model consists of two interacting populations with underlying Allee effects and stochastic dynamics, in separate patches connected by dispersal, which can generate bistability. From an ecological standpoint, we look for rescue effects whereby one population can prevent the collapse of a second population. As a way to investigate the stochastic dynamics, we use an individual-based modeling approach rooted in chemical reaction network theory. Then, using continuous-time Markov chains and the theory of first passage times, we essentially approximate, or emulate, the original high-dimensional model by a Markov chain with just three states, where each state corresponds to a combination of population thresholds. Analysis of this reduced model shows when the system is likely to recover, as well as when tipping cascades through the whole system.

Content caching is a promising way to overcome backhaul limitations in small cell networks. However, in such type of networks, small base stations (SBSs) are always deployed with limited cache storages. Thus, it is necessary for SBSs to adjust their contents for better caching efficiency, so as to reduce backhaul traffic. In this paper, we study the content replacement problem to minimize the traffic flowing into the costly backhaul links. However, in small cell networks where SBSs make up backhaul mesh networks, the effectiveness of reducing backhaul traffic depends on the hop distance from the content location to the requesting user. On this basis, we formulate a hop minimization problem that is inherently combinatorial. Through log-sum-exp approximation, we can solve the problem and arrive at a close-form solution with guaranteed performance gap to the optimal solution. By exploiting the properties of continuous-time Markov chain (CTMC), the solution can be implemented by designing a CTMC that can instruct the content replacement process. As a consequence, a concise, efficient, and flexible content replacement strategy is proposed. Simulation results verify our analysis and show that our proposed strategy outperforms the conventional strategies.

The state 0 of a birth and death process with state space E = {0, 1, 2, … } is a barrier which can be classified into four kinds: reflection, absorption, leaping reflection, quasi-leaping reflection. For the first, second and fourth barriers, the characteristic numbers of different forms have been introduced. In this paper unified characteristic numbers for birth and death processes with barriers were introduced, the related equations were solved and the solutions were expressed by unified characteristic numbers. This paper concerns work solving probability construction problem of birth and death processes with leaping reflection barrier and quasi-leaping reflection barrier.

Stochastic chemical reaction networks (CRNs) are complex systems that combine the features of concurrent transformation of multiple variables in each elementary reaction event and nonlinear relations between states and their rates of change. Most general results concerning CRNs are limited to restricted cases where a topological characteristic known as deficiency takes a value 0 or 1, implying uniqueness and positivity of steady states and surprising, low-information forms for their associated probability distributions. Here we derive equations of motion for fluctuation moments at all orders for stochastic CRNs at general deficiency. We show, for the standard base case of proportional sampling without replacement (which underlies the mass-action rate law), that the generator of the stochastic process acts on the hierarchy of factorial moments with a finite representation. Whereas simulation of high-order moments for many-particle systems is costly, this representation reduces the solution of moment hierarchies to a complexity comparable to solving a heat equation. At steady states, moment hierarchies for finite CRNs interpolate between low-order and high-order scaling regimes, which may be approximated separately by distributions similar to those for deficiency-zero networks and connected through matched asymptotic expansions. In CRNs with multiple stable or metastable steady states, boundedness of high-order moments provides the starting condition for recursive solution downward to low-order moments, reversing the order usually used to solve moment hierarchies. A basis for a subset of network flows defined by having the same mean-regressing property as the flows in deficiency-zero networks gives the leading contribution to low-order moments in CRNs at general deficiency, in a 1/n expansion in large particle numbers. Our results give a physical picture of the different informational roles of mean-regressing and non-mean-regressing flows and clarify the dynamical meaning of deficiency not only for first-moment conditions but for all orders in fluctuations.

A novel condition-based maintenance framework for parallel manufacturing systems based on bivariate birth/birth–death processes

Continuous Time Markov chains (CTMCs) are widely used to model and analyze networked systems. A common analysis approach is to solve the system of balance equations governing the state transitions of a CTMC to obtain its steadystate probability distribution, and use the state probabilities to derive or compute various performance measures. In many systems, the state space of the underlying CTMC is infinite and multi-dimensional with state-dependent transitions; exact analysis of such models is challenging. For example, the exact probability distribution of the number of jobs in the Discriminatory Processor Sharing (DPS) system, first proposed by Kleinrock in 1967 [4], is still an open challenge. Likewise, obtaining the exact state probabilities of quasi-birth-and-death (QBD) processes with leveldependent transitions is known to be challenging [1]; QBDs are infinite state space multi-dimensional Markov chains in which states are organized into levels and transitions are skip-free between the levels.