Continuous Time Markov Chain Models for Chemical Reaction Networks

Stochastic effects play an important role in modeling the time evolution of chemical reaction systems in fields such as systems biology, where the concentrations of some constituent molecules can be low. The most common stochastic models for these systems are continuous time Markov chains, which track the molecular abundance of each chemical species. Often, these stochastic models are studied by computer simulations, which can quickly become computationally expensive. A common approach to reduce computational effort is to approximate the discrete valued Markov chain by a continuous valued diffusion process. However, existing diffusion approximations either do not respect the constraint that chemical concentrations are never negative (linear noise approximation) or are typically only valid until the concentration of some chemical species first becomes zero (chemical Langevin equation). In this paper, we propose (obliquely) reflected diffusions, which respect the nonnegativity of chemical concentrations, as approximations for Markov chain models of chemical reaction networks. These reflected diffusions satisfy “constrained Langevin equations,” in that they behave like solutions of chemical Langevin equations in the interior of the positive orthant and are constrained to the orthant by instantaneous oblique reflection at the boundary. To motivate their form, we first illustrate our constrained Langevin approximations for two simple examples. We then describe the general form of our proposed approximation. We illustrate the performance of our approximations through comparison of their stationary distributions for the two examples with those of the Markov chain model and through simulations of more complex examples.

In this thesis, a single serotype (j = 1) and patch (i = 1) ordinary difference equation (ODE) model is formulated and analysed for the effects of direct and transplacental transmission on the probability of bluetongue virus (BTV) persistence. Using the next generation approach, the basic reproduction number (R0) is determined. When R0 < 1, the model exhibits a backward bifurcation indicating that the virus persists. When R0 > 1, a continuous-time Markov chain (CTMC) model derived from the ODE model is used to estimate the probability of BTV persistence. By approximating the CTMC model with a multitype branching process, it is shown that both direct and transplacental transmission can have a large effect on the probability of persistence in regions with temperature T < 12◦C and a small effect for those with T > 12◦C. The ODE and CTMC models are extended to include r serotypes and n patches with the aim of determining the effects of midge movement on the outbreak and coexistence of multiple BTV serotypes in an environment divided into patches depending on the risk of infection. An estimate for the probability of a major outbreak of two BTV serotypes in two patches is obtained by approximating the CTMC model with a multitype branching process. It is shown that without movement a major outbreak occurs in the high-risk patch, but with cattle or midge movement it occurs in both patches. When a major outbreak occurs, numerical simulations of the ODE model illustrate possible coexistence in both patches if the patches are connected by midge or cattle movement. The multi-patch single-serotype ODE model is then modified as an optimal control problem to evaluate the effectiveness of vaccination, quarantine, insecticide spraying and the use of repellent control strategies in reducing the within- and between-patches transmission. By using optimal control theory, the effectiveness of these strategies is established. In a single patch, vaccination, insecticide spraying and the use of a repellent are all highly effective in minimising transmission, but the most costeffective is vaccination. In patches connected by host and midge movements, if any of these controls is applied in the high-risk patch, a disease-free status is achieved in both patches, but if implemented in the low-risk patch, it is not attained in any patch. If hosts and midges move, quarantine has no effect, but for no midge movement, the effect can be large in the low-risk patch if it’s internally imposed.

Analysis of deterministic and stochastic mathematical models with resistant bacteria and bacteria debris for bacteriophage dynamics

Aerial computing: Enhancing mobile cloud computing with unmanned aerial vehicles as data bridges—A Markov chain based dependability quantification

Continuous time Markov chain (CTMC) models offer ethologists a powerful tool. The methods are based on well‐established procedures for estimating the rates at which one state (e.g. resting) changes to some other set of states (e.g. feeding, fighting, etc.). Unfortunately, ethological data typically differ in a very critical manner from the type of data to which these methods are usually applied: ethological data are usually heavily censored in the sense that each behavioral state shows frequent transitions to several other possible states. This occurs when several competing processes can each end a bout.We used computer simulation of various behavioral models with known transition rates to investigate the unknown performance of four of the most popular statistical tests for screening data prior to application of CTMC models; this included a modification of one of these tests derived under the assumption of random censoring. Two of the four tests failed completely and would result in rejection of nearly all data even if the model did fit the assumptions of the CTMC methods. Only Barlow's total‐time‐on test performed with an acceptable α error rate under all conditions. None of the tests were particularly effective at detecting certain types of departures from the CTMC assumptions.Guidelines are given as to how much confidence should be attached to apparent changes in transition rates.

In this article, we develop and analyze a continuous-time Markov chain (CTMC) model to study the resurgence of dengue. We also explore the large population asymptotic behavior of probabilistic model of dengue using the law of large numbers (LLN). Initially, we calculate and estimate the probabilities of dengue extinction and major outbreak occurrence using multi-type Galton-Watson branching processes. Subsequently, we apply the LLN to examine the convergence of the stochastic model towards the deterministic model. Finally, theoretical numerical simulations are conducted exploration to validate our findings. Under identical conditions, our numerical results demonstrate that dengue could vanish in the stochastic model while persisting in the deterministic model. The highlighting of the law of large numbers through numerical simulations indicates from what population size a deterministic model should be considered preferable.

BackgroundWhile many species have suffered from the detrimental impacts of increasing human population growth, some species, such as cougars (Puma concolor), have been observed using human-modified landscapes. However, human-modified habitat can be a source of both increased risk and increased food availability, particularly for large carnivores. Assessing preferential use of the landscape is important for managing wildlife and can be particularly useful in transitional habitats, such as at the wildland-urban interface. Preferential use is often evaluated using resource selection functions (RSFs), which are focused on quantifying habitat preference using either a temporally static framework or researcher-defined temporal delineations. Many applications of RSFs do not incorporate time-varying landscape availability or temporally-varying behavior, which may mask conflict and avoidance behavior.MethodsContemporary approaches to incorporate landscape availability into the assessment of habitat selection include spatio-temporal point process models, step selection functions, and continuous-time Markov chain (CTMC) models; in contrast with the other methods, the CTMC model allows for explicit inference on animal movement in continuous-time. We used a hierarchical version of the CTMC framework to model speed and directionality of fine-scale movement by a population of cougars inhabiting the Front Range of Colorado, U.S.A., an area exhibiting rapid population growth and increased recreational use, as a function of individual variation and time-varying responses to landscape covariates.ResultsWe found evidence for individual- and daily temporal-variability in cougar response to landscape characteristics. Distance to nearest kill site emerged as the most important driver of movement at a population-level. We also detected seasonal differences in average response to elevation, heat loading, and distance to roads. Motility was also a function of amount of development, with cougars moving faster in developed areas than in undeveloped areas.ConclusionsThe time-varying framework allowed us to detect temporal variability that would be masked in a generalized linear model, and improved the within-sample predictive ability of the model. The high degree of individual variation suggests that, if agencies want to minimize human-wildlife conflict management options should be varied and flexible. However, due to the effect of recursive behavior on cougar movement, likely related to the location and timing of potential kill-sites, kill-site identification tools may be useful for identifying areas of potential conflict.

This review article is concerned to provide a global context to several works on the fitting of continuous time nonhomogeneous Markov chains with finite state space and also to point out some selected aspects of two techniques previously introduced—estimation and calibration—relevant for applications and used to fit a continuous time Markov chain model to data by the adequate selection of parameters. The denomination estimation suits the procedure better when statistical techniques—e.g., maximum likelihood estimators—are employed, while calibration covers the case where, for instance, some optimisation technique finds a best approximation parameter to ensure good model fitting. For completeness, we provide a short summary of well-known important notions and results formulated for nonhomogeneous Markov chains that, in general, can be transferred to the homogeneous case. Then, as an illustration for the homogeneous case, we present a selected Billingsley’s result on parameter estimation for irreducible chains with finite state space. In the nonhomogeneous case, we quote two recent results, one of the calibration type and the other with more of a statistical flavour. We provide an ample set of bibliographic references so that the reader wanting to pursue her/his studies will be able to do so more easily and productively.

We propose a stochastic logistic model with mate limitation and stochastic immigration. Incorporating stochastic immigration into a continuous time Markov chain model, we derive and analyze the associated master equation. By a standard result, there exists a unique globally stable positive stationary distribution. We show that such stationary distribution admits a bimodal profile which implies that a strong Allee effect exists in the stochastic model. Such strong Allee effect disappears and threshold phenomenon emerges as the total population size goes to infinity. Stochasticity vanishes and the model becomes deterministic as the total population size goes to infinity. This implies that there is only one possible fate (either to die out or survive) for a species constrained to a specific community and whether population eventually goes extinct or persists does not depend on initial population density but on a critical inherent constant determined by birth, death and mate limitation. Such a conclusion interprets differently from the classical ordinary differential equation model and thus a paradox on strong Allee effect occurs. Such paradox illustrates the diffusion theory's dilemma.

Two new stochastic epidemic models, a continuous-time Markov chain model and a stochastic differential equation model, are formulated. These are based on a deterministic model that includes vaccination and is applicable to pertussis. For some parameter values, the deterministic model exhibits a backward bifurcation if the vaccine is imperfect. Thus a region of bistability exists in a subset of parameter space. The dynamics of the stochastic epidemic models are investigated in this region of bistability, and compared with those of the deterministic model. In this region the probability distribution associated with the infective population exhibits bimodality with one mode at the disease-free equilibrium and the other at the larger endemic equilibrium. For population sizes N=/>1000, the deterministic and stochastic models agree, but for small population sizes the stochastic models indicate that the backward bifurcation may have little effect on the disease dynamics.

The aim of this paper is to introduce a new model of a financial asset prices distribution. It is known that the probability distribution of an asset prices or returns is unknown in reality. The general model of asset prices based on continuous time Markov chains is proposed. For this reason the interarrivals between two price states are approximated by mixture of exponential distributions. Numerical-analytic approach is used to obtain the probability distribution of asset prices. The developed software allows creating the space of an asset prices, the matrix of transition rates among states, a system of equations to find the steady state probabilities of price states and solves the system of equations by method of imbedded Markov chains.

Modeling of a reaction network and its optimization by genetic algorithm

Bayesian phylogeographic inference is widely used in molecular epidemiological studies to reconstruct the dispersal history of pathogens. Discrete phylogeographic analysis treats geographic locations as discrete traits and infers lineage transition events among them, and is typically followed by a Bayes factor (BF) test to assess the statistical support. In the standard BF (BFstd) test, the relative abundance of the involved trait states is not considered, which can be problematic in the case of unbalanced sampling. Existing methods to correct sampling bias in discrete phylogeographic analyses using continuous-time Markov chain (CTMC) model, often require additional epidemiological information to balance the sampling effort among locations. As such data is not necessarily available, alternative approaches that rely solely on available genomic data are needed. In this perspective, we assess the performance of a modification of the BFstd, the adjusted Bayes factor (BFadj), which incorporates information on the relative abundance of samples by location when inferring support for transition events and root location inference without requiring additional data. Using a simulation framework, we assess the statistical performance of BFstd and BFadj under varying levels of sampling bias, estimating their type I and type II error rates. Our results show that BFadj complements the BFstd by reducing type I errors at the cost increasing type II errors for inferred transition events, while improving type I and type II errors in root location inference. Our findings provide guidelines for implementing the complementary BFadj to detect and mitigate sampling bias in discrete phylogeographic inference using CTMC modeling.

In molecular programming, the Chemical Reaction Network model is often used to describe systems of interacting molecules. This model can describe either real systems, allowing us to analyze and determine their computational function; or describe hypothetical systems, with known computational function but perhaps no known physical example. One significant breakthrough in the field is that any Chemical Reaction Network can be approximated by a system using DNA Strand Displacement mechanisms. This allows the Chemical Reaction Network model to be treated like a programming language, where programs can be written in the abstract and then compiled into physical molecules. Given a programming language and a proof-of-concept compiler, one would want to take the compiler from the proof-of-concept stage into a more reliable, more systematic, and better understood process. This thesis is made up of my contributions to that effort. First, given a programming language and a compiler, it would be useful to formally verify that the compiler is correct. My collaborators, Qing Dong and Erik Winfree, and I defined a Chemical Reaction Network-specific form of bisimulation equivalence, which can compare two such networks and verify that one is (or is not) a correct implementation of the other. For example, the compiler-produced DNA circuit can be verified as an implementation of its abstract program, although this is not the only possible use. After defining this concept of equivalence, we show that it can be checked by algorithm; although various parts of the problem are NP-complete or PSPACE-complete, we give algorithms that meet these lower bounds. We also prove a number of interesting properties of Chemical Reaction Network bisimulation equivalence, including transitivity and modularity properties which are particularly useful for stepwise checking of large systems. We further extend this bisimulation method to linear Polymer Reaction Networks, a strictly more powerful abstraction which has been occasionally used in molecular programming. Again we prove complexity hardness results, which in this case are as expected uncomputable in the general case; however, many practical systems can still be verified, and we give one such example. Finally, we use bisimulation to identify a class of single-locus networks that are practical to implement. Thus we show a method of verification which can simplify use of the above-mentioned compiler by proving general statements of correctness about its results. Second, given a programming language and a concept of compiling it, it would be useful to optimize the result of the compilation. One particular area of optimization is the number of DNA strands per prepared complex; some experiments suggest that systems with no more than 2 strands per complex are more robust. Lulu Qian and I developed some proposed DNA Strand Displacement schemes for general Chemical Reaction Network implementations with no more than 2 strands per complex, and a number of other desirable properties. Meanwhile, having been shown to be useful for many reasons, the mechanisms of DNA Strand Displacement have recently been formalized, abstracted, and analyzed. I show that this formalization, combined with the bisimulation methods above, can prove various statements about the limits of DNA Strand Displacement systems. For example, a set of desirable conditions including the 2-strand limit cannot be achieved by any general Chemical Reaction Network implementation scheme. I also observe that two of the new schemes we discovered, each meeting all but one condition of the impossible set, were found in the process of coming up with this proof. I thus argue that through formalization of DNA Strand Displacement we can have a more systematic method of finding and designing molecular programs, and of knowing when the programs we want do not exist.

A partition method for bounding continuous-time Markov chain models of general reaction network.