Markov Chains with Absorbing States: A Genetic Example

Unified theory for finite Markov chains

The graph transformation (GT) algorithm robustly computes the mean first-passage time to an absorbing state in a finite Markov chain. Here we present a concise overview of the iterative and block formulations of the GT procedure and generalize the GT formalism to the case of any path property that is a sum of contributions from individual transitions. In particular, we examine the path action, which directly relates to the path probability, and analyze the first-passage path ensemble for a model Markov chain that is metastable and therefore numerically challenging. We compare the mean first-passage path action, obtained using GT, with the full path action probability distribution simulated efficiently using kinetic path sampling, and with values for the highest-probability paths determined by the recursive enumeration algorithm (REA). In Markov chains representing realistic dynamical processes, the probability distributions of first-passage path properties are typically fat-tailed and therefore difficult to converge by sampling, which motivates the use of exact and numerically stable approaches to compute the expectation. We find that the kinetic relevance of the set of highest-probability paths depends strongly on the metastability of the Markov chain, and so the properties of the dominant first-passage paths may be unrepresentative of the global dynamics. Use of a global measure for edge costs in the REA, based on net productive fluxes, allows the total reactive flux to be decomposed into a finite set of contributions from simple flux paths. By considering transition flux paths, a detailed quantitative analysis of the relative importance of competing dynamical processes is possible even in the metastable regime.

Nowadays, standard proofs of the large deviation property (LDP) for i.i.d. processes are based on the `method of types' [4], [2]. However, for Markov chains, the proofs in standard texts are not based on the method of types. Instead they make use of more advanced concepts such as the Ga¿rtner-Ellis theorem (see e.g. [6]) or Varadhan's lemma (see e.g. [15]). There is indeed a proof of the LDP for finite state Markov chains based on the method of types [16], but it is not wellknown nor cited in [6]. The principal objective of this paper is therefore to present a first-principles derivation of the LDP for finite state Markov chains, using only simple combinatorial arguments (e.g. the method of types). The approach presented here extends naturally to multi-step Markov chains. We also relate the LDP rate function of a Markov chain to that of its time-reversed version.

For a finite Markov chain, by rotating the transition matrix by 180°, or relabelling the states, one can define a new Markov chain. This Markov chain in fact is the imbedded Markov chain of an inverse process. Duality properties about this Markov chain sometimes are not difficult to obtain. Similarly, one can discuss an infinite Markov chain with states 0,±1,±2,…, However, many applications involve transition matrices with various boundary modifications where rotation cannot directly apply. After introducing some duality properties for a boundary-free or finite Markov chain, we will mainly focus on some interesting application problems and show how to use the duality from rotation to these problems.

The known approaches to project management are typically based on the intuitive decisions by project managers, not on theoretical foundations, patterns and laws of project management. In order to solve this contradiction, it is necessary to conduct the generalization and develop applied aspects for using the Markov chains to represent and model the weakly structured project management systems. To construct the Markov chain, we used a modified system of project management, which is represented in the standard of project management and defines the scheme of interaction between project participants. A method for the transformation of this scheme into a homogeneous Markov chain with discrete states and time is developed. It is demonstrated that iteration solution of the system of equations of the Markov model makes it possible to construct a trajectory of development of virtual or actual project systems. We examined practical aspects in measuring the effectiveness of trajectory of project development by using the devised Markov model. An evaluation of results of changing the probability distribution of project states in the coordinates of probabilities of the system states by steps revealed a significant effect on the progress and efficiency of a project only in the case of variation of terms of cooperation within a project team.

The problem of random number generation from an uncorrelated random source (of unknown probability distribution) dates back to von Neumann's 1951 work. Elias (1972) generalized von Neumann's scheme and showed how to achieve optimal efficiency in unbiased random bits generation. Hence, a natural question is what if the sources are correlated? Both Elias and Samuelson proposed methods for generating unbiased random bits in the case of correlated sources (of unknown probability distribution), specifically, they considered finite Markov chains. However, their proposed methods are not efficient or have implementation difficulties. Blum (1986) devised an algorithm for efficiently generating random bits from degree-2 finite Markov chains in expected linear time, however, his beautiful method is still far from optimality on information-efficiency. In this paper, we generalize Blum's algorithm to arbitrary degree finite Markov chains and combine it with Elias's method for efficient generation of unbiased bits. As a result, we provide the first known algorithm that generates unbiased random bits from an arbitrary finite Markov chain, operates in expected linear time and achieves the information-theoretic upper bound on efficiency.

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

Markovian processes in aneuploids: I. Diploid populations

The paper deals with the problem of mathematical modeling of thermochemical destruction process. The apparatus of Markov's chains is used to synthesize a mathematical model. The authors of the study suggest to consider the destruction process as a random one, where the system state changes, which is characterized by the proportion of macromolecules in each fraction of the molecular- and weight distribution. The intensities of transitions from one state to another characterize the corresponding rates of destruction processes for each fraction of the molecular- and weight distribution. The processes of crosslinking and polymerization in this work were neglected, and it was accepted that there is a probability of transition from any state with a lower order index (corresponding to fractions with higher molecular weights) to any state with a higher index (corresponding to fractions with lower molecular weights). Markov's chain with discrete states and continuous time was taken as the mathematical model basis. Interactive graphical simulation environment MathWorksSimulink was used as a simulation environment. Experimental studies of polybutadiene destruction in solution were carried out to evaluate the mathematical model parameters. The GPC (gel-penetration chromatography) data of the polybutadiene solution were used as the initial (starting) data for estimating the polymer WMD (molecular weight distribution). Mean-square deviation of the calculated data from the experimental data for each fraction and at specified times was minimized for the numerical search of parameter values. The results of comparison of experimental and calculated on mathematical model data showed an error of calculations on the average about 5%, which indicates an acceptable error in estimating of polymer fractions proportions change during the process of destruction for the process under consideration and conditions.

Introduction. The development of methods for modeling the flows of events in the occurrence and development of emergency and critical states, designed to solve the problems of supporting the management of rapid response to incidents in conditions of incomplete information, as well as the generation of preventive algorithms for rapid response in emergency situations, seems to be a promising direction for improving information and analytical support of management in security systems. The purpose of the research is to create mathematical tools for predicting emergency and critical conditions in local incidents based on discrete-event modeling of the dynamics of emergency situations. Methods of stochastic modeling are based on the representation of a cascade scenario of the dynamics of the development of accidents and catastrophes by a Markov process using a three-parameter Weibull distribution to set the intensities of transitions between emergency and critical states of adjacent levels. Results and discussion. A general stochastic model is constructed that describes the dynamics of the occurrence and development of accidents and catastrophes of technogenic and natural origin initiated by local incidents according to a cascade scenario with a branching structure with a set of possible elementary states represented in the form of a stratified graph. The main stages of the emergence and development of emergency situations are considered within the framework of the proposed representation of the cascade model by a Markov process with discrete states and continuous time. The expediency of local modeling of individual stages is substantiated based on the description of transitions between states of adjacent levels. Local systems of differential equations corresponding to some of the most probable scenarios for the development of events in the event of incidents are given. Non-stationary solutions of differential equations of Kolmogorov-Chapman are obtained, with assignment of intensities of transitions by three-parameter Weibull distributions. Conclusions. The proposed stochastic model makes it possible to predict the dynamics of the development of emergency and critical states according to a cascade scenario, and also reflects the probability characteristics of the timeliness and effectiveness of possible impacts in a prompt response to incidents. Keywords: stochastic modeling of technogenic accidents, cascade development of accidents and catastrophes, Weibull distribution.

The Strong Chebyshev Distribution and Orthogonal Laurent Polynomials

We consider ergodic backward stochastic differential equations in a discrete time setting, where noise is generated by a finite state Markov chain. We show existence and uniqueness of solutions, along with a comparison theorem. To obtain this result, we use a Nummelin splitting argument to obtain ergodicity estimates for a discrete time Markov chain which hold uniformly under suitable perturbations of its transition matrix. We conclude with an application of this theory to a treatment of an ergodic control problem.

Given a finite Markov chain, we investigate the first minors of the transition matrix of a lifting of this Markov chain to covering trees. In a simple case we exhibit a nice factorisation of these minors, and we conjecture that it holds more generally.

This paper describes a Bayesian approach to determining the order of a finite state Markov chain whose transition probabilities are themselves governed by a homogeneous finite state Markov chain. It extends previous work on homogeneous Markov chains to more general and applicable hidden Markov models. The method we describe uses a Markov chain Monte Carlo algorithm to obtain samples from the (posterior) distribution for both the order of Markov dependence in the observed sequence and the other governing model parameters. These samples allow coherent inferences to be made straightforwardly in contrast to those which use information criteria. The methods are illustrated by their application to both simulated and real data sets.

This paper presents an efficient method for state classification of finite state Markov chains using BDD-based symbolic techniques. The method exploits the fundamental properties of a Markov chain and classifies the state space by iteratively applying reachability analysis. We compare our method with the current state-of-the-art technique which requires the computation of the transitive closure of the transition relation of a Markov chain. Experiments in over a dozen synchronous and asynchronous systems demonstrate that our method dramatically reduces the CPU time needed, and solves much larger problems because of reduced memory requiremen ts.