Evolving sets, mixing and heat kernel bounds

Here we develop an effective approach to simplify two-time-scale Markov chains with infinite state spaces by removal of states with fast leaving rates, which improves the simplification method of finite Markov chains. We introduce the concept of fast transition paths and show that the effective transitions of the reduced chain can be represented as the superposition of the direct transitions and the indirect transitions via all the fast transition paths. Furthermore, we apply our simplification approach to the standard Markov model of single-cell stochastic gene expression and provide a mathematical theory of random gene expression bursts. We give the precise mathematical conditions for the bursting kinetics of both mRNAs and proteins. It turns out that random bursts exactly correspond to the fast transition paths of the Markov model. This helps us gain a better understanding of the physics behind the bursting kinetics as an emergent behavior from the fundamental multiscale biochemical reaction kinetics of stochastic gene expression.

In this paper, we develop a new "robust mixing" framework for reasoning about adversarially modified Markov Chains (AMMC). Let $\mathbb{P}$ be the transition matrix of an irreducible Markov Chain with stationary distribution $\pi$. An adversary announces a sequence of stochastic matrices $\{\mathbb{A}_t\}_{t > 0}$ satisfying $\pi\mathbb{A}_t = \pi$. An AMMC process involves an application of $\mathbb{P}$ followed by $\mathbb{A}_t$ at time $t$. The robust mixing time of an ergodic Markov Chain $\mathbb{P}$ is the supremum over all adversarial strategies of the mixing time of the corresponding AMMC process. Applications include estimating the mixing times for certain non-Markovian processes and for reversible liftings of Markov Chains. Non-Markovian card shuffling processes: The random-to-cyclic transposition process is a non-Markovian card shuffling process, which at time $t$, exchanges the card at position $L_t := t {\pmod n}$ with a random card. Mossel, Peres and Sinclair (2004) showed a lower bound of $(0.0345+o(1))n\log n$ for the mixing time of the random-to-cyclic transposition process. They also considered a generalization of this process where the choice of $L_t$ is adversarial, and proved an upper bound of $C n\log n + O(n)$ (with $C \approx 4\times 10^5$) on the mixing time. We reduce the constant to $1$ by showing that the random-to-top transposition chain (a Markov Chain) has robust mixing time $\leq n\log n + O(n)$ when the adversarial strategies are limited to holomorphic strategies, i.e. those strategies which preserve the symmetry of the underlying Markov Chain. We also show a $O(n\log^2 n)$ bound on the robust mixing time of the lazy random-to-top transposition chain when the adversary is not limited to holomorphic strategies. Reversible liftings: Chen, Lovasz and Pak showed that for a reversible ergodic Markov Chain $\mathbb{P}$, any reversible lifting $\mathbb{Q}$ of $\mathbb{P}$ must satisfy $\mathcal{T}(\mathbb{P}) \leq \mathcal{T}(\mathbb{Q})\log (1/\pi_*)$ where $\pi_*$ is the minimum stationary probability. Looking at a specific adversarial strategy allows us to show that $\mathcal{T}(\mathbb{Q}) \geq r(\mathbb{P})$ where $r(\mathbb{P})$ is the relaxation time of $\mathbb{P}$. This gives an alternate proof of the reversible lifting result and helps identify cases where reversible liftings cannot improve the mixing time by more than a constant factor.

In this paper, we develop a new “robust mixing” framework for reasoning about adversarially modified Markov Chains (AMMC). Let ℙ be the transition matrix of an irreducible Markov Chain with stationary distribution π. An adversary announces a sequence of stochastic matrices $\{{\mathbb{A}}_t\}_{t > 0}$ satisfying $\pi{\mathbb{A}}_t = \pi$. An AMMC process involves an application of ℙ followed by ${\mathbb{A}}_t$ at time t. The robust mixing time of an irreducible Markov Chain ℙ is the supremum over all adversarial strategies of the mixing time of the corresponding AMMC process. Applications include estimating the mixing times for certain non-Markovian processes and for reversible liftings of Markov Chains. Non-Markovian card shuffling processes: The random-to-cyclic transposition process is a non-Markovian card shuffling process, which at time t, exchanges the card at position $t {\pmod n}$ with a random card. Mossel, Peres and Sinclair (2004) showed that the mixing time of this process lies between (0.0345+o(1))nlogn and Cnlogn + O(n) (with C ≈4 ×105). We reduce the constant C to 1 by showing that the random-to-top transposition chain (a Markov Chain) has robust mixing time ≤nlogn + O(n) when the adversarial strategies are limited to those which preserve the symmetry of the underlying Markov Chain. Reversible liftings: Chen, Lovasz and Pak showed that for a reversible ergodic Markov Chain ℙ, any reversible lifting ℚ of ℙ must satisfy ${\mathcal{T}}({\mathbb{P}}) \leq {\mathcal{T}}(\mathbb{Q})\log (1/\pi_*)$ where π* is the minimum stationary probability. Looking at a specific adversarial strategy allows us to show that ${\mathcal{T}}(\mathbb{Q}) \geq r({\mathbb{P}})$ where r(ℙ) is the relaxation time of ℙ. This helps identify cases where reversible liftings cannot improve the mixing time by more than a constant factor.

Article Free Access Share on Exact sampling and approximate counting techniques Author: Mark Huber School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NYView Profile Authors Info & Claims STOC '98: Proceedings of the thirtieth annual ACM symposium on Theory of computingMay 1998 Pages 31–40https://doi.org/10.1145/276698.276709Published:23 May 1998Publication History 25citation666DownloadsMetricsTotal Citations25Total Downloads666Last 12 Months66Last 6 weeks15 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

We present in this paper a mathematical method based on stochastic comparisons of Markov chains in order to compute in a mobile network performance measures that are important Q parameters for users: the dropping handover of voice and the blocking probability of a new voice call. The key idea of this methodology is that given a complex system represented by a multi-dimensional Markov chain which is too large to be solved, we propose to reduce the state space, and so to define a new Markov chain which is a simplified version of the original one. This reduced Markov chain is defined as an aggregated one, which represents a stochastic bound for performance measures written as increasing reward function on the stationary distribution. The main steps of the construction of the aggregated Markov chain applied into mobile networks are presented in this paper. As the number of mobile users with different kinds of applications increases, the associated model is more complex and it can be represented by multi-dimensional continuous time Markov chain with a very large size. Thus, we define an aggregated Markov chain represented by a multi-dimensional birth and death process which is very easy to solve. We have proved that there is a weak ordering between the original Markov chain and the aggregated one using increasing set formalism. We have computed upper bounds of dropping handover and blocking probability for different values of input parameters. Numerical results prove that upper bounds give good results and so the stochastic methodology is an interesting mathematical tool for the performance evaluation of complex systems. Keywords: Mobile Networks, Quality of Service, Stochastic ordering, stochastic comparisons, Continuous Time Markov Chains.

The problem of sampling from the stationary distribution of a Markov chain finds widespread applications in a variety of fields. The time required for a Markov chain to converge to its stationary distribution is known as the classical mixing time. In this article, we deal with analog quantum algorithms for mixing. First, we provide an analog quantum algorithm that, given a Markov chain, allows us to sample from its stationary distribution in a time that scales as the sum of the square root of the classical mixing time and the square root of the classical hitting time. Our algorithm makes use of the framework of interpolated quantum walks and relies on Hamiltonian evolution in conjunction with von Neumann measurements. There also exists a different notion for quantum mixing: the problem of sampling from the limiting distribution of quantum walks, defined in a time-averaged sense. In this scenario, the quantum mixing time is defined as the time required to sample from a distribution that is close to this limiting distribution. Recently, we provided an upper bound on the quantum mixing time for Erd\ifmmode \mbox{\H{o}}\else \H{o}\fi{}s-R\'enyi random graphs [Phys. Rev. Lett. 124, 050501 (2020)]. Here, we also extend and expand upon our findings therein. Namely, we provide an intuitive understanding of the state-of-the-art random matrix theory tools used to derive our results. In particular, for our analysis we require information about macroscopic, mesoscopic, and microscopic statistics of eigenvalues of random matrices which we highlight here. Furthermore, we provide numerical simulations that corroborate our analytical findings and extend this notion of mixing from simple graphs to any ergodic, reversible, Markov chain.

In this work, we characterize the fixed points of the doubly stochastic operators defined on discrete spaces ℓ p ( I ) , where I is a nonempty set and p ∈ [ 1 , + ∞ ) . We use our results to characterize all Markov chains with infinite state space and doubly stochastic transition probability matrix that have at least one stationary distribution vector.

Formal verification is becoming a fundamental step in assuring thecorrectness of safety-critical systems. Since these systems are oftenasynchronous and even distributed, their verification requires methodsthat can deal with huge or even infinite state spaces. Model checkingis one of the current techniques to analyse the behaviour of systems,as part of the verification process. In this paper a symbolic boundedmodel checking algorithm is presented that relies on efficient saturation-basedmethods. The previous approaches are extended with new bounded statespace exploration strategies. In addition, constrained saturationis also introduced to improve the efficiency of bounded model checking.Our measurements confirm that these approaches do not only offera solution to deal with infinite state spaces, but in many casesthey even outperform the original methods.

The past few decades have seen robust research on questions regarding the existence, form, and properties of stationary distributions of stochastically modeled reaction networks. When a stochastic model admits a stationary distribution an important practical question is: what is the rate of convergence of the distribution of the process to the stationary distribution? With the exception of [1] pertaining to models whose state space is restricted to the non-negative integers, there has been a notable lack of results related to this rate of convergence in the reaction network literature. This paper begins the process of filling that hole in our understanding. In this paper, we characterize this rate of convergence, via the mixing times of the processes, for two classes of stochastically modeled reaction networks. Specifically, by applying a Foster-Lyapunov criteria we establish exponential ergodicity for two classes of reaction networks introduced in [2]. Moreover, we show that for one of the classes the convergence is uniform over the initial state.

Parameterized systems are systems that involve numerous instantiations of finite-state processes, and depend on parameters which define their size. The verification of parameterized systems is to decide if a property holds in its every size instance, essentially a problem with an infinite state space, and thus poses a great challenge to the community. Starting with a set of undesired states represented by an upward-closed set, the backward reachability analysis will always terminate because of the well-quasi-orderingness. As a result, backward reachability analysis has been widely used in the verification of parameterized systems. However, many existing approaches are facing with the dimensionality problem, which describes the phenomenon that the memory used for storing the symbolic state space grows extremely fast when the number of states of the finite-state process increases, making the verification rather inefficient. Based on bounded backward reachability graphs, a novel abstraction for parameterized systems, we have developed an approach for building abstractions with incrementally increased dimensions and thus improving the precision until a property is proven or a counterexample is detected. The experiments show that the verification efficiencies have been significantly improved because conclusive results tend to be drawn on abstractions with much lower dimensions.

Quadratic stochastic operator (QSO) theory has advanced significantly since the early 1920s and is still growing due to its numerous applications in a variety of fields, particularly mathematics, where QSOs have inspired mathematicians to use and integrate various mathematical knowledge and concepts to better understand their properties and behaviors. Motivated by the relationship between the number of partitions on an infinite state space and the development of the system of equations corresponding to QSOs, this work sought to investigate the dynamics of QSOs formed by three partitions. First, we define and construct the 3-partition QSOs, which result in a system of equations with three variables. We then provide the formulation of the fixed point form and discuss its behavior using Jacobian matrix analysis. Some scenarios of three-partition QSOs with three different parameters are considered to readily investigate the type of fixed point in such systems. It is demonstrated that the operators can have either an attracting or a saddle fixed point but can never be repelling. We show how the saddle fixed point behaves, by identifying a set of points known as the fixed point’s stable manifold.

In this paper we introduce a discrete event system model of immune systems of mammalians using Markov Decision Processes based on Petri Nets Models. Based on immunological principles we propose an approach in order to study the mechanisms that govern the immune system's functionality. A two-module algorithm that launches a specific action against an anomalous situation is developed. The Petri nets tools are assumed in this approach. Also, Markov Decision Processes (MDPs) with a truncated state space to the problem with infinite state space is considered. We also propose a new algorithm to build a large model (e.g., a macro-model) of immune mechanisms of mammalians. We show that an optimal stationary policy exists and we apply the results of [1] to a dynamic scheduling problem of the immunological response to external stimuli.

A quadratic stochastic operator (QSO) is frequently acknowledged as the analysis source to investigate dynamical properties and modeling in numerous areas. Countless classes of QSO have been investigated since the operator was introduced in 1920s. The study of QSOs is still an open problem in the nonlinear operator theory field, especially QSOs on infinite state space. We are interested in the dynamics of Geometric QSO generated by a 2-partition defined on countable state space. We first show the system of equations formed from defined Geometric QSO with infinite-dimensional space can be simplified into a one-dimensional setting, corresponding to the number of defined partitions. The trajectory behavior of such a system is investigated by using functional analysis approach, where the operator either converges to a unique fixed point or has a second-order cycle. It is shown that such an operator can be either regular or nonregular for arbitrary initial points depending on the value of parameters. In this research, we present two cases, i.e., two different parameters and three different parameters. We display the form of the fixed point and periodic points of period-2. Moreover, an example for the nonregular transformation will be provided, where such QSO has 2-periodic points.

In the literature concerning Dirichlet forms and its applications, closability plays a crucial role. In fact, closedness is one of the defining properties of a Dirichlet form. According to this, a number of closability criterions are known in particular cases. An important question is under which conditions closability is kept after changing the reference measure. M. Fukushima, K. Sato, and S. Taniguchi [5] treated this problem for a regular Dirichlet form (E (E )) which is defined on a locally compact separable metric state space. Under technical conditions on some core C ⊆ (E ), they presented a complete solution if the Dirichlet form is either irreducible or transient. An earlier paper dealing with this subject is M. Rockner and N. Wielens [13]. Related results on Lusinean separable metric spaces were published in I. Shigekawa and S. Taniguchi [16]. The aim of this paper is to give general analytical conditions in order to keep closability when turning to a new reference measure. One particular purpose is to present an extension of an assertions in [5] (namely, Corollary 4.2) within a purely measure theoretic framework, i.e., the state space ( B) is just a measurable space. In particular, the set C is defined exclusively in terms of the initial form (E (E )) on 2( μ). The main results are Theorems 2.3, 2.4, and 2.5. We proceed to give some basic definitions.

Asynchronous reactive systems form the basis of a wide range of software systems, for instance in the telecommunications domain. It is highly desirable to rigorously show that these systems are correctly designed. However, traditional formal approaches to the verification of these systems are often difficult because asynchronous reactive systems usually possess extremely large or even infinite state spaces. We propose an integer linear program (ILP) solving-based property checking framework that concentrates on the local analysis of the cyclic behavior of each individual component of a system. We apply our framework to the checking of the buffer boundedness and livelock freedom properties, both of which are undecidable for asynchronous reactive systems with an infinite state space. We illustrate the application of the proposed checking methods to Promela, the input language of the SPIN model checker. While the precision of our framework remains an issue, we propose a counterexample guided abstraction refinement procedure based on the discovery of dependences among control flow cycles. We have implemented prototype tools with which we obtained promising experimental results on real-life system models.