Robust Mixing

In this paper, we develop a new “robust mixing” framework for reasoning about adversarially modified Markov Chains (AMMC). Let ℙ be the transition matrix of an irreducible Markov Chain with stationary distribution π. An adversary announces a sequence of stochastic matrices $\{{\mathbb{A}}_t\}_{t > 0}$ satisfying $\pi{\mathbb{A}}_t = \pi$. An AMMC process involves an application of ℙ followed by ${\mathbb{A}}_t$ at time t. The robust mixing time of an irreducible Markov Chain ℙ is the supremum over all adversarial strategies of the mixing time of the corresponding AMMC process. Applications include estimating the mixing times for certain non-Markovian processes and for reversible liftings of Markov Chains. Non-Markovian card shuffling processes: The random-to-cyclic transposition process is a non-Markovian card shuffling process, which at time t, exchanges the card at position $t {\pmod n}$ with a random card. Mossel, Peres and Sinclair (2004) showed that the mixing time of this process lies between (0.0345+o(1))nlogn and Cnlogn + O(n) (with C ≈4 ×105). We reduce the constant C to 1 by showing that the random-to-top transposition chain (a Markov Chain) has robust mixing time ≤nlogn + O(n) when the adversarial strategies are limited to those which preserve the symmetry of the underlying Markov Chain. Reversible liftings: Chen, Lovasz and Pak showed that for a reversible ergodic Markov Chain ℙ, any reversible lifting ℚ of ℙ must satisfy ${\mathcal{T}}({\mathbb{P}}) \leq {\mathcal{T}}(\mathbb{Q})\log (1/\pi_*)$ where π* is the minimum stationary probability. Looking at a specific adversarial strategy allows us to show that ${\mathcal{T}}(\mathbb{Q}) \geq r({\mathbb{P}})$ where r(ℙ) is the relaxation time of ℙ. This helps identify cases where reversible liftings cannot improve the mixing time by more than a constant factor.

Despite intensive studies on epidemic spreading in metapopulation networks, the infection process is normally assumed to be a Markovian process without considering time-dependent infectivity. In this paper, a reaction-diffusion SIRS epidemic model with non-Markovian infection processes is constructed on metapopulation networks, where the infection rate function associated with the sojourn time in the infected state is obtained by constructing the generation time distribution of the emerging infection process. Meanwhile, the recurrent mobility patterns are incorporated by the metapopulation model. By Lyapunov stability analysis, the epidemic outbreak condition is derived theoretically. The results show that the non-Markovian infection characteristics can significantly alter the transient behavior of epidemic spreading under the same basic reproduction number. Furthermore, with the increase in the average time for an infected individual to generate secondary cases in the non-Markovian process, the impact of mobility rate on transient epidemic spreading behavior will be reduced. Finally, mobility can have a suppressing effect on epidemic spreading dynamics when the population distribution exhibits significant asymmetry. The results can sharpen our understanding of the role of real-world non-Markovian infection processes in epidemic outbreaks.

The problem of sampling from the stationary distribution of a Markov chain finds widespread applications in a variety of fields. The time required for a Markov chain to converge to its stationary distribution is known as the classical mixing time. In this article, we deal with analog quantum algorithms for mixing. First, we provide an analog quantum algorithm that, given a Markov chain, allows us to sample from its stationary distribution in a time that scales as the sum of the square root of the classical mixing time and the square root of the classical hitting time. Our algorithm makes use of the framework of interpolated quantum walks and relies on Hamiltonian evolution in conjunction with von Neumann measurements. There also exists a different notion for quantum mixing: the problem of sampling from the limiting distribution of quantum walks, defined in a time-averaged sense. In this scenario, the quantum mixing time is defined as the time required to sample from a distribution that is close to this limiting distribution. Recently, we provided an upper bound on the quantum mixing time for Erd\ifmmode \mbox{\H{o}}\else \H{o}\fi{}s-R\'enyi random graphs [Phys. Rev. Lett. 124, 050501 (2020)]. Here, we also extend and expand upon our findings therein. Namely, we provide an intuitive understanding of the state-of-the-art random matrix theory tools used to derive our results. In particular, for our analysis we require information about macroscopic, mesoscopic, and microscopic statistics of eigenvalues of random matrices which we highlight here. Furthermore, we provide numerical simulations that corroborate our analytical findings and extend this notion of mixing from simple graphs to any ergodic, reversible, Markov chain.

We consider the problem of recursively estimating the transition matrix of a regular Markov chain with a finite number of states using only noisy measurements of the state. The measurement noise sequence is assumed to be independent with known mean and unknown variance. We also discuss the convergence rates and computational aspects of the algorithms and the methods of accelerating them.

Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic process. A standard way to compute this distribution for a random walk on a finite set of states is to compute the Perron vector of the associated transition matrix. There are algebraic analogues of this Perron vector in terms of transition probability tensors of higher-order Markov chains. These vectors are nonnegative, have dimension equal to the dimension of the state space, and sum to one and are derived by making an algebraic substitution in the equation for the joint-stationary distribution of a higher-order Markov chains. Here, we present the spacey random walk, a non-Markovian stochastic process whose stationary distribution is given by the tensor eigenvector. The process itself is a vertex-reinforced random walk, and its discrete dynamics are related to a continuous dynamical system. We analyze the convergence properties of these dynamics and discuss numerical methods for computing the stationary distribution. Finally, we provide several applications of the spacey random walk model in population genetics, ranking, and clustering data, and we use the process to analyze taxi trajectory data in New York. This example shows definite non-Markovian structure.

A phenomenological interpolation model for the transition operator of a stationary Markov process is shown to be equivalent to the simplest difference approximation in the master equation for the conditional density. Comparison with the formal solution of the Fokker-Planck equation yields a criterion for the choice of the correlation time in the approximate solution. The interpolation model is shown to be form-invariant under an iteration-cum-rescaling scheme. Next, we go beyond Markov processes to find the effective time-development operator (the counterpart of the conditional density) in the following very general situation: the stochastic interruption of the systematic evolution of a variable by an arbitrary stationary sequence of randomizing pulses. Continuous-time random walk theory with a distinct first-waiting-time distribution is used, along with the interpolation model for the transition operator, to obtain the solution. Convenient closed-form expressions for the ‘averaged’ time-development operator and the autocorrelation function are presented in various special cases. These include (i) no systematic evolution, but correlated pulses; (ii) systematic evolution interrupted by an uncorrelated (Poisson) sequence of pulses.

Markov and non-Markov processes in complex systems by the dynamical information entropy

We show that a new probabilistic technique, recently introduced by the first author, yields the sharpest bounds obtained to date on mixing times of Markov chains in terms of isoperimetric properties of the state space (also known as conductance bounds or Cheeger inequalities). We prove that the bounds for mixing time in total variation obtained by Lovasz and Kannan, can be refined to apply to the maximum relative deviation |p n (x,y)/π(y)−1| of the distribution at time n from the stationary distribution π. We then extend our results to Markov chains on infinite state spaces and to continuous-time chains. Our approach yields a direct link between isoperimetric inequalities and heat kernel bounds; previously, this link rested on analytic estimates known as Nash inequalities.

We present in this paper a mathematical method based on stochastic comparisons of Markov chains in order to compute in a mobile network performance measures that are important Q parameters for users: the dropping handover of voice and the blocking probability of a new voice call. The key idea of this methodology is that given a complex system represented by a multi-dimensional Markov chain which is too large to be solved, we propose to reduce the state space, and so to define a new Markov chain which is a simplified version of the original one. This reduced Markov chain is defined as an aggregated one, which represents a stochastic bound for performance measures written as increasing reward function on the stationary distribution. The main steps of the construction of the aggregated Markov chain applied into mobile networks are presented in this paper. As the number of mobile users with different kinds of applications increases, the associated model is more complex and it can be represented by multi-dimensional continuous time Markov chain with a very large size. Thus, we define an aggregated Markov chain represented by a multi-dimensional birth and death process which is very easy to solve. We have proved that there is a weak ordering between the original Markov chain and the aggregated one using increasing set formalism. We have computed upper bounds of dropping handover and blocking probability for different values of input parameters. Numerical results prove that upper bounds give good results and so the stochastic methodology is an interesting mathematical tool for the performance evaluation of complex systems. Keywords: Mobile Networks, Quality of Service, Stochastic ordering, stochastic comparisons, Continuous Time Markov Chains.

Coupling and mixing times in a Markov chain

Yuval Peres and Perla Sousi showed that the mixing times and average mixing times of reversible Markov chains on finite state spaces are equal up to some universal multiplicative constant. We use tools from nonstandard analysis to extend this result to reversible Markov chains on compact state spaces that satisfy the strong Feller property.

The standard coalescent is widely used in evolutionary biology and population genetics to model the ancestral history of a sample of molecular sequences as a rooted and ranked binary tree. In this paper we present a representation of the space of ranked trees as a space of constrained ordered matched pairs. We use this representation to define ergodic Markov chains on labeled and unlabeled ranked tree shapes analogously to transposition chains on the space of permutations. We show that an adjacent-swap chain on labeled and unlabeled ranked tree shapes has a mixing time at least of order $n^3$ , and at most of order $n^{4}$ . Bayesian inference methods rely on Markov chain Monte Carlo methods on the space of trees. Thus it is important to define good Markov chains which are easy to simulate and for which rates of convergence can be studied.

<p>A complex fractional derivative can be derived by formally extending the integer k in the kth derivative of a function, computed via Cauchy's integral, to complex α. This straightforward approach reveals fundamental problems due to inherent nonanalyticity. A consequence is that the complex fractional derivative is not uniquely defined. We explain in detail the anomalies (not closed paths, branch cut jumps) and try to interpret their meaning physically in terms of entropy, friction and deviations from ideal vector fields. Next, we present a class of non-Markovian continuous-time processes by replacing the standard derivative by a Caputo fractional derivative in the classical Chapman-Kolmogorov governing equation of a continuous-time Markov process. The fractional derivative leads to a replacement of the set of exponential base functions by a set of Mittag-Leffler functions, but also creates a complicated dependence structure between states. This fractional non-Markovian process may be applied to generalize the Markovian SIS epidemic process on a contact graph to a more realistic setting. </p>

This work is concerned with weak convergence of non-Markov random processes modulated by a Markov chain. The motivation of our study stems from a wide variety of applications in actuarial science, communication networks, production planning, manufacturing and financial engineering. Owing to various modelling considerations, the modulating Markov chain often has a large state space. Aiming at reduction of computational complexity, a two-time-scale formulation is used. Under this setup, the Markov chain belongs to the class of nearly completely decomposable class, where the state space is split into several subspaces. Within each subspace, the transitions of the Markov chain varies rapidly, and among different subspaces, the Markov chain moves relatively infrequently. Aggregating all the states of the Markov chain in each subspace to a single super state leads to a new process. It is shown that under such aggregation schemes, a suitably scaled random sequence converges to a switching diffusion process.

This work analyzes distributed linear averaging within a connected network of sensors that each track the stationary distribution of an ergodic Markov chain with a slowly switching regime. Our approach is based on a two-time-scale stochastic approximation. A hyperparameter modeled as a Markov chain on a slower time-scale modulates the regime of each observed Markov chain. The average of all currently observed stationary distributions constitutes the average-consensus estimate to be reached by all sensors. Assuming the Markov chains do not share a common stationary distribution conditioned on their regime, then under the proposed linear averaging algorithm, the exchange graph conditions required for the sequence of sensor state values to converge weakly to the average-consensus are obtained. Estimation of a weighted average of all observed stationary distributions, not only the current ones, is proved feasible over a long-run time horizon, provided an additional communication condition holds. The sensor st...