Cutoff for nonbacktracking random walks on sparse random graphs

The cutoff phenomenon describes a sharp transition in the convergence of a family of ergodic finite Markov chains to equilibrium. Many natural families of chains are believed to exhibit cutoff, and yet establishing this fact is often extremely challenging. An important such family of chains is the random walk on G(n,d), a random d-regular graph on n vertices. It is well known that almost every such graph for d≥3 is an expander, and even essentially Ramanujan, implying a mixing time of O(logn). According to a conjecture of Peres, the simple random walk on G(n,d) for such d should then exhibit cutoff with high probability (whp). As a special case of this, Durrett conjectured that the mixing time of the lazy random walk on a random 3-regular graph is whp (6+o(1))log2n. In this work we confirm the above conjectures and establish cutoff in total-variation, its location, and its optimal window, both for simple and for non-backtracking random walks on G(n,d). Namely, for any fixed d≥3, the simple random walk on G(n,d) whp has cutoff at (d/(d−2))logd−1n with window order logn. Surprisingly, the non-backtracking random walk on G(n,d) whp has cutoff already at logd−1n with constant window order. We further extend these results to G(n,d) for any d=no(1) that grows with n (beyond which the mixing time is O(1)), where we establish concentration of the mixing time on one of two consecutive integers

Unified theory for finite Markov chains

In this chapter, we examine some of the properties of discrete-time finite Markov chains. These processes are discrete valued, and to keep the analysis as elementary as possible, we restrict our attention to homogeneous, i.e., time-invariant, and finite chains, i.e., chains where the process X(t) can take only a finite number of values. Under these simplifying assumptions, Markov chains can be analyzed by using tools of linear algebra. Yet, the class of finite homogeneous Markov chains covers a wide range of engineering systems, since it is in essence identical to the class of synchronous stochastic automata.

Nowadays, standard proofs of the large deviation property (LDP) for i.i.d. processes are based on the `method of types' [4], [2]. However, for Markov chains, the proofs in standard texts are not based on the method of types. Instead they make use of more advanced concepts such as the Ga¿rtner-Ellis theorem (see e.g. [6]) or Varadhan's lemma (see e.g. [15]). There is indeed a proof of the LDP for finite state Markov chains based on the method of types [16], but it is not wellknown nor cited in [6]. The principal objective of this paper is therefore to present a first-principles derivation of the LDP for finite state Markov chains, using only simple combinatorial arguments (e.g. the method of types). The approach presented here extends naturally to multi-step Markov chains. We also relate the LDP rate function of a Markov chain to that of its time-reversed version.

Emergence of new diseases and elimination of existing diseases is a key public health issue. In mathematical models of epidemics, such phenomena involve the process of infections and recoveries passing through a critical threshold where the basic reproductive ratio is 1. In this paper, we study near-critical behaviour in the context of a susceptible-infective-recovered epidemic on a random (multi)graph on n vertices with a given degree sequence. We concentrate on the regime just above the threshold for the emergence of a large epidemic, where the basic reproductive ratio is 1 + omega (n) n^{-1/3}, with omega (n) tending to infinity slowly as the population size, n, tends to infinity. We determine the probability that a large epidemic occurs, and the size of a large epidemic. Our results require basic regularity conditions on the degree sequences, and the assumption that the third moment of the degree of a random susceptible vertex stays uniformly bounded as n rightarrow infty . As a corollary, we determine the probability and size of a large near-critical epidemic on a standard binomial random graph in the ‘sparse’ regime, where the average degree is constant. As a further consequence of our method, we obtain an improved result on the size of the giant component in a random graph with given degrees just above the critical window, proving a conjecture by Janson and Luczak.

The switch Markov chain has been extensively studied as the most natural Markov Chain Monte Carlo approach for sampling graphs with prescribed degree sequences. We use comparison arguments with other---less natural but simpler to analyze---Markov chains, to show that the switch chain mixes rapidly in two different settings. We first study the classic problem of uniformly sampling simple undirected, as well as bipartite, graphs with a given degree sequence. We apply an embedding argument, involving a Markov chain defined by Jerrum and Sinclair (TCS, 1990) for sampling graphs that almost have a given degree sequence, to show rapid mixing for degree sequences satisfying strong stability, a notion closely related to P-stability. This results in a much shorter proof that unifies the currently known rapid mixing results of the switch chain and extends them up to sharp characterizations of P-stability. In particular, our work resolves an open problem posed by Greenhill (SODA, 2015). Secondly, in order to illustrate the power of our approach, we study the problem of uniformly sampling graphs for which, in addition to the degree sequence, a joint degree distribution is given. Although the problem was formalized over a decade ago, and despite its practical significance in generating synthetic network topologies, small progress has been made on the random sampling of such graphs. The case of a single degree class reduces to sampling of regular graphs, but beyond this almost nothing is known. We fully resolve the case of two degree classes, by showing that the switch Markov chain is always rapidly mixing. Again, we first analyze an auxiliary chain for strongly stable instances on an augmented state space and then use an embedding argument.

A general critical condition for the emergence of a giant component in random graphs with given degrees

The problem of random number generation from an uncorrelated random source (of unknown probability distribution) dates back to von Neumann's 1951 work. Elias (1972) generalized von Neumann's scheme and showed how to achieve optimal efficiency in unbiased random bits generation. Hence, a natural question is what if the sources are correlated? Both Elias and Samuelson proposed methods for generating unbiased random bits in the case of correlated sources (of unknown probability distribution), specifically, they considered finite Markov chains. However, their proposed methods are not efficient or have implementation difficulties. Blum (1986) devised an algorithm for efficiently generating random bits from degree-2 finite Markov chains in expected linear time, however, his beautiful method is still far from optimality on information-efficiency. In this paper, we generalize Blum's algorithm to arbitrary degree finite Markov chains and combine it with Elias's method for efficient generation of unbiased bits. As a result, we provide the first known algorithm that generates unbiased random bits from an arbitrary finite Markov chain, operates in expected linear time and achieves the information-theoretic upper bound on efficiency.

Adaptive control of Markov chains with local updates

For a finite Markov chain, by rotating the transition matrix by 180°, or relabelling the states, one can define a new Markov chain. This Markov chain in fact is the imbedded Markov chain of an inverse process. Duality properties about this Markov chain sometimes are not difficult to obtain. Similarly, one can discuss an infinite Markov chain with states 0,±1,±2,…, However, many applications involve transition matrices with various boundary modifications where rotation cannot directly apply. After introducing some duality properties for a boundary-free or finite Markov chain, we will mainly focus on some interesting application problems and show how to use the duality from rotation to these problems.

We study the largest component of a random (multi)graph on n vertices with a given degree sequence. We let n → ∞. Then, under some regularity conditions on the degree sequences, we give conditions on the asymptotic shape of the degree sequence that imply that with high probability all the components are small, and other conditions that imply that with high probability there is a giant component and the sizes of its vertex and edge sets satisfy a law of large numbers; under suitable assumptions these are the only two possibilities. In particular, we recover the results by Molloy and Reed on the size of the largest component in a random graph with a given degree sequence.We further obtain a new sharp result for the giant component just above the threshold, generalizing the case of G(n,p) with np = 1 + ω(n)n−1/3, where ω(n) → ∞ arbitrarily slowly.Our method is based on the properties of empirical distributions of independent random variables, and leads to simple proofs. © 2008 Wiley Periodicals, Inc. Random Struct. Alg., 2009

We consider weak lumpability of finite homogeneous Markov chains, which is when a lumped Markov chain with respect to a partition of the initial state space is also a homogeneous Markov chain. We show that weak lumpability is equivalent to the existence of a direct sum of polyhedral cones that is positively invariant by the transition probability matrix of the original chain. It allows us, in a unified way, to derive new results on lumpability of reducible Markov chains and to obtain spectral properties associated with lumpability.

As a continuation of a recent paper, dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact metric space killed in finite time. The idea is to run the process until extinction and then to bring it back to life at a position randomly chosen according to the (possibly weighted) empirical occupation measure of its past positions. General conditions are given ensuring the convergence of this measure to the quasi-stationary distribution of the chain. We then apply this method to the numerical approximation of the quasi-stationary distribution of a diffusion process killed on the boundary of a compact set. Finally, the sharpness of the assumptions is illustrated through the study of the algorithm in a nonirreducible setting.

We consider approximate policy evaluation for finite state and action Markov de- cision processes (MDP) with the least squares temporal difference (LSTD) algorithm, LSTD(λ), in an exploration-enhanced learning context, where policy costs are computed from observations of a Markov chain different from the one corresponding to the policy under evaluation. We establish for the discounted cost criterion that LSTD(λ) converges almost surely under mild, minimal conditions. We also analyze other properties of the iterates involved in the algorithm, including convergence in mean and boundedness. Our analysis draws on theories of both finite space Markov chains and weak Feller Markov chains on a topological space. Our results can be applied to other temporal difference algorithms and MDP models. As examples, we give a convergence analysis of a TD(λ) algorithm and extensions to MDP with compact state and action spaces, as well as a convergence proof of a new LSTD algorithm with state-dependent λ-parameters.

This paper describes a Bayesian approach to determining the order of a finite state Markov chain whose transition probabilities are themselves governed by a homogeneous finite state Markov chain. It extends previous work on homogeneous Markov chains to more general and applicable hidden Markov models. The method we describe uses a Markov chain Monte Carlo algorithm to obtain samples from the (posterior) distribution for both the order of Markov dependence in the observed sequence and the other governing model parameters. These samples allow coherent inferences to be made straightforwardly in contrast to those which use information criteria. The methods are illustrated by their application to both simulated and real data sets.