7 - Random Limit Theorems via Strong Invariance Principles

In Kifer and Varadhan (Nonconventional limit theorems in discrete and continuous time via martingales, 2010) we obtained a functional central limit theorem (known also as a weak invariance principle) for sums of the form \({\sum_{n=1}^{[Nt]} F\big(X(n), X(2n), .\, .\, .\, .\, X(kn), X(q_{k+1}(n)), X(q_{k+2}(n)), .\, .\, .\, , X(q_\ell(n))\big)}\) (normalized by \({1/\sqrt N}\)) where X(n), n ≥ 0 is a sufficiently fast mixing vector process with some moment conditions and stationarity properties, F is a continuous function with polynomial growth and certain regularity properties and qi, i > k are positive functions taking on integer values on integers with some growth conditions which are satisfied, for instance, when qi’s are polynomials of growing degrees. This paper deals with strong invariance principles (known also as strong approximation theorems) for such sums which provide their uniform in time almost sure approximation by processes built out of Brownian motions with error terms growing slower than \({\sqrt N}\) . This yields, in particular, an invariance principle in the law of iterated algorithm for the above sums. Among motivations for such results are their applications to multiple recurrence for stochastic processes and dynamical systems as well, as to some questions in metric number theory and they can be considered as a natural follow up of a series of papers dealing with nonconventional ergodic averages.

Many of the fundamental results in probability theory are formulated as limit theorems. Bernoulli’s law of large numbers was formulated as a limit theorem; so was the De Moivre-Laplace theorem, which can fairly be called the origin of a genuine theory of probability and, in particular, which led the way to numerous investigations that clarified the conditions for the validity of the central limit theorem. Poisson’s theorem on the approximation of the binomial distribution by the “Poisson” distribution in the case of rare events was formulated as a limit theorem. After the example of these propositions, and of results on the rapidity of convergence in the De Moivre-Laplace and Poisson theorems, it became clear that in probability it is necessary to deal with various kinds of convergence of distributions, and to establish the rapidity of convergence connected with the introduction of various “natural” measures of the distance between distributions. In the present chapter we shall discuss some general features of the convergence of probability distributions and of the distance between them. In this section we take up questions in the general theory of weak convergence of probability measures in metric spaces. The De Moivre-Laplace theorem, the progenitor of the central limit theorem, finds a natural place in this theory. From §3, it is clear that the method of characteristic functions is one of the most powerful means for proving limit theorems on the weak convergence of probability distributions in R n. In §6, we consider questions of metrizability of weak convergence. Then, in §8, we turn our attention to a different kind of convergence of distributions (stronger than weak convergence), namely convergence in variation. Proofs of the simplest results on the rapidity of convergence in the central limit theorem and Poisson’s theorem will be given in §§10 and 11.KeywordsProbability MeasureCharacteristic FunctionCentral Limit TheoremWeak ConvergenceIndependent Random VariableThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Limit theorems for cumulative processes

The functional law of the iterated logarithm for the empirical process of some long-range dependent sequences

In this paper, we develop theory and tools for studying U-processes, a natural higher-order generalization of the empirical processes. We introduce a class of random discrete U-measures that generalize the empirical U-measure. We establish a Glivenko-Cantelli and a Donsker theorem under conditions on entropy numbers prevalent in the theory of empirical processes. These results are proved under some standard structural conditions on the Vapnik-Chervonenkis classes of functions and some mild conditions on the model. The uniform limit theorems discussed in this paper are key tools for many further developments involving empirical process techniques. Our results are applied to prove the asymptotic normality of Liu's simplicial median. We conclude this paper by extending Anscombe's central limit theorem to encompass randomly stopped U-processes, building upon its application to randomly stopped sums of independent random variables.

About forty years ago it was realized by several researchers that the essential features of certain objects of Probability theory, notably Gaussian processes and limit theorems, may be better understood if they are considered in settings that do not impose structures extraneous to the problems at hand. For instance, in the case of sample continuity and boundedness of Gaussian processes, the essential feature is the metric or pseudometric structure induced on the index set by the covariance structure of the process, regardless of what the index set may be. This point of view ultimately led to the Fernique-Talagrand majorizing measure characterization of sample boundedness and continuity of Gaussian processes, thus solving an important problem posed by Kolmogorov. Similarly, separable Banach spaces provided a minimal setting for the law of large numbers, the central limit theorem and the law of the iterated logarithm, and this led to the elucidation of the minimal (necessary and/or sufficient) geometric properties of the space under which different forms of these theorems hold. However, in light of renewed interest in Empirical processes, a subject that has considerably influenced modern Statistics, one had to deal with a non-separable Banach space, namely $\mathcal{L}_{\infty}$. With separability discarded, the techniques developed for Gaussian processes and for limit theorems and inequalities in separable Banach spaces, together with combinatorial techniques, led to powerful inequalities and limit theorems for sums of independent bounded processes over general index sets, or, in other words, for general empirical processes.

On the Chebyshev-Cramér Asymptotic Expansions

The Lorenz curve of the distribution of ‘wealth’ is a graph of cumulative proportion of total ‘wealth’ owned, against cumulative proportion of the population owning it. This paper uses Gastwirth's definition of the Lorenz curve which applies to a general probability distribution on (0, ∞) having finite mean; thus it applies both to a ‘population’ distributionand to empirical distributions obtained on sampling. The Lorenz curves of the latter are proved to converge, with probability 1, uniformly to the former, and similarly for their inverses. Modified Lorenz curves are also defined, which treat atoms ofdifferently, and these and their inverses are proved strongly consistent. Functional central limit theorems are then proved for empirical Lorenz curves and their inverses, under condition thatbe continuous and have finite variance. A mild variation condition is also needed in some circumstances. If the support ofis connected, the weak convergence is relative toC[0, 1] with uniform topology, otherwise toD[0, 1] withM1topology. Selected applications are discussed, one being to the Gini coefficient.

Let (X k) k≥1 be a sequence of random variables with common distribution function F(x) = P(X 1 ≤ x). Define the empirical distribution function $$ {{F}_{n}}(x) = \frac{1}{n}\# \{ 1 \leqslant i \leqslant n:{{X}_{1}} \leqslant x\} , $$ and the empirical process by $$ \sqrt {n} ({{F}_{n}}(x) - F(x)) $$ In this chapter we provide a survey of classical as well as modern techniques in the study of empirical processes of dependent data. We begin with a sketch of the early roots of the field in the theory of uniform distribution mod 1, of sequences defined by X k = {n k ω}, ω ∈ [0, 1], dating back to Weyl’s celebrated 1916 paper. In the second section we provide the essential tools of empirical process theory, and we prove Donsker’s classical empirical process invariance principle for i.i.d. processes. The third section provides an introduction to the subject of weakly dependent random variables. We introduce a variety of mixing concepts, provide necessary technical tools like correlation and moment inequalities, and prove central limit theorems for partial sums. The empirical process of weakly dependent data is investigated in the fourth section, where we put special emphasis on almost sure approximation techniques. The fifth section is devoted to the empirical distribution of U-statistics, defined as $$ Un(x) = {{\left( {\begin{array}{*{20}{c}} n \\ 2 \\ \end{array} } \right)}^{{ - 1}}}\# \{ 1 \leqslant i < j \leqslant n:h({{X}_{i}},{{X}_{j}}) \leqslant x\} $$ for some symmetric kernel h. We give some applications, e.g., to dimension estimation in the analysis of time series, and prove weak convergence of the corresponding empirical process. Empirical processes of long-range dependent data are the topic of the sixth section. We give an introduction to the area of long-range dependent processes, provide important technical tools for the study of their partial sums and investigate the limit behavior of the empirical process. It turns out that the limit process is of a completely different type as in the case of independent or weakly dependent data, and that this has important consequences for various functionals of the empirical process. The final section is devoted to pair correlations, i.e., U-statistics empirical processes over short intervals associated with the kernel h(x, y) = |x − y|

We study a multiclass open-queueing network with a set of single-server stations that operate under a combination of FIFO (first-in-first out) and priority service disciplines, and are subject to random breakdowns. Assuming that the primitive processes—in particular, external arrivals, service requirements, service capacities (up and down times), and the routing mechanism—follow two-moment approximations (based on functional central limit theorems), we develop a semi-martingale reflected Brownian motion (SRBM) approximation for the performance processes such as workload, queue lengths, and sojourn times. We illustrate through numerical examples in comparison against simulation that the SRBM approximation, while not always supported by a limit theorem, exhibits good accuracy in most cases. Through analyzing special networks, we also discuss the existence of the SRBM approximation in relation to the stability and the heavy traffic limits of the networks.

In this paper, we consider functional limit theorems for Poisson cluster processes. We first present a maximal inequality for Poisson cluster processes. Then we establish a functional central limit theorem under the second moment and a functional moderate deviation principle under the Cramér condition for Poisson cluster processes. We apply these results to obtain a functional moderate deviation principle for linear Hawkes processes.

We study a transportation system consisting of S vehicles of unit capacity and N passenger terminals. Customers arrive stochastically at terminal i, 1 ≦ i ≦ N, seeking transportation to a terminal j, 1 ≦ j ≦ N, with probability Pij . Customers at each terminal are served as vehicles become available. Each vehicle is dispatched from a terminal when loaded, whereupon it travels to the destination of its passenger, according to a stochastic travel time. It is shown under mild conditions that the system is unstable, due to random fluctuations of independent customer arrival processes. We obtain limit theorems, in certain special cases, for the customer queue size processes. Where a steady-state limit exists, this limit is expressed in terms of the corresponding limit in a related GI/G/S queue. In other cases, functional central limit theorems are obtained for appropriately normalized random functions.

Cycle symmetry, limit theorems, and fluctuation theorems for diffusion processes on the circle

We study shot noise processes with cluster arrivals, in which entities in each cluster may experience random delays (possibly correlated), and noises within each cluster may be correlated. We prove functional limit theorems for the process in the large-intensity asymptotic regime, where the arrival rate gets large while the shot shape function, cluster sizes, delays, and noises are unscaled. In the functional central limit theorem, the limit process is a continuous Gaussian process (assuming the arrival process satisfies a functional central limit theorem with a Brownian motion limit). We discuss the impact of the dependence among the random delays and among the noises within each cluster using several examples of dependent structures. We also study infinite-server queues with cluster/batch arrivals where customers in each batch may experience random delays before receiving service, with similar dependence structures.

A Non-Uniform Estimate for the Convergence Speed in the Multi-Dimensional Central Theorem