Remarks on univariate elliptical distributions

1. The main notions.- 2. The main lemmas.- 2.1. General lemmas on the approximation of distribution of an arbitrary random variable by the normal distribution.- 2.2. Proof of lemmas 2.1-2.4.- 3. Theorems on large deviations for the distributions of sums of independent random variables.- 3.1. Theorems on large deviations under Bernstein's condition.- a) Sums of non-identically distributed random variables.- b) Sums of weighted random variables.- 3.2. A theorem of large deviations in terms of Lyapunov's fractions.- 4. Theorems of large deviations for sums of dependent random variables.- 4.1. Estimates of the kth order centered moments of random processes with mixing.- 4.2. Estimates of mixed cumulants of random processes with mixing.- 4.3. Estimates of cumulants of sums of dependent random variables.- 4.4. Theorems and inequalities of large deviations for sums of dependent random variables.- 5. Theorems of large deviations for polynomial forms, multiple stochastic integrals and statistical estimates.- 5.1. Estimates of cumulants and theorems of large deviations for polynomial forms, polynomial Pitman estimates and U-statistics.- 5.2. Cumulants of multiple stochastic integrals and theorems of large deviations.- 5.3. Large deviations for estimates of the spectrum of a stationary sequence.- 6. Asymptotic expansions in the zones of large deviations.- 6.1. Asymptotic expansion for distribution density of an arbitrary random variable.- 6.2. Estimates for characteristic functions.- 6.3. Asymptotic expansion in the Cramer zone for distribution density of sums of independent random variables.- 6.4. Asymptotic expansions in integral theorems with large deviations.- 7. Probabilities of large deviations for random vectors.- 7.1. General lemmas on large deviations for a random vector with regular behaviour of cumulants.- 7.2. Theorems on large deviations for sums of random vectors and quadratic forms.- a) Sums of non-identically distributed random vectors.- b) Sums of weighted random vectors.- c) Sums of random number of random vectors.- d) Quadratic forms.- Appendices.- Appendix 1. Proof of inequalities for moments and Lyapunov's fractions.- Appendix 2. Proof of the lemma on the representation of cumulants.- Appendix 3. Leonov - Shiryaev's formula.- References.

This chapter introduces probability theory as a system of models, based on measure theory, of some real-world phenomena. The models are measure spaces of total measure 1 and usually have certain distinguished measurable functions defined on them. Section 1 begins by establishing the measure-theoretic framework and a short dictionary for passing back and forth between terminology in measure theory and terminology in probability theory. The latter terminology includes events, random variables, mean, probability distribution of a random variable, and joint probability distribution of several random variables. An important feature of probability is that it is possible to work with random variables without any explicit knowledge of the underlying measure space, the joint probability distributions of random variables being the objects of importance. Section 2 introduces conditional probability and uses that to motivate the mathematical definition of independence of events. In turn, independence of events leads naturally to a definition of independent random variables. Independent random variables are of great importance in the subject and play a much larger role than their counterparts in abstract measure theory. Examples at the end of the section indicate the extent to which functions of independent random variables can remain independent. The techniques in the examples are of use in the subject of statistical inference, which is introduced in Section 10. Section 3 states and proves the Kolmogorov Extension Theorem, a foundational result allowing one to create stochastic processes involving infinite sets of times out of data corresponding to finite subsets of those times. A special case of the theorem provides the existence of infinite sets of independent random variables with specified probability distributions. Section 4 establishes the celebrated Strong Law of Large Numbers, which says that the Cesaro sums of a sequence of identically distributed independent random variables with finite mean converge almost everywhere to a constant random variable, the constant being the mean. This is a theorem that is vaguely known to the general public and is widely misunderstood. The proof is based on Kolmogorov’s inequality. Sections 5–8 provide background for the Central Limit Theorem, whose statement and proof are in Section 9. Section 5 discusses three successively weaker kinds of convergence for random variables—almost sure convergence, convergence in probability, and convergence in distribution. Convergence in distribution will be the appropriate kind for the Central Limit Theorem. Section 6 contains the Portmanteau Lemma, which gives some equivalent formulations of convergence in distribution, Section 7 introduces characteristic functions as Fourier transforms of probability distributions, and Section 8 proves the Levy Continuity Theorem, which formulates convergence in distribution in terms of characteristic functions. Section 9 contains the statement and proof of the Central Limit Theorem, followed by some simple examples. This theorem is the most celebrated result in probability theory and has many applications in mathematics and other fields. Section 10 is a brief introduction to the subject of statistical inference, showing how the Central Limit theorem plays a role in practice through the $t$ test of W. S. Gosset.

Over the past few years, a family of interesting new inequalities for the entropies of sums and differences of random variables has been developed by Ruzsa, Tao and others, motivated by analogous results in additive combinatorics. The present work extends these earlier results to the case of random variables taking values in $\mathbb{R}^n$ or, more generally, in arbitrary locally compact and Polish abelian groups. We isolate and study a key quantity, the Ruzsa divergence between two probability distributions, and we show that its properties can be used to extend the earlier inequalities to the present general setting. The new results established include several variations on the theme that the entropies of the sum and the difference of two independent random variables severely constrain each other. Although the setting is quite general, the result are already of interest (and new) for random vectors in $\mathbb{R}^n$. In that special case, quantitative bounds are provided for the stability of the equality conditions in the entropy power inequality; a reverse entropy power inequality for log-concave random vectors is proved; an information-theoretic analog of the Rogers-Shephard inequality for convex bodies is established; and it is observed that some of these results lead to new inequalities for the determinants of positive-definite matrices. Moreover, by considering the multiplicative subgroups of the complex plane, one obtains new inequalities for the differential entropies of products and ratios of nonzero, complex-valued random variables.

Weighted averages with random proportions that are jointly uniformly distributed over the unit simplex

This paper continues a study, initiated in [1], of the stochastic independence of $V = X - Y$ and the mean of order $\gamma$, \begin{equation*}\tag{1}M_\gamma\begin{align*} = (1/\gamma) \log \lbrack(e^{\gamma X} + e^{\gamma Y})/2\rbrack\quad\text{for} \gamma \neq 0 \text{and} \gamma \neq \pm \infty\\ = (X + Y)/2\quad\text{for} \gamma = 0\\ = \max (X, Y)\quad\text{for} \gamma = + \infty\\ = \min (X, Y)\quad \text{for} \gamma = - \infty\end{align*}\end{equation*} for independent $X$ and $Y$. The case $\gamma = 0$ is solved by the well known result, if, for independent $X$ and $Y$, the variables $X + Y$ and $X - Y$ are independent, then both $X$ and $Y$ have normal distributions with a common variance. The first proof of this result may be found in Kac [6]. (Although the statement of this result there is much weaker, Kac's proof applies exactly to the statement above.) A corresponding result for the gamma distribution, proved in restricted form by Hogg [5], and proved without restrictions by Lukacs [7], is as follows: if $Z_1$ and $Z_2$ are independent positive random variables for which $Z_1 + Z_2$ and $Z_1/Z_2$ are independent, then both $Z_1$ and $Z_2$ have gamma distributions with a common scale parameter. This result may be used to complete the study of the independence of $M_\gamma$ and $V$ for independent $X$ and $Y$ when $\gamma$ is finite and non-zero, by letting $Z_1 = \exp (\gamma X)$ and $Z_2 = \exp (\gamma Y)$. It is clear, then, from the result of Hogg and Lukacs that both $\exp (\gamma Y)$ and $\exp (\gamma Y)$ must have gamma distributions with a common scale parameter. The resulting distributions of $X$ and $Y$ have been studied in [1]. Therefore, of this study, there remain yet to be completely solved only the cases $\gamma = +\infty$ and $\gamma = - \infty$. These two cases are essentially the same since if $M_{+\infty}$ and $V$ are independent, then so are $\min (-X, -Y)$ and $V$. Thus, the negatives of the distributions for which $M_{+\infty}$ and $V$ are independent will yield independent $M_{-\infty}$ and $V$. We consider only the case $\gamma = -\infty$. This problem is considered in a previous paper, [2], in which the distributions were restricted to be discrete. The main result of that paper is as follows: if $X$ and $Y$ are independent, non-degenerate, discrete random variables, and if $U = \min (X, Y)$ and $V = X - Y$ are independent, then $X$ and $Y$ both have geometric distributions with common location and scale parameters, but possibly different geometric parameters. If the probability mass function of the geometric distribution is written as \begin{equation*}\tag{2}p(x) = (1 - p)p^{(x-\theta)/c}\quad\text{for} x = \theta, \theta + c, \theta + 2c, \cdots\end{equation*} then, the parameter $\theta$ is a location parameter, $c$ is a scale parameter, and $p$ is called the geometric parameter, $c > 0, 0 < p < 1$. In this paper, the distributions of $X$ and $Y$ are restricted to be absolutely continuous. This will result in a characterization of the so-called exponential distribution whose density is \begin{equation*}\tag{3}f(x) = (1/\sigma) \exp (-(x - \theta)/\sigma)\quad\text{if} x > \theta\end{equation*} and zero otherwise. The parameter $\theta$ is a location parameter, and the parameter $\sigma > 0$ is a scale parameter, which is also the standard deviation of the distribution. In these terms the main result of this paper may be stated as follows: if $X$ and $Y$ are independent random variables with absolutely continuous distributions, and if $U = \min (X, Y)$ and $V = X - Y$ are independent, then both $X$ and $Y$ have exponential distributions with a common location parameter but with possibly different scale parameters. This result then gives a characterization of the exponential distribution, since it is easy to show the converse, that if $X$ and $Y$ are independent and if each has an exponential distribution with a common location parameter but with possibly different scale parameters, then $U$ and $V$ are independent. For related results, the reader is referred to papers by Fisz [3] and Rogers [8].

The prophet inequality for a sequence of independent nonnegative random variables shows that the ratio of the mean of the maximum of the sequence to the optimal expected return using stopping times is always bounded by 2; i.e., on average, the proportional advantage of a prophet with complete foresight over a gambler using nonanticipating stopping rules is at most 2. Here, an inequality linking the mean of the sum of the $k$ largest order statistics of the sequence and the optimal expected return is derived. This implies that if the $k$ largest order statistics are close to the maximum in mean then the proportional advantage of the prophet is at most of order $(k + 1)/k$. An extension of the additive prophet inequality for uniformly bounded independent random variables is also given.

Measuring association and dependence between random vectors

Given a sequence $$(X_n)$$ of symmetrical random variables taking values in a Hilbert space, an interesting open problem is to determine the conditions under which the series $$\sum _{n=1}^\infty X_n$$ is almost surely convergent. For independent random variables, it is well known that if $$\sum _{n=1}^\infty \mathbb {E}(\Vert X_n\Vert ^2) <\infty $$ , then $$\sum _{n=1}^\infty X_n$$ converges almost surely. This has been extended to some cases of dependent variables (namely negatively associated random variables), but in the general setting of dependent variables, the problem remains open. This paper considers the case where each variable $$X_n$$ is given as a linear combination $$a_{n,1}Z_1+ \cdots +a_{n,n}Z_n$$ where $$(Z_n)$$ is a sequence of independent symmetrical random variables of unit variance and $$(a_{n,k})$$ are constants. For Gaussian random variables, this is the general setting. We obtain a sufficient condition for the almost sure convergence of $$\sum _{n=1}^\infty X_n$$ which is also sufficient for the almost sure convergence of $$\sum _{n=1}^\infty \pm X_n$$ for all (non-random) changes of sign. The result is based on an important bound of the mean of the random variable $$\sup (\Vert X_1 + \cdots +X_k\Vert : 1\le k \le n)$$ which extends the classical Levy’s inequality and has some independent interest.

In stochastic simulation, we construct mathematical models to imitate the behavior of real systems, use computers to sample behavioral histories (sample paths) of these models, and exploit those samples to improve decision making with the real system. The imitation part can be very challenging, in particular for modeling uncertainty. Fitting univariate probability distribution to data is far from sufficient. Modeling the dependence is very important and much more challenging. It involves multivariate distributions, copulas, stochastic processes, and other complicated stochastic objects. Simulating the model on a computer also involves an imitation game, to simulate the realizations of random variables and stochastic processes with deterministic algorithms on a computer. Random number generation involves writing deterministic computer programs that can imitate simple probabilistic models such as independent uniform random variables uniformly distributed over the interval (0, 1). An "exact" algorithmic implantation of such models is theoretically impossible, so we settle for a reasonable fake. The talk will give snapshots and expose ideas collected from the author's journey thought stochastic simulation. The tour will start with random number generation and visit some challenging problems such as stochastic modeling, simulation-based optimization, rare events, simulation on parallel processors, and future challenges.

Determining distributions of the functions of random variables is a very important problem with a wide range of applications in Risk Management, Finance, Economics, Science, and many other areas. This paper develops the theory on both density and distribution functions for the quotient Y = X 1 X 2 and the ratio of one variable over the sum of two variables Z = X 1 X 1 + X 2 of two dependent or independent random variables X 1 and X 2 by using copulas to capture the structures between X 1 and X 2 . Thereafter, we extend the theory by establishing the density and distribution functions for the quotients Y = X 1 X 2 and Z = X 1 X 1 + X 2 of two dependent normal random variables X 1 and X 2 in the case of Gaussian copulas. We then develop the theory on the median for the ratios of both Y and Z on two normal random variables X 1 and X 2 . Furthermore, we extend the result of median for Z to a larger family of symmetric distributions and symmetric copulas of X 1 and X 2 . Our results are the foundation of any further study that relies on the density and cumulative probability functions of ratios for two dependent or independent random variables. Since the densities and distributions of the ratios of both Y and Z are in terms of integrals and are very complicated, their exact forms cannot be obtained. To circumvent the difficulty, this paper introduces the Monte Carlo algorithm, numerical analysis, and graphical approach to efficiently compute the complicated integrals and study the behaviors of density and distribution. We illustrate our proposed approaches by using a simulation study with ratios of normal random variables on several different copulas, including Gaussian, Student-t, Clayton, Gumbel, Frank, and Joe Copulas. We find that copulas make big impacts from different Copulas on behavior of distributions, especially on median, spread, scale and skewness effects. In addition, we also discuss the behaviors via all copulas above with the same Kendall’s coefficient. The approaches developed in this paper are flexible and have a wide range of applications for both symmetric and non-symmetric distributions and also for both skewed and non-skewed copulas with absolutely continuous random variables that could contain a negative range, for instance, generalized skewed-t distribution and skewed-t Copulas. Thus, our findings are useful for academics, practitioners, and policy makers.

Many randomized algorithms run successfully even when the random choices they utilize are not fully independent. For the analysis some limited amount of independence, like k-wise independence for some fixed k, often suffices. In these cases, it is possible to replace the appropriate exponentially large sample spaces required to simulate all random choices of the algorithms by ones of polynomial size. This enables one to derandomize the algorithms, that is, convert them into deterministic ones, by searching the relatively small sample spaces deterministically. If a random variable attains a certain value with positive probability, then we can actually search and find a point in which it attains such a value.The observation that n − 1 pairwise independent nontrivial random variables can be defined over a sample space of size n has been mentioned already long ago, see [11], [23]. The pairwise independent case has been a crucial ingredient in the construction of efficient hashing schemes in [14], [17]. A more general construction, of small sample spaces supporting k-wise independent random variables, appeared in [19]. For the case of binary, uniform random variables this is treated under the name orthogonal arrays in the Coding Theory literature, see, e.g., [27]. Most constructions are based on some simple properties of polynomials over a finite field or on certain explicit error correcting codes.Several researchers realized that constructions of this type are useful for derandomizing parallel algorithms, since one may simply check all points of the sample space in parallel. Papers pursuing this idea include [1], [22], [24], and papers dealing with the properties of the constructions in which the sample spaces are not necessarily uniform include [20], [21]. It can be shown that for fixed k, the minimum size of a sample space supporting n k-wise independent random variables is Ω(n ⌊k/2⌋. For the binary uniform case this is essentially the Rao bound [30] (see also [12], [16]), whereas for the general case it is shown inThe above techniques have been applied in numerous papers dealing with derandomization, and we make no attempt to list all of them here. Examples include parallelization of derandomized geometric algorithms in [10], [18], and various parallel graph algorithms [1], [9], [22], [24], [28]. It turned out that some variants of the techniques are also useful in derandomizing sequential algorithmsIn the talk I will survey the basic ideas in the constructions of small sample spaces and discuss some of the applications, focusing on various recent results that illustrate the somewhat surprising relevance of the techniques to the solutions of several algorithmic problems.

Reliability and non-reliability studies of Poisson variables in series and parallel systems

The authors investigate four approaches that can be used to compute the distribution of a sum of independent lognormal random variables (RVs). Simulation results for the complementary cumulative distribution function (CDF) of sums of independent lognormal RVs are used for verification and comparison. The aim is to determine which method is best for computing the complementary cumulative distribution function. The problem is formally stated and some observations regarding different approaches to the problem are given. Three related methods based on lognormal approximation of the sum CDF are described and assessed. An assessment of a different approach. Farley's method, that has been reported in the literature is described. An intuitive basis for the method is also given. A comparative examination of the different methods is presented. This leads to an explanation of the differences of the approximations. >

Let $(x_{nk}), k = 1, \cdots, k_n; n = 1, 2, \cdots$ be a double sequence of infinitesimal random variables which are rowwise independent (i.e., $\lim_{n\rightarrow\infty} \max_{1\leqq k \leqq k_n} P (| x_{nk} | > \epsilon) = 0$ for every $\epsilon > 0,$ and for each $n x_{n1}, \cdots, x_{nk_n}$ are independent). Let $S_n = x_{n1} + \cdots + x_{nk_n} - A_n$ where the $A_n$ are constants and let $F_n(x)$ be the distribution function of $S_n.$ Necessary and sufficient conditions for $F_n(x)$ to converge to a distribution function $F(x)$ are known, and in particular we know that $F(x)$ is infinitely divisible. In this paper we shall investigate the system of infinitesimal, rowwise independent random variables $(| x_{nk} | ^r), r \geqq 1.$ In particular we shall be interested in large values of $r$. Specifically, let $S^r_n = | x_{n1} | ^r + \cdots + | x_{n1} | ^r - B_n(r),$ where $B_n(r)$ are suitably chosen constants. Let $F_n^r(x)$ be the distribution function of $S^r_n.$ Necessary and sufficient conditions for $F_n^r(x)$ to converge $(n \rightarrow \infty)$ to a distribution function $F^r(x)$ are given, and also necessary and sufficient conditions for $F^r(x)$ to converge $(r \rightarrow \infty)$ to a distribution function $H(x)$ are given. The form that $H(x)$ must take is obtained and under rather general conditions it is shown that $H(x)$ is a Poisson distribution. In any case it is shown that $H(x)$ is the sum of two independent random variables, one Gaussian and the other Poisson (including their degenerate cases).

The workload of a generalized n-site asymmetric simple inclusion process (ASIP) is investigated. Three models are analyzed. The first model is a serial network for which the steady-state Laplace–Stieltjes transform (LST) of the total workload in the first k sites (\(k\le n\)) just after gate openings and at arbitrary epochs is derived. In a special case, the former (just after gate openings) turns out to be an LST of the sum of k independent random variables. The second model is a 2-site ASIP with leakage from the first queue. Gate openings occur at exponentially distributed intervals, and the external input processes to the stations are two independent subordinator Lévy processes. The steady-state joint workload distribution right after gate openings, right before gate openings and at arbitrary epochs is derived. The third model is a shot-noise counterpart of the second model where the workload at the first queue behaves like a shot-noise process. The steady-state total amount of work just before a gate opening turns out to be a sum of two independent random variables.