3 - Random Variables and Their Probability Distributions

Copulas have become an important analytic tool for characterizing multivariate distributions and dependence. One is often interested in simulating data from copula estimates. The process can be analytically and computationally complex and usually involves steps that are unique to a given parametric copula. We describe an alternative approach that uses \probability{proportional{to{size" (PPS) random sampling with weights formed from the copula likelihood function. The method is flexible and can be applied to parametric and nonparametric marginal density estimates. The precision of the simulation can be calibrated by adjusting the density of the multidimensional grid used in the simulation process. The approach is fully transparent to any copula function with continuous random variables. An example evaluates a number of goodness-of-fit criteria and provides strong support for the validity and practicality of the method.

Decisions are often based on imprecise, uncertain or vague information. Likewise, the consequences of an action are often equally unpredictable, thus putting the decision maker into a twofold jeopardy. Assuming that the effects of an action can be modeled by a random variable, then the decision problem boils down to comparing different effects (random variables) by comparing their distribution functions. Although the full space of probability distributions cannot be ordered, a properly restricted subset of distributions can be totally ordered in a practically meaningful way. We call these loss-distributions, since they provide a substitute for the concept of loss-functions in decision theory. This article introduces the theory behind the necessary restrictions and the hereby constructible total ordering on random loss variables, which enables decisions under uncertainty of consequences. Using data obtained from simulations, we demonstrate the practical applicability of our approach.

In the past, residual factor R in strain-softening soil slopes has been included, either directly or indirectly, as a deterministic variable in both deterministic and probabilistic studies. This paper discusses the uncertainties associated with R and outlines a systematic approach for the reliability analysis of a natural slope in which shear strength parameters and pore pressure ratio are random variables, each assumed with a lognormal probability distribution. For the residual factor R, seven probability distribution options under the generalized beta-distribution system are considered. Slope reliability is computed based on the first order reliability method (FORM) and validated against Monte-Carlo simulation (MCS). Results obtained from two illustrative examples indicate that the probability of failure, with R as one of six random variables, can be orders of magnitude higher than that based on five random variables with R considered as a deterministic parameter. The magnitude of influence of R as a random variable is, however, highly dependent on its probability distribution, the left-skewed triangular distribution having the most significant influence in both the examples. Results of sensitivity analyses reveal that, for almost all of its assumed probability distributions, R is the most dominant among the six random variables. Effects of variation of some of the statistical and correlation properties of the other random variables, viz. the shear strength parameters and the pore pressure ratio, on the results of reliability analyses are also studied.

This chapter introduces probability theory as a system of models, based on measure theory, of some real-world phenomena. The models are measure spaces of total measure 1 and usually have certain distinguished measurable functions defined on them. Section 1 begins by establishing the measure-theoretic framework and a short dictionary for passing back and forth between terminology in measure theory and terminology in probability theory. The latter terminology includes events, random variables, mean, probability distribution of a random variable, and joint probability distribution of several random variables. An important feature of probability is that it is possible to work with random variables without any explicit knowledge of the underlying measure space, the joint probability distributions of random variables being the objects of importance. Section 2 introduces conditional probability and uses that to motivate the mathematical definition of independence of events. In turn, independence of events leads naturally to a definition of independent random variables. Independent random variables are of great importance in the subject and play a much larger role than their counterparts in abstract measure theory. Examples at the end of the section indicate the extent to which functions of independent random variables can remain independent. The techniques in the examples are of use in the subject of statistical inference, which is introduced in Section 10. Section 3 states and proves the Kolmogorov Extension Theorem, a foundational result allowing one to create stochastic processes involving infinite sets of times out of data corresponding to finite subsets of those times. A special case of the theorem provides the existence of infinite sets of independent random variables with specified probability distributions. Section 4 establishes the celebrated Strong Law of Large Numbers, which says that the Cesaro sums of a sequence of identically distributed independent random variables with finite mean converge almost everywhere to a constant random variable, the constant being the mean. This is a theorem that is vaguely known to the general public and is widely misunderstood. The proof is based on Kolmogorov’s inequality. Sections 5–8 provide background for the Central Limit Theorem, whose statement and proof are in Section 9. Section 5 discusses three successively weaker kinds of convergence for random variables—almost sure convergence, convergence in probability, and convergence in distribution. Convergence in distribution will be the appropriate kind for the Central Limit Theorem. Section 6 contains the Portmanteau Lemma, which gives some equivalent formulations of convergence in distribution, Section 7 introduces characteristic functions as Fourier transforms of probability distributions, and Section 8 proves the Levy Continuity Theorem, which formulates convergence in distribution in terms of characteristic functions. Section 9 contains the statement and proof of the Central Limit Theorem, followed by some simple examples. This theorem is the most celebrated result in probability theory and has many applications in mathematics and other fields. Section 10 is a brief introduction to the subject of statistical inference, showing how the Central Limit theorem plays a role in practice through the $t$ test of W. S. Gosset.

In mathematics and statistics, the desire to eliminate mathematical tedium and facilitate exploration has lead to computer algebra systems. These computer algebra systems allow students and researchers to perform more of their work at a conceptual level. The design of generic algorithms for tedious computations allows modelers to push current modeling boundaries outward more quickly. Probability theory, with its many theorems and symbolic manipulations of random variables is a discipline in which automation of certain processes is highly practical, functional, and efficient. There are many existing statistical software packages, such as SPSS, SAS, and S-Plus, that have numeric tools for statistical applications. There is a potential for a probability package analogous to these statistical packages for manipulation of random variables. The software package being developed as part of this dissertation, referred to as “A Probability Programming Language” (APPL) is a random variable manipulator and is proposed to fill a technology gap that exists in probability theory. My research involves developing algorithms for the manipulation of discrete random variables. By defining data structures for random variables and writing algorithms for implementing common operations, more interesting and mathematically intractable probability problems can be solved, including those not attempted in undergraduate statistics courses because they were deemed too mechanically arduous. Algorithms for calculating the probability density function of order statistics, transformations, convolutions, products, and minimums/maximums of independent discrete random variables are included in this dissertation.

Consider the sample X1, X2, ..., XN of N independent and identically distributed (iid) random variables with common cumulative distribution function (cdf)F, and let Fu be their conditional excess distribution function F. We define the ordered sample by . Pickands (1975), Balkema and de Haan (1974) posed that for a large class of underlying distribution functions F , and large u ,Fu is well approximated by the Generalized Pareto Distribution.The mixed method is a method for determining thresholds. This method consists in minimizing the variance of a convex combination of other thresholds.The objective of the mixed method is to determine by which probability distribution one can approach this conditional distribution. In this article, we propose a theorem which specifies the conditional distribution of excesses when the deterministic threshold tends to the end point.

<p><span>Measurement uncertainty analysis based on combining the state-of-knowledge distributions requires evaluation of the probability density function (PDF), the cumulative distribution function (CDF), and/or the quantile function (QF) of a random variable reasonably associated with the measurand. This can be derived from the characteristic function (CF), which is defined as a Fourier transform of its probability distribution function. Working with CFs provides an alternative and frequently much simpler route than working directly with PDFs and/or CDFs. In particular, derivation of the CF of a weighted sum of independent random variables is a simple and trivial task. However, the analytical derivation of the PDF and/or CDF by using the inverse Fourier transform is available only in special cases. Thus, in most practical situations, a numerical derivation of the PDF/CDF from the CF is an indispensable tool. In metrological applications, such approach can be used to form the probability distribution for the output quantity of a measurement model of additive, linear or generalized linear form. In this paper we propose new original algorithmic implementations of methods for numerical inversion of the characteristic function which are especially suitable for typical metrological applications. The suggested numerical approaches are based on the Gil-Pelaez inverse formulae and on using the approximation by discrete Fourier transform and the fast Fourier transform (FFT) algorithm for computing PDF/CDF of the univariate continuous random variables. As illustrated here, for typical metrological applications based on linear measurement models the suggested methods are an efficient alternative to the standard Monte Carlo methods.</span></p>

This chapter reviews some subjects in mathematics and physics that are used in different contexts throughout this book. The selection of subjects and the level of their coverage reflect the author’s perception of what potential users of this text were exposed to in their earlier studies. Therefore, only brief overview is given of some subjects while somewhat more comprehensive discussion is given of others. In neither case can the coverage provided substitute for the actual learning of these subjects that are covered in detail by many textbooks. A random variable is an observable whose repeated determination yields a series of numerical values (“realizations” of the random variable) that vary from trial to trial in a way characteristic of the observable. The outcomes of tossing a coin or throwing a die are familiar examples of discrete random variables. The position of a dust particle in air and the lifetime of a light bulb are continuous random variables. Discrete random variables are characterized by probability distributions; Pn denotes the probability that a realization of the given random variable is n. Continuous random variables are associated with probability density functions P(x): P(x1)dx denotes the probability that the realization of the variable x will be in the interval x1 . . . x1+dx.

A method was proposed for calculating the probability distribution of a discrete random variable (DRV) using a d-ary tree. The tree has n leaves, which is equivalent to the number of elements in the DRV probability distribution, and the remaining vertices represent the DRV generators with the probability distribution including d elements (d-DRV). Variation in the DRV probability distribution by zeroing one of its elements reduces to the recalculation of the d-DRV probability distributions, the number of which increases logarithmically with respect to the value of n.

Input modeling is the selection of a probability distribution to capture the uncertainty in the input environment of a stochastic system. Example applications of input modeling include the representation of the randomness in the time to failure for a machining process, the time between arrivals of calls to a call center, and the demand received for a product of an inventory system. Building simulations of stochastic systems requires the development of input models that adequately represent the uncertainty in such random variables. Since there are an abundance of probability distributions that can be used for this purpose, a natural question to ask is how to identify the probability distribution that best represents the particular situation under study. For example, is the exponential distribution a reasonable choice to represent the time to failure for a machining process, or is it better to use an empirical distribution function obtained from the historical time-to-failure data? Recognizing the fact that there is no true input model waiting to be found, the goal of stochastic input modeling is to obtain an approximation that captures the key characteristics of the system inputs. The development of a good input model requires the collection of as much information as possible about the relevant randomness in the system as well as the historical data consisting of the past realizations of the random variables of interest. In the presence of a data set, the input model can be identified by fitting a probability distribution to the historical data. However, it may be difficult and/or costly to collect data for the stochastic system under study; it can also be impossible to properly collect any data at all such as when the proposed system does not exist. In the absence of historical data, any relevant information (e.g., expert opinion and the conventional bounds suggested by the underlying physical situation) can be used for input modeling. This article addresses the key issues that arise in stochastic input modeling both in the presence and in the absence of historical data. The first step in input modeling is to identify the sources of randomness in the input environment of the system under study. Many stochastic systems contain multiple sources of uncertainty, e.g., the completion time of an item on a particular machine, the potential breakdown of the machine, and the percentage of defective items produced by the machine might be among the sources of uncertainty in a manufacturing setting. Throughout, the random vector X = (X1, X2, …, X K )′ is used to represent the collection of K different inputs of a stochastic system, where X k is the random variable denoting the kth system input. The K components of this random vector might also be correlated with each other. Therefore, the stochastic properties of the random inputs X k , k = 1, 2, …, K, are captured in the joint probability

In this paper, in work strongly related with that of Coffman et al. [5], Bruss and Robertson [2], and Rhee and Talagrand [15], we focus our interest on an asymptotic distributional comparison between numbers of ‘smallest’ i.i.d. random variables selected by either on-line or off-line policies. Let X1,X2,… be a sequence of i.i.d. random variables with distribution function F(x), and let X1,n,…,Xn,n be the sequence of order statistics of X1,…,Xn. For a sequence (cn)n≥1 of positive constants, the smallest fit off-line counting random variable is defined by Ne(cn) := max {j ≤ n : X1,n + … + Xj,n ≤ cn}. The asymptotic joint distributional comparison is given between the off-line count Ne(cn) and on-line counts Nnτ for ‘good’ sequential (on-line) policies τ satisfying the sum constraint ∑j≥1XτjI(τj≤n) ≤ cn. Specifically, for such policies τ, under appropriate conditions on the distribution function F(x) and the constants (cn)n≥1, we find sequences of positive constants (Bn)n≥1, (Δn)n≥1 and (Δ'n)n≥1 such that for some non-degenerate random variables W and W'. The major tools used in the paper are convergence of point processes to Poisson random measure and continuous mapping theorems, strong approximation results of the normalized empirical process by Brownian bridges, and some renewal theory.

In this paper, in work strongly related with that of Coffman et al. [5], Bruss and Robertson [2], and Rhee and Talagrand [15], we focus our interest on an asymptotic distributional comparison between numbers of ‘smallest’ i.i.d. random variables selected by either on-line or off-line policies. Let X 1,X 2,… be a sequence of i.i.d. random variables with distribution function F(x), and let X 1,n ,…,X n,n be the sequence of order statistics of X 1,…,X n . For a sequence (c n ) n≥1 of positive constants, the smallest fit off-line counting random variable is defined by N e (c n ) := max {j ≤ n : X 1,n + … + X j,n ≤ c n }. The asymptotic joint distributional comparison is given between the off-line count N e (c n ) and on-line counts N n τ for ‘good’ sequential (on-line) policies τ satisfying the sum constraint ∑ j≥1 X τ j I (τ j ≤n) ≤ c n . Specifically, for such policies τ, under appropriate conditions on the distribution function F(x) and the constants (c n ) n≥1, we find sequences of positive constants (B n ) n≥1, (Δ n ) n≥1 and (Δ' n ) n≥1 such that for some non-degenerate random variables W and W'. The major tools used in the paper are convergence of point processes to Poisson random measure and continuous mapping theorems, strong approximation results of the normalized empirical process by Brownian bridges, and some renewal theory.

Chapter Three - PROBABILITY DISTRIBUTIONS

The asymptotic cost of many algorithms and combinatorial structures is related to the extreme-value Gumbel distribution exp(-exp(-x)). The following list is not exhaustive: Trie, Digital Search Tree, Leader Election, Adaptive Sampling, Counting Algorithms, trees related to the Register Function, Composition of Integers, some structures represented by Markov chains (Column-Convex Polyominoes, Carlitz Compositions), Runs and number of distinct values of some multiplicity in sequences of geometrically distributed random variables. Sometimes we can start from an exact (discrete) probability distribution, sometimes from an asymptotic analysis of the discrete objects (e.g., urn models) before establishing the relationship with the Gumbel distribution function. Also some Markov chains are either exactly and directly given by the structure itself, or as a limiting Markov process. The main motivation of the paper is to compute the asymptotic distribution and the moments of the random variables in question. The moments are usually given by a dominant part and a small fluctuating part. We use Laplace and Mellin transforms and singularity analysis and aim for a unified treatment in all cases. Furthermore, our goal is a purely mechanical computation of dominant and fluctuating components, with the help of a computer algebra system. We provide each time the first three moments, but the treatment is (almost) completely automatic. We need some real analysis for the approximations and apart from that only easy complex analysis; simple poles and a few special functions.

An important component in the assessment of the economic risk of a prospect prior to development is the calculation of in-place and recoverable reserves and their associated uncertainty. This calculation requires the multiplication of the probability distributions of volume, porosity, net-pay to gross etc. to form the probability distribution of reserves. Two methods have been advocated for this calculation; Parametric methods and Monte-Carlo methods In (i) component distributions are combined analytically to give measures of average and confidence levels in reserves. whereas in (ii) the distribution of the components are randomly sampled to build-up the distribution of reserves. Although at present the Monte-Carlo approach has been widely advocated in the petroleum industry, due to their general applicability, the parametric methods have many adherents citing their ease and speed of use as principle reasons. However, there is a seeming lack of rigorous analysis in either method which would make the selection of the most appropriate method for a particular problem more objective. In this paper we address this area and show how the errors in both methods can. be reduced. Additionally, we suggest a new approach which combines the strengths of both methods into a flexible algorithm for reserves calculations.