Rate estimates for total variation distance with applications

The total variation distance is a core statistical distance between probability measures that satisfies the metric axioms, with value always falling in [0,1]. Since the total variation distance does not admit closed-form expressions for statistical mixtures, one often has to rely in practice on costly numerical integrations or on fast Monte Carlo approximations that however do not guarantee deterministic bounds. In this work, we consider two methods for bounding the total variation of univariate mixture models: The first is based on the information monotonicity property of the total variation to design guaranteed nested deterministic lower bounds. The second method relies on computing the geometric lower and upper envelopes of weighted mixture components to derive deterministic bounds based on density ratio. We demonstrate the tightness of our bounds through simulating Gaussian, Gamma and Rayleigh mixture models.

We consider the problem of discrete distribution estimation under $\ell _{1}$ loss. We provide tight upper and lower bounds on the maximum risk of the empirical distribution (the maximum likelihood estimator), and the minimax risk in regimes where the support size $S$ may grow with the number of observations $n$ . We show that among distributions with bounded entropy $H$ , the asymptotic maximum risk for the empirical distribution is $2H/\ln n$ , while the asymptotic minimax risk is $H/\ln n$ . Moreover, we show that a hard-thresholding estimator oblivious to the unknown upper bound $H$ , is essentially minimax. However, if we constrain the estimates to lie in the simplex of probability distributions, then the asymptotic minimax risk is again $2H/\ln n$ . We draw connections between our work and the literature on density estimation, entropy estimation, total variation distance ( $\ell _{1}$ divergence) estimation, joint distribution estimation in stochastic processes, normal mean estimation, and adaptive estimation.

The aim of this paper is to address optimality of stochastic control strategies via dynamic programming subject to total variation distance ambiguity on the conditional distribution of the controlled process. We formulate the stochastic control problem using minimax theory, in which the control minimizes the payoff while the conditional distribution, from the total variation distance set, maximizes it. First, we investigate the maximization of a linear functional on the space of probability measures on abstract spaces, among those probability measures which are within a total variation distance from a nominal probability measure, and then we give the maximizing probability measure in closed form. Second, we utilize the solution of the maximization to solve minimax stochastic control with deterministic control strategies, under a Markovian and a non-Markovian assumption, on the conditional distributions of the controlled process. The results of this part include (1) minimax optimization subject to total va...

The Kolmogorov and total variation distance between the laws of random variables have upper bounds represented by the L1-norm of densities when random variables have densities. In this paper, we derive an upper bound, in terms of densities such as the Kolmogorov and total variation distance, for several probabilistic distances (e.g., Kolmogorov distance, total variation distance, Wasserstein distance, Forter–Mourier distance, etc.) between the laws of F and G in the case where a random variable F follows the invariant measure that admits a density and a differentiable random variable G, in the sense of Malliavin calculus, and also allows a density function.

An (n, k)-Poisson Multinomial Distribution (PMD) is the distribution of the sum of n independent random vectors supported on the set Bk={e1,...,ek} of standard basis vectors in Rk. We prove a structural characterization of these distributions, showing that, for all ε > 0, any (n, k)-Poisson multinomial random vector is ε-close, in total variation distance, to the sum of a discretized multidimensional Gaussian and an independent (poly(k/ε), k)-Poisson multinomial random vector. Our structural characterization extends the multi-dimensional CLT of Valiant and Valiant, by simultaneously applying to all approximation requirements ε. In particular, it overcomes factors depending on log n and, importantly, the minimum Eigen value of the PMD's covariance matrix. We use our structural characterization to obtain an ε-cover, in total variation distance, of the set of all (n, k)-PMDs, significantly improving the cover size of Daskalakis and Papadimitriou, and obtaining the same qualitative dependence of the cover size on n and ε as the k=2 cover of Daskalakis and Papadimitriou. We further exploit this structure to show that (n, k)-PMDs can be learned to within ε in total variation distance from Õk(1/ε) samples, which is near-optimal in terms of dependence on ε and independent of n. In particular, our result generalizes the single-dimensional result of Daskalakis, Diakonikolas and Servedio for Poisson binomials to arbitrary dimension. Finally, as a corollary of our results on PMDs, we give a Õk(1/ε2) sample algorithm for learning (n, k)-sums of independent integer random variables (SIIRVs), which is near-optimal for constant k.

This paper starts with a study of the minimum of the Renyi divergence, of an arbitrary order α > 0, subject to a fixed (or minimal) value of the total variation distance. Relying on the solution of this minimization problem, we determine the exact region of the points (D(Q∥P 1 ), D(Q∥P 2 )) where P 1 and P 2 are any probability distributions whose total variation distance is not below a fixed value, and the probability distribution Q is arbitrary (none of these three distributions is assumed to be fixed). It is further shown that all the points of this convex region are attained by a triple of 2-element probability distributions. As a byproduct of this characterization, we provide a geometric interpretation of the minimal Chernoff information subject to a minimal total variation distance. A full paper version, which includes more results and proofs, is available at http://arxiv.org/abs/1501.03616.

We derive new upper bounds for the total variation distance between compound Poisson distributions as well as between a random sum and a compound Poisson distribution, and as a result we also obtain upper bounds for the total variation distance between compound Poisson distributions and a sum of independent random variables. These bounds are generalizations and refinements of some well-known bounds in the literature. We also derive upper bounds for the total variation distance between negative binomial distributions of order k and between the negative binomial distributions of order k and compound Poisson distributions. Upper bounds for the total variation distances between the number of success runs of length k in binary Markovian trials and its limiting distributions for several enumeration schemes, are also given.

Article Free Access Share on Exact sampling and approximate counting techniques Author: Mark Huber School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NY School of Operations Research and Industrial Engineering, Cornell University, Ithaca, NYView Profile Authors Info & Claims STOC '98: Proceedings of the thirtieth annual ACM symposium on Theory of computingMay 1998 Pages 31–40https://doi.org/10.1145/276698.276709Published:23 May 1998Publication History 25citation666DownloadsMetricsTotal Citations25Total Downloads666Last 12 Months66Last 6 weeks15 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

We consider extremal values of the characteristics of nonrandomized statistical criteria in the following case: there are $n$ hypotheses with given total variation distances. It is shown that finding the extremal values of a function of error probabilities may be reduced to solving a finite-dimensional linear programming problem. We found exact formulas for extreme values of the sum of all error probabilities in the case n=3.

We explore a class of random combinatorial structures called weighted multisets. Their components are taken from an initial set satisfying general boundedness conditions posed on the number of elements with a given weight. The component vector of a multiset of weight n taken with equal probability has dependent coordinates, nevertheless, up to r = o(n) of them as n→∞, we approximate by an appropriate vector comprised from independent negative binomial random variables. The main result is an estimate of the total variation distance.

Random graphs with latent geometric structure are popular models of social and biological networks, with applications ranging from network user profiling to circuit design. These graphs are also of purely theoretical interest within computer science, probability and statistics. A fundamental initial question regarding these models is: when are these random graphs affected by their latent geometry and when are they indistinguishable from simpler models without latent structure, such as the Erdős–Rényi graph $${\mathcal {G}}(n, p)$$ ? We address this question for two of the most well-studied models of random graphs with latent geometry—the random intersection and random geometric graph. Our results are as follows: (a) The random intersection graph is defined by sampling n random sets $$S_1, S_2, \ldots , S_n$$ by including each element of a set of size d in each $$S_i$$ independently with probability $$\delta $$ , and including the edge $$\{i, j\}$$ if $$S_i \cap S_j \ne \emptyset $$ . We prove that the random intersection graph converges in total variation to an Erdős–Rényi graph if $$d = {\tilde{\omega }}(n^3)$$ , and does not if $$d = o(n^3)$$ , for both dense and sparse edge densities p. This resolves an open problem in Fill et al. (Random Struct Algorithms 16(2):156–176, 2000), Rybarczyk (Random Struct Algorithms 38(1–2):205–234, 2011) and Kim et al. (Random Struct Algorithms 52(4):662–679, 2018). The same result was obtained simultaneously and independently by Bubeck et al. (When random intersection graphs lose geometry. Manuscript, 2019). (b) We strengthen the preceding argument to show that the matrix of random intersection sizes $$|S_i \cap S_j|$$ converges in total variation to a symmetric matrix with independent Poisson entries. This yields the first total variation convergence result for $$\tau $$ -random intersection graphs, where the edge $$\{i, j\}$$ is included if $$|S_i \cap S_j| \ge \tau $$ . More precisely, our results imply that, if p is bounded away from 1, then the $$\tau $$ -random intersection graph with edge density p converges to $${\mathcal {G}}(n, p)$$ if $$d = \omega (\tau ^3 n^3)$$ . (c) The random geometric graph on $${\mathbb {S}}^{d - 1}$$ is defined by sampling $$X_1, X_2, \dots , X_n$$ uniformly at random from $${\mathbb {S}}^{d - 1}$$ and including the edge $$\{i, j\}$$ if $$\Vert X_i - X_j \Vert _2 \le \tau $$ . A result of Bubeck et al. (Random Struct Algorithms 49:503–532, 2016) showed that this model converges to $${\mathcal {G}}(n, p)$$ in total variation, where p is chosen so that the models have matching edge densities, as long as $$d = \omega (n^3)$$ . It was conjectured in Bubeck et al. (2016) that this threshold decreases drastically for p small. We make the first progress towards this conjecture by showing convergence if $$d = {\tilde{\omega }}\left( \min \{ pn^3, p^2 n^{7/2} \} \right) $$ . Our proofs are a hybrid of combinatorial arguments, direct couplings and applications of information inequalities. Previous upper bounds on the total variation distance between random graphs with latent geometry and $${\mathcal {G}}(n, p)$$ have typically not been both combinatorial and information-theoretic, while this interplay is essential to the sharpness of our bounds.

This dissertation addresses a central methodological limitation of smartphone-based travel surveys (SBTS): systematic missingness in passively collected location data. While smart surveys that combine self-reported information with sensor-based measurements offer substantial advantages over conventional travel diaries—including reduced recall bias, higher trip detection rates, and lower respondent burden—their analytical validity depends critically on how incomplete trajectory data are handled. The research investigates three interrelated questions: (1) what mechanisms generate missing location data in SBTS, (2) how missingness affects commonly used mobility indicators, and (3) whether imputation methods can produce complete datasets suitable for reliable mobility analysis. These questions are examined using empirical data from a large-scale field test conducted by Statistics Netherlands in 2018, in which participants used the Tabi travel app to record and annotate their movements, complemented by extensive simulation studies. Empirical analysis shows that missing data are structural rather than incidental. On average, participants recorded approximately one-third of the intended daily observation time, with data completeness varying systematically by device type, operating system behavior, and user characteristics. Comparisons with the national travel diary survey (ODiN) indicate that smartphone data capture more trips—particularly short and incidental movements—but produce biased estimates of total travel distance when missingness is ignored. These findings demonstrate that SBTS introduce new forms of measurement heterogeneity that are not addressed by conventional data-processing practices. Simulation experiments quantify the conditions under which missing data become analytically problematic. Short gaps in location data (under approximately ten minutes) can be addressed using linear interpolation with limited bias across standard mobility metrics, including total distance traveled and radius of gyration. In contrast, longer gaps produce non-linear bias amplification, with naive interpolation substantially underestimating travel behavior, especially when missingness is temporally clustered. To address this limitation, the dissertation introduces Dynamic Time Warping–Based Multiple Imputation (DTWBMI), a trajectory-specific imputation framework designed to reconstruct long gaps in mobility data. DTWBMI matches incomplete daily trajectories to reference days based on temporal movement similarity and generates multiple plausible realizations to propagate uncertainty. Two variants are developed for data-rich and data-poor contexts, respectively. Evaluation results show that DTWBMI substantially outperforms linear interpolation for long gaps, with the low-information variant exhibiting unexpectedly robust performance. The proposed methods are integrated into a hierarchical imputation framework that combines interpolation, DTWBMI, and multiple imputation depending on gap duration. Application to real SBTS data yields stable mobility estimates with reduced bias and uncertainty compared to common practices such as listwise deletion. Overall, the dissertation reframes missing data in SBTS from a prohibitive flaw into a manageable methodological challenge, providing empirically validated tools that support the integration of smartphone-based mobility measurement into transport research and official statistics.

The purpose of this application paper is to apply the Stein-Chen (SC) method to provide a Poisson-based approximation and corresponding total variation distance bounds in a time series context. The SC method that is used approximates the probability density function (PDF) defined on how many times a pattern such as It,It+1,It+2 = {1 0 1} occurs starting at position t in a time series of length N that has been converted to binary values using a threshold. The original time series that is converted to binary is assumed to consist of a sequence of independent random variables, and could, for example, be a series of residuals that result from fitting any type of time series model. Note that if {1 0 1} is known to not occur, for example, starting at position t = 1, then this information impacts the probability that {1 0 1} occurs starting at position t = 2 or t = 3, because the trials to obtain {1 0 1} are overlapping and thus not independent, so the Poisson distribution assumptions are not met. Nevertheless, the results shown in four examples demonstrate that Poisson-based approximation (that is strictly correct only for independent trials) can be remarkably accurate, and the SC method provides a bound on the total variation distance between the true and approximate PDF.

We study the distribution of the matrix product G₁ G₂ ⋯ G_r of r independent Gaussian matrices of various sizes, where G_i is d_{i-1} × d_i, and we denote p = d₀, q = d_r, and require d₁ = d_{r-1}. Here the entries in each G_i are standard normal random variables with mean 0 and variance 1. Such products arise in the study of wireless communication, dynamical systems, and quantum transport, among other places. We show that, provided each d_i, i = 1, …, r, satisfies d_i ≥ C p ⋅ q, where C ≥ C₀ for a constant C₀ > 0 depending on r, then the matrix product G₁ G₂ ⋯ G_r has variation distance at most δ to a p × q matrix G of i.i.d. standard normal random variables with mean 0 and variance ∏_{i = 1}^{r-1} d_i. Here δ → 0 as C → ∞. Moreover, we show a converse for constant r that if d_i < C' max{p,q}^{1/2}min{p,q}^{3/2} for some i, then this total variation distance is at least δ', for an absolute constant δ' > 0 depending on C' and r. This converse is best possible when p = Θ(q).

This paper studies the fundamental limits of covert communications covered by randomly activated overt users in both single-frame and multi-frame transmission scenarios. While traditional covert communications mainly consider concealing signal power characteristics, the existence of overt users provides opportunities such that covert communications can be achieved through the confusion between the users. This benefit is first revealed in single-frame transmission scenario. The major obstacle in analyzing performance limits is that the conventional Kullback-Leibler divergence based covertness measurement becomes infinite. To overcome the intractability, a tighter upper bound of the total variation distance (TVD) is then developed using a novel recursive-iterative approximation. On this basis, the collapse effect of the TVD is derived, which shows that the TVD is strictly less than 1 if the covert user sets the transmit power to be an integer multiple of that of the overt users. Then, we find that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(N)$ </tex-math></inline-formula>-bit information can be transmitted over N channel uses under the above setting, which breaks the well-known square root law. If the above setting is violated, the TVD instantly approaches 1 as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N\rightarrow \infty $ </tex-math></inline-formula>, and only <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(\sqrt {N})$ </tex-math></inline-formula>-bit information can be covertly transmitted. To prove this, the detection method of the warden is modified to cope with the random activation of overt users. These conclusions also hold for the transmission with uncertain powers or in fading channels, which resembles realistic wireless transmissions. In multi-frame transmission scenario, however, the access characteristics of overt users can be exposed from a statistical perspective, such that the rate gain disappears and the covert transmission rate drops to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(\sqrt {N})$ </tex-math></inline-formula> bits per frame. To obtain a positive covert transmission rate, we propose a rate-splitting based covert transmission scheme that introduces an opportunistic access branch to bring randomness, through which the covert user can transmit up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(NL)$ </tex-math></inline-formula>-bit information over L frames.