General Noise Models

Traditional stochastic approximation (SA) schemes employ a single gradient or a fixed batch of noisy gradients in computing a new iterate. We consider SA schemes in which Nk samples are utilized at step k and the total simulation budget is M, where [EQUATION] Nk ≤ M and K denotes the terminal step. This paper makes the following contributions in the strongly convex regime: (I) We conduct an error analysis for constant batches (Nk = N) under constant and diminishing steplengths and prove linear convergence in terms of expected error in solution iterates based on prescribing Nk in terms of simulation and computational budgets; (II) we extend the linear convergence rates to the setting where Nk is increased at a prescribed rate dependent on simulation and computational budgets; (III) finally, when steplengths are constant, we obtain the optimal number of projection steps that minimizes the bound on the mean-squared error.

We obtain new transport-entropy inequalities and, as a by-product, new deviation estimates for the laws of two kinds of discrete stochastic approximation schemes. The first one refers to the law of an Euler like discretization scheme of a diffusion process at a fixed deterministic date and the second one concerns the law of a stochastic approximation algorithm at a given time-step. Our results notably improve and complete those obtained in [Frikha, Menozzi, 2012]. The key point is to properly quantify the contribution of the diffusion term to the concentration regime. We also derive a general non-asymptotic deviation bound for the difference between a function of the trajectory of a continuous Euler scheme associated to a diffusion process and its mean. Finally, we obtain non-asymptotic bound for stochastic approximation with averaging of trajectories, in particular we prove that averaging a stochastic approximation algorithm with a slow decreasing step sequence gives rise to optimal concentration rate.

Chapter 16 - Long range dependence in heavy tailed stochastic processes

In this paper, we consider the maximizing of the probability $${\mathbb {P}}\left\{ \, \zeta \, \mid \, \zeta \, \in \, {\mathbf {K}}({\mathbf {x}}) \, \right\} $$ over a closed and convex set $${\mathcal {X}}$$ , a special case of the chance-constrained optimization problem. Suppose $${\mathbf {K}}({\mathbf {x}}) \, \triangleq \, \left\{ \, \zeta \, \in \, {\mathcal {K}}\, \mid \, c({\mathbf {x}},\zeta ) \, \ge \, 0 \right\} $$ , and $$\zeta $$ is uniformly distributed on a convex and compact set $${\mathcal {K}}$$ and $$c({\mathbf {x}},\zeta )$$ is defined as either $$c({\mathbf {x}},\zeta )\, \triangleq \, 1-\left| \zeta ^T{\mathbf {x}}\right| ^m$$ where $$m\ge 0$$ (Setting A) or $$c({\mathbf {x}},\zeta ) \, \triangleq \, T{\mathbf {x}}\, - \, \zeta $$ (Setting B). We show that in either setting, by leveraging recent findings in the context of non-Gaussian integrals of positively homogenous functions, $${\mathbb {P}}\left\{ \,\zeta \, \mid \, \zeta \, \in \, {\mathbf {K}}({\mathbf {x}}) \, \right\} $$ can be expressed as the expectation of a suitably defined continuous function $$F(\bullet ,\xi )$$ with respect to an appropriately defined Gaussian density (or its variant), i.e. $${\mathbb {E}}_{{{\tilde{p}}}} \left[ \, F({\mathbf {x}},\xi )\, \right] $$ . Aided by a recent observation in convex analysis, we then develop a convex representation of the original problem requiring the minimization of $$g\left( {\mathbb {E}}\left[ \, F(\bullet ,\xi )\, \right] \right) $$ over $${\mathcal {X}}$$ , where g is an appropriately defined smooth convex function. Traditional stochastic approximation schemes cannot contend with the minimization of $$g\left( {\mathbb {E}}\left[ F(\bullet ,\xi )\right] \right) $$ over $$\mathcal X$$ , since conditionally unbiased sampled gradients are unavailable. We then develop a regularized variance-reduced stochastic approximation (r-VRSA) scheme that obviates the need for such unbiasedness by combining iterative regularization with variance-reduction. Notably, (r-VRSA) is characterized by almost-sure convergence guarantees, a convergence rate of $$\mathcal {O}(1/k^{1/2-a})$$ in expected sub-optimality where $$a > 0$$ , and a sample complexity of $$\mathcal {O}(1/\epsilon ^{6+\delta })$$ where $$\delta > 0$$ . To the best of our knowledge, this may be the first such scheme for probability maximization problems with convergence and rate guarantees. Preliminary numerics on a portfolio selection problem (Setting A) and a set-covering problem (Setting B) suggest that the scheme competes well with naive mini-batch SA schemes as well as integer programming approximation methods.

Wireless traffic and packet-based traffic in general possess heavy-tail marginal distributions and long-range dependence (LRD). The tail parameter /spl alpha/ of a heavy-tail distribution controls the variability of its realizations. Several traffic models, statistical test and resource management algorithms rely on the accurate estimation of the tail parameter. Conventional estimators for the tail parameter only work well when the data is short range dependent. In this paper we propose a new method to estimate the tail parameter from LRD data. This is achieved by utilizing the wavelet transform and extreme value theory. The algorithm is then applied to two stochastic processes that possess both heavy-tail marginal distributions and LRD. Results from our simulation show that the proposed method gives good estimates of /spl alpha/. We then estimate the tail parameter from a recently collected IEEE 802.11b traffic trace.

In recent years, empirical studies on network traffic both in various network configurations including local area networks (LAN) and wide area networks (WAN) convincingly show that the actual traffic exhibit self-similarity and long range dependence (LRD), which are very different from that predicted by traditional telecommunication traffic models, such as Poisson process. Self-similarity in nature brings about the long range dependent burstiness of network traffic. The experimental evidences reveal that the heavy tailness is the key cause for the self-similarity of the network traffic. We present two distinctive predictors based on α-stable innovation for the LRD traffic. The two predictors can minimize the dispersion according to the minimum dispersion criteria with infinite variance. The final predicted values are obtained by combining the previous two individual predicted values. The predicted results for the actual traces show that the two individual predictors are precise and effective, the last compound predictors can enhance the final predicted accuracy.

We consider the problem of scheduling a wireless channel among multiple users. A slot is given to a user with a highest metric (e.g., channel gain) in that slot. The scheduler may not know the channel states of all the users at the beginning of each slot. In this scenario opportunistic splitting is an attractive solution. However this algorithm requires that the metrics of different users form independent, identically distributed (iid) sequences with same distribution and that their distribution and number be known to the scheduler. This limits the usefulness of opportunistic splitting. In this paper we develop a parametric version of this algorithm. The optimal parameters of the algorithm are learnt online through a stochastic approximation scheme. Our algorithm does not require the metrics of different users to have the same distribution. The statistics of these metrics and the number of users can be unknown and also vary with time. We prove the convergence of the algorithm and show its utility by scheduling the channel to maximize its throughput while satisfying some fairness and/or quality of service constraints.

The so‐called accelerated convergence is an ingenuous idea to improve the asymptotic accuracy in stochastic approximation (gradient based) algorithms. The estimates obtained from the basic algorithm are subjected to a second round of averaging, which leads to optimal accuracy for estimates of time‐invariant parameters. In this contribution, some simple calculations are used to get some intuitive insight into these mechanisms. Of particular interest is to investigate the properties of accelerated convergence schemes in tracking situations. It is shown that a second round of averaging leads to the recursive least‐squares algorithm with a forgetting factor. This also means that in case the true parameters are changing as a random walk, accelerated convergence does not, typically, give optimal tracking properties. Copyright © 2001 John Wiley & Sons, Ltd.

Monte-Carlo estimation of time-dependent statistical characteristics of random dynamical systems

The most important desidered characteristic for today's networks is quality of service (QoS). Currently DiffServ-based MPLS networks are able to the development of efficient methods for QoS provisioning. This article analysis, through computer simulation, the performance of WFQ schedulers in a DiffServ node loaded by long range dependence (LRD) traffic. The adopted process for the traffic generation consists in the aggregation of the traffic produced by several TCP connections transmitting files with a size that is modeled by heavy tail distribution. This distribution was obtained, from the Internet, by one-week long measurements. The objective is to observe the bandwidth distribution obtained with the WFQ scheduler, as well as the latency, i.e., the file transfer average delay, in an environment with different priorities. Simulations were carried out with the network simulator NS-2, and several simulation results are presented and analyzed.

Spatial modeling and monitoring are critical in geometric characterization and quality control of material/product surfaces. With advances in metrology technology, a long-range dependence (LRD) effect has recently been detected in spatial data over different fields. The spatial LRD refers to a type of dependence that decays slowly over the distance with heavy tails and non-summable autocovariances so that the correlation is high among surface measurements across long spatial distances. Physically, the spatial LRD effect can be caused by specific spatial patterns such as certain material textures, surface profiles, or manufacturing defects. In literature, although various Markovian and non-Markovian spatial models have been proposed to study material surfaces, none of them has yet considered the LRD effect, which can lead to inefficient surface characterization and inaccurate surface quality control. To overcome the challenge, in this article, we first propose a novel spatial model that can capture the spatial LRD on material surfaces. Both isotropic and anisotropic scenarios of the model are developed based on the Lévy fractional Brownian random field and the fractional Brownian sheet, respectively. Subsequently, based on the proposed spatial model we develop an LRD-integrated quality control framework to monitor surface quality via generalized likelihood ratio test. Comprehensive simulation studies and a real case study using images of wood surfaces are conducted to validate the proposed approach. Results show that the proposed model that integrates LRD significantly outperforms multiple existing models in anomaly detection, and traditional models mis-detect out-of-control surfaces when the spatial LRD actually presents.

We consider the resolution of the structured stochastic convex program: $\min \mathbb{E}[\tilde f(x,\xi)]+\mathbb{E}[\tilde g(y,\xi)]$ such that $Ax + By = b$. To exploit problem structure and allow for developing distributed schemes, we propose a stochastic inexact ADMM ({\bf SI-ADMM}) in which the subproblems are solved inexactly via stochastic approximation schemes. Based on this framework, we prove the following: (i) under suitable assumptions on the number of samples utilized in each iteration, the proposed SI-ADMM produces a sequence that converges to the unique solution almost surely; (ii) If the number of gradient steps (samples) increases at a geometric rate, the mean-squared error under a certain norm diminishes to zero at a prescribed geometric rate. Moreover, a canonical overall sample complexity is demonstrated. Applications of lasso and distributed regression are considered to compare SI-ADMM with other approaches.

We consider a subclass of N-player stochastic Nash games in which each player solves a parametrized stochastic optimization problem. In deterministic regimes, best response schemes have been shown to be convergent under a suitable spectral property associated with the proximal-response map. However, a direct application of this scheme to stochastic settings requires obtaining exact solutions to stochastic optimization problems at every step. Instead, we propose an inexact generalization of this scheme in which an inexact solution to the best response problem is computed where the player-specific inexactness sequence is assumed to be separable. Notably, this scheme is an implementable single-loop scheme that requires a fixed (but increasing) number of stochastic gradient steps to compute an inexact solution to the best response problem. On the basis of this framework, we make several contributions: (i) The presented inexact best-response scheme produces iterates that converge to the unique equilibrium in mean; (ii) Surprisingly, we show that the iterates converge at a prescribed linear rate with a prescribed constant rather than a sub-linear rate; and (iii) Finally, by assuming that an inexact solution is computed by a stochastic approximation scheme, the overall iteration complexity for computing an ∊-Nash equilibrium less that O(√N/∊)2+δ where δ is a positive scalar. Additionally, we show that the upper bound of this effort is shown to be Ω(N=∊2).

The long-range dependence (LRD) of the stochastic part of GPS-derived topocentric coordinates change (North, East, Up) results with relatively high autocorrelation values with a focus on self-similarity. One of the reasons for such self-similarity in the GPS time series are noises that are commonly recognised to prevail in the form of the flicker noise model. To prove the self-similarity of the stochastic part of GPS time series we used more than 130 ASG-EUPOS (active geodetic network EUPOS) stations from an area of Poland with a 5-year span of the daily topocentric coordinate changes. The deterministic part of time series was removed by means of the least-squares (LS) method, median absolute deviation (MAD) criterion and the sequential t-test algorithm, respectively. Then the self-similarity of the residue was proved by the results of the Ljung–Box test, whose values close to zero showed the dependence of the stochastic part of the GPS time series. The residue was analysed by means of the rescaled range (R/S) method with the H (Hurst) parameter and the detrended fluctuation analysis (DFA) with the scaling exponent α. Both H and α values ranged within assumed LRD limits of 0.5 and 1. This analysis was followed by noise investigation with a maximum likelihood estimation (MLE). The white plus the power-law noise models were assumed a priori, which gave us a spectral indices κ between − 0.4 and − 1.2 for all of the time series. It proved that fractional white noise outweighs other types of noises in GPS time series. Authors found here, that the LRD methods by omitting the noise amplitude data led to an underestimation of H values and their misinterpretation. The larger the omitted amplitude is, the greater the difference between the noise characters estimated with R/S values in comparison to the reference values of κ are. Some of these differences exceed even the value of 0.6, which may result in the estimation of false noise character in GPS data thus eliciting wrong conclusions.

In this paper, we investigate a stochastic Galerkin approximation scheme for an optimal control problem governed by an elliptic PDE with random field in its coefficients. The optimal control minimizes the expectation of a cost functional with mean-state constraints. We first represent the stochastic elliptic PDE in terms of the generalized polynomial chaos expansion and obtain the parameterized optimal control problems. By applying the Slater condition in the subdifferential calculus, we obtain the necessary and sufficient optimality conditions for the state-constrained stochastic optimal control problem for the first time in the literature. We then establish a stochastic Galerkin scheme to approximate the optimality system in the spatial space and the probability space. Then the a priori error estimates are derived for the state, the co-state and the control variables. A projection algorithm is proposed and analyzed. Numerical examples are presented to illustrate our theoretical results.