Nonparametric density estimation by exact leave- [formula omitted]-out cross-validation

Graphs are commonly used in various applications such as online social networks (OSNs), E-commerce systems and social recommender systems. Random walk sampling is often used to conduct statistical estimation over such graphs. This paper develops an algorithmic framework to reduce the mean square error of such statistical estimation. Our algorithmic framework is inspired by that the mean square error can be decomposed into a sum of the bias and variance of the estimator. More specifically, we apply the bootstrapping technique to design a bias reduction algorithm. A new feature of this bias reduction algorithm is that it allows the variance to increase whenever the bias can be further reduced. The increased variance may lead to a large mean square error of the estimator. We use multiple parallel random walks to reduce this variance such that it can be reduced to arbitrarily small by deploying a sufficient number of random walks. Our algorithmic framework enables one to attain different trade-offs between the sample complexity (i.e., number of parallel random walks) and the mean square error of the statistical estimation. Also, the proposed bias reduction algorithm is generic and can be applied to optimize a large class of random walk sampling algorithms. To demonstrate the versatility of the framework, we apply it to optimize the Metropolis random walk and simple random walk sampling. Extensive experiments confirm the effectiveness and efficiency of our proposed algorithmic framework.

The extensive use of maximum likelihood estimates underscores the importance of the problem of statistical estimation of their errors. These estimates are of utmost importance in cases where the family of normal distributions and the families related to the normal distributions are considered [1, 2, 4]. The mean square errors of the maximum likelihood estimates of the normal density were investigated in the author's paper [3]. The mean square errors of statistical estimates of some families of densities related to the normal distributions were considered in the papers [4–6]. In the present paper, we obtain an asymptotic expansion of the mean square error of the maximum likelihood estimates of the densities of the joint distribution of sufficient statistics of the family of multivariate normal distributions. The results obtained allow us to construct the mean square errors of the maximum likelihood estimates for the chi-square density and Wishart's density.

Co-efficient of variation is a unitless measure of dispersion and is very frequently used in scientific investigations. This has motivated several researchers to propose estimators and tests concerning the co-efficient of variation of normal distribution(s). While proposing a class of estimators for the co-efficient of variation of a finite population, Tripathi et al., (2002) suggested that the estimator of co-efficient of variation of a finite population can also be used as an estimator of C.V for any distribution when the sampling design is SRSWR. This has motivated us to propose 28 estimators of finite population co-efficient of variation as estimators of co-efficient of variation of one component of a bivariate normal distribution when prior information is available regarding the second component. Cramer Rao type lower bound is derived to the mean square error of these estimators. Extensive simulation is carried out to compare these estimators. The results indicate that out of these 28 estimators, eight estimators have larger relative efficiency compared to the sample co-efficient of variation. The asymptotic mean square errors of the best estimators are derived to the order of  for the benefit of users of co-efficient of variation.

Density estimation in the presence of noise

In this paper we propose a criterion to estimate the density function by means of a nonparametric and fuzzy set estimator, based on $n$ i.i.d random variable, obtaining a reduction of the integrated mean square error of the fuzzy set estimator regarding the integrated mean squared error of the classic kernel estimators. This reduction shows that the fuzzy set estimator has better performance than the kernel estimations. Also, the convergence rate of the optimal scaling factor is computed, which coincides with the convergence rate in classic kernel estimation. Finally, these theoretical findings are illustrated using a numerical example.

We consider the problem of nonparametric estimation of unknown smooth functions in the presence of restrictions on the shape of the estimator and on its support using polynomial splines. We provide a general computational framework that treats these estimation problems in a unified manner, without the limitations of the existing methods. Applications of our approach include computing optimal spline estimators for regression, density estimation, and arrival rate estimation problems in the presence of various shape constraints. Our approach can also handle multiple simultaneous shape constraints. The approach is based on a characterization of nonnegative polynomials that leads to semidefinite programming (SDP) and second-order cone programming (SOCP) formulations of the problems. These formulations extend and generalize a number of previous approaches in the literature, including those with piecewise linear and B-spline estimators. We also consider a simpler approach in which nonnegative splines are approximated by splines whose pieces are polynomials with nonnegative coefficients in a nonnegative basis. A condition is presented to test whether a given nonnegative basis gives rise to a spline cone that is dense in the space of nonnegative continuous functions. The optimization models formulated in the article are solvable with minimal running time using off-the-shelf software. We provide numerical illustrations for density estimation and regression problems. These examples show that the proposed approach requires minimal computational time, and that the estimators obtained using our approach often match and frequently outperform kernel methods and spline smoothing without shape constraints. Supplementary materials for this article are provided online.

ABSTRACTThe estimation of a multivariate function from a stationary m-dependent process is investigated, with a special focus on the case where m is large or unbounded. We develop an adaptive estimator based on wavelet methods. Under flexible assumptions on the nonparametric model, we prove the good performances of our estimator by determining sharp rates of convergence under two kinds of errors: the pointwise mean squared error and the mean integrated squared error. We illustrate our theoretical result by considering the multivariate density estimation problem, the derivatives density estimation problem, the density estimation problem in a GARCH-type model and the multivariate regression function estimation problem. The performance of proposed estimator has been shown by a numerical study for a simulated and real data sets.

In this paper, we consider the problem of bandwidth-constrained distributed estimation of a Gaussian vector with linear observation model. Each sensor makes a scalar noisy observation of the unknown vector, employs a multi-bit scalar quantizer to quantize its observation, and maps it to a digitally modulated symbol. Sensors transmit their symbols over orthogonal-power-constrained fading channels to a fusion center (FC). The FC is tasked with fusing the received signals from sensors and estimating the unknown vector. We derive the Bayesian Fisher Information Matrix (FIM) for three types of receivers: (i) coherent receiver; (ii) noncoherent receiver with known channel envelopes; and (iii) noncoherent receiver with known channel statistics only. We also derive the Weiss–Weinstein bound (WWB). We formulate two constrained optimization problems, namely maximizing trace and log-determinant of Bayesian FIM under network transmit power constraint, with sensors’ transmit powers being the optimization variables (we refer to as FIM-max schemes). We show that for coherent receiver, these problems are concave. However, for noncoherent receivers, they are not necessarily concave. The solution to the trace of Bayesian FIM maximization problem can be implemented in a distributed fashion, in the sense that each sensor calculates its own transmit power using its local parameters. On the other hand, the solution to the log-determinant of Bayesian FIM maximization problem cannot be implemented in a distributed fashion and the FC needs to find the powers (using parameters of all sensors) and inform the active sensors of their transmit powers. We numerically investigate how the FIM-max power allocation across sensors depends on the sensors observation qualities and physical layer parameters as well as the network transmit power constraint. Moreover, we evaluate the system performance in terms of mean square error (MSE) using the solutions of FIM-max schemes, and compare it with the solution obtained from minimizing the MSE of the LMMSE estimator (MSE-min scheme), and that of uniform power allocation. These comparisons illustrate that, although the WWB is tighter than the inverse of Bayesian FIM, it is still suitable to use FIM-max schemes, since the performance loss in terms of the MSE of the LMMSE estimator is not significant. Furthermore, comparing the performance of different receivers, our numerical results reveal that coherent receiver and noncoherent receiver with known channel statistics have the best and the worst performance, respectively.

Estimating the cumulative distribution function of lead-time demand using bootstrapping with and without replacement

We deal with deconvolution problems in density estimation. Assume that the data follow a density, which is a convolution of the original density f being of interest with a noise density f ϵ. In order to estimate the density f, one usually should know f ϵ completely and then uses some technique for deconvolution. In contrast, the so-called TAYLEX and SIMEX methods introduced by Carroll and Hall and Cook and Stefanski, respectively use partial information on f ϵ only and correct the naive density estimator towards the deconvoluted one. In the present paper, we assume that we have more and more information on the noise density when the sample size increases. We show that by applying these methods, one can achieve almost optimal rates and optimal rates respectively for densities f belonging to certain Sobolev classes.

Problems with censored data arise frequently in survival analyses and reliability applications. The estimation of the density function of the failure times is often of interest. Two inherent problems in density estimation for lifetime data are the spill-over at the origin and the smoothing parameter selection. To address these issues, we propose the use of asymmetric kernels (like inverse Gaussian) with bandwidths selected by a Bayes criterion. We show strong pointwise consistency of the density estimator, and for suitable choices of the prior, we show that one can obtain meaningful bandwidths with the same rates of convergence as for the classical asymptotically optimal bandwidths.

The aim of this thesis is to provide two extensions to the theory of nonparametric kernel density estimation that increase the scope of the technique. The basic ideas of kernel density estimation are not new, having been proposed by Rosenblatt [20] and extended by Parzen [17]. The objective is that for a given set of data, estimates of functions of the distribution of the data such as probability densities are derived without recourse to rigid parametric assumptions and allow the data themselves to be more expressive in the statistical outcome. Thus kernel estimation has captured the imagination of statisticians searching for more flexibility and eager to utilise the computing revolution. The abundance of data and computing power have revealed distributional shapes that are difficult to model by traditional parametric approaches and in this era, the computer intensive technique of kernel estimation can be performed routinely. Also we are aware that computing power can be harnessed to give improved statistical analyses. Thus a lot of modern statistical research involves kernel estimation from complex data sets and our research is concordant with that momentum. The thesis contains three chapters. In Chapter 1 we provide an introduction to kernel density estimation and we give an outline to our two research topics. Our first extension to the theory is given in Chapter 2 where we investigate density estimation from independent data, using high order kernel functions. These kernel functions are designed for bias reduction but they have the penalty of yielding negative density estimates where data are sparse. In common practice, the negative estimates would arise in the tails of the density and we provide four ways of correcting this negativity to give bona fide estimates of the probability density. Our theory shows that the effects of these corrections are asymptotically negligible and thus opens the way for the regular use of bias reducing, high order kernel functions. We also consider density estimation of continuous stationary stochastic processes and this is the content of Chapter 3. With this problem, the dependent nature of the data influences the accuracy of the kernel density estimator and we provide theory regarding the convergence of the kernel estimators of the density and its derivatives to the true functions. An important result from this study is that nonparametric density estimators from dependent processes can have the same rates of convergence as their parametric counterparts yet retain the flexibility of being independent of parametric assumptions. Our other results indicate that the convergence rate of the density estimator can be quite slow if there are large lag dependencies amongst the data and suggests that large samples would be required for reliable inference about such data. The flexibility of kernel density estimation for continuous and discrete data, independent and dependent observations, means that it is a useful statistical tool. The techniques given in this thesis are not restricted to the analysis of simple sets of data but may be employed in the construction of statistical models for complex data with a high degree of structure.

In deriving the bias and mean square error of the ratio estimator , it is generally assumed that for all possible samples. This assumption is, however, unlikely to be satisfied except in special situations. This article presents a justification for the use of the usual approximations to the bias and mean square error of . Bounds for the deviations of the approximations from the true values are also given.

Kernel estimation of $f(0)$ is considered where $f$ is a density in some class $\mathscr{F}$ of $d$-dimensional densities, described in terms of a Taylor series expansion. A sequence of kernels which asymptotically minimizes the maximum mean square error of estimation over $\mathscr{F}$ is given. The shape of the kernel is fixed, the size of the window depends on $f(0)$, and an easily computed estimate is obtained to efficiently adapt the sequence to the unknown value of $f(0)$.

Discrete power distributions are proposed and studied, by considering the positive jumps on the discontinuities of an original discrete distribution function. Inequalities in moments and distribution functions are studied, allowing the definition of discrete intermediate distributions that lie between an original distribution and a power distribution. Original uniform, binomial, Poisson, negative binomial, and hypergeometric distributions are considered, to propose new power and intermediate distributions. Stochastic orders and unimodality are discussed. Estimation problems using likelihood are investigated. Simulation experiments are performed, to evaluate the bias and the mean square error of the maximum likelihood estimates, that are numerically calculated, with classic tools for numerical optimization.