Estimation of density functionals via cross‐validation

On the boundary properties of Bernstein polynomial estimators of density and distribution functions

The topics of orthogonality and Fourier series occupy a central position in analysis. Nevertheless, there is surprisingly little statistical literature, with the exception of that of time series and regression, which involves Fourier analysis. In the last decade however, several papers have appeared which deal with the estimation of orthogonal expansions of distribution densities and cumulatives. Cencov [1] and Van Ryzin [6] considered general properties of orthogonal expansion based density estimators and the latter applied these properties to obtain classification procedures. Schwartz [3] and the authors [2] and [4] investigated respectively the Hermite and Trigonometric special cases. The authors also obtained certain general results which apply not only to estimators of the population density but also to estimators of the population cumulative [2], [4] and [5]. In this paper several results derived for the univariate case are extended to the multivariate case. Also a new relationship is obtained which involves general Fourier expansions and estimators. Although there is some reason for calling the Gram-Charlier estimation of distribution densities a Fourier method, one fundamental aspect of Fourier methods is not shared by Gram-Charlier estimation. Gram-Charlier techniques make no use of Parseval's Formula or related error relationships of Fourier analysis. The ease with which the mean integrated square error (MISE) is evaluated, when Fourier methods are applied, accounts for most of the recent interest in this area. Section 1 of this paper deals with an investigation of two general MISE relationships for multivariate estimates of Fourier expansions. The relationship given in Theorem 2 is particularly simple and yet includes the four MISE's which are involved in the estimation problem. In Section 2 the choice of orthogonal functions is restricted to the trigonometric polynomials. It is shown that the MISE of multidimensional trigonometric polynomial estimators are related in a simple way to the Fourier coefficients of the distribution which is being estimated. This result is of considerable utility since it yields a rule for deciding which terms should be included in the estimate of the multivariate density.

Do a random censorship and/or order restrictions (e.g., nonnegativity, monotonicity, convexity) affect estimation of a smooth density under mean integrated squared error (MISE)? Under mild assumptions, the known asymptotic results, which are concerned only with rates, answer “no.” This answer, especially for censored data, contradicts practical experience and statistical intuition. So what can be said about constants of MISE convergence? It is shown that asymptotically (a) censorship does affect the constant, and this allows one to find a relationship between sample sizes of directly observed and censored datasets that implies the same precision of estimation, and (b) an order restriction does not affect the constant, and thus no isotonic estimation is needed. Intensive Monte Carlo simulations show that the lessons of the sharp asymptotics are valuable for small sample sizes. Also, the estimator developed is illustrated both on simulated data and a dataset of lifetimes of conveyer blades used at wastewater treatment plants.

Kernel smoothers are among the most popular nonparametric functional estimates. These estimates depend on a bandwidth that controls the smoothness of the estimate. While the literature for a bandwidth choice in a univariate density estimate is quite extensive, the progress in the multivariate case is slower. The authors focus on a bandwidth matrix selection for a bivariate kernel density estimate provided that the bandwidth matrix is diagonal. A common task is to find entries of the bandwidth matrix which minimizes the Mean Integrated Square Error (MISE). It is known that in this case there exists explicit solution of an asymptotic approximation of MISE (Wand and Jones, 1995). In the present paper we pay attention to the visualization and optimizers are presented as intersection of bivariate functional surfaces derived from this explicit solution and we develop the method based on this visualization. A simulation study compares the least square cross-validation method and the proposed method. Theoretical results are applied to real data.

Given a set of point-process event times with constant dispersion ratio, we are interested in estimating the rate function by grouping the event times into count data from equal-width time intervals. We group in order to smooth the resulting piecewise-constant rate function using one of our two existing methods: I-SMOOTH and MNO-PQRS. Using the mean integrated squared error (MISE) for piecewise-constant rate functions, we create two estimators; minimizing the estimated MISE function yields the chosen number of intervals. The MISE function provides insights into the optimal number of intervals as a function of the rate-function shape and expected number of events. Across several examples, our two number-of-intervals estimators perform well and similarly; nevertheless, one dominates in terms of realized MISE value.

In this paper, the estimation of a quantile density function in the presence of right censored data is investigated. A new wavelet-based methodology for the estimation of the quantile function will is proposed. In particular, an adaptive hard thresholding wavelet estimator is constructed. Under mild assumptions on the model, we prove that it enjoys powerful mean integrated squared error properties over Besov balls. While existing estimators of the quantile density function are not good at the tails, our proposed estimators perform well at the tails. The comparison of the proposed estimator has been made with estimators given by Jones (1992) Ann Inst Stat Math 44(4):721–727 and Soni et al. (2012) Comput Stat Data Anal 56(12):3876–3886 graphically and in terms of the mean integrated square error (MISE) for the uncensored case.

An algorithm is given for estimation of the scaling parameter Λ in the kernel-type density estimate derived from the Fourier integral. A Monte Carlo study of five different densities compares the integrated square error (ISE) for the resulting estimate to the ISE for the Fourier integral estimate (FIE) with mean integrated squared error (MISE) optimal Λ and to the ISE for the normal kernel with asymptotically MISE optimal Λ. The study shows the FIE with estimated Λ compares favorably with the other estimates and demonstrates the asymptotic optimality of the FIE for smooth densities.

To estimate the density f of a conditional expectation μ(Z) = E[X|Z], Steckley and Henderson (2003) sample independent copies Z <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> ,...,Z <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</sub> ; then, conditional on Z <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> , they sample n independent samples of X, and their sample mean ̅X <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> is an approximate sample of μ(Z <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> ). For a kernel density estimate ̂f of f based on such samples and a bandwidth (smoothing parameter) h, they consider the mean integrated squared error (MISE), ∫(̂f(x)-f(x)) <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> dx, and find rates of convergence of m, n and h that optimize the rate of convergence of MISE to zero. Inspired by the cross-validation approach in classical density estimation, we develop an estimate of MISE (up to a constant) and select the h that minimizes this estimate. While a convergence analysis is lacking, numerical results suggest that our method is promising.

Some results on random design regression with long memory errors and predictors

This paper proposes a novel feature ranking method, DensityRank, based on kernel estimation on the feature spaces to improve the classification performance. As the availability of raw data in many of today's applications continues to grow at an explosive rate, it is critical to assess the learning capabilities of different features and select the important subset of features to improve learning accuracy as well as reduce computational cost. In our approach, kernel methods are used to estimate the probability density function for each feature across different class labels. Discrepancy analysis based on the mean integrated square error (MISE) between pairs of such density estimations is used to provide the ranking values. Then, the ranked subspace method is adopted to select subsets of important features that are used to develop the learning models. Comparative study of this method with those of traditional ranking methods related to Fisher's discrimination ratio and information gain theory, as well as the random subspace algorithm and the bootstrap aggregating (bagging), are presented in this paper. Simulation results on various real-world data sets illustrate the effectiveness of the proposed method.

Cost-effectiveness analysis in cardiac surgery: A review of its concepts and methodologies

We consider nonlinear function estimation using Radial Basis Function Networks. We analytically determine the optimal radial kernel minimizing the Mean Integrated Square Error (MISE) and the optimal MISE rate of convergence. The rates of convergence for various classes of nonlinear functions and input densities are also considered.

Nonlinear wavelet-based estimation to spectral density for stationary non-Gaussian linear processes

Let h(.) be a continuous, strictly positive probability density function over an interval [a, b] and H(.) its associated cumulative distribution function (cdf). Given a sample set \(X_{1},\ldots ,X_{n}\) of independent identically distributed variables, we want to estimate H(.) from this sample set. The present work has two goals. The first one is to propose an estimator of a cdf based on an orthogonal trigonometric series and to give its statistical and asymptotic proprieties (bias, variance, mean square error, mean integrated squared error, convergence of the bias, convergence of variance, convergence of the mean squared error, convergence of the mean integrated squared error, uniform convergence in probability and the rate of convergence of the mean integrated squared error). The second is to introduce a new method for the selection of a “smoothing parameter”. The comparison by simulation between this method and Kronmal–Tarter’s method, shows that the new method is more performant in the sense of the mean integrated square error.

We observe $n$ heteroscedastic stochastic processes $\{Y_v(t)\}_{v}$, where for any $v\in\{1,\ldots,n\}$ and $t \in [0,1]$, $Y_v(t)$ is the convolution product of an unknown function $f$ and a known blurring function $g_v$ corrupted by Gaussian noise. Under an ordinary smoothness assumption on $g_1,\ldots,g_n$, our goal is to estimate the $d$-th derivatives (in weak sense) of $f$ from the observations. We propose an adaptive estimator based on wavelet block thresholding, namely the "BlockJS estimator". Taking the mean integrated squared error (MISE), our main theoretical result investigates the minimax rates over Besov smoothness spaces, and shows that our block estimator can achieve the optimal minimax rate, or is at least nearly-minimax in the least favorable situation. We also report a comprehensive suite of numerical simulations to support our theoretical findings. The practical performance of our block estimator compares very favorably to existing methods of the literature on a large set of test functions.