Optimal spatial adaptation to inhomogeneous smoothness: an approach based on kernel estimates with variable bandwidth selectors

On the block thresholding wavelet estimators with censored data

We consider the wavelet-based estimators of mean regression function with long memory moving average errors and investigate their asymptotic rates of convergence based on thresholding of empirical wavelet coefficients. We show that these estimators achieve nearly optimal minimax convergence rates within a logarithmic term over a large range of Besov function classes \(B^{s}_{p,q}\). Therefore, in the presence of long memory non-Gaussian moving average noise, wavelet estimators still achieve nearly optimal convergence rates and provide explicitly the extraordinary local adaptability. The theory is illustrated with some numerical examples.

We propose an adaptive estimator for the stationary distribution of a bifurcating Markov Chain on . Bifurcating Markov chains (BMC for short) are a class of stochastic processes indexed by regular binary trees. A kernel estimator is proposed whose bandwidths are selected by a method inspired by the works of Goldenshluger and Lepski [(2011), ‘Bandwidth Selection in Kernel Density Estimation: Oracle Inequalities and Adaptive Minimax Optimality’, The Annals of Statistics 3: 1608–1632). Drawing inspiration from dimension jump methods for model selection, we also provide an algorithm to select the best constant in the penalty. Finally, we investigate the performance of the method by simulation studies and application to real data.

Thresholding of empirical wavelet coefficients was discussed in Section 7.1, along with some examples of global thresholds. This chapter will pick up where Section 7.1 left off, focusing on the nonparametric regression situation. The ideas described here could also be adapted for use in density estimation or other types of function estimation. To focus attention on the methods described in this chapter, it will be assumed throughout that an orthogonal wavelet transform on the unit interval is used, and that the sample size is a power of two: n = 2J for some integer J > 0. When this condition is not met, the methods described herein may be adapted, using techniques described in Chapter 6.KeywordsWavelet CoefficientThreshold SelectionGlobal ThresholdWavelet EstimatorShrinkage FunctionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A kernel method for the estimation of quantal dose-response curves is considered. In contrast to parametric modeling, this local smoothing method does not require any assumptions beyond smoothness of the dose-response curve and, in this sense, is nonparametric. In finite-sample situations, the kernel estimate of the dose-response curve is not necessarily monotone and, therefore, if the additional assumption of monotonicity of the dose-response curve is made, a monotonized version is discussed. Bias, variance, asymptotic normality, and uniform consistency, including rates of convergence of kernel estimates, are derived and applied to establish consistency and limiting distribution of kernel estimates of the EDα. Here, EDα is the effective dose at level α, that is, the dose where 100α% of the subjects show a response. The properties of the kernel estimated EDα are compared with the corresponding properties of the maximum likelihood estimator assuming the probit model. Practical application of the kernel method requires choice of a kernel function and of a bandwidth (smoothing window), and methods for global as well as local bandwidth selection are discussed. Another point of interest is the construction of confidence intervals for the estimated EDα. For the kernel method, two asymptotically consistent approaches are presented. These are compared in a simulation study, where the behavior of kernel and probit estimates of the EDα, α = .01, .05, .1, and .5, and of corresponding confidence intervals is observed for four different models (two probit models, one Weibull model, and one normal mixture model). The choice of different kernels and of local bandwidths was compared in another Monte Carlo study. The observed finite sample behavior of the kernel estimate of the ED50 was considerably better (gain of 40%-70% in terms of mean squared error) than that of the probit method under the two nonsigmoid models, whereas it was not drastically worse under the probit models (loss of 20%-30% in terms of mean squared error). Application of higher-order kernels and local bandwidth choice turned out to yield favorable versions of the kernel method. The results concerning observed coverage probabilities for the confidence intervals based on different approaches were more mixed.

Quantile and semiparametric M estimation are methods for estimating a censored linear regression model without assuming that the distribution of the random component of the model belongs to a known parametric family. Both methods require estimating derivatives of the unknown cumulative distribution function of the random component. The derivatives can be estimated consistently using kernel estimators in the case of quantile estimation and finite difference quotients in the case of semiparametric M estimation. However, the resulting estimates of derivatives, as well as parameter estimates and inferences that depend on the derivatives, can be highly sensitive to the choice of the kernel and finite difference bandwidths. This paper discusses the theory of asymptotically optimal bandwidths for kernel and difference quotient estimation of the derivatives required for quantile and semiparametric M estimation, respectively. We do not present a fully automatic method for bandwidth selection.

Bayesian bandwidth selections in continuous associated kernel estimation of probability density function are very good alternatives to classical methods like cross-validation techniques. In this paper, we examined the behavior of Bayesian variable bandwidths in gamma kernel estimation, developed theoretically in Wansouwé et al. [Ake: An R Package for Discrete and Continuous Associated Kernel Estimations, The R journal 8 (2016), pp. 259–276], and appropriated to smooth densities of support Simulations studies point out remarkable performance of the proposed approach, comparing to the global cross-validation bandwidth selection, and under integrated squared errors. Two applications related to CO 2 emissions and medical bills of bodily injury claims, respectively, are finally made.

Four powder diffraction patterns taken under different experimental conditions were denoised by a new method, i.e., thresholding of wavelet coefficients. The patterns were transformed by discrete wavelet transform applying Coiflet4 wavelet function. WLS refinements of peaks’ positions, FWHM, and intensity showed that wavelet denoising, in contrast to previously used polynomial smoothing, did not shift the maxima and preserved peak and integrated intensities. This method may therefore represent an useful alternative to polynomial filters or filters based on Fourier transform.

SUMMARY We consider estimation of a variance function g in regression problems. Such estimation requires simultaneous estimation of the mean function f. We obtain clear results on the extent to which the smoothness of f influences best rates of convergence for estimating g. For example, in nonparametric regression with two derivatives on g, ‘classical' rates of convergence are possible if and only if the unknown f satisfies a Lipschitz condition of order 13 or more. If a parametric model is known for g, then g may be estimated n1/2 consistently if and only if f is Lipschitz of order 12 or more. Optimal rates of convergence are attained by Kernel estimators.

Recently, several works have shown that the Bayesian approaches to global bandwidth and variable (local and adaptive) bandwidths selection in binomial kernel estimation for probability mass functions (pmfs) outperform the common classical methods, such as mean integrated squared error (MISE) and cross-validation (CV) methods. In this article, we first review the global, local, and adaptive binomial kernel estimator combined with the Bayesian approaches for selecting the bandwidths. Then we compare them by using several count data sets with different designs, in particular for small and moderate sample sizes. All the Bayesian bandwidth selection approaches are also applied to a real count data sets. In general, in terms of integrated squared error (ISE) and execution times, the local Bayesian approach outperforms the other Bayesian approaches.

A variable bandwidth selector in multivariate kernel density estimation

Wavelet denoising is commonly used for speech enhancement because of the simplicity of its implementation. However, the conventional methods generate the presence of musical residual noise while thresholding the background noise. The unvoiced components of speech are often eliminated from this method. In this paper, a novel algorithm of wavelet coefficient threshold (WCT) based on time-frequency adaptation is proposed. In addition, an unvoiced speech enhancement algorithm is also integrated into the system to improve the intelligibility of speech. The wavelet coefficient threshold (WCT) of each subband is first temporally adjusted according to the value of a posterior signal-to-noise ratio (SNR). To prevent the degradation of unvoiced sounds during noise, the algorithm utilizes a simple speech/noise detector (SND) and further divides speech signal into unvoiced and voiced sounds. Then, we apply appropriate wavelet thresholding according to voiced/unvoiced (V/U) decision. Based on the masking properties of human auditory system, a perceptual gain factor is adopted into wavelet thresholding for suppressing musical residual noise. Simulation results show that the proposed method is capable of reducing noise with little speech degradation and the overall performance is superior to several competitive methods.

The problem of bandwidth selection for kernel density estimation at a point is considered. Asymptotic lower bounds are established for the relative rate of convergence of data-driven bandwidth selectors to their optimal values. It is noted that some existing methods of local bandwidth selection, using high order kernel functions, attain these rates. Nevertheless, a simulation study indicates that improved performance predicted by asymptotic theory may not occur in practice for sample sizes as large as 104 or 105. The paper finishes with a comparison of local and global bandwidth selection, observing that in some sense the local problem is the more difficult. Keywords: Bandwidth selectorkernel density estimationmean squared errorrelative rate of convergence ∗Address for correspondence: Department of Statistical Science, University College London, Gower Street, London WC1E 6BT ∗Address for correspondence: Department of Statistical Science, University College London, Gower Street, London WC1E 6BT Notes ∗Address for correspondence: Department of Statistical Science, University College London, Gower Street, London WC1E 6BT

Bandwidth selection is a central issue in kernel density estimation. For large datasets, classical selectors such as cross-validation and bootstrap become computationally intensive and may yield bandwidths with high variability. This paper proposes subagging-based versions of several popular selectors, including cross-validation, direct plug-in, and bootstrap methods. These selectors are constructed by computing bandwidths over multiple subsamples (without replacement), rescaling them, and averaging the results. We also introduce a novel regression-based approach, Regression Subbagging (RSB), which extrapolates the optimal bandwidth via a log-log regression, avoiding the need to assume a known convergence rate. We assess statistical accuracy in terms of the mean squared error of the selectors and the corresponding MISE of the resulting kernel estimators, using the optimal bandwidth as a benchmark. Computational efficiency is evaluated via parallel implementations using the parallel and foreach packages in R, reporting speedups as a function of the number of CPU cores. The results confirm that subagging improves or preserves statistical performance while yielding substantial runtime reductions, especially for demanding selectors like cross-validation and bootstrap. The RSB variant, in particular, stands out as a scalable, flexible, and robust solution. The core methods are implemented in the R package baggingbwsel, available on CRAN.

Kernel density estimation is an important data smoothing technique. It has been applied most successfully for univariate data whilst for multivariate data its development and implementation have been relatively limited. The performance of kernel density estimators depends crucially on the bandwidth selection. Bandwidth selection in the univariate case involves selecting a scalar parameter which controls the amount of smoothing. In the multivariate case, the bandwidth matrix controls both the degree and direction of smoothing so its selection is more difficult. So far most of the research effort has been expended on automatic, data-driven selectors for univariate data. There is, on the other hand, a relative paucity of multivariate counterparts. Most of these multivariate bandwidth selectors are focused on the restricted case of diagonal matrices. In this thesis practical algorithms are constructed, with supporting theoretical justifications, for unconstrained bandwidth matrices. The two main classes of univariate bandwidth selectors are plug-in and cross validation. These unidimensional selectors are generalised to the multidimensional case. The univariate framework for theoretically analysing kernel density estimators is extended to a general multivariate version. This framework has at its core the quantification of the relative rates of convergence which provide a guide to the asymptotic behaviour of bandwidth selectors. Simulation studies and real data analysis are employed to illustrate their finite sample behaviour. It is found that unconstrained selectors possess good asymptotic and finite sample properties in a wide range of situations. Buoyed by this success, two extensions are embarked upon. The first is variable bandwidth selection, generalising the above case where the bandwidth is fixed throughout the sample space. The variation of the bandwidths is controlled by the local properties of the data. The novel contribution is to use non-parametric clustering to summarise these local properties, along with unconstrained bandwidth matrices. The second is in kernel discriminant analysis where unconstrained bandwidth matrices are shown to produce more accurate discrimination.