Parameter estimation for diffusion process from perturbed discrete observations

This paper develops a nonparametric density estimator with parametric overtones. Suppose f(x, θ) is some family of densities, indexed by a vector of parameters θ. We define a local kernel-smoothed likelihood function which, for each x, can be used to estimate the best local parametric approximant to the true density. This leads to a new density estimator of the form f(x, o(x)), thus inserting the best local parameter estimate for each new value of x. When the bandwidth used is large, this amounts to ordinary full likelihood parametric density estimation, while for moderate and small bandwidths the method is essentially nonparametric, using only local properties of data and the model. Alternative ways more general than via the local likelihood are also described. The methods can be seen as ways of nonparametrically smoothing the parameter within a parametric class. Properties of this new semiparametric estimator are investigated. Our preferred version has approximately the same variance as the ordinary kernel method but potentially a smaller bias. The new method is seen to perform better than the traditional kernel method in a broad nonparametric vicinity of the parametric model employed, while at the same time being capable of not losing much in precision to full likelihood methods when the model is correct. Other versions of the method are approximately equivalent to using particular higher order kernels in a semiparametric framework. The methodology we develop can be seen as the density estimation parallel to local likelihood and local weighted least squares theory in nonparametric regression.

The systematic risk estimation models: A different perspective

In this article we introduce a nonparametric estimator of the spectral density by smoothing the periodogram using beta kernel density. The estimator is proved to be bounded for short memory data and diverges at the origin for long memory data. The convergence in probability of the relative error and Monte Carlo simulations show that the proposed estimator automatically adapts to the long‐ and the short‐range dependency of the process. A cross‐validation procedure is studied in order to select the nuisance parameter of the estimator. Illustrations on historical as well as most recent returns and absolute returns of the S&P500 index show the performance of the beta kernel estimator.The Canadian Journal of Statistics48: 582–595; 2020 © 2020 Statistical Society of Canada

The Fisher information matrix (FIM) is a widely used measure for applications including statistical inference, information geometry, experiment design, and the study of criticality in biological systems. The FIM is defined for a parametric family of probability distributions and its estimation from data follows one of two paths: either the distribution is assumed to be known and the parameters are estimated from the data or the parameters are known and the distribution is estimated from the data. We consider the latter case which is applicable, for example, to experiments where the parameters are controlled by the experimenter and a complicated relation exists between the input parameters and the resulting distribution of the data. Since we assume that the distribution is unknown, we use a nonparametric density estimation on the data and then compute the FIM directly from that estimate using a finite-difference approximation to estimate the derivatives in its definition. The accuracy of the estimate depends on both the method of nonparametric estimation and the difference Δθ between the densities used in the finite-difference formula. We develop an approach for choosing the optimal parameter difference Δθ based on large deviations theory and compare two nonparametric density estimation methods, the Gaussian kernel density estimator and a novel density estimation using field theory method. We also compare these two methods to a recently published approach that circumvents the need for density estimation by estimating a nonparametric f divergence and using it to approximate the FIM. We use the Fisher information of the normal distribution to validate our method and as a more involved example we compute the temperature component of the FIM in the two-dimensional Ising model and show that it obeys the expected relation to the heat capacity and therefore peaks at the phase transition at the correct critical temperature.

There is a long-standing debate in the statistical, epidemiological, and econometric fields as to whether nonparametric estimation that uses machine learning in model fitting confers any meaningful advantage over simpler, parametric approaches in finite sample estimation of causal effects. We address the question: when estimating the effect of a treatment on an outcome, how much does the choice of nonparametric vs parametric estimation matter? Instead of answering this question with simulations that reflect a few chosen data scenarios, we propose a novel approach to compare estimators across a large number of data-generating mechanisms drawn from nonparametric models with semi-informative priors. We apply this proposed approach and compare the performance of two nonparametric estimators (Bayesian adaptive regression tree and a targeted minimum loss-based estimator) to two parametric estimators (a logistic regression-based plug-in estimator and a propensity score estimator) in terms of estimating the average treatment effect across thousands of data-generating mechanisms. We summarize performance in terms of bias, confidence interval coverage, and mean squared error. We find that the two nonparametric estimators can substantially reduce bias as compared to the two parametric estimators in large-sample settings characterized by interactions and nonlinearities while compromising very little in terms of performance even in simple, small-sample settings.

Methods of density estimation on the Grassmann manifold

Single-catch traps are frequently used in live-trapping studies of small mammals. Thus far, a likelihood for single-catch traps has proven elusive and usually the likelihood for multicatch traps is used for spatially explicit capture-recapture (SECR) analyses of such data. Previous work found the multicatch likelihood to provide a robust estimator of average density. We build on a recently developed continuous-time model for SECR to derive a likelihood for single-catch traps. We use this to develop an estimator based on observed capture times and compare its performance by simulation to that of the multicatch estimator for various scenarios with nonconstant density surfaces. While the multicatch estimator is found to be a surprisingly robust estimator of average density, its performance deteriorates with high trap saturation and increasing density gradients. Moreover, it is found to be a poor estimator of the height of the detection function. By contrast, the single-catch estimators of density, distribution, and detection function parameters are found to be unbiased or nearly unbiased in all scenarios considered. This gain comes at the cost of higher variance. If there is no interest in interpreting the detection function parameters themselves, and if density is expected to be fairly constant over the survey region, then the multicatch estimator performs well with single-catch traps. However if accurate estimation of the detection function is of interest, or if density is expected to vary substantially in space, then there is merit in using the single-catch estimator when trap saturation is above about 60%. The estimator's performance is improved if care is taken to place traps so as to span the range of variables that affect animal distribution. As a single-catch likelihood with unknown capture times remains intractable for now, researchers using single-catch traps should aim to incorporate timing devices with their traps.

Probability density estimation is the problem of inferring an underlying probability from a sampling of points. This study introduces smoothDE, an algorithm that uses Bayesian Field Theory to optimize non-parametric density estimation. smoothDE deterministically finds an optimal density function based on the probability of observed data, subject to a smoothing constraint and its associated prior probability. smoothDE's predicted densities have almost universally lower Kullback-Leibler divergences from simulated Gaussian Mixtures densities when compared to similar Bayesian Field Theory methods and Kernel Density Estimators. smoothDE was even able to outperform a specialized Bayesian Gaussian Mixture density estimator at lower samplings. smoothDE's ability to quickly fit arbitrary densities allowed it to be used as a preprocessing step for classification algorithm, in certain cases boosting classifier performance.

Nonparametric estimation for a probability density function that describes multivariate data has typically been addressed by kernel density estimation (KDE). A novel density estimator recently developed by Farmer and Jacobs offers an alternative high-throughput automated approach to univariate nonparametric density estimation based on maximum entropy and order statistics, improving accuracy over univariate KDE. This article presents an extension of the single variable case to multiple variables. The univariate estimator is used to recursively calculate a product array of one-dimensional conditional probabilities. In combination with interpolation methods, a complete joint probability density estimate is generated for multiple variables. Good accuracy and speed performance in synthetic data are demonstrated by a numerical study using known distributions over a range of sample sizes from 100 to 106 for two to six variables. Performance in terms of speed and accuracy is compared to KDE. The multivariate density estimate developed here tends to perform better as the number of samples and/or variables increases. As an example application, measurements are analyzed over five filters of photometric data from the Sloan Digital Sky Survey Data Release 17. The multivariate estimation is used to form the basis for a binary classifier that distinguishes quasars from galaxies and stars with up to 94% accuracy.

In this article, a semiparametric time‐varying nonlinear vector autoregressive (NVAR) model is proposed to model nonlinear vector time series data. We consider a combination of parametric and nonparametric estimation approaches to estimate the NVAR function for both independent and dependent errors. We use the multivariate Taylor series expansion of the link function up to the second order which has a parametric framework as a representation of the nonlinear vector regression function. After the unknown parameters are estimated by the maximum likelihood estimation procedure, the obtained NVAR function is adjusted by a nonparametric diagonal matrix, where the proposed adjusted matrix is estimated by the nonparametric kernel estimator. The asymptotic consistency properties of the proposed estimators are established. Simulation studies are conducted to evaluate the performance of the proposed semiparametric method. A real data example on short‐run interest rates and long‐run interest rates of United States Treasury securities is analyzed to demonstrate the application of the proposed approach.The Canadian Journal of Statistics47: 668–687; 2019 © 2019 Statistical Society of Canada

The aim of this paper is to present goodness-of-fit tests of a specified density function, based on a quadratic measure of deviation of a nonparametric kernel estimator. The results presented here, considering degenerate U-statistics technics, are generalisations, to the multidimensional case in a context of asymptotic independence, of the corresponding ones obtained by Bickel and Rosenblatt [1973]. A simulation is also proposed.

Estimators for derivatives associated with a density function can be useful in identifying its modes and inflection points. In addition, these estimators play an important role in plug-in methods associated with bandwidth selection in nonparametric kernel density estimation. In this paper, we extend the nonparametric class of density estimators proposed by Mynbaev and Martins-Filho (2010) to the estimation of m-order density derivatives. Contrary to some existing derivative estimators, the estimators in our proposed class have a full asymptotic characterization, including uniform consistency and asymptotic normality. An expression for the bandwidth that minimizes an asymptotic approximation for the estimators’ integrated squared error is provided. A Monte Carlo study sheds light on the finite sample performance of our estimators and contrasts it with that of density derivative estimators based on the classical Rosenblatt–Parzen approach.

Mixture distribution analysis provides us with a tool for identifying unlabeled clusters that naturally arise in a data set. In this paper, we demonstrate how to use the information criteria AIC and BIC to choose the optimal number of clusters for a given set of univariate Poisson data. We give an empirical comparison between minimum Hellinger distance (MHD) estimation and EM estimation for finding parameters in a mixture of Poisson distributions with artificial data. In addition, we discuss Bayes error in the context of classification problems with mixture of 2, 3, 4, and 5 Poisson models. Finally, we provide an example with real data, taken from a study that looked at sudden infant death syndrome (SIDS) count data from 100 North Carolina counties (Symons et al., 1983). This gives us an opportunity to demonstrate the advantages of the proposed model framework in comparison with the original analysis.

We address the problem of estimating and removing localized image blur, as it for example arises from moving objects in a scene, or when the depth of field is insufficient to sharply render all objects of interest. Unlike the case of camera shake, such blur changes abruptly at the object boundaries. To cope with this, we propose an automated sharp image recovery method that simultaneously determines blurred regions and estimates their responsible blur kernels. To address a wide range of different scenarios, our model is not restricted to a discrete set of candidate blurs, but allows for arbitrary, non-parametric blur kernels. Moreover, our approach does not require specialized hardware, an alpha matte, or user annotation of the blurred region. Unlike previous methods, we show that localized blur estimation can be accomplished by incorporating a pixel-wise latent variable to indicate the active blur kernel. Furthermore, we generalize the marginal likelihood technique of blind deblurring to the case of localized blur. Specifically, we integrate out the latent image derivatives to permit marginal density estimates of both blur kernels and their regions of influence. We obtain sharp images in applications to both object motion blur and defocus blur removal. Quantitative results on two novel datasets as well as qualitative results comparing to a range of specialized methods demonstrate the versatility and effectiveness of our non-parametric approach.

We study the estimation of certain functionals of the conditional distribution function with two classes of nonparametric estimators—the rank nearest neighbour (RNN) type estimators and the Nadaraya-Watson (NW) kernel type estimators. We obtain sharp pointwise rates of strong consistency by establishing laws of the iterated logarithm for these two classes of estimators. The results parallel those of Hall [Hall, P., 1981, Laws of the iterated logarithm for nonparametric density estimators. Zeitschrift Fur Wahrscheinlichkeitstheorie Und Verwandte Gebiete, 56, 47–61] and Härdle [Härdle, W., 1984, A law of the iterated logarithm for nonparametric regression function estimators. The Annals of Statistics, 12, 624–635] for certain density and regression function estimators respectively, and extend those of Mehra et al. [Mehra, K.L., Rama Krishnaiah, Y.S. and Rao, S.M., 1992a, Asymptotic properties of smoothed vs. unsmoothed conditional distribution function estimators, Bulletin of Informatics and Cybernetics, 25, 71–97] on the strong consistency of smooth conditional distribution function estimators.