Fully Data-driven Normalized and Exponentiated Kernel Density Estimator with Hyvärinen Score

Recently Hazelton and Turlach (2009) proposed a weighted kernel density estimatorfor the deconvolution problem. In the case of Gaussian kernels and measurement er-ror, they argued that the weighted kernel density estimator is a competitive estimatorover the classical deconvolution kernel estimator. In this paper we consider weightedkernel density estimators when sample observations are contaminated by double expo-nentially distributed errors. The performance of the weighted kernel density estimatorsis compared over the classical deconvolution kernel estimator and the kernel density es-timator based on the support vector regression method by means of a simulation study.The weighted density estimator with the Gaussian kernel shows numerical instabilityin practical implementation of optimization function. However the weighted densityestimates with the double exponential kernel has very similar patterns to the classicalkernel density estimates in the simulations, but the shape is less satisfactory than theclassical kernel density estimator with the Gaussian kernel.Keywords: Deconvolution, kernel density estimator, support vector regression, weightedkernel density estimator.

Nonparametric estimation for a probability density function that describes multivariate data has typically been addressed by kernel density estimation (KDE). A novel density estimator recently developed by Farmer and Jacobs offers an alternative high-throughput automated approach to univariate nonparametric density estimation based on maximum entropy and order statistics, improving accuracy over univariate KDE. This article presents an extension of the single variable case to multiple variables. The univariate estimator is used to recursively calculate a product array of one-dimensional conditional probabilities. In combination with interpolation methods, a complete joint probability density estimate is generated for multiple variables. Good accuracy and speed performance in synthetic data are demonstrated by a numerical study using known distributions over a range of sample sizes from 100 to 106 for two to six variables. Performance in terms of speed and accuracy is compared to KDE. The multivariate density estimate developed here tends to perform better as the number of samples and/or variables increases. As an example application, measurements are analyzed over five filters of photometric data from the Sloan Digital Sky Survey Data Release 17. The multivariate estimation is used to form the basis for a binary classifier that distinguishes quasars from galaxies and stars with up to 94% accuracy.

Density estimation for pre-binned data is challenging due to the loss of exact position information of the original observations. Traditional kernel density estimation methods cannot be applied when data are pre-binned in unequally spaced bins or when one or more bins are semi-infinite intervals. We propose a novel density estimation approach using the generalized lambda distribution (GLD) for data that have been pre-binned over a sequence of consecutive bins. This method enjoys the high power of the parametric model and the great shape flexibility of the GLD. The performances of the proposed estimators are benchmarked via simulation studies. Both simulation results and a real data application show that the proposed density estimators work well for data of moderate or large sizes.

This paper introduces an approach to select the bandwidth or smoothing parameter in multiresolution (MR) density estimation and nonparametric density estimation. It is based on the evolution of the second, third and fourth central moments and the shape of the estimated densities for different bandwidths and resolution levels. The proposed method has been applied to density estimation by means of multiresolution densities as well as kernel density estimation (MRDE and KDE respectively). The results of the simulations and the empirical application demonstrate that the level of resolution resulting from the moments method performs better with multimodal densities than the Bayesian Information Criterion (BIC) for multiresolution densities estimation and the plug-in for kernel densities estimation.

This paper considers the problem of nonparametric deconvolution density estimation when sample observa-tions are contaminated by double exponentially distributed errors. Three different deconvolution density estima-tors are introduced: a weighted kernel density estimator, a kernel density estimator based on the support vector regression method in a RKHS, and a classical kernel density estimator. The performance of these deconvolution density estimators is compared by means of a simulation study.

Probability density estimation is the problem of inferring an underlying probability from a sampling of points. This study introduces smoothDE, an algorithm that uses Bayesian Field Theory to optimize non-parametric density estimation. smoothDE deterministically finds an optimal density function based on the probability of observed data, subject to a smoothing constraint and its associated prior probability. smoothDE's predicted densities have almost universally lower Kullback-Leibler divergences from simulated Gaussian Mixtures densities when compared to similar Bayesian Field Theory methods and Kernel Density Estimators. smoothDE was even able to outperform a specialized Bayesian Gaussian Mixture density estimator at lower samplings. smoothDE's ability to quickly fit arbitrary densities allowed it to be used as a preprocessing step for classification algorithm, in certain cases boosting classifier performance.

The issue of monitoring non-normally distributed data with principal component analysis (PCA) is addressed through the application of density estimation for evaluating the quality of the principal component scores. Although kernel density estimation has been previously cited as a method for monitoring such data, mixture models are proposed here in order to reduce model complexity and computational effort. Furthermore, several adaptation strategies for the density estimators are developed and suggestions are provided on their use. A rapid thermal anneal case study demonstrates how the estimators outperform the traditional Hotelling's T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> statistic due to the presence of a first wafer effect.

Composite spatial data on administrative area level are often presented by maps. The aim is to detect regional differences in the concentration of subpopulations, like elderly persons, ethnic minorities, low-educated persons, voters of a political party or persons with a certain disease. Thematic collections of such maps are presented in different atlases. The standard presentation is by Choropleth maps where each administrative unit is represented by a single value. These maps can be criticized under three aspects: the implicit assumption of a uniform distribution within the area, the instability of the resulting map with respect to a change of the reference area and the discontinuities of the maps at the borderlines of the reference areas which inhibit the detection of regional clusters.In order to address these problems we use a density approach in the construction of maps. This approach does not enforce a local uniform distribution. It does not depend on a specific choice of area reference system and there are no discontinuities in the displayed maps. A standard estimation procedure of densities are Kernel density estimates. However, these estimates need the geo-coordinates of the single units which are not at disposal as we have only access to the aggregates of some area system. To overcome this hurdle, we use a statistical simulation concept. This can be interpreted as a Simulated Expectation Maximisation (SEM) algorithm of Celeux et al (1996). We simulate observations from the current density estimates which are consistent with the aggregation information (S-step). Then we apply the Kernel density estimator to the simulated sample which gives the next density estimate (E-Step).This concept has been first applied for grid data with rectangular areas, see Groß et al (2017), for the display of ethnic minorities. In a second application we demonstrated the use of this approach for the so-called “change of support” (Bradley et al 2016) problem. Here Groß et al (2020) used the SEM algorithm to recalculate case numbers between non-hierarchical administrative area systems. Recently Rendtel et al (2021) applied the SEM algorithm to display spatial-temporal clusters of Corona infections in Germany.Here we present three modifications of the basic SEM algorithm: 1) We introduce a boundary correction which removes the underestimation of kernel density estimates at the borders of the population area. 2) We recognize unsettled areas, like lakes, parks and industrial areas, in the computation of the kernel density. 3) We adapt the SEM algorithm for the computation of local percentages which are important especially in voting analysis.We evaluate our approach against several standard maps by means of the local voting register with known addresses. In the empirical part we apply our approach for the display of voting results for the 2016 election of the Berlin parliament. We contrast our results against Choropleth maps and show new possibilities for reporting spatial voting results.

ABSTRACTKernel density estimation has been applied in many computational subjects. In this paper, we propose a density estimation procedure from a Bayesian nonparametric perspective using Dirichlet process prior for the length-biased data under an unknown kernel function. In this situation, the kernel within the Dirichlet process mixture model will be approximated by the kernel density estimator. We present a Bayesian nonparametric method for finding the bandwidth parameter in the kernel density estimation using a Markov chain Monte Carlo approach. Then, this approach is used to the simulated and real data set. Finally, we compare the proposed bandwidth estimation with the other estimations like cross-validation and Bayes based on the mean integrated squared error criterion.

In this paper, we consider using nonparametric mixtures for density estimation. The mixture density estimation problem simply reduces to the problem of estimating a mixing distribution in the nonparametric mixture model. We focus on the least squares method for mixture density estimation problem. In a simulation experiment, the performance of the least squares mixture density estimator (MDE) and the kernel density estimator (KDE) is assessed by the mean integrated squared error. The performance improvement of MDE over KDE for some common densities is achieved by using cross-validation method for bandwidth selection.

Kernel Density Estimators (KDEs) are a non-parametric density estimation technique that has recently been applied to Monte Carlo radiation transport simulations. Kernel density estimators are an alternative to histogram tallies for obtaining global solutions in Monte Carlo tallies. With KDEs, a single event, either a collision or particle track, can contribute to the score at multiple tally points with the uncertainty at those points being independent of the desired resolution of the solution. Thus, KDEs show potential for obtaining estimates of a global solution with reduced variance when compared to a histogram. Previously, KDEs have been applied to neutronics for one-group reactor physics problems and fixed source shielding applications. However, little work was done to obtain reaction rates using KDEs. This paper introduces a new form of the MFP KDE that is capable of handling general geometries. Furthermore, extending the MFP KDE to 2-D problems in continuous energy introduces inaccuracies to the solution. An ad-hoc solution to these inaccuracies is introduced that produces errors smaller than 4% at material interfaces.

Density estimation is an important aspect of statistics. Statistical inference often requires the knowledge of observed data density. A common method of density estimation is the kernel density estimation (KDE). It is a nonparametric estimation approach which requires a kernel function and a window size (smoothing parameter H). It aids density estimation and pattern recognition. So, this work focuses on the use of a modified intersection of confidence intervals (MICIH) approach in estimating density. The Nigerian crime rate data reported to the Police as reported by the National Bureau of Statistics was used to demonstrate this new approach. This approach in the multivariate kernel density estimation is based on the data. The main way to improve density estimation is to obtain a reduced mean squared error (MSE), the errors for this approach was evaluated. Some improvements were seen. The aim is to achieve adaptive kernel density estimation. This was achieved under a sufficiently smoothing technique. This adaptive approach was based on the bandwidths selection. The quality of the estimates obtained of the MICIH approach when applied, showed some improvements over the existing methods. The MICIH approach has reduced mean squared error and relative faster rate of convergence compared to some other approaches. The approach of MICIH has reduced points of discontinuities in the graphical densities the datasets. This will help to correct points of discontinuities and display adaptive density. Keywords: approach, bandwidth, estimate, error, kernel density

We propose a method for nonparametric density estimation that exhibits robustness to contamination of the training sample. This method achieves robustness by combining a traditional kernel density estimator (KDE) with ideas from classical M-estimation. We interpret the KDE based on a positive semi-definite kernel as a sample mean in the associated reproducing kernel Hilbert space. Since the sample mean is sensitive to outliers, we estimate it robustly via M-estimation, yielding a robust kernel density estimator (RKDE). An RKDE can be computed efficiently via a kernelized iteratively re-weighted least squares (IRWLS) algorithm. Necessary and sufficient conditions are given for kernelized IRWLS to converge to the global minimizer of the M-estimator objective function. The robustness of the RKDE is demonstrated with a representer theorem, the influence function, and experimental results for density estimation and anomaly detection.

Problems with censored data arise frequently in survival analyses and reliability applications. The estimation of the density function of the failure times is often of interest. Two inherent problems in density estimation for lifetime data are the spill-over at the origin and the smoothing parameter selection. To address these issues, we propose the use of asymmetric kernels (like inverse Gaussian) with bandwidths selected by a Bayes criterion. We show strong pointwise consistency of the density estimator, and for suitable choices of the prior, we show that one can obtain meaningful bandwidths with the same rates of convergence as for the classical asymptotically optimal bandwidths.

Introduction. Some controversy exists concerning whether the onset of schizophrenia and some clinical characteristics of the disorder are different between males and females.Objective. The relationship between sex, age at onset and negative symptoms was evaluated in patients with schizophrenic spectrum disorders.Materials and methods. A sample of 225 patients (89 women and 136 men) were diagnosed for schizophrenia between 2008 and 2009. Each was compared for age at onset of symptoms and SANS score (Scale for the Assessment of Negative Symptoms). Kernel density estimators were used to evaluate characteristics of age of onset with respect to gender. Parameters of the mixed distributions were estimated via maximum-likelihood algorithms. Relationships between negative symptoms score and age of onset were evaluated using multiple regression analysis.Results. A significant difference was found in age at onset across gender (mean age of 24.5 years in men, 27.5 years in women). An association was found between gender and early onset of symptoms, with early onset occurring more frequently in male patients. Density estimates for age at onset suggested a mixture model with three components having as parameters: m1=21.55 +/- SD 5.25; m2=29.54 +/-SD 7.22; m3=40.01 +/- 3.98. When density estimates took into account gender, two bimodal structureswere found--(1) men with the lowest mean (18.0 years) and (2) the highest mean in middle-aged women (41.0 years). Regression coefficients suggested an increase in negative symptoms as time of disease increased.Conclusion. The hypothesis was supported that a relationship exists between age of onset of symptoms, gender and clinical characteristics in patients with schizophrenic spectrum disorders, showing that men have an early onset an a more deteriorating course than women.