A fast and objective multidimensional kernel density estimation method: fastKDE

Particle density estimation with grid-projected and boundary-corrected adaptive kernels

In many data stream mining applications, traditional density estimation methods such as kernel density estimation, reduced set density estimation can not be applied to the density estimation of data streams because of their high computational burden, processing time and intensive memory allocation requirement. In order to reduce the time and space complexity, a novel density estimation method Dm-KDE over data streams based on the proposed algorithm m-KDE which can be used to design a KDE estimator with the fixed number of kernel components for a dataset is proposed. In this method, Dm-KDE sequence entries are created by algorithm m-KDE instead of all kernels obtained from other density estimation methods. In order to further reduce the storage space, Dm-KDE sequence entries can be merged by calculating their KL divergences. Finally, the probability density functions over arbitrary time or entire time can be estimated through the obtained estimation model. In contrast to the state-of-the-art algorithm SOMKE, the distinctive advantage of the proposed algorithm Dm-KDE exists in that it can achieve the same accuracy with much less fixed number of kernel components such that it is suitable for the scenarios where higher on-line computation about the kernel density estimation over data streams is required.We compare Dm-KDE with SOMKE and M-kernel in terms of density estimation accuracy and running time for various stationary datasets. We also apply Dm-KDE to evolving data streams. Experimental results illustrate the effectiveness of the proposed method.

In kernel density estimation methods, an approximation of the data probability density function is achieved by locating a kernel function at each data location. The smoothness of the functional approximation and the modelling ability are controlled by the kernel bandwidth. In this paper we propose a Bayesian estimation method for finding the kernel bandwidth. The distribution corresponding to the bandwidth is estimated from distributions characterizing the second order statistics estimates calculated from local neighbourhoods. The proposed bandwidth estimation method is applied in three different kernel density estimation based approaches: scale space, mean shift and quantum clustering. The third method is a novel pattern recognition approach using the principles of quantum mechanics.

Abstract. Unacceptably high levels of aflatoxin in shelled corn drastically reduce its market value and result in substantial economic losses. This study looked at the reduction in aflatoxin levels that could be achieved by removal of kernels discolored by mold and presumed to contain aflatoxin. The study related kernel physical characteristics to the reduction achieved using screening and density separation techniques which are currently in use in the grain industry. Physical properties of corn kernels discolored by mold and healthy corn kernels were determined in a sample taken from a 737 kg corn lot known to have relatively high levels of aflatoxin. Properties measured included kernel diameters, sphericity, and kernel density. Differences between discolored and healthy kernels in major diameter, minor diameter, sphericity, and density were statistically significant. Tests were also conducted at a commercial inbred seed corn processing facility using a screen cleaner and a gravity table. Removal of fine material (approximately 10% by weight) reduced aflatoxin levels by 84%. In two successive passes through a screen cleaner, removal of smaller kernels and kernel pieces with a round-hole sieve further reduced aflatoxin levels by 1.8% and 9.4%, respectively. After the smaller kernels and kernel pieces were added back to the lot, two successive passes through a gravity table reduced aflatoxin levels by 12.6% and 16.4%, respectively. Kernel density distributions of a sample taken before gravity table density sorting, and of samples of the higher and lower density fractions after sorting, were also determined. The effect of screening with a slotted sieve was studied by screening a 3 kg sample from the corn lot using a table top Clipper cleaner. This 3 kg sample had been previously screened to remove fines. The results indicate that, at least for some corn lots, aflatoxin levels can be reduced using a screen cleaner and a gravity table. The authors also concluded that insight into the effectiveness of aflatoxin reduction in corn by size and density sorting could be gained by conducting a size and density analysis on a representative sample of kernels taken from the corn being sorted. Keywords: Aflatoxin, Corn, Density, Gravity table, Physical properties, Screening, Sieves.

This paper introduces a supervised metric learning algorithm, called kernel density metric learning (KDML), which is easy to use and provides nonlinear, probability-based distance measures. KDML constructs a direct nonlinear mapping from the original input space into a feature space based on kernel density estimation. The nonlinear mapping in KDML embodies established distance measures between probability density functions, and leads to correct classification on datasets for which linear metric learning methods would fail. It addresses the severe challenge to kNN when features are from heterogeneous domains and, as a result, the Euclidean or Mahalanobis distance between original feature vectors is not meaningful. Existing metric learning algorithms can then be applied to the KDML features. We also propose an integrated optimization algorithm that learns not only the Mahalanobis matrix but also kernel bandwidths, the only hyper-parameters in the nonlinear mapping. KDML can naturally handle not only numerical features, but also categorical ones, which is rarely found in previous metric learning algorithms. Extensive experimental results on various datasets show that KDML significantly improves existing metric learning algorithms in terms of kNN classification accuracy.

This paper proposes a novel approach for multimedia signal reconstruction based on kernel density estimation (KDE). We make use of a vector formalism in which vectors consist of a first subvector containing a set of missing samples and a second one containing a set of available context samples. The missing subvector is reconstructed by a minimum mean square error estimator which employs a probability density function (pdf) obtained by KDE. As in any kernel-based method, the main issue to deal with is the estimation of an appropriate kernel bandwidth. We propose an adaptive procedure for bandwidth estimation (BE) especially conceived for signal reconstruction. Thus, unlike general KDE or kernel-based regression, which try to obtain a general fit, the focus of this BE procedure is on the specific missing subvector. Also, in order to exploit local signal correlations, our BE proposal adopts a scaling approach in which the bandwidth is computed as the local covariance matrix scaled by two factors. These two scale factors are obtained by minimization of two different approximations to the reconstruction error. The resulting reconstruction methodology is tested on a spatial error concealment (EC) application in which intracoded images have been transmitted through an error prone channel. The experimental results show the superiority of the proposed approach over a wide range of existing EC techniques.

Kernel Density Estimators (KDEs) are a non-parametric density estimation technique that has recently been applied to Monte Carlo radiation transport simulations. Kernel density estimators are an alternative to histogram tallies for obtaining global solutions in Monte Carlo tallies. With KDEs, a single event, either a collision or particle track, can contribute to the score at multiple tally points with the uncertainty at those points being independent of the desired resolution of the solution. Thus, KDEs show potential for obtaining estimates of a global solution with reduced variance when compared to a histogram. Previously, KDEs have been applied to neutronics for one-group reactor physics problems and fixed source shielding applications. However, little work was done to obtain reaction rates using KDEs. This paper introduces a new form of the MFP KDE that is capable of handling general geometries. Furthermore, extending the MFP KDE to 2-D problems in continuous energy introduces inaccuracies to the solution. An ad-hoc solution to these inaccuracies is introduced that produces errors smaller than 4% at material interfaces.

Recently Hazelton and Turlach (2009) proposed a weighted kernel density estimatorfor the deconvolution problem. In the case of Gaussian kernels and measurement er-ror, they argued that the weighted kernel density estimator is a competitive estimatorover the classical deconvolution kernel estimator. In this paper we consider weightedkernel density estimators when sample observations are contaminated by double expo-nentially distributed errors. The performance of the weighted kernel density estimatorsis compared over the classical deconvolution kernel estimator and the kernel density es-timator based on the support vector regression method by means of a simulation study.The weighted density estimator with the Gaussian kernel shows numerical instabilityin practical implementation of optimization function. However the weighted densityestimates with the double exponential kernel has very similar patterns to the classicalkernel density estimates in the simulations, but the shape is less satisfactory than theclassical kernel density estimator with the Gaussian kernel.Keywords: Deconvolution, kernel density estimator, support vector regression, weightedkernel density estimator.

Genetic associations have been quantified using a number of statistical measures. Entropy-based mutual information may be one of the more direct ways of estimating the association, in the sense that it does not depend on the parametrization. For this purpose, both the entropy and conditional entropy of the phenotype distribution should be obtained. Quantitative traits, however, do not usually allow an exact evaluation of entropy. The estimation of entropy needs a probability density function, which can be approximated by kernel density estimation. We have investigated the proper sequence of procedures for combining the kernel density estimation and entropy estimation with a probability density function in order to calculate mutual information. Genotypes and their interactions were constructed to set the conditions for conditional entropy. Extensive simulation data created using three types of generating functions were analyzed using two different kernels as well as two types of multifactor dimensionality reduction and another probability density approximation method called m-spacing. The statistical power in terms of correct detection rates was compared. Using kernels was found to be most useful when the trait distributions were more complex than simple normal or gamma distributions. A full-scale genomic dataset was explored to identify associations using the 2-h oral glucose tolerance test results and γ-glutamyl transpeptidase levels as phenotypes. Clearly distinguishable single-nucleotide polymorphisms (SNPs) and interacting SNP pairs associated with these phenotypes were found and listed with empirical p-values.

Multivariate online kernel density estimation with Gaussian kernels

The Probability Density Function (PDF) is a key concept in statistics. Constructing the most adequate PDF from the observed data is still an important and interesting scientific problem, especially for large datasets. PDFs are often estimated using nonparametric data-driven methods. One of the most popular nonparametric method is the Kernel Density Estimator (KDE). However, a very serious drawback of using KDEs is the large number of calculations required to compute them, especially to find the optimal bandwidth parameter. In this paper we investigate the possibility of utilizing Graphics Processing Units (GPUs) to accelerate the finding of the bandwidth. The contribution of this paper is threefold: (a) we propose algorithmic optimization to one of bandwidth finding algorithms, (b) we propose efficient GPU versions of three bandwidth finding algorithms and (c) we experimentally compare three of our GPU implementations with the ones which utilize only CPUs. Our experiments show orders of magnitude improvements over CPU implementations of classical algorithms.

Abstract: Home‐range estimators are commonly tested with simulated animal locational data in the laboratory before the estimators are used in practice. Although kernel density estimation (KDE) has performed well as a home‐range estimator for simulated data, several recent studies have reported its poor performance when used with data collected in the field. This difference may be because KDE and other home‐range estimators are generally tested with simulated point locations that follow known statistical distributions, such as bivariate normal mixtures, which may not represent well the space‐use patterns of all wildlife species. We used simulated animal locational data of 5 point pattern shapes that represent a range of wildlife utilization distributions to test 4 methods of home‐range estimation: 1) KDE with reference bandwidths, 2) KDE with least‐squares cross‐validation, 3) KDE with plug‐in bandwidths, and 4) minimum convex polygon (MCP). For the point patterns we simulated, MCP tended to produce more accurate area estimates than KDE methods. However, MCP estimates were markedly unstable, with bias varying widely with both sample size and point pattern shape. The KDE methods performed best for concave distributions, which are similar to bivariate normal mixtures, but still overestimated home ranges by about 40–50% even in the best cases. For convex, linear, perforated, and disjoint point patterns, KDE methods overestimated home‐range sizes by 50–300%, depending on sample size and method of bandwidth selection. These results indicate that KDE does not produce home‐range estimates that are as accurate as the literature suggests, and we recommend exploring other techniques of home‐range estimation.

Estimating the kernel density function of a random vector taking values on Riemannian manifolds are considered. More Precisely, we consider the problem of the estimation of the probability density of n i.i.d. random objects on the product of two compact Riemannian manifolds without boundary. The proposed methodology adapts the Pelletier’s approach which is the kernel density estimation on non Euclidean setting. Under sufficient regularity assumptions on the underlying density, L2 convergence rates are obtained. Riemannian products are the most natural and fruitful generalization of Cartesian products. These manifold models are of interest to many fields of applied mathematics, natural sciences and technology. Let us recall here as examples, the use of these products in the domain of computer vision and in biomedical sciences. In this note, we construct an estimator of the density on a product manifold from one of the estimators existing in the literature. Among the estimation methods, we mainly cite a method based on Fourier developments proposed in (Hendriks, 1990). Another method based on geodesic distance is proposed later by B. Pelletier in (Pelletier, 2005). This method used in the present work, is a generalization of the kernel method in the affine case. Another method recently developed by Kim et al. in (Kim, 2013) use the concept of exponential map to define a kernel estimator of the density on a Riemannian manifold. The aim in the present work, is to show that Pelletier's method is the best suited to the case of the Riemannian product.

Nonparametric estimation for a probability density function that describes multivariate data has typically been addressed by kernel density estimation (KDE). A novel density estimator recently developed by Farmer and Jacobs offers an alternative high-throughput automated approach to univariate nonparametric density estimation based on maximum entropy and order statistics, improving accuracy over univariate KDE. This article presents an extension of the single variable case to multiple variables. The univariate estimator is used to recursively calculate a product array of one-dimensional conditional probabilities. In combination with interpolation methods, a complete joint probability density estimate is generated for multiple variables. Good accuracy and speed performance in synthetic data are demonstrated by a numerical study using known distributions over a range of sample sizes from 100 to 106 for two to six variables. Performance in terms of speed and accuracy is compared to KDE. The multivariate density estimate developed here tends to perform better as the number of samples and/or variables increases. As an example application, measurements are analyzed over five filters of photometric data from the Sloan Digital Sky Survey Data Release 17. The multivariate estimation is used to form the basis for a binary classifier that distinguishes quasars from galaxies and stars with up to 94% accuracy.

In this paper, a novel CFAR algorithm for detecting layover and shadow areas in Interferometric synthetic aperture radar (InSAR) images is proposed. Firstly, the probability density function (PDF) of the square root amplitude of InSAR image is estimated by the kernel density estimation. Then, a CFAR algorithm combining with the morphological method for detecting both layover and shadow is presented. Finally, the proposed algorithm is evaluated on a real InSAR image obtained by TerraSAR-X system. The experimental results have validated the effectiveness of the proposed method.