Bin width selection in multivariate histograms by the combinatorial method

Optimal [formula omitted] bandwidth selection for variable kernel density estimates

Methods of density estimation on the Grassmann manifold

Density estimation by the penalized combinatorial method

For eight years (2007–2014), Greeneridge Sciences deployed, collected, and analyzed an immense passive acoustic dataset to study the response of bowhead whales (Balaena mysticetus) to oil exploration activities along a 280 km swath of the Alaskan Arctic continental shelf. Each year, up to 40 Directional Autonomous Seafloor Acoustic Recorders (DASARs) detected bowhead calls during the whales’ annual fall migration, resulting in over 2.4 million localized calls over the study period. These enormous sample sizes, combined with the unique localization capability of the DASARs, have enabled the investigation and cross-validation of several acoustic density estimation (DE) methods. Here, we compared results among three density estimation methods: (i) direct census, (ii) distance sampling, and (iii) spatially explicit capture recapture (SECR). In the course of our investigation, we encountered and addressed two challenges: (i) how to apply point transect theory to distributed array systems, and (ii) how to implement SECR in situations where detections between sensors are not statistically independent or where false positives exist on single sensors. These density estimates were then used to estimate relative abundance of the Western Arctic bowhead whale population over multiple years. [Work sponsored by the Joint Industry Programme.]

Multivariate density estimation is a fundamental problem in Applied Statistics and Machine Learning. Given a collection of data sampled from an unknown distribution, the task is to approximately reconstruct the generative distribution. There are two different approaches to the problem, the parametric approach and the non-parametric approach. In the parametric approach, the approximate distribution is represented by a model from a predetermined family. In this thesis, we adopt the parametric approach and investigate the use of a model family called latent tree models for the task of density estimation. Latent tree models are tree-structured Bayesian networks in which leaf nodes represent observed variables, while internal nodes represent hidden variables. Such models can represent complex relationships among observed variables, and in the meantime, admit efficient inference among them. Consequently, they are a desirable tool for density estimation. While latent tree models are studied for the first time in this thesis for the purpose of density estimation, they have been investigated earlier for clustering and latent structure discovery. Several algorithms for learning latent tree models have been proposed. The state-of-the-art is an algorithm called EAST. EAST determines model structures through principled and systematic search, and determines model parameters using the EM algorithm. It has been shown to be capable of achieving good trade-off between fit to data and model complexity. It is also capable of discovering latent structures behind data. Unfortunately, it has a high computational complexity, which limits its applicability to density estimation problems. In this thesis, we propose two latent tree model learning algorithms specifically for density estimation. The two algorithms have distinct characteristics and are suitable for different applications. The first algorithm is called HCL. HCL assumes a predetermined bound on model complexity and restricts to binary model structures. It first builds a binary tree structure based on mutual information and then runs the EM algorithm once on the resulting structure to determine the parameters. As such, it is efficient and can deal with large applications. The second algorithm is called Pyramid. Pyramid does not assume predetermined bounds on model complexity and does not restrict to binary tree structures. It builds model structures using heuristics based on mutual information and local search. It is slower than HCL. However, it is faster than EAST and is only slightly inferior to EAST in terms of the quality of the resulting models. In this thesis, we also study two applications of the density estimation techniques that we develop. The first application is to approximate probabilistic inference in Bayesian networks. A Bayesian network represents a joint distribution over a set of random variables. It often happens that the network structure is very complex and making inference directly on the network is computational intractable. We propose to approximate the joint distribution using a latent tree model and exploit the latent tree model for faster inference. The idea is to sample data from the Bayesian network, learn a latent tree model from the data offline, and when online, make inference with the latent tree model instead of the original Bayesian network. HCL is used here because the sample size needs to be large to produce accurate approximation and it is possible to predetermine a bound on the online running. Empirical evidence shows that this method can achieve good approximation accuracy at low online computational cost. The second application is classification. A common approach to this task is to formulate it as a density estimation problem: One constructs the class-conditional density for each class and then uses the Bayes rule to make classification. We propose to estimate those class-conditional densities using either EAST or Pyramid. Empiricalevidence shows that this method yields good classification performances. Moreover, the latent tree models built for the class-conditional densities are often meaningful, which is conducive to user confidence. A comparison between EAST and Pyramid reveals that Pyramid is significantly more efficient than EAST, while it results in more or less the same classification performance as the latter.

In many data stream mining applications, traditional density estimation methods such as kernel density estimation, reduced set density estimation can not be applied to the density estimation of data streams because of their high computational burden, processing time and intensive memory allocation requirement. In order to reduce the time and space complexity, a novel density estimation method Dm-KDE over data streams based on the proposed algorithm m-KDE which can be used to design a KDE estimator with the fixed number of kernel components for a dataset is proposed. In this method, Dm-KDE sequence entries are created by algorithm m-KDE instead of all kernels obtained from other density estimation methods. In order to further reduce the storage space, Dm-KDE sequence entries can be merged by calculating their KL divergences. Finally, the probability density functions over arbitrary time or entire time can be estimated through the obtained estimation model. In contrast to the state-of-the-art algorithm SOMKE, the distinctive advantage of the proposed algorithm Dm-KDE exists in that it can achieve the same accuracy with much less fixed number of kernel components such that it is suitable for the scenarios where higher on-line computation about the kernel density estimation over data streams is required.We compare Dm-KDE with SOMKE and M-kernel in terms of density estimation accuracy and running time for various stationary datasets. We also apply Dm-KDE to evolving data streams. Experimental results illustrate the effectiveness of the proposed method.

A method is proposed for achieving fast—O(n –8/9) and faster—asymptotic mean integrated squared error (AMISE) convergence rates in density estimation using data condensed to standard histogram bin counts and edges. The method involves the two elements of B splines and the correct mass proportions property, in which the probability mass for the histogram bins equals exactly the fraction of the data found in those bins. It is shown that the asymptotic bias of such an estimate using a spline of order p and bin width h will be O(hP ), resulting in optimal AMISE of O(n –2p/2p+1) form. Theoretical and numeric comparisons to kernel-based and other estimators show that the histospline estimator is usually comparable and often superior. An extension to the multivariate density estimation setting is described.

This study compared the results from three freeway density estimation methods based on point detector data with the results obtained from the density estimation procedure of the Highway Capacity Manual 2010 (HCM 2010). The three methods were the cumulative volume–based method, the occupancy-based method, and the fundamental relationship–based method. The study also developed and tested a new method that integrated data from point traffic detectors and automatic vehicle identification readers to estimate the density of freeway segments for offline and real-time applications. The four density estimation methods were compared with each other and the HCM 2010 method by using two case studies based on simulation modeling and real-world data. Results showed that the density estimates based on the proposed segmentation method, cumulative volume method, and HCM 2010 method were generally closer to each other compared with the estimates based on the other two tested methods. The simulation case study showed that the density estimates from these three methods were also closer to density measurements obtained on the basis of vehicle trajectories from simulation. The two case study results indicate that the selection of the density estimation method affects mainly the level-of-service value during intermediate congested conditions.

Nonparametric estimation for a probability density function that describes multivariate data has typically been addressed by kernel density estimation (KDE). A novel density estimator recently developed by Farmer and Jacobs offers an alternative high-throughput automated approach to univariate nonparametric density estimation based on maximum entropy and order statistics, improving accuracy over univariate KDE. This article presents an extension of the single variable case to multiple variables. The univariate estimator is used to recursively calculate a product array of one-dimensional conditional probabilities. In combination with interpolation methods, a complete joint probability density estimate is generated for multiple variables. Good accuracy and speed performance in synthetic data are demonstrated by a numerical study using known distributions over a range of sample sizes from 100 to 106 for two to six variables. Performance in terms of speed and accuracy is compared to KDE. The multivariate density estimate developed here tends to perform better as the number of samples and/or variables increases. As an example application, measurements are analyzed over five filters of photometric data from the Sloan Digital Sky Survey Data Release 17. The multivariate estimation is used to form the basis for a binary classifier that distinguishes quasars from galaxies and stars with up to 94% accuracy.

The elements that constitute the literary space can mainly be described by categories such as settings, zones where actions take place and routes along which characters move. Apart from the presentation of the specific locations and spatial distribution of literary places, we are also interested in the spatial pattern such places form, the boundaries that separate the literarily populated from the void regions and the varying density of the literary space. From a GIS point of view, the elements of the literary space correspond to point, line and area data types. However, the established method used to calculate the spatially varying density – the method of density estimation – is restricted to point data only. In this paper, we will present an improved method that is able to estimate the density regardless of the underlying data type. Our approach aims at adopting the typically radially symmetric kernel function to approximate also linear and areal features. We claim that this method treats point, line and area data in a consistent way by taking equivalent density contributions into account. The different steps of the improved method can visually be examined by the accompanying map examples.

We introduce a recursive method for estimating a probability density subject to constraints of unimodality or monotonicity. It uses an empirical estimate of the probability transform to construct a sequence of maps of a known template, which satisfies the constraints. The algorithm may be employed without a smoothing step, in which case it produces step-function approximations to the sampling density. More satisfactorily, a certain amount of smoothing may be interleaved between each recursion, in which case the estimate is smooth. The amount of smoothing may be chosen using a standard cross-validation algorithm. Unlike other methods for density estimation, however, the recursive approach is robust against variation of the amount of smoothing, and so choice of bandwidth is not critical.

Nonparametric density estimation is a fundamental problem of statistics and data mining. Even though kernel density estimation is the most widely used method, its performance highly depends on the choice of the kernel bandwidth, and it can become computationally expensive for large data sets. We present an adaptive sparse-grid-based density estimation method which discretizes the estimated density function on basis functions centered at grid points rather than on kernels centered at the data points. Thus, the costs of evaluating the estimated density function are independent from the number of data points. We give details on how to estimate density functions on sparse grids and develop a cross validation technique for the parameter selection. We show numerical results to confirm that our sparse-grid-based method is well-suited for large data sets, and, finally, employ our method for the classification of astronomical objects to demonstrate that it is competitive to current kernel-based density estimation approaches with respect to classification accuracy and runtime.

BackgroundIn the Point-Centred Quarter Method (PCQM), the mean distance of the first nearest plants in each quadrant of a number of random sample points is converted to plant density. It is a quick method for plant density estimation. In recent publications the estimator equations of simple PCQM (PCQM1) and higher order ones (PCQM2 and PCQM3, which uses the distance of the second and third nearest plants, respectively) show discrepancy. This study attempts to review PCQM estimators in order to find the most accurate equation form. We tested the accuracy of different PCQM equations using Monte Carlo Simulations in simulated (having ‘random’, ‘aggregated’ and ‘regular’ spatial patterns) plant populations and empirical ones.Principal FindingsPCQM requires at least 50 sample points to ensure a desired level of accuracy. PCQM with a corrected estimator is more accurate than with a previously published estimator. The published PCQM versions (PCQM1, PCQM2 and PCQM3) show significant differences in accuracy of density estimation, i.e. the higher order PCQM provides higher accuracy. However, the corrected PCQM versions show no significant differences among them as tested in various spatial patterns except in plant assemblages with a strong repulsion (plant competition). If N is number of sample points and R is distance, the corrected estimator of PCQM1 is 4(4N − 1)/(π ∑ R2) but not 12N/(π ∑ R2), of PCQM2 is 4(8N − 1)/(π ∑ R2) but not 28N/(π ∑ R2) and of PCQM3 is 4(12N − 1)/(π ∑ R2) but not 44N/(π ∑ R2) as published.SignificanceIf the spatial pattern of a plant association is random, PCQM1 with a corrected equation estimator and over 50 sample points would be sufficient to provide accurate density estimation. PCQM using just the nearest tree in each quadrant is therefore sufficient, which facilitates sampling of trees, particularly in areas with just a few hundred trees per hectare. PCQM3 provides the best density estimations for all types of plant assemblages including the repulsion process. Since in practice, the spatial pattern of a plant association remains unknown before starting a vegetation survey, for field applications the use of PCQM3 along with the corrected estimator is recommended. However, for sparse plant populations, where the use of PCQM3 may pose practical limitations, the PCQM2 or PCQM1 would be applied. During application of PCQM in the field, care should be taken to summarize the distance data based on ‘the inverse summation of squared distances’ but not ‘the summation of inverse squared distances’ as erroneously published.

In the linear model y = x′β + u where x is a regressor vector and E(u | x)= 0, we estimate β in E(y | x)= x′β However, the assumption of the linear model, or any nonlinear model for that matter, is a strong one. In nonparametric regression, we try to estimate E(y | x) without specifying the functional form. Since $$E(y{\kern 1pt} |{\kern 1pt} x) = \smallint y \cdot f(y|x)dy = \smallint y \cdot \{ f(y,x)/f(x)\} dy $$ (1.1) if we can estimate f(y, x) and f(x), we can also estimate E(y | x). In this chapter, we study nonparametric density estimation for x as a prelude to nonparametric regression in the next chapter. We will assume that x has a continuous density function f (x) If x is discrete, one can estimate P(x = x 0) either with the same estimation method used for the continuous case or with the number of observations with x i = x 0. There are several nonparametric density estimation methods available. The most popular is “kernel density estimation method” which we explore mainly. Other methods will be examined briefly in the last section. See Prakasa Rao (1983), Silverman (1986), Izenman (1991), Rosenblatt (1991) and Scott (1992) for more on nonparametric density estimation in general.

Kernel density estimation (KDE) is one of the most widely used nonparametric density estimation methods. The fact that it is a memory-based method, i.e., it uses the entire training data set for prediction, makes it unsuitable for most current big data applications. Several strategies, such as tree-based or hashing-based estimators, have been proposed to improve the efficiency of the kernel density estimation method. The novel density kernel density estimation method (DMKDE) uses density matrices, a quantum mechanical formalism, and random Fourier features, an explicit kernel approximation, to produce density estimates. This method has its roots in the KDE and can be considered as an approximation method, without its memory-based restriction. In this paper, we systematically evaluate the novel DMKDE algorithm and compare it with other state-of-the-art fast procedures for approximating the kernel density estimation method on different synthetic data sets. Our experimental results show that DMKDE is on par with its competitors for computing density estimates and advantages are shown when performed on high-dimensional data. We have made all the code available as an open source software repository.