A fast algorithm for computing least-squares cross-validations for nonparametric conditional kernel density functions

A conditional beta probability density function is employed to model time-delay measurement errors associated with sensors used in triangulation ranging applications. This type of density function is less restrictive than the beta density used in previous analyses in that end points can be extended out beyond the imposed range constraints and data editing is allowed before range calculations. The resulting conditional range density function is computed along with the mean and variance. Comparisons are made of the pertinent range statistics associated with the beta and conditional beta density functions. Finally, the conditional beta density function range statistics are compared with those obtained from a gated Gaussian density function for equivalent values of time-delay measurement error standard deviation. The Gaussian statistics are determined via Monte-Carlo simulation.

In thiswork, a newalgorithmis proposed for fast estimation of nonparametricmultivariate kernel density, based on principal direction divisive partitioning (PDDP) of the data space.The goal of the proposed algorithm is to use the finite support property of kernels for fast estimation of density. Compared to earlier approaches, this work explains the need of using boundaries (for partitioning the space) instead of centroids (used in earlier approaches), for better unsupervised nature (less user incorporation), and lesser (or atleast same) computational complexity. In earlier approaches, the finite support of a fixed kernel varies within the space due to the use of cluster centroids. It has been argued that if one uses boundaries (for partitioning) rather than centroids, the finite support of a fixed kernel does not change for a constant precision error. This fact introduces better unsupervision within the estimation framework. Themain contributionof thiswork is the insight gained in the kernel density estimation with the incorporation of clustering algortihm and its application in texture synthesis. Texture synthesis through nonparametric, noncausal, Markov random field (MRF), has been implemented earlier through estimation of and sampling from nonparametric conditional density. The incorporation of the proposed kernel density estimation algorithm within the earlier texture synthesis algorithm reduces the computational complexity with perceptually same results. These results provide the efficacy of the proposed algorithm within the context of natural texture synthesis.

In this paper kernel estimates of the joint and conditional probability density functions are used to generate synthetic streamflow sequences. Streamflow is assumed to be a Markov process with time dependence characterized by a multivariate probability density function. Kernel methods are used to estimate this multivariate density function. Simulation proceeds by sequentially resampling from the conditional density function derived from the kernel estimate of the underlying multivariate probability density function. This is a nonparametric method for the synthesis of streamflow that is data‐driven and avoids prior assumptions as to the form of dependence (e.g., linear or nonlinear) and the form of the probability density functions (e.g., Gaussian). We show, using synthetic examples with known underlying models, that the nonparametric method presented is more flexible than the conventional models used in stochastic hydrology and is capable of reproducing both linear and nonlinear dependence. The effectiveness of this model is illustrated through its application to simulation of monthly streamflow from the Beaver River in Utah.

A kernel-based parametric method for conditional density estimation

Strong consistency and rates for recursive nonparametric conditional probability density estimates under (α, β)-mixing conditions

Estimators for derivatives associated with a density function can be useful in identifying its modes and inflection points. In addition, these estimators play an important role in plug-in methods associated with bandwidth selection in nonparametric kernel density estimation. In this paper, we extend the nonparametric class of density estimators proposed by Mynbaev and Martins-Filho (2010) to the estimation of m-order density derivatives. Contrary to some existing derivative estimators, the estimators in our proposed class have a full asymptotic characterization, including uniform consistency and asymptotic normality. An expression for the bandwidth that minimizes an asymptotic approximation for the estimators’ integrated squared error is provided. A Monte Carlo study sheds light on the finite sample performance of our estimators and contrasts it with that of density derivative estimators based on the classical Rosenblatt–Parzen approach.

We propose a consistent kernel-based specification test for conditional density models when the dependent variable is categorical/discrete. The method is applicable to popular parametric binary choice models such as the logit and probit specification and their multinomial and ordered counterparts, along with parametric count models, among others. The test is valid when the conditional density function contains both categorical and real-valued covariates. Theoretical support for the test and for a bootstrap-based version of the test is provided. Monte Carlo simulations are conducted to assess the finite-sample performance of the proposed method.

We suggest two improved methods for conditional density estimation. The first is based on locally fitting a log-linear model, and is in the spirit of recent work on locally parametric techniques in density estimation. The second method is a constrained local polynomial estimator. Both methods always produce non-negative estimators. We propose an algorithm suitable for selecting the two bandwidths for either estimator. We also develop a new bootstrap test for the symmetry of conditional density functions. The proposed methods are illustrated by both simulation and application to a real data set.

SUMMARY We extend the idea of crossvalidation to choose the smoothing parameters of the 'double-kernel' local linear regression for estimating a conditional density. Our selection rule optimises the estimated conditional density function by minimising the integrated squared error. We also discuss three other bandwidth selection rules, an ad hoc method used by Fan et al. (1996), a bootstrap method of Hall et al. (1999) for bandwidth selection in the estimation of conditional distribution functions, modified by Bashtannyk & Hyndman (2001) to cover conditional density functions, and finally a simple approach proposed by Hyndman & Yao (2002). The performance of the new approach is compared with these three methods by simulation studies, and our method performs outstandingly well. The method is illustrated by an application to estimating the transition density and the Value-at-Risk of treasury-bill data.

Redshift is a key quantity for inferring cosmological model parameters. In photometric redshift estimation, cosmologists use the coarse data collected from the vast majority of galaxies to predict the redshift of individual galaxies. To properly quantify the uncertainty in the predictions, however, one needs to go beyond standard regression and instead estimate the full conditional density $f(z|\mathbf{x})$ of a galaxy’s redshift $z$ given its photometric covariates $\mathbf{x}$. The problem is further complicated by selection bias: usually only the rarest and brightest galaxies have known redshifts, and these galaxies have characteristics and measured covariates that do not necessarily match those of more numerous and dimmer galaxies of unknown redshift. Unfortunately, there is not much research on how to best estimate complex multivariate densities in such settings. Here we describe a general framework for properly constructing and assessing nonparametric conditional density estimators under selection bias, and for combining two or more estimators for optimal performance. We propose new improved photo-$z$ estimators and illustrate our methods on data from the Sloan Data Sky Survey and an application to galaxy–galaxy lensing. Although our main application is photo-$z$ estimation, our methods are relevant to any high-dimensional regression setting with complicated asymmetric and multimodal distributions in the response variable.

ABSTRACTThe paper investigates various nonparametric models including regression, conditional distribution, conditional density and conditional hazard function, when the covariates are infinite dimensional. The main contribution is to prove uniform in bandwidth asymptotic results for kernel estimators of these functional operators. Then, the application issues, involving data-driven bandwidth selection, are discussed.

This paper studies the minimax rate of nonparametric conditional density estimation under a weighted absolute value loss function in a multivariate setting. We first demonstrate that conditional density estimation is impossible if one only requires that $p_{X|Z}$ is smooth in $x$ for all values of $z$. This motivates us to consider a sub-class of absolutely continuous distributions, restricting the conditional density $p_{X|Z}(x|z)$ to not only be H\"older smooth in $x$, but also be total variation smooth in $z$. We propose a corresponding kernel-based estimator and prove that it achieves the minimax rate. We give some simple examples of densities satisfying our assumptions which imply that our results are not vacuous. Finally, we propose an estimator which achieves the minimax optimal rate adaptively, i.e., without the need to know the smoothness parameter values in advance. Crucially, both of our estimators (the adaptive and non-adaptive ones) impose no assumptions on the marginal density $p_Z$, and are not obtained as a ratio between two kernel smoothing estimators which may sound like a go to approach in this problem.

We have developed and tested a new algorithm, which we call the filter, that solves the nonlinear filtering problem with discrete-time measurements by solving the Fokker-Planck equation for the conditional probability density function using a split-step technique. The wave filter uses fast convolution to compute the effect of process noise at discrete times. Between measurements, the conditional density is propagated by solving a system of ordinary differential equations. Measurement update is carried out via Bayes' rule. We propose the adjoint method to reduce the computational complexity of the algorithm by adaptively varying the mesh size.

Reliable long-term streamflow forecasting is invaluable for water resource planning and management which allocates water supply according to the demand of water users. It is necessary to get probabilistic forecasts to establish risk-based reservoir operation policies. Probabilistic forecasts may be useful for the users who assess and manage risks according to decision-making responding forecasting results. Probabilistic forecasting of seasonal inflow to Andong dam is performed and assessed using selected predictors from sea surface temperature and 500 hPa geopotential height data. Categorical probability forecast by Piechota's method and logistic regression analysis, and probability forecast by conditional probability density function are used to forecast seasonal inflow. Kernel density function is used in categorical probability forecast by Piechota's method and probability forecast by conditional probability density function. The results of categorical probability forecasts are assessed by Brier skill score. The assessment reveals that the categorical probability forecasts are better than the reference forecasts. The results of forecasts using conditional probability density function are assessed by qualitative approach and transformed categorical probability forecasts. The assessment of the forecasts which are transformed to categorical probability forecasts shows that the results of the forecasts by conditional probability density function are much better than those of the forecasts by Piechota's method and logistic regression analysis except for winter season data.

A criticism of using the Gaussian density function to model the classes in multispectral remote sensing data is that the classes may be multimodal, and therefore, not well represented by the Gaussian density function. The normal mixture density function, which is the sum of one or more weighted Gaussian components, is a compromise between Gaussian and non-parametric density functions. It can model multimodal density functions, yet requires fewer parameters to be estimated than non-parametric density functions, which is an advantage in remote sensing applications where training samples are difficult and expensive to obtain. Usually in practice, the number of components is not known, and must be estimated from the training samples. A new approach for estimating the appropriate number of components in a normal mixture density is described. The approach is to divide the data from each class into various numbers of clusters using the nearest means algorithm, compute a measure of fit which measures how well the training data are represented by the clusters, and to select the number of clusters that best fits the data. The measure of fit is computed by leaving one sample out and estimating the mean vectors and covariance matrices of the mixture density with the rest of the samples. Then the likelihood of the left-out sample is computed, and the process is repeated, each time leaving out a different sample. In this way, the samples used to test the estimates are independent of the samples used to compute estimates, and so the measure of fit, which is the average log likelihood of the left-out samples, does not increase monotonically like the joint likelihood, but will reach a maximum. The number of clusters that results in the maximum value of the measure of fit is selected as the estimate of the number of components in the mixture density.