Adaptive wavelet estimation of a function from an m-dependent process with possibly unbounded m

We consider the problem of nonparametric estimation of unknown smooth functions in the presence of restrictions on the shape of the estimator and on its support using polynomial splines. We provide a general computational framework that treats these estimation problems in a unified manner, without the limitations of the existing methods. Applications of our approach include computing optimal spline estimators for regression, density estimation, and arrival rate estimation problems in the presence of various shape constraints. Our approach can also handle multiple simultaneous shape constraints. The approach is based on a characterization of nonnegative polynomials that leads to semidefinite programming (SDP) and second-order cone programming (SOCP) formulations of the problems. These formulations extend and generalize a number of previous approaches in the literature, including those with piecewise linear and B-spline estimators. We also consider a simpler approach in which nonnegative splines are approximated by splines whose pieces are polynomials with nonnegative coefficients in a nonnegative basis. A condition is presented to test whether a given nonnegative basis gives rise to a spline cone that is dense in the space of nonnegative continuous functions. The optimization models formulated in the article are solvable with minimal running time using off-the-shelf software. We provide numerical illustrations for density estimation and regression problems. These examples show that the proposed approach requires minimal computational time, and that the estimators obtained using our approach often match and frequently outperform kernel methods and spline smoothing without shape constraints. Supplementary materials for this article are provided online.

Problems with censored data arise frequently in survival analyses and reliability applications. The estimation of the density function of the failure times is often of interest. Two inherent problems in density estimation for lifetime data are the spill-over at the origin and the smoothing parameter selection. To address these issues, we propose the use of asymmetric kernels (like inverse Gaussian) with bandwidths selected by a Bayes criterion. We show strong pointwise consistency of the density estimator, and for suitable choices of the prior, we show that one can obtain meaningful bandwidths with the same rates of convergence as for the classical asymptotically optimal bandwidths.

Multivariate density estimation is a fundamental problem in Applied Statistics and Machine Learning. Given a collection of data sampled from an unknown distribution, the task is to approximately reconstruct the generative distribution. There are two different approaches to the problem, the parametric approach and the non-parametric approach. In the parametric approach, the approximate distribution is represented by a model from a predetermined family. In this thesis, we adopt the parametric approach and investigate the use of a model family called latent tree models for the task of density estimation. Latent tree models are tree-structured Bayesian networks in which leaf nodes represent observed variables, while internal nodes represent hidden variables. Such models can represent complex relationships among observed variables, and in the meantime, admit efficient inference among them. Consequently, they are a desirable tool for density estimation. While latent tree models are studied for the first time in this thesis for the purpose of density estimation, they have been investigated earlier for clustering and latent structure discovery. Several algorithms for learning latent tree models have been proposed. The state-of-the-art is an algorithm called EAST. EAST determines model structures through principled and systematic search, and determines model parameters using the EM algorithm. It has been shown to be capable of achieving good trade-off between fit to data and model complexity. It is also capable of discovering latent structures behind data. Unfortunately, it has a high computational complexity, which limits its applicability to density estimation problems. In this thesis, we propose two latent tree model learning algorithms specifically for density estimation. The two algorithms have distinct characteristics and are suitable for different applications. The first algorithm is called HCL. HCL assumes a predetermined bound on model complexity and restricts to binary model structures. It first builds a binary tree structure based on mutual information and then runs the EM algorithm once on the resulting structure to determine the parameters. As such, it is efficient and can deal with large applications. The second algorithm is called Pyramid. Pyramid does not assume predetermined bounds on model complexity and does not restrict to binary tree structures. It builds model structures using heuristics based on mutual information and local search. It is slower than HCL. However, it is faster than EAST and is only slightly inferior to EAST in terms of the quality of the resulting models. In this thesis, we also study two applications of the density estimation techniques that we develop. The first application is to approximate probabilistic inference in Bayesian networks. A Bayesian network represents a joint distribution over a set of random variables. It often happens that the network structure is very complex and making inference directly on the network is computational intractable. We propose to approximate the joint distribution using a latent tree model and exploit the latent tree model for faster inference. The idea is to sample data from the Bayesian network, learn a latent tree model from the data offline, and when online, make inference with the latent tree model instead of the original Bayesian network. HCL is used here because the sample size needs to be large to produce accurate approximation and it is possible to predetermine a bound on the online running. Empirical evidence shows that this method can achieve good approximation accuracy at low online computational cost. The second application is classification. A common approach to this task is to formulate it as a density estimation problem: One constructs the class-conditional density for each class and then uses the Bayes rule to make classification. We propose to estimate those class-conditional densities using either EAST or Pyramid. Empiricalevidence shows that this method yields good classification performances. Moreover, the latent tree models built for the class-conditional densities are often meaningful, which is conducive to user confidence. A comparison between EAST and Pyramid reveals that Pyramid is significantly more efficient than EAST, while it results in more or less the same classification performance as the latter.

Let us consider the problem of robust spectral density estimation. Conventional methods to obtain estimates of spectral density function are not robust in the presence of outlying observations. We present different methods to robustly estimate the spectral density function that are insensitive against outliers. The proposed methods are applied to simulated and real data and the results are compared. As a special practical application we focus on the frequency-domain analysis of short-term heart rate variability measurements of diabetes patients.

We deal with deconvolution problems in density estimation. Assume that the data follow a density, which is a convolution of the original density f being of interest with a noise density f ϵ. In order to estimate the density f, one usually should know f ϵ completely and then uses some technique for deconvolution. In contrast, the so-called TAYLEX and SIMEX methods introduced by Carroll and Hall and Cook and Stefanski, respectively use partial information on f ϵ only and correct the naive density estimator towards the deconvoluted one. In the present paper, we assume that we have more and more information on the noise density when the sample size increases. We show that by applying these methods, one can achieve almost optimal rates and optimal rates respectively for densities f belonging to certain Sobolev classes.

Density estimation in the presence of noise

Nonparametric density estimation by exact leave- [formula omitted]-out cross-validation

The aim of this thesis is to provide two extensions to the theory of nonparametric kernel density estimation that increase the scope of the technique. The basic ideas of kernel density estimation are not new, having been proposed by Rosenblatt [20] and extended by Parzen [17]. The objective is that for a given set of data, estimates of functions of the distribution of the data such as probability densities are derived without recourse to rigid parametric assumptions and allow the data themselves to be more expressive in the statistical outcome. Thus kernel estimation has captured the imagination of statisticians searching for more flexibility and eager to utilise the computing revolution. The abundance of data and computing power have revealed distributional shapes that are difficult to model by traditional parametric approaches and in this era, the computer intensive technique of kernel estimation can be performed routinely. Also we are aware that computing power can be harnessed to give improved statistical analyses. Thus a lot of modern statistical research involves kernel estimation from complex data sets and our research is concordant with that momentum. The thesis contains three chapters. In Chapter 1 we provide an introduction to kernel density estimation and we give an outline to our two research topics. Our first extension to the theory is given in Chapter 2 where we investigate density estimation from independent data, using high order kernel functions. These kernel functions are designed for bias reduction but they have the penalty of yielding negative density estimates where data are sparse. In common practice, the negative estimates would arise in the tails of the density and we provide four ways of correcting this negativity to give bona fide estimates of the probability density. Our theory shows that the effects of these corrections are asymptotically negligible and thus opens the way for the regular use of bias reducing, high order kernel functions. We also consider density estimation of continuous stationary stochastic processes and this is the content of Chapter 3. With this problem, the dependent nature of the data influences the accuracy of the kernel density estimator and we provide theory regarding the convergence of the kernel estimators of the density and its derivatives to the true functions. An important result from this study is that nonparametric density estimators from dependent processes can have the same rates of convergence as their parametric counterparts yet retain the flexibility of being independent of parametric assumptions. Our other results indicate that the convergence rate of the density estimator can be quite slow if there are large lag dependencies amongst the data and suggests that large samples would be required for reliable inference about such data. The flexibility of kernel density estimation for continuous and discrete data, independent and dependent observations, means that it is a useful statistical tool. The techniques given in this thesis are not restricted to the analysis of simple sets of data but may be employed in the construction of statistical models for complex data with a high degree of structure.

Accurate Estimation of Multivariable Frequency Response Functions

Nonparametric estimation of the survival function for either incident or prevalent cohort failure time data, exclusively, has been well studied in the literature; the Kaplan‐Meier (KM) estimator is routinely used for right‐censored incident cohort failure time data, whereas a modified form of the KM estimator, sometimes referred to as the Tsai–Jewell–Wang (TJW) estimator, is the default estimator used for prevalent cohort data with follow‐up. Often, failure time data comprise observations from a combination of incident and prevalent cohorts. In this note, we justify the use of the TJW estimator for a combined sample of incident and prevalent cohort data with follow‐up. We suggest how the TJW estimator forms the basis for density estimation and hypothesis testing problems, when incident and prevalent cohorts are combined.

Non-parametric Bayesian models of response function in dynamic image sequences

In this paper, we study the problem of density estimation from data that contains small measurement errors. The only assumption on these errors is that the maximal measurement error is bounded by some real number converging to zero for sample size tending to infinity. In particular, we do not assume that the measurement errors are independent with expectation zero. We estimate the density by a standard kernel density estimate applied to data with measurement errors and derive a data-driven method to choose its bandwidth. We derive an adaptation result for this estimate and analyze the expected L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> error of our density estimate depending on the smoothness of the density and the size of the maximal measurement error. The results are applied in a density estimation problem in a simulation model, where we show under suitable assumptions that the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> error of our newly proposed estimate converges to zero much faster than the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> error of the standard kernel density estimate if both are based on the same number of observations in the simulation model. The performance of the method in case of finite sample size is analyzed using simulated data.

In this article, we propose a new adaptive estimator for multivariate density functions defined on a bounded domain in the framework of multivariate mixing processes. Several procedures have been proposed in the literature to tackle the boundary bias issue encountered using classical kernel estimators. Most of them are designed to work in dimension $d=1$ or on the unit $d$-dimensional hypercube. We extend such results to more general bounded domains such as simple polygons or regular domains that satisfy a rolling condition. We introduce a specific family of kernel-type estimators devoid of boundary bias. We then propose a data-driven Goldenshluger and Lepski type procedure to jointly select a kernel and a bandwidth. We prove the optimality of our procedure in the adaptive framework, stating an oracle-type inequality. We illustrate the good behavior of our new class of estimators on simulated data. Finally, we apply our procedure to a real dataset.

The large literature on software reliability assessment and prediction is essentially concerned with parametric models. During the last decade non-parametric models have been developed. This paper presents a new non-parametric model which is based on the idea of separation of concern between the long term trend in reliability growth and the local behaviour The trend is captured using moving averages while the local behaviour is estimated using a kernel estimator. The predictions obtained from the new model are compared with predictions that come from earlier models on some real data sets. A simulation study is also conducted.

Bitcoin has rapidly gained much attention by media, investors and scholars, since it is widely used for investment purposes as an alternative to regular currencies. The price of bitcoin is characterized with high volatility, which makes firms that hold large amounts of it prone to risks. To prevent those risks and to gain insights into its behavior and trading strategies, it is necessary to forecast bitcoin volatility. In that context, this study reviews the impact of different factors on BTC price, returns and volatility. Moreover, it examines the sensitivity of GARCH family models to standardized residuals distributions in forecasting BTC volatility based on the leverage effect. Then, it compares the best GARCH-type model against the best ANN model with respect to short-term and long-term horizons. Results outline that APARCH, TGARCH and EGARCH are very sensitive to standardized residuals distributions, such that TGARCH run with the normal distribution is the best model that captures BTC volatility. Further, MLP outperforms all the parametric and nonparametric models, while its accuracy weakens with the level of forecasting horizons. Consequently, nonparametric models prevail parametric models in BTC volatility forecasting due to their high degree of flexibility and strong generalization abilities, whereas MLP is only effective in short-term forecasting.