Shape-Constrained Estimation Using Nonnegative Splines

In the problem of nonparametric regression for a fixed design model, we may want to use additional information about the shape of the regression function, when available, to improve the estimation. The regression function may, for example, be convex or monotone or more generally belong to a cone in some functional space. We devise a method for improving any ordinary consistent estimate by projecting it onto a discretized version of the cone, using the theory of reproducing kernel Hilbert spaces and convex optimization techniques. The initial estimate can be chosen as a smoothing spline or a convolution type kernel estimate. The latter is shown to be mean square consistent in a Sobolev norm sense. The consistency (in the same sense) of the constrained estimate follows.

This article concerns datasets in which variables are in the form of intervals, which are obtained by aggregating information about variables from a larger dataset. We propose to view the observed set of hyper-rectangles as an empirical histogram, and to use a Gaussian kernel type estimator to approximate its underlying distribution in a nonparametric way. We apply this idea to both univariate density estimation and regression problems. Unlike many existing methods used in regression analysis, the proposed method can estimate the conditional distribution of the response variable for any given set of predictors even when some of them are not interval-valued. Empirical studies show that the proposed approach has a great flexibility in various scenarios with complex relationships between the location and width of intervals of the response and predictor variables.

ABSTRACTThe estimation of a multivariate function from a stationary m-dependent process is investigated, with a special focus on the case where m is large or unbounded. We develop an adaptive estimator based on wavelet methods. Under flexible assumptions on the nonparametric model, we prove the good performances of our estimator by determining sharp rates of convergence under two kinds of errors: the pointwise mean squared error and the mean integrated squared error. We illustrate our theoretical result by considering the multivariate density estimation problem, the derivatives density estimation problem, the density estimation problem in a GARCH-type model and the multivariate regression function estimation problem. The performance of proposed estimator has been shown by a numerical study for a simulated and real data sets.

Joining multiple decision-makers together is a powerful way to obtain more sophisticated decision-making systems, but requires to address the questions of division of labor and specialization. We investigate in how far information constraints in hierarchies of experts not only provide a principled method for regularization but also to enforce specialization. In particular, we devise an information-theoretically motivated on-line learning rule that allows partitioning of the problem space into multiple sub-problems that can be solved by the individual experts. We demonstrate two different ways to apply our method: (i) partitioning problems based on individual data samples and (ii) based on sets of data samples representing tasks. Approach (i) equips the system with the ability to solve complex decision-making problems by finding an optimal combination of local expert decision-makers. Approach (ii) leads to decision-makers specialized in solving families of tasks, which equips the system with the ability to solve meta-learning problems. We show the broad applicability of our approach on a range of problems including classification, regression, density estimation, and reinforcement learning problems, both in the standard machine learning setup and in a meta-learning setting.

A directional error estimator for adaptive limit analysis

To enhance the self-healing control capability of soft open points with energy storage (E-SOPs) and optimize the fault recovery performance in flexible interconnected distribution networks, this paper proposes a novel power supply restoration method based on E-SOP. The methodology begins with a comprehensive analysis of the E-SOP’s fundamental architecture and loss model. Subsequently, a dual-objective optimization function is formulated to maximize the sum of nodal active load restoration while minimizing network losses. The optimization problem is transformed into a second-order cone programming formulation under comprehensive operational constraints. To solve this complex optimization model, an innovative hybrid approach combining the Improved Whale Optimization Algorithm (IWOA) with second-order cone programming is developed. The proposed methodology is extensively validated using the IEEE 33-node test system. The experimental results demonstrate that this approach significantly enhances the power supply restoration capability of distribution networks while maintaining practical feasibility.

We deal with deconvolution problems in density estimation. Assume that the data follow a density, which is a convolution of the original density f being of interest with a noise density f ϵ. In order to estimate the density f, one usually should know f ϵ completely and then uses some technique for deconvolution. In contrast, the so-called TAYLEX and SIMEX methods introduced by Carroll and Hall and Cook and Stefanski, respectively use partial information on f ϵ only and correct the naive density estimator towards the deconvoluted one. In the present paper, we assume that we have more and more information on the noise density when the sample size increases. We show that by applying these methods, one can achieve almost optimal rates and optimal rates respectively for densities f belonging to certain Sobolev classes.

Density estimation in the presence of noise

Nonparametric density estimation by exact leave- [formula omitted]-out cross-validation

Problems with censored data arise frequently in survival analyses and reliability applications. The estimation of the density function of the failure times is often of interest. Two inherent problems in density estimation for lifetime data are the spill-over at the origin and the smoothing parameter selection. To address these issues, we propose the use of asymmetric kernels (like inverse Gaussian) with bandwidths selected by a Bayes criterion. We show strong pointwise consistency of the density estimator, and for suitable choices of the prior, we show that one can obtain meaningful bandwidths with the same rates of convergence as for the classical asymptotically optimal bandwidths.

The aim of this thesis is to provide two extensions to the theory of nonparametric kernel density estimation that increase the scope of the technique. The basic ideas of kernel density estimation are not new, having been proposed by Rosenblatt [20] and extended by Parzen [17]. The objective is that for a given set of data, estimates of functions of the distribution of the data such as probability densities are derived without recourse to rigid parametric assumptions and allow the data themselves to be more expressive in the statistical outcome. Thus kernel estimation has captured the imagination of statisticians searching for more flexibility and eager to utilise the computing revolution. The abundance of data and computing power have revealed distributional shapes that are difficult to model by traditional parametric approaches and in this era, the computer intensive technique of kernel estimation can be performed routinely. Also we are aware that computing power can be harnessed to give improved statistical analyses. Thus a lot of modern statistical research involves kernel estimation from complex data sets and our research is concordant with that momentum. The thesis contains three chapters. In Chapter 1 we provide an introduction to kernel density estimation and we give an outline to our two research topics. Our first extension to the theory is given in Chapter 2 where we investigate density estimation from independent data, using high order kernel functions. These kernel functions are designed for bias reduction but they have the penalty of yielding negative density estimates where data are sparse. In common practice, the negative estimates would arise in the tails of the density and we provide four ways of correcting this negativity to give bona fide estimates of the probability density. Our theory shows that the effects of these corrections are asymptotically negligible and thus opens the way for the regular use of bias reducing, high order kernel functions. We also consider density estimation of continuous stationary stochastic processes and this is the content of Chapter 3. With this problem, the dependent nature of the data influences the accuracy of the kernel density estimator and we provide theory regarding the convergence of the kernel estimators of the density and its derivatives to the true functions. An important result from this study is that nonparametric density estimators from dependent processes can have the same rates of convergence as their parametric counterparts yet retain the flexibility of being independent of parametric assumptions. Our other results indicate that the convergence rate of the density estimator can be quite slow if there are large lag dependencies amongst the data and suggests that large samples would be required for reliable inference about such data. The flexibility of kernel density estimation for continuous and discrete data, independent and dependent observations, means that it is a useful statistical tool. The techniques given in this thesis are not restricted to the analysis of simple sets of data but may be employed in the construction of statistical models for complex data with a high degree of structure.

This letter addresses the problem of source localization in non-line of sight (NLOS) environment using received signal strength (RSS) and time of arrival (TOA) measurements when both transmission time and power are unknown. To mitigate the NLOS bias, a robust weighted least square (RWLS) problem is formulated from original non-convex maximum-likelihood (ML) estimation problem using a set of tight approximations. The RWLS problem is further tightened by a novel convex-hull constraint and is ultimately solved using second-order cone programming (SOCP). Simulation results confirm the optimal performance of the proposed robust method in various network scenarios and also validate its excellence over state-of-art methods.

In this paper, we study the problem of density estimation from data that contains small measurement errors. The only assumption on these errors is that the maximal measurement error is bounded by some real number converging to zero for sample size tending to infinity. In particular, we do not assume that the measurement errors are independent with expectation zero. We estimate the density by a standard kernel density estimate applied to data with measurement errors and derive a data-driven method to choose its bandwidth. We derive an adaptation result for this estimate and analyze the expected L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> error of our density estimate depending on the smoothness of the density and the size of the maximal measurement error. The results are applied in a density estimation problem in a simulation model, where we show under suitable assumptions that the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> error of our newly proposed estimate converges to zero much faster than the L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> error of the standard kernel density estimate if both are based on the same number of observations in the simulation model. The performance of the method in case of finite sample size is analyzed using simulated data.

Hierarchical age estimation with dissimilarity-based classification

Head pose estimation is an important problem in the field of computer vision and facial analysis. We model the problem of head pose estimation as a regression problem, where the three rotation angles (yaw, pitch, roll) are functions of the face appearance. We make use of that fact and learn the appearance of the face using a tree cascade of sparse Multi-Variate Relevance Vector Machines (MVRVM). Our method is fast and suitable for real-time applications as it is not computationally expensive. Our method learns the face appearance to estimate the head rotation angles. We evaluated our approach on two challenging datasets, the YouTube Faces and the Point and Shoot Challenging (PaSC) dataset. We achieved results of head pose estimation (yaw, pitch, roll) with mean error less than 5\(\circ \) and with error tolerance less than ±4 on the PaSC dataset. In terms of speed, one prediction takes around 6 milliseconds, which is suitable for real-time applications and also with high frame rate.