• Home
  • Search
  • Sample-Optimal Density Estimation in Nearly-Linear Time
  • Cite Icon60
  • https://doi.org/10.1137/1.9781611974782.83Copy DOI Icon

Sample-Optimal Density Estimation in Nearly-Linear Time

  • Jan 1, 2017
  • Jayadev Acharya +3 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

We design a new, fast algorithm for agnostically learning univariate probability distributions whose densities are well-approximated by piecewise polynomial functions. Let ƒ be the density function of an arbitrary univariate distribution, and suppose that ƒ is OPT-close in Li- distance to an unknown piecewise polynomial function with t interval pieces and degree d. For any γ > 0, our algorithm draws n = Õγ(t(d + 1)/ ∊2) samples from ƒ, runs in time Õ(n), and with probability at least 9/10 outputs an Ογ (t)-piecewise degree-d hypothesis H that is (3 + γ) · OPT + ∊ close to f. Our approximation factor almost matches the best known information-theoretic (but computationally inefficient) upper bound of 3.Our general algorithm yields (nearly) sample- optimal and nearly-linear time estimators for a wide range of structured distribution families over both continuous and discrete domains in a unified way. For most of our applications, these are the first sample-optimal and nearly-linear time estimators in the literature. As a consequence, our work resolves the sample and computational complexities of a broad class of inference tasks via a single “meta-algorithm”. Moreover, we demonstrate that our algorithm performs very well in experiments.Our algorithm consists of three levels: (i) At the top level, we employ an iterative greedy algorithm for finding a good partition of the real line into the pieces of a piecewise polynomial. (ii) For each piece, we show that the sub-problem of finding a good polynomial fit on the current interval can be solved efficiently with a separation oracle method. (iii) We reduce the task of finding a separating hyperplane to a combinatorial problem and design a nearly-linear algorithm for this problem. Combining these three procedures gives a density estimation algorithm with the claimed guarantees.

Similar Papers
  • PDF
  • Research Article
  • Citations6

Deceptive Path Planning via Count-Based Reinforcement Learning under Specific Time Constraint

  • Jun 26, 2024
  • Mathematics
  • Dejun Chen +5
  • Research Article
  • Citations10

Solving parametric piecewise polynomial systems

  • May 11, 2011
  • Journal of Computational and Applied Mathematics
  • Yisheng Lai +2
  • Conference Article
  • Citations8

A Study on Piecewise Polynomial Smooth Approximation to the Plus Function

  • Jan 01, 2006
  • Linkai Luo +3
  • Book Chapter
  • Citations1

Finite Element Methods for Elliptic Equations

  • Nov 18, 2008
  • Research Article
  • Citations2

PPV-Based Modeling and Event-Driven Simulation of Injection-Locked Oscillators in SystemVerilog

  • Aug 01, 2015
  • IEEE Transactions on Circuits and Systems I: Regular Papers
  • Jieun Jang +1
  • Conference Article
  • Citations5

Piecewise polynomial approximation based on taylor series with efficient realization using Farrow structure

  • Oct 01, 2009
  • Djordje Babic
  • Conference Article
  • Citations1

E-spline analysis for de-noising and wavelet compression applications

  • Jul 01, 2013
  • G Fahmy +1
  • Research Article
  • Citations6

Existence, uniqueness, and convergence rates for gradient flows in the training of artificial neural networks with ReLU activation

  • Jan 01, 2023
  • Electronic Research Archive
  • Simon Eberle +3
  • Book Chapter
  • Citations2

Gathering in the Discrete Domain: State of the Art

  • Jul 06, 2018
  • Arun Sadhu +3
  • Research Article
  • Citations1

Direct and Inverse Theorems on Approximation by Piecewise Polynomial Functions

  • Jul 19, 2015
  • Journal of Mathematical Sciences
  • A S Kochurov
  • Research Article
  • Citations7

A multi-level discontinuous Galerkin method for solving the stationary Navier–Stokes equations

  • Aug 30, 2006
  • Nonlinear Analysis: Theory, Methods & Applications
  • Yinnian He +1
  • Book Chapter
  • Citations8

A Solving Procedure for Stochastic Satisfiability Modulo Theories with Continuous Domain

  • Jan 01, 2015
  • Yang Gao +1
  • Research Article
  • Citations17

On the degree of convergence of piecewise polynomial approximation on optimal meshes

  • Jan 01, 1977
  • Transactions of the American Mathematical Society
  • H G Burchard
  • Research Article
  • Citations2

Stationary distribution functions for ohmic Tokamak-plasmas in the weak-collisional transport regime by MaxEnt principle

  • Sep 10, 2014
  • Journal of Plasma Physics
  • Giorgio Sonnino +4
  • Book Chapter
  • Citations188

Remarks on Some Nonparametric Estimates of a Density Function

  • Jan 01, 2011
  • Richard A Davis +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.