Polynomial approximations of multivariate smooth functions from quasi-random data

In this paper, we introduce a method for multivariate function approximation using function evaluations, Chebyshev polynomials, and tensor-based compression techniques via the Tucker format. We develop novel randomized techniques to accomplish the tensor compression, provide a detailed analysis of the computational costs, provide insight into the error of the resulting approximations, and discuss the benefits of the proposed approaches. We also apply the tensor-based function approximation to develop low-rank matrix approximations to kernel matrices that describe pairwise interactions between two sets of points; the resulting low-rank approximations are efficient to compute and store (the complexity is linear in the number of points). We present an adaptive version of the function and kernel approximation that determines an approximation that satisfies a user-specified relative error over a set of random points. We extend our approach to the case where the kernel requires repeated evaluations for many values of (hyper)parameters that govern the kernel. We give detailed numerical experiments on example problems involving multivariate function approximation, low-rank matrix approximations of kernel matrices involving well-separated clusters of sources and target points, and a global low-rank approximation of kernel matrices with an application to Gaussian processes. We observe speedups up to 18X over standard matrix-based approaches.

Recently, smooth rank function was proposed for matrix completion problem. The main idea of this method was based on a continuous and differentiable approximation of the rank function. In this paper, a new approach for solving low-rank matrix recovery problem based on smooth function is proposed. It not only uses a smooth function to approximate the rank function, but also approximates the l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0</sub> -norm with a continuous and differentiable function. In addition, gradient decreasing approach can be used to solve the minimization problem. Finally, experimental results show that this new algorithm provides accurate results in almost all of our testing scenarios with a reasonable running time. Especially, it has higher approximation performance than other methods.

Direct theorems on the approximation of smooth functions by polynomials with integer coefficients are strengthened. First theorems on approximation by polynomials with natural coefficients are obtained.

ABSTRACTThis paper considers deep neural networks for learning weakly dependent processes in a general framework that includes, for instance, regression estimation, time series prediction, time series classification. The ‐weak dependence structure considered is quite large and covers other conditions such as mixing, association, and so on. Firstly, the approximation of smooth functions by deep neural networks with a broad class of activation functions is considered. We derive the required depth, width and sparsity of a deep neural network to approximate any Hölder smooth function, defined on any compact set . Secondly, we establish a bound of the excess risk for the learning of weakly dependent observations by deep neural networks. When the target function is sufficiently smooth, this bound is close to the usual .

On approximation of smooth functions from samples of partial derivatives with application to phase unwrapping

In this paper, we explore the concept of dimension preserving approximation of continuous multivariate functions defined on the domain [Formula: see text] (q-times) where [Formula: see text] is a natural number). We establish a few well-known multivariate constrained approximation results in terms of dimension preserving approximants. In particular, we indicate the construction of multivariate dimension preserving approximants using the concept of [Formula: see text]-fractal interpolation functions. We also prove the existence of one-sided approximation of multivariate function using fractal functions. Moreover, we provide an upper bound for the fractal dimension of the graph of the [Formula: see text]-fractal function. Further, we study the approximation aspects of [Formula: see text]-fractal functions and establish the existence of the Schauder basis consisting of multivariate fractal functions for the space of all real valued continuous functions defined on [Formula: see text] and prove the existence of multivariate fractal polynomials for the approximation.

Multivariate function approximation is a fundamental problem in machine learning. Classic multivariate function approximations rely on hand-crafted basis functions (e.g., polynomial basis and Fourier basis), which limits their approximation ability and data adaptation ability, resulting in unsatisfactory performance. To address these challenges, we introduce the neural basis function by leveraging an untrained neural network as the basis function. Equipped with the proposed neural basis function, we suggest the neural approximation (NeuApprox) paradigm for multivariate function approximation. Specifically, the underlying multivariate function behind the multi-dimensional data is decomposed into a sum of block terms. The clear physically-interpreted block term is the product of expressive neural basis functions and their corresponding learnable coefficients, which allows us to faithfully capture distinct components of the underlying data and also flexibly adapt to new data by readily fine-tuning the neural basis functions. Attributed to the elaborately designed block terms, the suggested NeuApprox enjoys strong approximation ability and flexible data adaptation ability over the hand-crafted basis function-based methods. We also theoretically prove that NeuApprox can approximate any multivariate continuous function to arbitrary accuracy. Extensive experiments on diverse multi-dimensional datasets (including multispectral images, light field data, videos, traffic data, and point cloud data) demonstrate the promising performance of NeuApprox in terms of both approximation capability and adaptability.

A convergent smoothing algorithm for training max –min fuzzy neural networks

The aim of the current paper is to propose an efficient method for finding the approximate solution of fractional delay differential equations. This technique is based on hybrid functions of block-pulse and fractional-order Fibonacci polynomials. First, we define fractional-order Fibonacci polynomials. Next, using Fibonacci polynomials of fractional-order, we introduce a new set of basis functions. These new functions are called fractional-order Fibonacci-hybrid functions (FFHFs) which are appropriate for the approximation of smooth and piecewise smooth functions. The Riemann–Liouville integral operational matrix and delay operational matrix of the FFHFs are obtained. Then, using these matrices and collocation method, the problem is reduced to a system of algebraic equations. Using Newton’s iterative method, we solve this system. Some examples are proposed to test the efficiency and effectiveness of the present method. Given the application of these kinds of fractional equations in the modeling of many phenomena, a numerical example of this work includes the Hutchinson model which describes the rate of population growth.

On the approximation of smooth functions using generalized digital nets

Let f be a function we wish to approximate on the interval [x 1 ,x N ] knowing p 1 > 1,p 2 , . . . ,p N coefficients of expansion of f at the points x 1 ,x 2 , . . . ,x N . We start by computing two neighboring N -point Padé approximants (NPAs) of f, namely f 1 = [m/n] and f 2 = [m − 1/n] of f. The second NPA is computed with the reduced amount of information by removing the last coefficient from the expansion of f at x 1 . We assume that f is sufficiently smooth, (e.g. convex-like function), and (this is essential) that f 1 and f 2 bound f in each interval]x i ,x i+1[ on the opposite sides (we call the existence of such two-sided approximants the two-sided estimates property of f ). Whether this is the case for a given function f is not necessarily known a priori, however, as illustrated by examples below it holds for many functions of practical interest. In this case, further steps become relatively simple. We select a known function s having the two-sided estimates property with values s(x i ) as close as possible to the values f(x i ). We than compute the approximants s 1 = [m/n] and s 2 = [m − 1/n] using the values at points x i and determine for all x the weight function α from the equation s = αs 1 + (1 − α)s 2 . Applying this weight to calculate the weighted mean αf 1 + (1 − α)f 2 we obtain significantly improved approximation of f.

Many applications of neural networks are based on their universal approximation property. For example, a common approach in the prediction of time series t 1,t 2,... is to consider each t n as an unknown fuction of a certain (fixed) number of previous values. A neural network is then trained to approximate this unknown function. We note that one of the reasons for the popularity of neural networks over their precursors, perceptrons, is their universal approximation property.KeywordsNeural NetworkHide LayerActivation FunctionTarget FunctionRadial Basis Function NetworkThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Quasi-random points keep their distance

The paper describes some results on nonlinear spline approximation of univariate and multivariate functions with singularities obtained by the author partly in collaboration with I.P. Irodova.

Universal approximation capability is an important research topic in artificial neural networks. The purpose of this study is to investigate universal approximation capability of a single hidden layer feed forward multiple 2π-periodic approximate identity neural networks in two function spaces. We present the notion of multiple 2π-periodic approximate identity. With respect to this notion, we prove two theorems in the space of continuous multivariate 2π-periodic functions. The second theorem shows that the above networks have universal approximation capability. The proof of the theorem uses a technique based on the notion of epsilon-net. Moreover, we discuss the universal approximation capability of the networks in the space of Lebesgue integrable multivariate 2π-periodic functions. The results of this study will be able to extend the standard theory of the universal approximation capability of feedforward neural networks.