Nonlinear Approximation Theory

This paper presents a novel approach to recognize images based on nonlinear wavelet approximation. The mathematical theory on nonlinear wavelet approximation is introduced, which shows that nonlinear approximation contains much more information of the original image than linear approximation. Based on this theory, a scheme to obtain the basic information of images with less data is provided. Experiments on face recognition produce effective matching rates.

Previous article Next article Uniform Generalized Weight Function Polynomial Approximation with InterpolationH. L. Loeb, D. G. Moursund, L. L. Schumaker, and G. D. TaylorH. L. Loeb, D. G. Moursund, L. L. Schumaker, and G. D. Taylorhttps://doi.org/10.1137/0706026PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] S. Paszkowski, On approximation with nodes, Rozprawy Mat., 14 (1957), 63– MR0092888 0083.29002 Google Scholar[2] Frank Deutsch, On uniform approximation with interpolatory constraints, J. Math. Anal. Appl., 24 (1968), 62–79 10.1016/0022-247X(68)90049-8 MR0230017 0174.35501 CrossrefISIGoogle Scholar[3] E. W. Cheney and , H. L. Loeb, Generalized rational approximation, J. Soc. Indust. Appl. Math. Ser. B Numer. Anal., 1 (1964), 11–25 MR0178288 0138.04403 LinkGoogle Scholar[4] Samuel Karlin and , William J. Studden, Tchebycheff systems: With applications in analysis and statistics, Pure and Applied Mathematics, Vol. XV, Interscience Publishers John Wiley & Sons, New York-London-Sydney, 1966xviii+586 MR0204922 0153.38902 Google Scholar[5] David G. Moursund, Chebyshev approximation using a generalized weight function, SIAM J. Numer. Anal., 3 (1966), 435–450 10.1137/0703038 MR0204936 0163.07302 LinkGoogle Scholar Previous article Next article FiguresRelatedReferencesCited ByDetails A general framework for the optimal approximation of circular arcs by parametric polynomial curvesJournal of Computational and Applied Mathematics, Vol. 345 | 1 Jan 2019 Cross Ref Strong unicity of order 2 in C(T)Journal of Approximation Theory, Vol. 56, No. 3 | 1 Mar 1989 Cross Ref The Best Interpolating Approximation is a Limit of Best Weighted ApproximationsCanadian Mathematical Bulletin, Vol. 25, No. 4 | 20 November 2018 Cross Ref The best interpolating approximation and weighted approximationJournal of Approximation Theory, Vol. 35, No. 1 | 1 Oct 1982 Cross Ref Remez algorithm for Chebyshev approximation with interpolationComputing, Vol. 28, No. 1 | 1 Mar 1982 Cross Ref Discrete chebyshev approximation with interpolationInternational Journal of Computer Mathematics, Vol. 11, No. 3-4 | 19 March 2007 Cross Ref Approximation with interpolatory constraintsJournal of Mathematical Analysis and Applications, Vol. 64, No. 2 | 1 Jun 1978 Cross Ref Approximation with constraints in normed linear spacesJournal of Approximation Theory, Vol. 21, No. 3 | 1 Nov 1977 Cross Ref An alternation theory for copositive approximationJournal of Approximation Theory, Vol. 19, No. 2 | 1 Feb 1977 Cross Ref The Remez exchange algorithm for approximation with linear restrictionsTransactions of the American Mathematical Society, Vol. 223, No. 0 | 1 January 1976 Cross Ref Best uniform approximation with Hermite-Birkhoff interpolatory side conditionsJournal of Approximation Theory, Vol. 15, No. 2 | 1 Oct 1975 Cross Ref Optimal rational starting approximationsJournal of Approximation Theory, Vol. 12, No. 2 | 1 Oct 1974 Cross Ref Optimal starting approximations for iterative schemesJournal of Approximation Theory, Vol. 9, No. 1 | 1 Sep 1973 Cross Ref Approximation with Convex ConstraintsJames T. LewisSIAM Review, Vol. 15, No. 1 | 18 July 2006AbstractPDF (2399 KB)A generalization of the varisolvency and unisolvency propertiesJournal of Approximation Theory, Vol. 7, No. 1 | 1 Jan 1973 Cross Ref Nonlinear Tchebycheff approximation with constraintsJournal of Approximation Theory, Vol. 6, No. 3 | 1 Oct 1972 Cross Ref A unified approach to uniform real approximation by polynomials with linear restrictionsTransactions of the American Mathematical Society, Vol. 166, No. 0 | 1 January 1972 Cross Ref Kolmogoroff's criterion for constrained rational approximationJournal of Approximation Theory, Vol. 4, No. 2 | 1 Jun 1971 Cross Ref Uniform rational approximation with osculatory interpolationJournal of Computer and System Sciences, Vol. 4, No. 6 | 1 Dec 1970 Cross Ref Approximation by functions having restricted ranges: Equality caseNumerische Mathematik, Vol. 14, No. 1 | 1 Dec 1969 Cross Ref Volume 6, Issue 2| 1969SIAM Journal on Numerical Analysis161-315 History Submitted:30 September 1968Published online:14 July 2006 InformationCopyright © 1969 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/0706026Article page range:pp. 284-293ISSN (print):0036-1429ISSN (online):1095-7170Publisher:Society for Industrial and Applied Mathematics

Geophysical Inverse Theory and Regularization Problems

This paper studies constrained optimization problems in Banach spaces without usual differentiability and convexity assumptions on the functionals involved in the data. The aim is to give optimality conditions for the problems from which one can derive the characterization of best approximations. The objective and the inequality constraint functionals are assumed to have one-sided directional derivatives. First-order necessary conditions are given in terms of subdifferentials of the directional derivatives. The notion of “max-pseudoconvexity” weaker than pseudoconvexity is introduced for sufficiency. The optimality conditions are applied to linear and nonlinear Tchebycheff approximation problems to derive the characterization of best approximations.

Many computational problems can be solved with the aid of contour integrals containing e z in the integrand: examples include inverse Laplace transforms, special functions, functions of matrices and operators, parabolic PDEs, and reaction-diffusion equations. One approach to the numerical quadrature of such integrals is to apply the trapezoid rule on a Hankel contour defined by a suitable change of variables. Optimal parameters for three classes of such contours have recently been derived: (a) parabolas, (b) hyperbolas, and (c) cotangent contours, following Talbot in 1979. The convergence rates for these optimized quadrature formulas are very fast: roughly O(3-N ), where N is the number of sample points or function evaluations. On the other hand, convergence at a rate apparently about twice as fast, O(9.28903-N ), can be achieved by using a different approach: best supremum-norm rational approximants to e z for z∈(–∞,0], following Cody, Meinardus and Varga in 1969. (All these rates are doubled in the case of self-adjoint operators or real integrands.) It is shown that the quadrature formulas can be interpreted as rational approximations and the rational approximations as quadrature formulas, and the strengths and weaknesses of the different approaches are discussed in the light of these connections. A MATLAB function is provided for computing Cody–Meinardus–Varga approximants by the method of Caratheodory–Fejer approximation.

Synthesis from examples enables non-expert users to generate programs by specifying examples of their behavior. A domain-specific form of such synthesis has been recently deployed in a widely used spreadsheet software product. In this paper we contribute to foundations of such techniques and present a complete algorithm for synthesis of a class of recursive functions defined by structural recursion over a given algebraic data type definition. The functions we consider map an algebraic data type to a string; they are useful for, e.g., pretty printing and serialization of programs and data. We formalize our problem as learning deterministic sequential top-down tree-to-string transducers with a single state (1STS). The first problem we consider is learning a tree-to-string transducer from any set of input/output examples provided by the user. We show that, given a set of input/output examples, checking whether there exists a 1STS consistent with these examples is NP-complete in general. In contrast, the problem can be solved in polynomial time under a (practically useful) closure condition that each subtree of a tree in the input/output example set is also part of the input/output examples. Because coming up with relevant input/output examples may be difficult for the user while creating hard constraint problems for the synthesizer, we also study a more automated active learning scenario in which the algorithm chooses the inputs for which the user provides the outputs. Our algorithm asks a worst-case linear number of queries as a function of the size of the algebraic data type definition to determine a unique transducer. To construct our algorithms we present two new results on formal languages. First, we define a class of word equations, called sequential word equations, for which we prove that satisfiability can be solved in deterministic polynomial time. This is in contrast to the general word equations for which the best known complexity upper bound is in linear space. Second, we close a long-standing open problem about the asymptotic size of test sets for context-free languages. A test set of a language of words L is a subset T of L such that any two word homomorphisms equivalent on T are also equivalent on L. We prove that it is possible to build test sets of cubic size for context-free languages, matching for the first time the lower bound found 20 years ago.

The following results are obtained for the metric of a sign sensitive weight: necessary and sufficient conditions imposed on the weight in order to ensure the fulfillment of the complete analogue of Jackson's theorem on the estimate of the best approximation of an arbitrary continuous function by means of its modulus of continuity; the analogue of Bernstein's inequality on the estimate of the derivative of a trigonometric polynomial; the analogue of Stechkin's theorem on the connection between the modulus of continuity of a function and the rate of its approximation by polynomials; the analogue of Dolzhenko's inequality on the estimate of the variation of a rational function; and the analogue of Dolzhenko's theorem on the estimate of the variation of a function by means of its best rational approximation.

Optimal approximation of elliptic problems by linear and nonlinear mappings II

This paper is concerned with the data-driven construction of fuzzy systems from the viewpoint of regularization and approximation theory, where we consider the important subclass of Sugeno controllers. Generally, we obtain a nonlinear constrained least squares approximation problem which is ill-posed. Therefore, nonlinear regularization theory has to be employed. We analyze a smoothing method, which is common in spline approximation, as well as Tikhonov regularization, along with rules how to choose the regularization parameters based on nonlinear regularization theory considering the error due to noisy data.For solving the regularized nonlinear least squares problem, we use a generalized Gauss-Newton like method. A typical numerical example shows that the regularized problem is not only more robust, but also favors solutions that are easily interpretable, which is an important quality criterion for fuzzy systems.

The identification of nonlinear systems can be posed as a nonlinear functional approximation problem. From the Weierstrass Theorem (Powell, 1981) and the Kolmogorov theorem (Sprecher, 1965) in approximation theory, it is shown that the polynomial and many other approximation schemes can approximate a continuous function arbitrarily well. In recent years, a number of nonlinear system identification approaches, particularly identification using neural networks, based on the universal approximation theorem (Cybenko, 1989), are applications of a similar mathematical approach.

A set of discrete data (xk, f (xk)) (k = 1, 2, …, m) may be fitted in any lp norm by a nonlinear form derived from a function g (L) of a linear form L = L(x). Such a nonlinear approximation problem may under appropriate conditions be (asymptotically) replaced by the fitting of g–1 (f) by L in any lp norm with respect to a weight function w = g′ (g–1 (f)). In practice this “direct method” can yield very good results, sometimes coming close to a best approximation. However, to ensure a near‐best approximation, by using an iterative procedure based on fitting L, two algorithms are proposed in the l2 norm ‐ one already established by Mason and Upton (1989) and one completely new, based on minimising the two algorithms and multiplicative combinations of errors, respectively. For a general g we prove they converge locally and linearly with small constants. Moreover it is established that they converge to different (nonlinear) “Galerkin type” approximations, the first based on making the explicit error ϵ ≡ f – g (L) orthogonal to a set of functions forming a basis for L, and the second based on making the implicit error ϵ* ≡ w(g–1 (f) – L) orthogonal to such a basis. Finally, and mainly for comparison purposes, the well known Gauss‐Newton algorithm is adopted for the determination of a best (nonlinear) approximation. Illustrative problems are tackled and numerical results show how effective all of the algorithms can be. To add a further novel feature, L is here chosen throughout to be a radial basis function (RBF), and, as far as we are aware, this is one of the first successful uses of a (nonlinear) function of an RBF as an approximation form in data fitting. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)

I.4 - Rational Approximation

Over the last decade the use of numerical techniques for the solution of the problems of physics, engineering, chemistry, biology and the social sciences has increased by leaps and bounds, and it was felt that the time was ripe for holding a Discussion Meeting on some topic in numerical analysis. This was intended not merely to provide an opportunity for experts in the field to get together, since there are many specialized meetings in numerical analysis these days. The aim was rather to give scientists in general who are interested in numerical methods a chance to find out what is being done, so that they can make greater use of this work and hopefully influence its future development. After some deliberation I decided on partial differential equations as the topic, in spite of the fact that it is not an area in which I have made any direct contribution in recent years. This is because I believe it to be one of the most important and challenging fields; indeed the solution of systems of p. d. es lies at the very heart of the problems of applied mathematics. Long after we have the more basic fields of linear and nonlinear algebra and approximation theory in good order the problems arising in the solution of p. d. es will still be with us. The work that has been done in numerical analysis may then appear as a preliminary sharpening up of the tools we are to use.

This paper presents a rational approximation method for fractional calculus operators <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mo>±</mml:mo><mml:mi mathvariant="normal">α</mml:mi></mml:mrow></mml:msup></mml:math>in the given frequency range and the error, which is based on the best rational approximation definition. The fractional integral operator is selected as an example to describe the construction of the rational approximation functions. An application case (<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msup><mml:mrow><mml:mi>s</mml:mi></mml:mrow><mml:mrow><mml:mn>0.3</mml:mn></mml:mrow></mml:msup></mml:math>) is used to illustrate the effectiveness of the proposed method. The obtained approximation function in the frequency domain is a best rational approximation function, which can further improve the accuracy of the approximation without increasing the orders. On the basis of the presented rational approximation method, a rational approximation equation of a fractional-order PID controller is obtained. Finally, the method for analyzing the optimization and frequency characteristics of the fractional-order controller is implemented to demonstrate the good frequency characteristic and best structure. The results from theoretical analysis and experimental verification show that the proposed method provides a new design idea for the effective application of the fractional-order PID controller in engineering.

A simplified nonlinear transient analysis method for gas bearings was recently published by the authors (Hassini, M. A., and Arghir, M., 2012, “Simplified Nonlinear Transient Analysis Method for Gas Bearings,” J. Tribol., 134(1), 011704). The method uses the fact that linearized dynamic characteristics of gas bearings, namely the impedances, can be approximated by rational transfer functions. The method gave good results if the rational transfer function approach approximated the linearized dynamic characteristics well. Indeed, each of the four complex impedances Zαβ,α,β={x,y} had one or two poles depending on the order of the rational function that were used. These poles appear as supplementary eigenvalues of the extended matrix of the homogeneous system of first order differential equations describing the model of the rotor. They govern the stability of the dynamic model in the same way as the original eigenvalues do and therefore they impose non-negligible constraints on the rational function approximation of the impedances of gas bearings. The present improvement of the method overrides this problem. The basic idea is to impose the same set of poles for Zxx, Zxy, Zyx, and Zyy. By imposing this constraint, the poles are stable and the introduction of artificial instability or erratic eigenvalues is avoided. Campbell and stability diagrams naturally taking into account the variation of the dynamic coefficients with the excitation frequency can now be easily plotted. For example, the method is used for analyzing the stability of rigid and flexible rotors supported by two identical gas bearings modeled with second order rational transfer functions. The method can be applied to any bearing or seal whose impedance is approximated by rational transfer functions.