Numerical Accuracy Improvement by Interprocedural Program Transformation

This chapter contains sections titled: Introduction Round-Off Error Propagation Induced by FP Computation The CESTAC Method, a Stochastic Approach for Analyzing Round-Off Error Propagation Stochastic Arithmetic Validation and Implementation of the CESTAC Method Discrete Stochastic Arithmetic (DSA) Taking into Account Data Errors The CADNA Library [36] The Use of the CADNA Library Can the CADNA Library Fail? Conclusion References

Whitebox fuzzing is a novel form of security testing based on dynamic symbolic execution and constraint solving. Over the last couple of years, whitebox fuzzers have found many new security vulnerabilities (buffer overflows) in Windows and Linux applications, including codecs, image viewers and media players. Those types of applications tend to use floating-point instructions available on modern processors, yet existing whitebox fuzzers and SMT constraint solvers do not handle floating-point arithmetic. Are there new security vulnerabilities lurking in floating-point code?A naive solution would be to extend symbolic execution to floating-point (FP) instructions (months of work), extend SMT solvers to reason about FP constraints (months of work or more), and then face more complex constraints and an even worse path explosion problem. Instead, we propose an alternative approach, based on the rough intuition that FP code should only perform memory safe data-processing of the payload of an image or video file, while the non-FP part of the application should deal with buffer allocations and memory address computations, with only the latter being prone to buffer overflows and other security critical bugs. Our approach combines (1) a lightweight local path-insensitive may static analysis of FP instructions with (2) a high-precision whole-program path-sensitive must dynamic analysis of non-FP instructions. The aim of this combination is to prove memory safety of the FP part of each execution and a form of non-interference between the FP part and the non-FP part with respect to memory address computations.We have implemented our approach using two existing tools for, respectively, static and dynamic x86 binary analysis. We present preliminary results of experiments with standard JPEG, GIF and ANI Windows parsers. For a given test suite of diverse input files, our mixed static/dynamic analysis is able to prove memory safety of FP code in those parsers for a small upfront static analysis cost and a marginal runtime expense compared to regular dynamic symbolic execution.

We carried out a theoretical investigation of the impact of the numerical errors caused by using floating point numbers (FPNs) in simulations, such as rounding errors. Under the presupposition that model variables can be written as the linear sum of the true value and the numerical error, equations governing the time evolution of numerical errors due to FPNs (FPN errors) are obtained by considering the total errors of the results of simulations of shallow‐water models and estimating the errors incurred by using FPNs with varying precision. We can use the time evolution equations to estimate the behavior of the FPN errors, then confirm these estimations by carrying out numerical simulations. In a geostrophic wind balance state, the FPN error oscillates and gradually increases in proportion to the square root of the number of time steps, like a random walk. We found that the error introduced by using FPNs can be considered as stochastic forcing. In a state of barotropic instability, the FPN error initially evolves as stochastic forcing, as in the case of the geostrophic wind balance state. However, it then begins to increase exponentially, like a barotropic instability wave. These numerical results are obtained by using a staggered‐grid arrangement and stable time‐integration method to retain near‐neutral numerical stability in the simulations. The FPN error tends to behave as theoretically predicted if the numerical stability is close to neutral.

This article introduces a new program transformation in order to enhance the numerical accuracy of floating-point computations. We consider that a program would return an exact result if the computations were carried out using real numbers. In practice, roundoff errors due to the finite representation of values arise during the execution. These errors are closely related to the way formulas are evaluated. Indeed, mathematically equivalent formulas, obtained using laws like associativity, distributivity, etc., may lead to very different numerical results in the computer arithmetic. We propose a semantics-based transformation in order to optimize the numerical accuracy of programs. This transformation is expressed in the abstract interpretation framework and it aims at rewriting pieces of numerical codes in order to obtain results closer to what the computer would output if it used the exact arithmetic.

International audience

Decreasing feature sizes allow additional functionality to be added to future microprocessors to improve the performance of important application domains. As a result of rapid growth in financial, commercial, and Internet-based applications, hardware support for decimal floating-point arithmetic is now being considered by various computer manufacturers and specifications for decimal floating-point arithmetic have been added to the draft revision of the IEEE-754 Standard for Floating-Point Arithmetic (IEEE-754R). This work presents an efficient arithmetic algorithm and hardware design for decimal floating-point division. The design uses an optimized piecewise linear approximation, a modified Newton-Raphson iteration, a specialized rounding technique, and a simplified combined decimal incrementer/decrementer. Synthesis results show that a 64-bit (16-digit) implementation of the decimal divider, which is compliant with IEEE-754R, has an estimated critical path delay of 0.69 ns when implemented using LSI Logic's 0.11 micron gflx-p standard cell library.

For scientific computations on a digital computer the set of real numbers is usually approximated by a finite set F of “floating-point numbers”. We compare the numerical accuracy possible with different choices of F having approximately the same range and requiring the same wordlength. In particular, we compare different choices of base (or radix) with the usual floating-point systems. The emphasis is on the choice of F, not on the details of the number representation or the arithmetic, but both rounded and truncated arithmetic are considered. Theoretical results are given, and some simulations of typical floating-point computations (forming sums, solving systems of linear equations, finding eigenvalues) are described. If the leading fraction bit of a normalized base-2 number is not stored explicitly (saving a bit), and the criterion is to minimize the mean square roundoff error, then base 2 is best. If unnormalized numbers are allowed, so the first bit must be stored explicitly, then base 4 (or sometimes base 8) is the best of the usual systems.

Many numerical problems require a higher computing precision than the one offered by standard floating-point (FP) formats. One common way of extending the precision is to represent numbers in a <i>multiple component</i> format. By using the so-called <i>floating-point expansions</i> , real numbers are represented as the unevaluated sum of standard machine precision FP numbers. This representation offers the simplicity of using directly available, hardware implemented and highly optimized, FP operations. It is used by multiple-precision libraries such as Bailey's QD or the analogue Graphics Processing Units (GPU) tuned version, GQD. In this article we briefly revisit algorithms for adding and multiplying FP expansions, then we introduce and prove new algorithms for normalizing, dividing and square rooting of FP expansions. The new method used for computing the reciprocal <inline-formula><tex-math notation="LaTeX">${a}^{-1}$</tex-math></inline-formula> and the square root <inline-formula><tex-math notation="LaTeX">$\sqrt{a}$</tex-math></inline-formula> of a FP expansion <inline-formula><tex-math notation="LaTeX">$a$</tex-math></inline-formula> is based on an adapted Newton-Raphson iteration where the intermediate calculations are done using “truncated” operations (additions, multiplications) involving FP expansions. We give here a thorough error analysis showing that it allows very accurate computations. More precisely, after <inline-formula><tex-math notation="LaTeX">$q$</tex-math> </inline-formula> iterations, the computed FP expansion <inline-formula><tex-math notation="LaTeX">$x=x_0+\ldots +x_{2^q-1}$</tex-math> </inline-formula> satisfies, for the reciprocal algorithm, the relative error bound: <inline-formula><tex-math notation="LaTeX"> $\left|({x-a^{-1}})/{a^{-1}}\right| \le 2^{-2^q(p-3)-1}$</tex-math></inline-formula> and, respectively, for the square root one: <inline-formula><tex-math notation="LaTeX"> $\left|x-{1}/{\sqrt{a}}\right| \le {2^{-2^q(p-3)-1}}/{\sqrt{a}}$</tex-math></inline-formula> , where <inline-formula> <tex-math notation="LaTeX">$p&gt; 2$</tex-math></inline-formula> is the precision of the FP representation used ( <inline-formula><tex-math notation="LaTeX">$p=24$</tex-math></inline-formula> for single precision and <inline-formula><tex-math notation="LaTeX">$p=53$</tex-math></inline-formula> for double precision).

Most of today's numerical computations are performed using floating-point data operations for representing real numbers. The precision of the related data types should be adapted in order to guarantee a desired overall rounding error and to strengthen the performance of programs. This adaptation lies to the precision tuning technique, which consists of finding the least floating-point formats enabling a program to compute some results with an accuracy requirement. Precision tuning is important for several applications from a wide variety of domains including embedded systems, Internet of Things (IoT), etc. In this work, we evaluate the performance of POP, our static analysis tool, in several manners. First, we compare two ways of optimizing programs in POP throughout the definition of different cost functions given to the solver. Second, we measure the runtime errors between the exact results given by an execution in multiple precision and the results of tuned programs by POP and show that the measured error is always less than the accuracy requirements given by the user for several examples. Third, we provide a detailed comparison of POP versus the prior state-of-the-art tool, Precimonious.

Field Programmable Gate Arrays (FPGAs) are commonly used to accelerate floating-point (FP) applications. Although researchers have extensively studied FPGA FP implementations, existing work has largely focused on standalone operators and frequency-optimized designs. These works are not suitable for FPGA soft processors which are more sensitive to latency, impose a lower frequency ceiling, and require IEEE FP standard compliance. We present an open-source floating-point unit (FPU) for FPGA RISC-V soft processors that is fully IEEE compliant with configurable levels of FP precision. Our design emphasizes runtime performance with 25% lower latency in the most common instructions compared to previous works while maintaining efficient resource utilization. Our FPU also allows users to explore various mantissa widths without having to rewrite or recompile their algorithms. We use this to investigate the scalability of our reduced-precision FPU across numerous microbenchmark functions as well as more complex case studies. Our experiments show that applications like the discrete cosine transformation and the Black-Scholes model can realize a speedup of more than 1.35x in conjunction with a 43% and 35% reduction in lookup table and flip-flop resources while experiencing less than a 0.025% average loss in numerical accuracy with a 16-bit mantissa width.

Airborne critical systems that use real numbers in their algorithms can be considerably affected by the floating-point discretized representation. This limitation in the representation is the cause of many problems already known while running software with floating-point variables such as loss of precision in arithmetic operations. This work presents a method for detecting floating-point absorption and cancellation phenomena using static analysis in software design models. Our method uses a range analysis tool that operates over SCADE model to propagate the range of the internal variables and, then, each addition and subtraction operation in the model is analyzed to check the occurrence of the floating-point absorption and cancellation phenomena. The results are presented in an HTML report containing absorption alerts and cancellations charts providing the number of canceled bits for each sub-range in order to permit better assessment of the impact of each detected cancellation. In order to validate the proposed method, we compare the results of our proposed method with the results obtained by an alternative validation method developed using the source code as input and the Frama-C EVA to propagate internal ranges. The method can be used as an early design analysis enabling the cheaper detection and treatment of floating-point anomalies.

Robust numerical calculation of tangent moduli at finite strains based on complex-step derivative approximation and its application to localization analysis

Numerical Validation of Compensated Summation Algorithms with Stochastic Arithmetic

This paper investigates the aspect-ratio locking of the isoparametric 8-node quadrilateral (QUAD8) element. An important finding is that, if finite element solution is carried out with in exact arithmetic (i.e., with no truncation and round off errors), the locking tendency of the element is completely avoided even for aspect-ratios as high as 100000. The current finite element codes mostly use floating point arithmetic. Thus, they can only avoid this locking for aspect-ratios up to 100 or 1000. A novel method is proposed in the paper to avoid aspect-ratio locking in floating point computations. In this method, the offending terms of the strain-displacement matrix (i.e., <TEX>$\mathbf{B}$</TEX>-matrix) are multiplied by suitable scaling factors to avoid ill-conditioning of stiffness matrix. Numerical examples are presented to demonstrate the efficacy of the method. The examples reveal that aspect-ratio locking is avoided even for aspect-ratios as high as 100000.

Ordinary differential equations are ubiquitous in scientific computing. Solving exactly these equations is usually not possible, except for special cases, hence the use of numerical schemes to get a discretized solution. We are interested in such numerical integration methods, for instance Euler's method or the Runge-Kutta methods. As they are implemented using floating-point arithmetic, round-off errors occur. In order to guarantee their accuracy, we aim at providing bounds on the round-off errors of explicit one-step numerical integration methods. Our methodology is to apply a fine-grained analysis to these numerical algorithms. Our originality is that our floating-point analysis takes advantage of the linear stability of the scheme, a mathematical property that vouches the scheme is well-behaved.