An Efficient Floating-Point Bit-Blasting API for Verifying C Programs
AAn Efficient Floating-Point Bit-Blasting API forVerifying C Programs
Mikhail R. Gadelha , Lucas C. Cordeiro , and Denis A. Nicole SIDIA Instituto de Ciˆencia e Tecnologia, Manaus, Brazil [email protected] University of Manchester, Manchester, UK [email protected] University of Southampton, UK, [email protected]
Abstract.
We describe a new SMT bit-blasting API for floating-pointsand evaluate it using different out-of-the-shelf SMT solvers during theverification of several C programs. The new floating-point API is part ofthe SMT backend in ESBMC, a state-of-the-art bounded model checkerfor C and C++. For the evaluation, we compared our floating-pointAPI against the native floating-point APIs in Z3 and MathSAT. Weshow that Boolector, when using floating-point API, outperforms thesolvers with native support for floating-points, correctly verifying moreprograms in less time. Experimental results also show that our floating-point API implemented in ESBMC is on par with other state-of-the-artsoftware verifiers. Furthermore, when verifying programs with floating-point arithmetic, our new floating-point API produced no wrong answers.
Keywords:
Floating-Point Arithmetic · Satisfiability Modulo Theories · Software Verification.
Software verification tools operate by converting their input (e.g., a programsource code) into a format understandable by an automated theorem prover, en-coding high-level program properties (e.g., arithmetic overflow) and algorithms(e.g., bounded model checking) into low-level equations (e.g., SMT). The encod-ing process of a program usually involves several intermediate steps, designedto generate a formula that can be efficiently solved by the theorem provers.In this domain, the analysis of programs with floating-point arithmetic has re-ceived much attention, primarily when safety depends on the correctness of theseprograms. In essence, the Ariane 5 rocket exploded mid-air in 1996 due to anexception thrown by an invalid floating-point conversion [42]. It is a complexproblem because the semantics may change beyond code level, including theoptimization performed by compilers [46].There exist various static analysis tools that are able to check for floating-point computations [9,10,21,22,30,52,56]. For example, Astr´ee is a static analysis a r X i v : . [ c s . L O ] A p r M. Gadelha et al. tool that considers all possible rounding errors when verifying C programs withfloating-point numbers [9]. It has been applied to verify embedded software in theflight control software of the Airbus. CBMC [21] is also another notable exampleof a software model checking tool, which implements a bit-precise decision proce-dure for the theory of floating-point arithmetic [11]. It has been applied to verifyindustrial applications from the automotive industry, which rely on floating-point reasoning [55]. CBMC is also the main verification engine employed byother software verifiers that efficiently verify C programs with floating-pointnumbers such as PeSCo [53] and VeriAbs [19]. It is a challenging verificationtask to prove the correctness of C programs with floating-points mainly becauseof 32/64 bits floating-point computations. Given the current knowledge in soft-ware verification, there exists no other study that shows a thorough comparativeevaluation of software verifiers and SMT solvers concerning the verification of Cprograms that contain floating-points.Here we present the new floating-point technologies developed in one boundedmodel checker, ESBMC [31], and evaluate it using a large set of floating-pointbenchmarks [7]. In particular, we evaluate a new floating-point API on top of ourSMT backend that extends the floating-point feature to all solvers supported byESBMC (including Boolector [14] and Yices [26] that currently do not supportthe SMT FP logic [13]). For evaluation, we used the benchmarks of the 2020International Competition on Software Verification (SV-COMP) [7], from thefloating-point sub-category. The five different solvers supported by ESBMC wereevaluated (Z3 [25], Yices [26], Boolector [14], MathSAT [20], and CVC4 [4]) andESBMC is able to evaluate more benchmarks within the usual time and memorylimits (15 minutes and 15GB, respectively) when using Boolector. In particular,results show that Boolector can solve more floating-point problems using the newfloating-point API than MathSAT or Z3, which have native floating-point APIs.Our experimental results also show that our floating-point API implementedin ESBMC is competitive to other state-of-the-art software verifiers, includingCBMC [21], PeSCo [53], and VeriAbs [19]. The manipulation of real values in programs is a necessity in many fields, e.g.,scientific programming [46]. The set of real numbers, however, is infinite, andsome numbers cannot be represented with finite precision, e.g., irrational num-bers. Over the years, computer manufacturers have experimented with differentmachine representations for real numbers [34]. The two fundamental ways to en-code a real number are the fixed-point representation, usually found in embeddedmicroprocessors and microcontrollers [28], and the floating-point representation,in particular, the IEEE floating-point standard (IEEE 754-2008 [36]), which hasbeen formally accepted by many processors [35].Each encoding can represent a range of real numbers depending on the word-length and how the bits are distributed. A fixed-point representation of a numberconsists of an integer component, a fractional component, and a bit for the sign. n Efficient Floating-Point Bit-Blasting API for Verifying C Programs 3
In contrast, the floating-point representation consists of an exponent component,a significand component, and a bit for the sign. Floating-point has a higherdynamic range than fixed-point (e.g., a float in C has 24 bits of precision,but can have values up to 2 ), while fixed-point can have higher precisionthan floating-point [49]. Furthermore, the IEEE floating-point standard definesvalues that have no equivalent in a fixed-point or real encoding, e.g., positive andnegative infinities. In general, IEEE floating-points are of the following kinds: zeroes , NaNs , infinities , normal , denormal (or subnormal) [36]. Definition 1 (Infinities)
Both +inf and -inf are defined in the standard.These floating-points represent overflows or the result of non-zero floating-pointdivisions by zero (Annex F of the C language specification [38]).
Definition 2 (Zeroes)
Both +0 and -0 are defined in the standard. Most ofthe operations will behave identically when presented with +0 or -0 except whenextracting the sign bit or dividing by zero (usual rules about signedness applyand will result in either +inf or -inf ). Equalities will even be evaluated to truewhen comparing positive against negative zeroes. Definition 3 (NaNs)
The N ot a N umber special values represent undefined orunrepresentable values, e.g., √− or . As a safety measure, most of theoperations will return NaN if at least one operator is NaN, as a way to indicatethat the computation is invalid. NaNs are not comparable: except for the not equaloperator ( != ), all other comparisons will evaluate to false (even comparing a NaNagainst itself ). Furthermore, casting NaNs to integers is undefined behavior. Definition 4 (Normal)
A non-zero floating-point that can be represented withinthe range supported by the encoding.
Definition 5 (Denormal (or subnormal))
A non-zero floating-point repre-senting values very close to zero, filling the gap between what can be usuallyrepresented by the encoding and zero.
The IEEE standard also defines five kinds of exceptions, to be raised un-der specific conditions, which are: invalid operation , overflow , division by zero , underflow , and inexact [36]. Exception 1 (Invalid Operation)
This exception is raised when the operationproduces a NaN as a result.
Exception 2 (Overflow)
This exception is raised when the result of an oper-ation is too large to be represented by the encoding. By default, these operationsreturn ± inf . Exception 3 (Division By Zero)
It is raised by x/ ± , for x (cid:54) = . By default,these operations return ± inf . Exception 4 (Underflow)
Raised when the result is too small to be repre-sented by the encoding. The result is a denormal floating-point.
M. Gadelha et al.
Exception 5 (Inexact)
This exception is raised when the encoding cannot rep-resent the result of an operation unless it is rounded. By default, these operationswill round the result.
The standard defines five rounding modes. Given a real number x , a roundedfloating-point r ( x ) will be rounded using: Round Toward Positive (RTP) , RoundToward Negative (RTN) , Round Toward Zero (RTZ) , Round to Nearest ties toEven (RNE) , and
Round to Nearest ties Away from zero (RNA) : Mode 1 (RTP) r ( x ) is the least floating-point value ≥ x . Mode 2 (RTN) r ( x ) is the greatest floating-point value ≤ x . Mode 3 (RTZ) r ( x ) is the floating-point with the same sign of x , such that | r ( x ) | is the greatest floating-point value ≤ | x | . Mode 4 (RNE) r ( x ) is the floating-point value closest to x ; if two floating-point values are equidistant to x , r ( x ) is the one which the least significant bitis zero. Mode 5 (RNA) r ( x ) is the floating-point value closest to x ; if two floating-point values are equidistant to x , r ( x ) is the one further away from zero. The standard also defines some arithmetic operations (add, subtract, mul-tiply, divide, square root, fused multiply-add, remainder), conversions (betweenformats, to and from strings), and comparisons and total ordering. In particular,the standard defines how floating-point operations are to be encoded using bit-vectors. Table 1 shows four primitive types usually available in the x86 family ofprocessors that follow the standard; each type is divided into three parts: one bitfor the sign, an exponent, and a significant part which depends on the bit lengthof the type. The significands also include a hidden bit: a 1 bit that is assumedto be the leading part of the significand, unless the floating-point is denormal.
Name Common Name Size(exponent + significand) fp16 Half precision 16 (5 + 10)fp32 Single precision 32 (8 + 23)fp64 Double precision 64 (11 + 53)fp128 Quadruple precision 128 (15 + 113)
Table 1: IEEE floating-point types.In Annex F of the C language specification [38], fp32 and fp64 are defined as float and double . The standard does not define any types for fp16, and com-pilers usually implement two formats: fp16 as defined in the ARM C languageextension (ACLE) [2] and
Float16 as defined by the ISO/IEC 18661-3:2015 n Efficient Floating-Point Bit-Blasting API for Verifying C Programs 5 standard [39]. While fp16 is only a storage and interchange format (meaningthat it is promoted when used in arithmetic operations),
Float16 is an actualtype, and arithmetic operations are performed using half-precision. The standardonly weakly specifies how an fp128 ( long double in C) should be implemented,and compilers usually implement it using an 80-bit long double extended preci-sion format [35].Floating-points are represented as ( − sign × signif icand × exponent . Here,1 ≤ signif icand ≤ exponent is the scaling factor [51]. Regular floating-points are encoded assuming that the leading hidden bit is 1 and the expo-nent is in the range [ − exponent max + 1 , exponent max ], e.g., the number 0.125is represented as (cid:104) (cid:105) in the floating-point format. Denormalsare encoded assuming that the leading hidden bit is zero and the exponent is − exponent max . Zeros are represented as an all-zero bit-vector (except for thesign bit if the zero is negative). Finally, a bit-vector with the exponent equal to exponent max and significand all zero is an infinity. In contrast, a bit-vector withan exponent equal to exponent max and significand not zero is a NaN. When ESBMC was created, all floating-point types and operations were encodedusing fixed-points [1,5,6,18,37]. A fixed-point number is represented in ESBMCas a pair ( m, n ) where m is the total number of bits and n ≤ m is the number offractional bits, e.g., the number 0.125 is represented as (cid:104) . (cid:105) (assuming itis 8 bits long) in the fixed-point format. The fixed-point arithmetic is performedsimilarly to the bit-vector arithmetic, except that the operations are applied sep-arately to the integral and fractional parts of the fixed-points and concatenatedat the end (overflow in the fractional parts are treated accordingly). Differentfrom floating-points, all bit-vectors represent one number in the real domain.The lack of proper floating-point encoding, however, meant that ESBMCwas unable to accurately verify an entire class of programs, such as the famousfloating-point “issue” [57] illustrated in Figure 1.The assertion in line 7 holds if the program is encoded using fixed-pointarithmetic, but fails if floating-point arithmetic is used. The assertion violationarises from the fact that floating-points in the IEEE standard are represented aswhole numbers × a power of two, so the only numbers that use a prime factorof the base two that can be correctly expressed as fractions. Since in binary (orbase 2) the only prime factor is 2, only , , , . . . would be correctly expressedas decimals, so the constants 0 .
1, 0 .
2, 0 . , , ) in the program are onlyapproximations. In the program in Figure 1, the constants are: – x is – y is – w is – z is M. Gadelha et al. int main ( ) { double x = 0 . 1 ; double y = 0 . 2 ; double w = 0 . 3 ; double z = x + y ; assert (w == z ) ; return } Fig. 1: The assertions in line 7 does not hold when using floating-point arithmetic.The discrepancy happens in the C program because the closest floatint-pointto is smaller than the real but the closest floating-point to is greaterthan the real , so adding the floating-points and results in a floating-point slightly greater than the constant floating-point .To address this limitation, ESBMC was extended to support floating-pointarithmetic [32] but was only able to encode it using SMT solvers that offerednative support for the floating-point theory, i.e., Z3 and MathSAT. A floating-point is represented in ESBMC following the IEEE-754 standard for the size ofthe exponent and significand precision. For example, a half-precision floating-point (16 bits) has 1 bit for the sign, 5 bits for the exponent, and 11 bits for thesignificand (1 hidden bit) [36].The work described in this paper, namely a new floating-point API in ourSMT backend, is the natural evolution of our research: the support of floating-point arithmetic for the remaining SMT solvers in ESBMC (Boolector [48],Yices [26], and CVC4 [4]). The new floating-point API works by converting allfloating-point types and operations to bit-vectors (a process called bit-blasting),thus extending the support for floating-point arithmetic to any solver that sup-ports bit-vector arithmetic [33].
The SMT FP logic is an addition to the SMT standard, first proposed in 2010by R¨ummer and Wahl [54]. The current version of the theory largely follows theIEEE standard 754-2008 [36]. It formalizes floating-point arithmetic, positiveand negative infinities and zeroes, NaNs, relational and arithmetic operators,and five rounding modes: round nearest with ties choosing the even value, roundnearest with ties choosing away from zero, round towards positive infinity, roundtowards negative infinity and round towards zero.There exist some functionalities from the IEEE standard that are not yet sup-ported by the FP logic as described by Brain et al. [13]; however, when encodingC programs using the FP logic, most of the process is a one-to-one conversion,as we described in our previous work [32]. n Efficient Floating-Point Bit-Blasting API for Verifying C Programs 7 Encoding programs using the SMT floating-point theory has several advan-tages over a fixed-point encoding. However, the main one is the correct modelingof ANSI-C/C++ programs that use the IEEE floating-point arithmetic. ESBMCships with models for most of the current C11 standard functions [38]; floating-point exception handling, however, is not yet supported.The encoding algorithms, however, can be very complex, and it is not uncom-mon to see the SMT solvers struggling to support every corner case [27,50]. Cur-rently, various SMT solvers support the SMT floating-point theory, e.g., Z3 [25],MathSAT [20], CVC4 [4], Colibri [44], Solonar [41], and UppSAT [58]; ESBMCimplements the floating-point encoding for all of them using their native API.Regarding the support from the solvers, Z3 implements all operators, Math-SAT implements all but two: fp.rem (remainder operator) and fp.fma (fusedmultiply-add) and CVC4 implements all but the conversions to other sorts.The three solvers offer two (non-standard) functions to reinterpret floating-points to and from bit-vectors: fp as ieeebv and fp from ieeebv , respectively.These functions can be used to circumvent any lack of operators, and only requirethe user to write the missing operators. Note that this is different from convertingfloating-points to bit-vectors and vice-versa: converting to bit-vectors follows therounding modes defined by the IEEE-754 standard while reinterpreting floating-point as bit-vectors returns the bit-vector format of the floating-point. We usethese functions in our backend to implement the fused multiply-add operator forMathSAT.The implementation of the floating-point API is based on the encoding ofMuller et al. [47], however, before we can discuss the algorithms in the floating-point API, we first need to describe the basic operations performed by most ofthem, the four-stage pipeline [12]: unpack, operate, round, and pack.1.
Unpack stage : the floating-point is split into three bit-vectors, one for thesign, one for the exponent, and one for the significand. In our floating-pointAPI, the unpack operation also adds the hidden bit to the significand, unbiasthe exponent. It offers an option to normalize subnormals exponents andsignificands if requested.2.
Operate stage : in this stage, conversion and arithmetic operations are per-formed in the three bit-vectors. Depending on the operation, the bit-vectorsneed to be extended, e.g., during a fused multiply-add operation, the signif-icand has length , and the exponent has length eb + 2 .3.
Round stage : since the previous stage was performed using extended bit-vectors, this stage needs to round the bit-vectors back to the nearest rep-resentable floating-point of the target format. Here, guard and sticky bitsin the significand are used to determine how far the bit-vector is from thenearest representable, and the rounding mode is used to determine in whichdirection the floating-point will be rounded. The exponent bit-vector is alsochecked for under- or overflow when rounding, to create the correct floating-point, e.g., infinity might be created if the exponent is too large for the targetformat.
M. Gadelha et al. Pack stage : in the final stage, the three bit-vectors are concatenated to formthe final floating-point.The four-stage pipeline will be used when performing operations with thefloating-points. We grouped the operations into seven groups: sorts construc-tors, rounding modes constructors, value constructors, classification operators,comparison operators, conversion operators, and arithmetic operators.In the three constructors groups (sorts, rounding modes, and value), thefloating-points are encoded using bit-vectors:
Sorts constructors.
The sorts follow the definitions in Table 1 for the bit-vectorsizes. We do not support the 80-bit long double extended precision format usedin some processors [35]; instead, we use 128 bits for quadruple precision.
Rounding mode constructors.
The floating-point API supports all roundingmodes described in Section 2, even though the C standard does not supportRNA [38]. These are encoded as 3-bits long bit-vectors.
Value constructors.
Floating-point literals, plus and minus infinity, plus andminus zeroes and NaNs can be created. For the later, the same NaN is alwayscreated (positive, the significand is 000 . . . , where eb is the number of exponent bits and sb isthe number of significand bits. All algorithms in the floating-point API assumeone hidden-bit in the significand.The remaining four operators groups use at least one of the stages in thepipeline to reason about floating-points: Classification operators.
Algorithms to classify normals, subnormals, zeros(regardless of sign), infinities (regardless of sign), NaNs, and negatives and pos-itives. The operators work by unpacking the floating-point and comparing thebit-vectors against the definitions.
Comparison operators.
The operators “greater than or equal to”, “greaterthan”, “less than or equal to”, “less than”, and “equality” are supported. Thefirst three are written in terms of the last two. All of them evaluate to false ifone of their arguments is NaN; this check is done using the NaN classificationoperator.
Conversion operators.
The floating-point API can convert: – Floating-points to signed bit-vectors and floating-points to unsigned bit-vectors: converts the floating-point to bit-vectors always rounding towardszero. These operations generate a free variable if it can not represent thefloating-point using the target bit-vector, i.e., if the floating-point is out-of-range, ± NaN or ± infinity. Minus zero is converted to zero. – Floating-points to another floating-point: converts the floating-point to adifferent format using a rounding mode. ± NaN, ± infinity, and ± zeroes arealways convertible between floating-points, but converting between formatsmight create ± infinity if the target format can not represent the originalfloating-point. – Signed bit-vectors to floating-points and unsigned bit-vectors to floating-points: converts bit-vectors to the nearest representable floating-point, using n Efficient Floating-Point Bit-Blasting API for Verifying C Programs 9 a rounding mode. It might create ± infinity if the target format can notrepresent the original bit-vector. Arithmetic operators.
The floating-point API implements: – Absolute value operator : sets the sign bit of the floating-point to . – Negation operator : flips the sign bit of the floating-point. – Addition operator : the significands are extended by 3 bits to perform theaddition and the exponent are extended by 2 bits to check for overflows.The algorithm first aligns the significands then it adds them. – Subtraction operator : negates the right-hand side of the expression and usesthe addition operator, i.e., x − y = x + ( − y ). – Multiplication operator : the length of the significand bit-vectors are doubledbefore multiplying them, and the exponents are added together. The finalsign bit is the result of xor’ing the sign of both operands of the multiplication. – Division operator : the length of both significand and exponent are extendedby 2 bits, then bit-vector subtractions are used to calculate the target sig-nificand and exponent. – Fused multiply-add : the significand is extended to length toaccommodate both the multiplication and the addition, and the exponent isextended by 2 bits. The first two operands are multiplied, and the result isaligned with the third operand before adding them. – Square root operator : neither the significand nor the exponent is extendedsince the result always fits the original format and can never underflow oroverflows as per the operator definition. Here, √ x = l ∗ d , where the finalexponent d is half the unbiased exponent minus the leading zeros, and l iscalculated using a restorative algorithm [47, Chapter 10].All operators but the absolute value and negation handle special values( ± NaN, ± infinity, and ± zeroes) before performing the operations, e.g., in themultiplication operator, if the left-hand side argument is positive infinity, theresult is NaN if the right-hand side argument is ; otherwise, the result is aninfinity with the right-hand side argument sign. Furthermore, all arithmetic op-erations in the floating-point API that take more than one floating-point as anargument assume that the floating-points have the same format. This assump-tion is not a problem when converting C programs, as type promotion rulesalready ensure this pre-condition [38].A detailed table with all the supported features of the floating-point API, andthe comparison with the features from other solvers can be found in Appendix A. Our experimental evaluation consists of three parts. In Section 4.1, we presentthe benchmarks used to evaluate the implementation of our floating-point API.In Section 4.2, we compare the verification results of the new floating-point APIin ESBMC using several solvers. In Section 4.3, we compare the best solver found in Section 4.2 against all the tools that competed in the
ReachSafety-Floats sub-category in SV-Comp 2020. Our experimental evaluation aims to answer tworesearch questions:
RQ1 (Soundness and completeness)
Is our floating-point API soundand complete?
RQ2 (Performance)
How does the implementation of our floating-pointAPI compare to other software verifiers?
We evaluate our approach using all the verification tasks in SV-COMP 2020 [7].In particular, we considered 466 benchmarks for the sub-category
ReachSafety-Floats , described as “containing tasks for checking programs with floating-pointarithmetics” .The
ReachSafety-Floats sub-category is part of the
ReachSafety category. Inthis category, a function call is checked for reachability; the property is formallydefined in the competition as
G ! call( VERIFIER error()) or “The function VERIFIER error() is not called in any finite execution of the program” .We have implemented our floating-point API in ESBMC. We run ESBMCon each benchmark in that sub-category once per solver, with the followingset of options: --no-div-by-zero-check , which disables the division by zerocheck (an SV-COMP requirement); --incremental-bmc , which enables the in-cremental BMC; --unlimited-k-steps , which removes the upper limit of it-eration steps in the incremental BMC algorithm; --floatbv , which enablesSMT floating-point encoding; --32 , which assumes a 32 bits architecture; and --force-malloc-success , which forces all dynamic allocations succeed to (alsoan SV-COMP requirement). We also disable pointer safety checks and arraybounds check ( --no-pointer-check , --no-bounds-check ) as, per the compe-tition definition, these benchmarks only have reachability bugs. Finaly, in orderto select an SMT solver for verification, the options --boolector , --z3 , --cvc , --mathsat , and --yices are used.All experiments were conducted on our mini cluster at the University ofManchester, UK. The compute node used are equipped with Intel(R) Xeon(R)CPU E5-2620 v4 @ 2.10GHz and 180GB of RAM, where nine instances of ES-BMC were executed in parallel. For each benchmark, we set time and memorylimits of 900 seconds and 15GB, respectively, as per the competition definitions.We, however, do not present the results as scores (as it is done in SV-COMP)but show the number of correct and incorrect results, and the verification time. Figure 2 shows the number of correctly verified programs out of the 466 bench-marks from the
ReachSafety-Floats sub-category, using several solvers and how n Efficient Floating-Point Bit-Blasting API for Verifying C Programs 11 long it took to complete the verification. There exists no case where ESBMCreports an incorrect result. C o rr ec t r e s u lt s Boolector (lingeling, fp2bv)Boolector (CaDiCaL, fp2bv)Z3YicesMathSATZ3 (fp2bv)MathSAT (fp2bv)CVC4 (fp2bv)
Fig. 2:
ReachSafety-Floats results for each solver, using the incremental BMC.The “fp2bv” next to the solver name means that our floating-point API wasused to bit-blast floating-point arithmetic.Boolector (lingeling, fp2bv) reports the highest number of correct results(421), followed by MathSAT using their native floating-point API (414). Thisevaluation produced a slightly better result than our previous one of thesesolvers, where MathSAT was able to solve floating-point problems quickly butsuffered slowdowns in programs with arrays [32]. MathSAT (fp2bv) presentedthe fewest number of correct results (329).The results show that Z3 with its native floating-point API and Z3 with ourfp2bv API produce very similar results: 390 and 387, respectively; this resultis expected since our fp2bv API is heavily based on the bit-blasting performedby Z3 when solving floating-points. The number of variables and clauses gen-erated in the CNF format, when using Z3 with its native floating-point API,is 1%-2% smaller than the number generated when using our fp2bv API. Thesmaller number explains the slightly better results: we assume this is the resultof optimizations when Z3 performs the bit-blasting internally.
MathSAT results show that their API can solve 85 more benchmarks thanMathSAT (fp2bv) within time and memory limits. These benchmarks containchains of multiplications. They thus require a high computational effort dur-ing the propositional satisfiability search. Given that we replace all higher-leveloperators by bit-level circuit equivalents (bit-blasting), we end up destroyingstructural word-level information in the problem formulation. Therefore, theseresults lead us to believe that the MathSAT ACDL algorithm is somehow op-timized for FP operations; unfortunately, MathSAT is a free but closed sourcetool, so we cannot confirm this.The total verification time for each solver is also illustrated in Figure 2,and again Boolector (lingeling, fp2bv) was the faster solver, thereby solvingall programs in 46100 seconds. It is followed by Boolector (CaDiCal, fp2bv)with 56900, and Yices (fp2bv) with 57400 seconds. Overall, Boolector (lingeling,fp2bv) presented the best results. It correctly verified more programs while alsobeing the faster solver, almost 20% faster than the second faster solver, whichis also Boolector but with a different SAT solver (CaDiCaL).ESBMC produced no incorrect result in this evaluation, which partiallyanswers
RQ1 : although we can not formally prove that our algorithm issound and complete, empirical evidence suggests it.
We compare the implementation of our floating-point API with other softwareverifiers: 2LS [43], CBMC [40], CPA-Seq [8], DIVINE [3], PeSCo [53], Pinaka [17],Symbiotic [16], VeriAbs [19]. Figure 3 illustrates the
ReachSafety-Floats resultsfrom our best approach against tools that participated in SV-COMP 2020. Inparticular, we have used the binary and scripts of these tools that are availableat the SV-COMP 2020 website under “Participating Teams”. Overall, VeriAbsachieved the highest number of correct results (435) in 53600 s followed by Pinaka(422) with 27800 s, ESBMC (421) with 46100 s, and CBMC (420) with 49200 s.VeriAbs can verify C programs with floating-points via abstraction usingSAT solvers. In particular, VeriAbs replaces loops in the original code by ab-stract loops of small known bounds; it performs value analysis to compute loopinvariants and then applies an iterative refinement using k -induction. The Veri-Abs tool uses CBMC as its backend to prove properties and find errors, whichthus allows VeriAbs to verify C programs with floating-points. By contrast, ES-BMC uses an iterative technique and verifies the program for each unwind bounduntil it exhausts the time or memory limits. Intuitively, ESBMC can either finda counterexample with up to k loop unwinding or fully unwinds all loops usingthe same unwinding bound so that it can provide a correct result. ESBMC alsorelies on SMT solvers to check the satisfiability of the verifications conditionsthat contain floating-point arithmetic. https://sv-comp.sosy-lab.org/2020/systems.phpn Efficient Floating-Point Bit-Blasting API for Verifying C Programs 13 C o rr ec t r e s u lt s Fig. 3:
ReachSafety-Floats results from our best approach against tools from SV-COMP 2020.Pinaka verifies C programs using CBMC, but it relies on an incrementalSAT solving coupled with eager state infeasibility checks. Additionally, Pinakaextends CBMC to support both Breadth-First Search and Depth-First Searchas state exploration strategies along with partial and full incremental modes.Here we have not evaluated the SMT incremental mode implemented in ESBMCsince this feature is currently supported for the SMT solver Z3 only. Other SMTsolvers do support incremental solving, but ESBMC does not provide supportfor incremental solving for other SMT solvers yet.CBMC [21] implements a bit-precise decision procedure for the theory offloating-point arithmetic [11]. Both VeriAbs and Pinaka rely on CBMC to verifythe underlying C program using that decision procedure. ESBMC originated asa fork of CBMC in 2008 with an improved SMT backend [24] and support for theverification of concurrent programs using an explicit interleaving approach [23].CBMC uses SAT solvers as their primary engine but offers support for the gen-eration of an SMT formula for an external SMT solver. ESBMC supports SMTsolvers directly, through their APIs, along with the option to output SMT for-mulae. As a result, the main difference between CBMC and ESBMC here relieson the encoding and checking of the verification conditions that contain floating-point arithmetic.
These results answer our
RQ2 : our floating-point API is on par with otherstate-of-the-art tools. VeriAbs and Pinaka implement several heuristics tosimplify the check for satisfiability using CBMC, while ESBMC used anincremental approach produced close results. ESBMC was also slightlyfaster and provided a few more results than CBMC, which lead us tobelieve that our tool would also greatly benefit VeriAbs and Pinaka ifused as backend.
Several symbolic execution tools try to verify programs with floating-point arith-metic by employing different strategies. CoverMe [30] reformulates floating-pointconstraints as mathematical optimization problems and uses a specially builtsolver called XSat [29] to check for satisfiability. Pex [56] uses a similar ap-proach and reasons for floating-point constraints as a search problem, and theyare solved by using meta-heuristics search methods. FPSE [10] models floating-point arithmetic by using an interval solver over real arithmetic combined withprojection functions.HSE [52] extends KLEE [15] to execute the program and convert floating-points into bit-vectors symbolically. It then uses SMT solvers to reason aboutsatisfiability. Astr´ee is a static analysis tool that considers all possible roundingerrors when verifying C programs with floating-point numbers [9]. It has beenapplied to verify embedded software in the flight control software of the Airbus.Bounded model checkers have also been applied to verify programs withfloating-point arithmetic: CBMC [21] and 2LS [55] convert floating-point opera-tions to bit-vectors and use SAT solvers to reason about satisfiability. CPBPV [22]uses bounded model checking combined with their FPCS [45] interval solver togenerate tests that violate output constraints in the program.Brain et al. [12] describe an approach called SymFPU for handling the the-ory of floating-point by reducing it to the theory of bit-vectors. In particular,the authors describe a library of encodings, which can be included in SMTsolvers to add support for the theory of floating-point by taking into accountfloating-point reasoning and the fundamentals of circuit design. Brain et al. haveintegrated SymFPU into the SMT solver CVC4 and evaluate it using a broadset of benchmarks; they conclude that SymFPU+CVC4 can substantially out-performs all previous systems despite using a straightforward bit-blasting ap-proach for floating-point problems. We could not compare our approach againstSymFPU because of bugs in the CVC4 C API; we contacted the author, and wewill create bug reports about the issues we identified.
We have described our new SMT floating-point API, which bit-blasts floating-point arithmetic and extends the floating-point support for SMT solvers that n Efficient Floating-Point Bit-Blasting API for Verifying C Programs 15 only support bit-vector arithmetic. The floating-point API was implemented inthe SMT backend of ESBMC. Our experimental results show that Boolector(with lingeling as SAT solver) presented the best results: the highest numberof correct results within the shortest verification time. We also show that ourfloating-point API implemented in ESBMC is on par with other state-of-the-artsoftware verifiers. VeriAbs and Pinaka implement several heuristics to simplifythe check for satisfiability using CBMC, while ESBMC with a straightforwardincremental approach produced close results.ESBMC was already extensively used to verify digital systems [1,5,6]. How-ever, these projects were limited to fixed-point arithmetic; supporting floating-point encoding will allow researchers to expand their activities in the scientificcommunity. The extensive evaluation performed during the development of thesetechnologies also identified areas to be improved in the solvers and other ver-ification tools. In particular, we submitted patches to Z3 to optimize the gen-eration of unsigned less-than operations during the bit-blast of floating-points (accepted, part of Z3 4.6.1). We also reported bugs to both CBMC and Math-SAT, concerning floating-point arithmetic issues, which were later confirmed bythe developers. References
1. Abreu, R.B., Gadelha, M.R., Cordeiro, L.C., Filho, E.B.d.L., da Silva Jr., W.S.:Bounded model checking for fixed-point digital filters. Journal of the BrazilianComputer Society (1), 1:1–1:20 (2016)2. ARM: ARM C Language Extensions 2.1 (2016), IHI 0053D3. Baranov´a, Z., Barnat, J., Kejstov´a, K., Kuˇcera, T., Lauko, H., Mr´azek, J., Roˇckai,P., ˇStill, V.: Model checking of C and C++ with DIVINE 4. In: Automated Tech-nology For Verification And Analysis. LNCS, vol. 10482, pp. 201–207 (2017)4. Barrett, C., Conway, C., Deters, M., Hadarean, L., Jovanovi´c, D., King, T.,Reynolds, A., Tinelli, C.: CVC4. In: Computer-Aided Verification. LNCS, vol. 6806,pp. 171–177 (2011)5. Bessa, I., Ismail, H., Cordeiro, L.C., Filho, J.E.C.: Verification of fixed-point dig-ital controllers using direct and delta forms realizations. Design Automation forEmbedded Systems (2), 95–126 (2016)6. Bessa, I., Ismail, H., Palhares, R., Cordeiro, L.C., Filho, J.E.C.: Formal non-fragilestability verification of digital control systems with uncertainty. IEEE Transactionson Computers (3), 545–552 (2017)7. Beyer, D.: Advances in automatic software verification: SV-COMP 2020. In: Toolsand Algorithms for the Construction and Analysis of Systems. LNCS, vol. 12079,pp. 347–367 (2020)8. Beyer, D., Keremoglu, M.E.: CPAchecker: A tool for configurable software verifi-cation. In: Computer-Aided Verification. LNCS, vol. 6806, pp. 184–190 (2011)9. Blanchet, B., Cousot, P., Cousot, R., Feret, J., Mauborgne, L., Min´e, A., Monniaux,D., Rival, X.: A static analyzer for large safety-critical software. In: ProgrammingLanguage Design and Implementation. pp. 196–207 (2004) https://github.com/Z3Prover/z3/pull/1501 https://github.com/diffblue/cbmc/issues/19446 M. Gadelha et al.10. Botella, B., Gotlieb, A., Michel, C.: Symbolic execution of floating-point computa-tions: Research articles. Software Testing, Verification & Reliability (2), 97–121(2006)11. Brain, M., D’Silva, V., Griggio, A., Haller, L., Kroening, D.: Deciding floating-point logic with abstract conflict driven clause learning. Formal Methods in SystemDesign (2), 213–245 (2014)12. Brain, M., Schanda, F., Sun, Y.: Building better bit-blasting for floating-pointproblems. In: Tools and Algorithms for the Construction and Analysis of Systems.LNCS, vol. 11427, pp. 79–98 (2019)13. Brain, M., Tinelli, C., Ruemmer, P., Wahl, T.: An automatable formal semanticsfor IEEE-754 floating-point arithmetic. In: Symposium On Computer Arithmetic.pp. 160–167 (2015)14. Brummayer, R., Biere, A.: Boolector: An efficient SMT solver for bit-vectors andarrays. In: Tools And Algorithms For The Construction And Analysis Of Systems.LNCS, vol. 5505, pp. 174–177 (2009)15. Cadar, C., Dunbar, D., Engler, D.: KLEE: Unassisted and automatic generation ofhigh-coverage tests for complex systems programs. In: Symposium On OperatingSystems Design And Implementation. pp. 209–224 (2008)16. Chalupa, M., Vitovsk´a, M., Jon´as, M., Slaby, J., Strejcek, J.: Symbiotic 4: Be-yond reachability - (competition contribution). In: Tools and Algorithms for theConstruction and Analysis of Systems. LNCS, vol. 10206, pp. 385–389 (2017)17. Chaudhary, E., Joshi, S.: Pinaka: Symbolic execution meets incremental solving- (competition contribution). In: Tools and Algorithms for the Construction andAnalysis of Systems. LNCS, vol. 11429, pp. 234–238 (2019)18. Chaves, L., Bessa, I., Cordeiro, L.C., Kroening, D., Filho, E.B.d.L.: Verifying dig-ital systems with MATLAB. In: Symposium On Software Testing And Analysis.pp. 388–391 (2017)19. Chimdyalwar, B., Darke, P., Chauhan, A., Shah, P., Kumar, S., Venkatesh, R.:Veriabs: Verification by abstraction (competition contribution). In: Tools and Al-gorithms for the Construction and Analysis of Systems. LNCS, vol. 10206, pp.404–408 (2017)20. Cimatti, A., Griggio, A., Schaafsma, B., Sebastiani, R.: The mathSAT5 SMTsolver. In: Tools And Algorithms For The Construction And Analysis Of Systems.LNCS, vol. 7795, pp. 93–107 (2013)21. Clarke, E., Kroening, D., Lerda, F.: A tool for checking ANSI-C programs. In:Tools And Algorithms For The Construction And Analysis Of Systems. LNCS,vol. 2988, pp. 168–176 (2004)22. Collavizza, H., Michel, C., Ponsini, O., Rueher, M.: Generating test cases insidesuspicious intervals for floating-point number programs. In: Constraints In SoftwareTesting Verification And Analysis. pp. 7–11 (2014)23. Cordeiro, L.C., Fischer, B.: Verifying multi-threaded software using SMT-basedcontext-bounded model checking. In: International Conference on Software Engi-neering. pp. 331–340 (2011)24. Cordeiro, L.C., Fischer, B., Marques-Silva, J.: SMT-based bounded model checkingfor embedded ANSI-C software. In: Automated Software Engineering. pp. 137–148(2009)25. De Moura, L., Bjørner, N.: Z3: An efficient SMT solver. In: Tools And AlgorithmsFor The Construction And Analysis Of Systems. LNCS, vol. 4963, pp. 337–340(2008)26. Dutertre, B.: Yices 2.2. In: Computer-Aided Verification. LNCS, vol. 8559, pp.737–744 (2014)n Efficient Floating-Point Bit-Blasting API for Verifying C Programs 1727. Erkk, L.: Bug in floating-point conversions. https://github.com/Z3Prover/z3/issues/1564 (2018), [Online; accessed January-2020]28. Frantz, G., Simar, R.: Comparing fixed- and floating-point DSPs. Tech. rep.,SPRY061, Texas Instruments (2004)29. Fu, Z., Su, Z.: XSat: A fast floating-point satisfiability solver. In: Computer-AidedVerification. LNCS, vol. 9780, pp. 187–209 (2016)30. Fu, Z., Su, Z.: Achieving high coverage for floating-point code via unconstrainedprogramming. In: Programming Language Design And Implementation. pp. 306–319 (2017)31. Gadelha, M.R., Monteiro, F., Cordeiro, L., Nicole, D.: ESBMC v6.0: Verifying Cprograms using k -induction and invariant inference. In: Tools And Algorithms ForThe Construction And Analysis Of Systems. LNCS, vol. 11429, pp. 209–213 (2019)32. Gadelha, M.Y.R., Cordeiro, L.C., Nicole, D.A.: Encoding floating-point numbersusing the SMT theory in ESBMC: An empirical evaluation over the SV-COMPbenchmarks. In: Simp´osio Brasileiro De M´etodos Formais. LNCS, vol. 10623, pp.91–106 (2017)33. Gadelha, M.Y.R., Menezes, R., Monteiro, F.R., Cordeiro, L.C., Nicole, D.A.: ES-BMC: scalable and precise test generation based on the floating-point theory -(competition contribution). In: Wehrheim, H., Cabot, J. (eds.) International Con-ference Fundamental Approaches to Software Engineering. LNCS, vol. 12076, pp.525–529. Springer (2020)34. Gerrity, G.W.: Computer representation of real numbers. IEEE Transactions onComputers C-31 (8), 709–714 (1982)35. Goldberg, D.: What every computer scientist should know about floating pointarithmetic. ACM Computing Surveys (3), 12:1–12:41 (2008)47. Muller, J.M., Brisebarre, N., Dinechin, F., Jeannerod, C.P., Lefvre, V., Melquiond,G., Revol, N., Stehl, D., Torres, S.: Handbook of Floating-Point Arithmetic.Birkhuser Boston, 1st edn. (2010)8 M. Gadelha et al.48. Niemetz, A., Preiner, M., Biere, A.: Boolector 2.0 system description. Journal onSatisfiability, Boolean Modeling and Computation , 53–58 (2014)49. Nikoli´c, Z., Nguyen, H.T., Frantz, G.: Design and implementation of numericallinear algebra algorithms on fixed point DSPs. European Association for SignalProcessing (1), 1–22 (2007)50. Noetzli, A.: Failing precondition when multiplying 4-bit significand/4-bit ex-ponent floats. https://github.com/CVC4/CVC4/issues/2182 (2018), [Online; ac-cessed January-2020]51. Patterson, D.A., Hennessy, J.L.: Computer Organization and Design - The Hard-ware / Software Interface (Revised 4th Edition). Academic Press (2012)52. Quan, M.: Hotspot symbolic execution of floating-point programs. In: SymposiumOn Foundations Of Software Engineering. pp. 1112–1114 (2016)53. Richter, C., Wehrheim, H.: Pesco: Predicting sequential combinations of verifiers- (competition contribution). In: Tools and Algorithms for the Construction andAnalysis of Systems. LNCS, vol. 11429, pp. 229–233 (2019)54. R¨ummer, P., Wahl, T.: An SMT-LIB theory of binary floating-point arithmetic.In: SMT Workshop (2010)55. Schrammel, P., Kroening, D., Brain, M., Martins, R., Teige, T., Bienm¨uller, T.:Incremental bounded model checking for embedded software (extended version).Formal Aspects of Computing (5), 911–931 (2017)56. Tillmann, N., De Halleux, J.: Pex: White box test generation for .NET. In: TestsAnd Proofs. pp. 134–153 (2008)57. Wiffin, E.: 0.30000000000000004.com. https://0.30000000000000004.com/ (2012),[Online; accessed April-2020]58. Zeljic, A., Backeman, P., Wintersteiger, C.M., R¨ummer, P.: Exploring approxima-tions for floating-point arithmetic using uppsat. In: International Joint Conferenceon Automated Reasoning. LNCS, vol. 10900, pp. 246–262. Springer (2018)n Efficient Floating-Point Bit-Blasting API for Verifying C Programs 19 A Support for the FP logic Z3 MathSAT CVC4 ESBMCSMT FP operations v4.7.1 v5.5.1 v1.6-prerelease FP APICreate floating point sort √ √ √ √ Create rounding mode sort √ √ √ √
Create floating point literal √ √ √ √
Create plus and minus infinity √ √ √ √
Create plus and minus zeroes √ √ √ √
Crete NaN √ √ √ √
Absolute value operator √ √ √ √
Negation operator √ √ √ √
Addition operator √ √ √ √
Subtraction operator √ √ √ √
Multiplication operator √ √ √ √
Division operator √ √ √ √
Fused multiply-add operator √ × √ √ Square root operator √ √ √ √
Remainder operator √ × √ ×
Rounding to Integral operator √ √ √ √
Minimum operator √ √ √ ×
Maximum operator √ √ √ ×
Less than or equal to operator √ √ √ √
Less than operator √ √ √ √
Greater than or equal to operator √ √ √ √
Greater than operator √ √ √ √
Equality operator √ √ √ √
IsNormal check √ √ √ √
IsSubnormal check √ √ √ √
IsZero check √ √ √ √
IsInfinite check √ √ √ √
IsNaN check √ √ √ √
IsNegative check √ √ √ √
IsPositive check √ √ √ √
Convert to FP from real √ √ × ×
Convert to FP from signed BV √ √ × √
Convert to FP from unsigned BV √ √ × √
Convert to FP from another FP √ √ × √
Convert to unsigned BV from FP √ √ × √
Convert to signed BV from FP √ √ × √
Convert to real from FP √ √ × × In ESBMC, the fused multiply-add operation uses the bit-blasting API when usingMathSAT.0 M. Gadelha et al.