Priyanka Darke

Automatic Test Data Generation for C Programs

Preparation of test data that adequately tests a given piece of code is very expensive and effort intensive. This paper presents a tool AutoGen that reduces this cost and effort by automatically generating test data for C code. AutoGen takes the C code and a criterion such as statement coverage, decision coverage, or Modified Condition/Decision Coverage (MCDC) and generates non-redundant test data that satisfies the specified criterion. This paper also presents our experience in using this tool to generate MCDC test data for three embedded reactive system applications. The effort required using the tool was one third of the manual effort required. The main contributions of this paper are a tool that can generate data for various kinds of coverage including MCDC and the experience of running this tool on real applications.

Test generation for large automotive models

Modeling tools such as Statemate, Simulink and Stateflow are widely used in the automotive industry to specify low level requirements and design. Systematic testing of models to achieve structural coverage such as state coverage, transition coverage or modified condition decision coverage (MCDC) helps in early defect detection. Automatic generation of test data can help in reducing the cost and improving the quality of systematic testing. Test data can be automatically generated either 1) directly from the models or 2) from the code generated from these models. In this paper we argue for and recommend the second approach. We propose generating test data from C, a formalism-independent intermediate language, as it is widely used in the embedded domain and most modeling tools have C code generators. Accurate representation of floating point number type is possible in C (that being the representation in the final executable) and there are various analysis tools that are available for C. A major challenge in using code to generate test data is scalability. To overcome this problem, we built a tool that combines available static slicing and model-checking techniques to generate test data. We conducted experiments to check if this tool can generate test data for large complex models from the automotive domain. To demonstrate formalism independence and scalability we chose industry size Statemate as well as Simulink/Stateflow models. The set up and the findings of these experiments are also presented in this paper. We successfully generated test data for code sizes as large as 50KLOC and detected several bugs in four already tested industry models thus proving the benefits of this approach.

Precise Analysis of Large Industry Code

Static analysis of code is very effective in finding common programmer errors but it comes at a price - a large number of false positives. Model checking, on the other hand, is very precise but does not scale up. We have developed a tool that combines both techniques and also implements a novel loop abstraction. The tool was run on 2 million lines of embedded code to analyze for two properties - division by zero and array index out of bounds. In other experiments we compared the precision of our tool to that achieved by tools implementing abstract interpretation. This paper presents details of the tool and the results of evaluations that we have carried out to measure the scalability and to compare the precision of our method on industry code against other static analysis tools.

Over-approximating loops to prove properties using bounded model checking

Bounded Model Checkers (BMCs) are widely used to detect violations of program properties up to a bounded execution length of the program. However when it comes to proving the properties, BMCs are unable to provide a sound result for programs with loops of large or unknown bounds. To address this limitation, we developed a new loop over-approximation technique LA. LA replaces a given loop in a program with an abstract loop having a smaller known bound by combining the techniques of output abstraction and a novel abstract acceleration, suitably augmented with a new application of induction. The resulting transformed program can then be fed to any bounded model checker to provide a sound proof of the desired properties. We call this approach, of LA followed by BMC, as LABMC. We evaluated the effectiveness of LABMC on some of the SV-COMP14 loop benchmarks, each with a property encoded into it. Well known BMCs failed to prove most of these properties due to loops of large, infinite or unknown bounds while LABMC obtained promising results. We also performed experiments on a real world automotive application on which the well known BMCs were able to prove only one of the 186 array accesses to be within array bounds. LABMC was able to successfully prove 131 of those array accesses to be within array bounds.

Eliminating Static Analysis False Positives Using Loop Abstraction and Bounded Model Checking

Sound static analyzers over-approximate the input program behaviour and thus imprecisely report many correct properties as potential errors (false warnings). Manual investigation of these warnings is cost intensive and error prone. To get an insight into the causes and explore the effectiveness of current solutions, we analyzed the code structure associated with warnings reported by sound state of the art static analyzers: Polyspace and TCS Embedded Code Analyzer, over six industrial embedded applications. We observed that most of the warnings were due to variables modified inside loops with large or unknown bounds.

VeriAbs: Verification by Abstraction Competition Contribution

VeriAbs verifies C programs by transforming them to abstract programs. The transformation replaces loops in the original code by abstract loops of small known bounds. Bounded model checkers can then be used to prove properties over such programs. To perform such a transformation, VeriAbs implements i a static value analysis to compute loop invariants, ii abstract acceleration and output abstraction for numerical loops, iii a novel array witness selection for loops that iterate over arrays, and iv an iterative refinement using an enhanced k-induction technique. To find errors, VeriAbs computes bounds of the original loops and then checks for errors within those bounds. VeriAbs can thus prove properties and find errors using bounded model checking. It uses the C Bounded Model Checker CBMC version 5.4 with MiniSat version 2.2.

Statically relating program properties for efficient verification (short WIP paper)

Efficient automatic verification of real world embedded software with numerous properties is a challenge. Existing techniques verify a sufficient subset of properties by identifying implication relations between their verification outcomes. We believe this is expensive and propose a novel complementary approach called grouping. Grouping does not consider the verification outcomes but uses data and control flow characteristics of the program to create disjoint groups of properties verifiable one group at a time.We present three grouping techniques, a framework, and experiments over open source and industrial applications to support our thesis. The experiments show a high gain in performance of a few state-of-the-art tools. This led to the integration of grouping into the verification process of an automotive software manufacturer.

Efficient Safety Proofs for Industry-Scale Code Using Abstractions and Bounded Model Checking

Loop Abstraction followed by Bounded Model Checking, or LABMC in short, is a promising recent technique for proving safety of large programs. In an experimental setup proposed last year [14], LABMC was combined with slicing and Iterative Context Extension (ICE) with the aim of achieving scalability over industrial code. In this paper, we address two major limitations of that set-up, namely i) the inability of ICE to prune redundant code in a verification context, and ii) the unavailability of a tool that implements the set-up. We propose an improvement over ICE called Iterative Function Level Slicing (IFLS) and incorporate it in our tool called ELABMC, to offer an efficient implementation of [14]. We substantiate our claim with two sets of experiments over industrial applications as well as academic benchmarks. Quantifying the benefits of IFLS over traditional ICE in one, our results report that IFLS leads to 34.9% increase in efficiency, 17.7% improvement in precision, and scales in 14.2% more cases. With the second experiment, we show that ELABMC outperforms state-of-the-art verification techniques in the task of identifying static analysis warnings as false alarms.