Kazuki Munakata

Test data generation for web application using a UML class diagram with OCL constraints

In this paper, we report on our current work toward efficient and effective verification of web application’s basic design. We use a UML class diagram with Object Constraint Language (OCL) to describe the application behaviors and data constraints. Then we generate test data from the formally represented specifications. We make the observation that key web application behaviors can be captured through table size constraints as well as data constraints like foreign key constraints. Based on the observation, we translate the OCL specification into the equivalent constraints using table size expressions. We present a scheme to generate test data from the translated constraints using a Satisfiability Modulo Theories solver. We employ two techniques to reduce constraints. The first is string handling and the other is decomposition of table structures. We also report on an experimental result of test data generation. The result indicates a potential that our scheme works well for real applications in reasonable times.

Enhancing Symbolic Execution to Test the Compatibility of Re-engineered Industrial Software

After a legacy system is re-engineered, it is important to perform compatibility testing so as to identify the difference and reduce the introduced bugs. We can first apply symbolic execution to obtain an exhaustive set of test cases, then use them to check the compatibility of the old system and the new old. However, the path explosion problem of symbolic execution makes it difficult to work on realistic non-trivial applications. We show in this paper how to enhance symbolic execution, e.g. with extra constraints, path cutting, variable grouping, and test case selection, to successfully test the compatibility of an SMTP library (used in embedded systems) with around 20K lines of code. Our experience indicates that these enhancements are essential to apply symbolic execution on realistic industrial applications.

Detection of unexpected situations by applying software reliability growth models to test phases

In software development, software reliability growth models (SRGMs) often provide values that do not meet expectations; sometimes the results of the SRGM and the actual data disagree and other times the SRGM overestimates the expected values. The former often occurs in model curves and the predicted number of faults. For example, the software reliability growth curve cannot describe the situation where developers stop testing multiple times because the equations in SRGMs cannot treat such information. The latter can arise when the total number of expected faults is 100, but the SRGM indicates 1000. If developers encounter such situations, they often doubt the SRGM results and hesitate using SRGMs for predictions. In this study, we apply two different cases of SRGM. Two projects of Fujitsu Labs Ltd. are analyzed using SRGM either for the entire dataset or each test phase. Based on the results and interviews with the developers, we found that the model using separate test phases provides a better fit because faults counted in each test phase have different viewpoints and the deviation between SRGM and expectations indicates a problem with development.

KLOVER: Automatic Test Generation for C and C Programs, Using Symbolic Execution

Fujitsu researchers have developed a methodology to automate testing of industrial-strength embedded software implemented in C or C++. The methodology’s core is a program analysis technique called symbolic execution, which the researchers have customized to automate testing. The methodology generates unit-level tests, greatly reducing test generation time and cost while providing excellent test coverage.

Defect Analysis and Prediction by Applying the Multistage Software Reliability Growth Model

In software development, defects are inevitable. To improve reliability, software reliability growth models are useful to analyze projects. Selecting an expedient model can also help with defect predictions, but the model must be well fitted to all the original data. A particular software reliability growth model may not fit all the data well. To overcome this issue, herein we use multistage modeling to fit defect data. In the multistage model, an evaluation is used to divide the data into several parts. Each part is fitted with its own growth model, and the separate models are recombined. As a case study, projects provided by a Japanese enterprise are analyzed by both traditional software reliability growth models and the multistage model. The multistage model has a better performance for data with a poor fit using a traditional software reliability growth model.

Model-based test case generation using symbolic execution

In this paper, we present a test case generation method in which test cases are generated from Excel-based functional specications, called error-check/update specications. This method has the following two characteristics. 1) Logical structures and constraint conditions in error-check/update specications are translated into Java code. Then the trans- lated Java code is executed symbolically. A solution found by solving combinations of the constraint conditions, during the symbolic execution can serve as a test case for the Excel- based specication. 2)We present test case selection method to extract a suitable test-suite from the large number of test cases typically generated by the above symbolic execution. Our test case selection is based on a path condition-based test case selection criterion, proposed in our previous work, in which the selected test cases covers all the atomic conditions of path conditions collected during symbolic execution. In this paper, we extend this method with classication of test cases and priority of variable values with the aim of improv- ing the quality of the selected test-suite. We implemented the test case generation method in our tool and evaluated this method with Excel-based test specications for a real- world nancial system.