Yoonkyu Jang

Industrial Application of Concolic Testing on Embedded Software: Case Studies

Current industrial testing practices often build test cases in a manual manner, which is slow and ineffective. To alleviate this problem, concolic testing generates test cases that can achieve high coverage in an automated fashion. However, due to a large number of possible execution paths, concolic testing might not detect bugs even after spending significant amount of time. Thus, it is necessary to check if concolic testing can detect bugs in embedded software in a practical manner through case studies. This paper describes case studies of applying the concolic testing tool CREST to embedded Applications. Through this project, we have detected new faults in the Samsung Linux Platform (SLP) file manager, Samsung security library, and busy box ls.

Industrial application of concolic testing approach: a case study on libexif by using CREST-BV and KLEE

As smartphones become popular, manufacturers such as Samsung Electronics are developing smartphones with rich functionality such as a camera and photo editing quickly, which accelerates the adoption of open source applications in the smartphone platforms. However, developers often do not know the detail of open source applications, because they did not develop the applications themselves. Thus, it is a challenging problem to test open source applications effectively in short time. This paper reports our experience of applying concolic testing technique to test libexif, an open source library to manipulate EXIF information in image files. We have demonstrated that concolic testing technique is effective and efficient at detecting bugs with modest effort in industrial setting. We also compare two concolic testing tools, CREST-BV and KLEE, in this testing project. Furthermore, we compare the concolic testing results with the analysis result of the Coverity Prevent static analyzer. We detected a memory access bug, a null pointer dereference bug, and four divide-by-zero bugs in libexif through concolic testing, none of which were detected by Coverity Prevent.

Automated unit testing of large industrial embedded software using concolic testing

Current testing practice in industry is often ineffective and slow to detect bugs, since most projects utilize manually generated test cases. Concolic testing alleviates this problem by automatically generating test cases that achieve high coverage. However, specialized execution platforms and resource constraints of embedded software hinder application of concolic testing to embedded software. To overcome these limitations, we have developed CONcrete and symBOLic (CONBOL) testing framework to unit test large size industrial embedded software automatically. To address the aforementioned limitations, CONBOL tests target units on a host PC platform by generating symbolic unit testing drivers/stubs automatically and applying heuristics to reduce false alarms caused by the imprecise drivers/stubs. We have applied CONBOL to four million lines long industrial embedded software and detected 24 new crash bugs. Furthermore, the development team of the target software adopted CONBOL to their development process to apply CONBOL to the revised target software regularly.

Concolic Testing Framework for Industrial Embedded Software

Recently, concolic testing has drawn a lot of attention from the research community as a practical solution of overcoming limitations of manual test case generation. CONBOL, a concolic testing framework for large-scale embedded industrial software, has shown effectiveness of discovering real defects even for commercial quality software. In this paper, we extend CONBOL to be applicable to a project written in C++. Front-end of CONBOL is re-implemented based on LLVM and concolic engine is also modified. We design an enhanced DFS search strategy to enlarge search space when there an infinite loop exists in the execution path or control flow is too complex to be traversed. After applying to real world project, we prove that enhanced search strategy is better to increase test coverage. Finally, we investigate reasons of segmentation faults during concolic execution and suggest several heuristics to decrease segmentation faults.