Xiaoguang Mao

The strength of random search on automated program repair

Automated program repair recently received considerable attentions, and many techniques on this research area have been proposed. Among them, two genetic-programming-based techniques, GenProg and Par, have shown the promising results. In particular, GenProg has been used as the baseline technique to check the repair effectiveness of new techniques in much literature. Although GenProg and Par have shown their strong ability of fixing real-life bugs in nontrivial programs, to what extent GenProg and Par can benefit from genetic programming, used by them to guide the patch search process, is still unknown. To address the question, we present a new automated repair technique using random search, which is commonly considered much simpler than genetic programming, and implement a prototype tool called RSRepair. Experiment on 7 programs with 24 versions shipping with real-life bugs suggests that RSRepair, in most cases (23/24), outperforms GenProg in terms of both repair effectiveness (requiring fewer patch trials) and efficiency (requiring fewer test case executions), justifying the stronger strength of random search over genetic programming. According to experimental results, we suggest that every proposed technique using optimization algorithm should check its effectiveness by comparing it with random search.

Using automated program repair for evaluating the effectiveness of fault localization techniques

Many techniques on automated fault localization (AFL) have been introduced to assist developers in debugging. Prior studies evaluate the localization technique from the viewpoint of developers: measuring how many benefits that developers can obtain from the localization technique used when debugging. However, these evaluation approaches are not always suitable, because it is difficult to quantify precisely the benefits due to the complex debugging behaviors of developers. In addition, recent user studies have presented that developers working with AFL do not correct the defects more efficiently than ones working with only traditional debugging techniques such as breakpoints, even when the effectiveness of AFL is artificially improved. In this paper we attempt to propose a new research direction of developing AFL techniques from the viewpoint of fully automated debugging including the program repair of automation, for which the activity of AFL is necessary. We also introduce the NCP score as the evaluation measurement to assess and compare various techniques from this perspective. Our experiment on 15 popular AFL techniques with 11 subject programs shipping with real-life field failures presents the evidence that these AFL techniques performing well in prior studies do not have better localization effectiveness according to NCP score. We also observe that Jaccard has the better performance over other techniques in our experiment.

Slice-based statistical fault localization

HighlightsWe propose a novel slice-based statistical fault localization approach to improve fault localization effectiveness.We present an approximate dynamic backward slicing approach to balance the size and accuracy of a slice.An experimental study on a sufficient number of faulty versions and fault localization techniques shows the high applicability and effectiveness of our approach. Recent techniques for fault localization statistically analyze coverage information of a set of test runs to measure the correlations between program entities and program failures. However, coverage information cannot identify those program entities whose execution affects the output and therefore weakens the aforementioned correlations. This paper proposes a slice-based statistical fault localization approach to address this problem. Our approach utilizes program slices of a set of test runs to capture the influence of a program entitys execution on the output, and uses statistical analysis to measure the suspiciousness of each program entity being faulty. In addition, this paper presents a new slicing approach called approximate dynamic backward slice to balance the size and accuracy of a slice, and applies this slice to our statistical approach. We use two standard benchmarks and three real-life UNIX utility programs as our subjects, and compare our approach with a sufficient number of fault localization techniques. The experimental results show that our approach can significantly improve the effectiveness of fault localization.

Efficient Automated Program Repair through Fault-Recorded Testing Prioritization

Most techniques for automated program repair use test cases to validate the effectiveness of the produced patches. The validation process can be time-consuming especially when the object programs ship with either lots of test cases or some long-running test cases. To alleviate the cost for testing, we first introduce regression test prioritization insight into the area of automated program repair, and present a novel prioritization technique called FRTP with the goal of reducing the number of test case executions in the repair process. Unlike most existing prioritization techniques frequently requiring additional cost for gathering previous test executions information, FRTP iteratively extracts that information just from the repair process, and thus incurs trivial performance lose. We also built a tool called TrpAutoRepair, which implements our FRTP technique and has the ability of automatically repairing C programs. To evaluate TrpAutoRepair, we compared it with GenProg, a state-of-the-art tool for automated C program repair. The experiment on the 5 subject programs with 16 real-life bugs provides evidence that TrpAutoRepair performs at least as good as GenProg in term of success rate, in most cases (15/16), TrpAutoRepair can significantly improve the repair efficiency by reducing efficiently the test case executions when searching a valid patch in the repair process.

Effective Statistical Fault Localization Using Program Slices

Recent techniques for fault localization statistically analyze coverage information of a set of test runs to measure the correlations between program entities and program failures. However, coverage information cannot identify those program entities whose execution affects the output, which weakens the aforementioned correlations. Thus, this paper proposes a novel statistical fault localization approach to address this problem. Our statistical approach utilizes program slices of a set of test runs to capture the influence of a program entitys execution on the output, and uses statistical analysis to measure the suspiciousness of program entities to be faulty. In addition, this paper presents a new slicing approach called approximate dynamic backward slice to balance the size and accuracy of a slice, and applies this slice to our statistical approach. The experimental results on two standard benchmarks show that our statistical approach significantly outperforms eight representative fault localization techniques.

More efficient automatic repair of large-scale programs using weak recompilation

Automatically repairing a bug can be a time-consuming process especially for large-scale programs owing to the significant amount of time spent recompiling and reinstalling the patched program. To reduce this time overhead and speed up the repair process, in this paper we present a recompilation technique called weak recompilation. In weak recompilation, we assume that a program consists of a set of components, and for each candidate patch only the altered components are recompiled to a shared library. The original program is then dynamically updated by a function indirection mechanism. The advantage of weak recompilation is that redundant recompilation cost can be avoided, and while the reinstallation cost is completely eliminated as the original executable program is not modified at all. For maximum applicability of weak recompilation we created WAutoRepair, a scalable system for fixing bugs with high efficiency in large-scale C programs. The experiments on real bugs in widely used programs show that our repair system significantly outperforms Genprog, a well-known approach to automatic program repair. For the wireshark program containing over 2 million lines of code, WAutoRepair is over 128 times faster in terms of recompilation cost than Genprog.

Does Genetic Programming Work Well on Automated Program Repair

Automated program repair has made some important progress in the recent decade. One well-known repair tool is GneProg, which automates the patch generation process according to the guidance of genetic programming. Although GenProg has successfully fixed many legacy faulty programs, the guidance effectiveness of genetic programming used by GenProg has not been even justified through the comparison with random search algorithm. In this paper we try to make the guidance effectiveness comparison between genetic programming and random search algorithm on program repair. The experimental results show that genetic programming does not perform better than random search algorithm on guiding the patch generation process.

Using ELECTRE TRI outranking method to evaluate trustworthy software

Trustworthy software evaluation is taken as the multicriteria decision aiding process in this paper. The use of ELECTRE TRI method for evaluating software trustworthiness is presented. Software under evaluation is compared with some predefined norms and is assigned to one of trust levels. The entire evaluating process is described, including definition of problem situation and formulation, determination of the model and its parameters, and the application of the model. A metric for trustworthiness and an attributes weighting method are also presented. Some practical considerations are discussed in the final part of the paper.

Making automatic repair for large-scale programs more efficient using weak recompilation

For large-scale programs, automatically repairing a bug by modifying source code is often a time-consuming process due to plenty of time spent on recompiling and reinstalling the patched program. To suppress the above time cost and make the repair process more efficient, a recompilation technique called weak recompilation is described in this paper. In weak recompilation, a program is assumed to be constructed from a set of components, and for each candidate patch only the changed code fragment in term of one component is recompiled to a shared library; the behaviors of patched program are observed by executing the original program with an instrumentation tool which can wrap specified function. The advantage of weak recompilation is that redundant recompilation cost can be suppressed, and reinstallation cost will be cut down completely. We also built WAutoRepair, a system which enables scalability to fix bugs in large-scale C programs with high efficiency. The experiments confirm that our repair system significantly outperforms Genprog, a famous approach for automatic program repair. For the wireshark program containing over 2 millions lines of code, WAutoRepair spent only 0.222 seconds in recompiling one candidate patch and 8.035 seconds in totally repairing the bug, compared to Genprog separately taking about 20.484 and 75.493 seconds, on average.

Light-Weight Test Oracles for Resource Leaks Based on Finalizers

Garbage collectors automatically manage memory in an elegant way. However, finalization performs poorly in reclaiming finite system resources such as file descriptors and database connections, especially in Java. Resource leaks degrade application performance and can even result in system crashes. Unfortunately, it is hard to test resource leaks because they have no immediate symptoms. In this paper, we first investigate Java API classes for the usage of finalizers. Based on the common usage pattern of these finalizers, we then propose a lightweight, easy-to-use yet effective resource leak testing approach called Orbaf. Orbaf instruments resource classes to inspect whether the cleanup method of a resource object is called before its finalizers execution. Test cases are also optionally instrumented to improve the incidence of object finalization. Orbaf can detect leaks of most system resources and does not require any formal specifications. It can be handily used by common software developers. Experiments show that Orbaf has high effectiveness and low overhead.