Andy Podgurski

A formal model of program dependences and its implications for software testing, debugging, and maintenance

A formal, general model of program dependences is presented and used to evaluate several dependence-based software testing, debugging, and maintenance techniques. Two generalizations of control and data flow dependence, called weak and strong syntactic dependence, are introduced and related to a concept called semantic dependence. Semantic dependence models the ability of a program statement to affect the execution behavior of other statements. It is shown that weak syntactic dependence is a necessary but not sufficient condition for semantic dependence and that strong syntactic dependence is necessary but not sufficient condition for a restricted form of semantic dependence that is finitely demonstrated. These results are used to support some proposed uses of program dependences, to controvert others, and to suggest new uses. >

Automated support for classifying software failure reports

This paper proposes automated support for classifying reported software failures in order to facilitate prioritizing them and diagnosing their causes. A classification strategy is presented that involves the use of supervised and unsupervised pattern classification and multivariate visualization. These techniques are applied to profiles of failed executions in order to group together failures with the same or similar causes. The resulting classification is then used to assess the frequency and severity of failures caused by particular defects and to help diagnose those defects. The results of applying the proposed classification strategy to failures of three large subject programs are reported These results indicate that the strategy can be effective.

A formal evaluation of data flow path selection criteria

The authors report on the results of their evaluation of path-selection criteria based on data-flow relationships. They show how these criteria relate to each other, thereby demonstrating some of their strengths and weaknesses. A subsumption hierarchy showing their relationship is presented. It is shown that one of the major weaknesses of all the criteria is that they are based solely on syntactic information and do not consider semantic issues such as infeasible paths. The authors discuss the infeasible-path problem as well as other issues that must be considered in order to evaluate these criteria more meaningfully and to formulate a more effective path-selection criterion. >

jRapture: A Capture/Replay tool for observation-based testing

We describe the design of jRapture: a tool for capturing and replaying Java program executions in the field. jRapture works with Java binaries (byte code) and any compliant implementation of the Java virtual machine. It employs a lightweight, transparent capture process that permits unobtrusive capture of a Java programs executions. jRapture captures interactions between a Java program and the system, including GUI, file, and console inputs, among other types, and on replay it presents each thread with exactly the same input sequence it saw during capture. In addition, jRapture has a profiling interface that permits a Java program to be instrumented for profiling ó after its executions have been captured. Using an XML-based profiling specification language a tester can specify various forms of profiling to be carried out during replay.

Retrieving reusable software by sampling behavior

A new method, called behavior sampling, is proposed for automated retrieval of reusable components from software libraries. Behavior sampling exploits the property of software that distinguished it from other forms of test: executability. Basic behavior sampling identifies relevant routines by executing candidates on a searcher-supplied sample of operational inputs and by comparing their output to output provided by the searcher. The probabilistic basis for behavior sampling is described, and experimental results are reported that suggest that basic behavior sampling exhibits high precision when used with small samples. Extensions to basic behavior sampling are proposed to improve its recall and to make it applicable to the retrieval of abstract data types and object classes.

Dex: a semantic-graph differencing tool for studying changes in large code bases

This paper describes an automated tool called Dex (difference extractor) for analyzing syntactic and semantic changes in large C-language code bases. It is applied to patches obtained from a source code repository, each of which comprises the code changes made to accomplish a particular task. Dex produces summary statistics characterizing these changes for all of the patches that are analyzed. Dex applies a graph differencing algorithm to abstract semantic graphs (ASGs) representing each version. The differences are then analyzed to identify higher-level program changes. We describe the design of Dex, its potential applications, and the results of applying it to analyze bug fixes from the Apache and GCC projects. The results include detailed information about the nature and frequency of missing condition defects in these projects.

Causal inference for statistical fault localization

This paper investigates the application of causal inference methodology for observational studies to software fault localization based on test outcomes and profiles. This methodology combines statistical techniques for counterfactual inference with causal graphical models to obtain causal-effect estimates that are not subject to severe confounding bias. The methodology applies Pearls Back-Door Criterion to program dependence graphs to justify a linear model for estimating the causal effect of covering a given statement on the occurrence of failures. The paper also presents the analysis of several proposed-fault localization metrics and their relationships to our causal estimator. Finally, the paper presents empirical results demonstrating that our model significantly improves the effectiveness of fault localization.

The Probabilistic Program Dependence Graph and Its Application to Fault Diagnosis

This paper presents an innovative model of a programs internal behavior over a set of test inputs, called the probabilistic program dependence graph (PPDG), which facilitates probabilistic analysis and reasoning about uncertain program behavior, particularly that associated with faults. The PPDG construction augments the structural dependences represented by a program dependence graph with estimates of statistical dependences between node states, which are computed from the test set. The PPDG is based on the established framework of probabilistic graphical models, which are used widely in a variety of applications. This paper presents algorithms for constructing PPDGs and applying them to fault diagnosis. The paper also presents preliminary evidence indicating that a PPDG-based fault localization technique compares favorably with existing techniques. The paper also presents evidence indicating that PPDGs can be useful for fault comprehension.

Tree-based methods for classifying software failures

Recent research has addressed the problem of providing automated assistance to software developers in classifying reported instances of software failures so that failures with the same cause are grouped together. In this paper, two new tree-based techniques are presented for refining an initial classification of failures. One of these techniques is based on the use of dendrograms, which are rooted trees used to represent the results of hierarchical cluster analysis. The second technique employs a classification tree constructed to recognize failed executions. With both techniques, the tree representation is used to guide the refinement process. We also report the results of experimentally evaluating these techniques on several subject programs.

An Empirical Study of Test Case Filtering Techniques Based on Exercising Information Flows

Some software defects trigger failures only when certain local or nonlocal program interactions occur. Such interactions are modeled by the closely related concepts of information flows, program dependences, and program slices. The latter concepts underlie a 78 variety of proposed test data adequacy criteria, and they form a potentially important basis for filtering existing test cases. We report the results of an empirical study of several test case filtering techniques that are based on exercising information flows. Both coverage-based and profile-distribution-based filtering techniques are considered. They are compared to filtering techniques based on exercising simpler program elements, such as basic blocks, branches, function calls, and call pairs, with respect to their effectiveness for revealing defects.