Joseph R. Ruthruff

Predicting accurate and actionable static analysis warnings: an experimental approach

Static analysis tools report software defects that may or may not be detected by other verification methods. Two challenges complicating the adoption of these tools are spurious false positive warnings and legitimate warnings that are not acted on. This paper reports automated support to help address these challenges using logistic regression models that predict the foregoing types of warnings from signals in the warnings and implicated code. Because examining many potential signaling factors in large software development settings can be expensive, we use a screening methodology to quickly discard factors with low predictive power and cost-effectively build predictive models. Our empirical evaluation indicates that these models can achieve high accuracy in predicting accurate and actionable static analysis warnings, and suggests that the models are competitive with alternative models built without screening.

An empirical study of fault localization for end-user programmers

End users develop more software than any other group of programmers, using software authoring devices such as e-mail filtering editors, by-demonstration macro builders, and spreadsheet environments. Despite this, there has been little research on finding ways to help these programmers with the dependability of their software. We have been addressing this problem in several ways, one of which includes supporting end-user debugging activities through fault localization techniques. This paper presents the results of an empirical study conducted in an end-user programming environment to examine the impact of two separate factors in fault localization techniques that affect technique effectiveness. Our results shed new insights into fault localization techniques for end-user programmers and the factors that affect them, with significant implications for the evaluation of those techniques.

Strategies and behaviors of end-user programmers with interactive fault localization

End-user programmers are writing an unprecedented number of programs, due in large part to the significant effort put forth to bring programming power to end users. Unfortunately, this effort has not been supplemented by a comparable effort to increase the correctness of these often faulty programs. To address this need, we have been working towards bringing fault localization techniques to end users. In order to understand how end users are affected by and interact with such techniques, we conducted a think-aloud study, examining the interactive, human-centric ties between end-user debugging and a fault localization technique. Our results provide insights into the contributions such techniques can make to an interactive end-user debugging process.

Interactive, visual fault localization support for end-user programmers

End-user programmers are writing an unprecedented number of programs, primarily using languages and environments that incorporate a number of interactive and visual programming techniques. To help these users debug these programs, we have developed an entirely visual, interactive approach to fault localization. This paper presents the approach. We also present the results of a think-aloud study that examined interactive, human-centric issues that arise in end-user debugging using a fault localization approach. Our results provide insights into the contributions such approaches can make to the end-user debugging process.

Garbage in, garbage out? An empirical look at oracle mistakes by end-user programmers

End-user programmers, because they are human, make mistakes. However, past research has not considered how visual end-user debugging devices could be designed to ameliorate the effects of mistakes. This paper empirically examines oracle mistakes - mistakes users make about which values are right and which are wrong - to reveal differences in how different types of oracle mistakes impact the quality of visual feedback about bugs. We then consider the implications of these empirical results for designers of end-user software engineering environments.

Experimental program analysis: a new program analysis paradigm

Program analysis techniques are used by software engineers to deduce and infer characteristics of software systems. Recent research has suggested that a new form of program analysis technique can be created by incorporating characteristics of experimentation into analyses. This paper reports the results of research exploring this suggestion. Building on principles and methodologies underlying the use of experimentation in other fields, we provide descriptive and operational definitions of experimental program analysis, illustrate them by example, and describe several differences between experimental program analysis and experimentation in other fields. We show how the use of an experimental program analysis paradigm can help researchers identify limitations of analysis techniques, improve existing experimental program analysis techniques, and create new experimental program analysis techniques.

Interactive fault localization techniques in a spreadsheet environment

End-user programmers develop more software than any other group of programmers, using software authoring devices such as multimedia simulation builders, e-mail filtering editors, by-demonstration macro builders, and spreadsheet environments. Despite this, there has been only a little research on finding ways to help these programmers with the dependability of the software they create. We have been working to address this problem in several ways, one of which includes supporting end-user debugging activities through interactive fault localization techniques. This paper investigates fault localization techniques in the spreadsheet domain, the most common type of end-user programming environment. We investigate a technique previously described in the research literature and two new techniques. We present the results of an empirical study to examine the impact of two individual factors on the effectiveness of fault localization techniques. Our results reveal several insights into the contributions such techniques can make to the end-user debugging process and highlight key issues of interest to researchers and practitioners who may design and evaluate future fault localization techniques.

Six challenges in supporting end-user debugging

This paper summarizes six challenges in end-user programming that can impact the debugging efforts of end users. These challenges have been derived through our experiences and empirical investigation of interactive fault localization techniques in the spreadsheet paradigm. Our contributions reveal several insights into debugging techniques for end-user programmers, particularly fault localization techniques, that can help guide the direction of future end-user software engineering research.

Experimental program analysis

Program analysis techniques are used by software engineers to deduce and infer characteristics of software systems. Recent research has suggested that certain program analysis techniques can be formulated as formal experiments. This article reports the results of research exploring this suggestion. Building on principles and methodologies underlying the use of experimentation in other fields, we provide descriptive and operational definitions of experimental program analysis, illustrate them by example, and describe several differences between experimental program analysis and experimentation in other fields. We also explore the applicability of experimental program analysis to three software engineering problems: program transformation, program debugging, and program understanding. Our findings indicate that experimental program analysis techniques can provide new and potentially improved solutions to these problems, and suggest that experimental program analysis offers a promising new direction for program analysis research.

Experimental program analysis: a new paradigm for program analysis

Program analysis techniques are used by software engineers to deduce and infer targeted characteristics of software systems for tasks such as testing, debugging, maintenance, and program comprehension. Recently, some program analysis techniques have been designed to leverage characteristics of traditional experimentation in order to analyze software systems. We believe that the use of experimentation for program analysis constitutes a new program analysis paradigm: experimental program analysis. This research seeks to accomplish four goals: to precisely define experimental program analysis, to provide a means for classifying experimental program analysis techniques, to identify existing experimental program analysis techniques in the research literature, and to enhance the use of experimental program analysis by improving existing, and by creating new, experimental program analysis techniques.