Brittany Johnson

Why don't software developers use static analysis tools to find bugs?

Using static analysis tools for automating code inspections can be beneficial for software engineers. Such tools can make finding bugs, or software defects, faster and cheaper than manual inspections. Despite the benefits of using static analysis tools to find bugs, research suggests that these tools are underused. In this paper, we investigate why developers are not widely using static analysis tools and how current tools could potentially be improved. We conducted interviews with 20 developers and found that although all of our participants felt that use is beneficial, false positives and the way in which the warnings are presented, among other things, are barriers to use. We discuss several implications of these results, such as the need for an interactive mechanism to help developers fix defects.

Questions developers ask while diagnosing potential security vulnerabilities with static analysis

Security tools can help developers answer questions about potential vulnerabilities in their code. A better understanding of the types of questions asked by developers may help toolsmiths design more effective tools. In this paper, we describe how we collected and categorized these questions by conducting an exploratory study with novice and experienced software developers. We equipped them with Find Security Bugs, a security-oriented static analysis tool, and observed their interactions with security vulnerabilities in an open-source system that they had previously contributed to. We found that they asked questions not only about security vulnerabilities, associated attacks, and fixes, but also questions about the software itself, the social ecosystem that built the software, and related resources and tools. For example, when participants asked questions about the source of tainted data, their tools forced them to make imperfect tradeoffs between systematic and ad hoc program navigation strategies.

Compiler error notifications revisited: an interaction-first approach for helping developers more effectively comprehend and resolve error notifications

Error notifications and their resolutions, as presented by modern IDEs, are still cryptic and confusing to developers. We propose an interaction-first approach to help developers more effectively comprehend and resolve compiler error notifications through a conceptual interaction framework. We propose novel taxonomies that can serve as controlled vocabularies for compiler notifications and their resolutions. We use preliminary taxonomies to demonstrate, through a prototype IDE, how the taxonomies make notifications and their resolutions more consistent and unified.

From Quick Fixes to Slow Fixes: Reimagining Static Analysis Resolutions to Enable Design Space Exploration

Quick Fixes as implemented by IDEs today prioritize the speed of applying the fix as a primary criteria for success. In this paper, we argue that when tools over-optimize this criteria, such tools neglect other dimensions that are important to successfully applying a fix, such as being able to explore the design space of multiple fixes. This is especially true in cases where a fix only partially implements the intention of the developer. In this paper, we implement an extension to the FindBugs defect finding tool, called FixBugs, an interactive resolution approach within the Eclipse development environment that prioritizes other design criteria to the successful application of suggested fixes. Our empirical evaluation method of 12 developers suggests that FixBugs enables developers to explore alternative designs and balances the benefits of manual fixing with automated fixing, without having to compromise in either effectiveness or efficiency. Our analytic evaluation method with six usability experts identified trade-offs between FixBugs and Quick Fix, and suggests ways in which FixBugs and Quick Fix can offer complementary capabilities to better support developers.

A study on improving static analysis tools: why are we not using them?

Using static analysis tools for automating code inspections can be beneficial for software engineers. Despite the benefits of using static analysis tools, research suggests that these tools are underused. In this research, we propose to investigate why developers are not widely using static analysis tools and how current tools could potentially be improved to increase usage.

I heart hacker news: expanding qualitative research findings by analyzing social news websites

Grounded theory is an important research method in empirical software engineering, but it is also time consuming, tedious, and complex. This makes it difficult for researchers to assess if threats, such as missing themes or sample bias, have inadvertently materialized. To better assess such threats, our new idea is that we can automatically extract knowledge from social news websites, such as Hacker News, to easily replicate existing grounded theory research --- and then compare the results. We conduct a replication study on static analysis tool adoption using Hacker News. We confirm that even a basic replication and analysis using social news websites can offer additional insights to existing themes in studies, while also identifying new themes. For example, we identified that security was not a theme discovered in the original study on tool adoption. As a long-term vision, we consider techniques from the discipline of knowledge discovery to make this replication process more automatic.

A cross-tool communication study on program analysis tool notifications

Program analysis tools use notifications to communicate with developers, but previous research suggests that developers encounter challenges that impede this communication. This paper describes a qualitative study that identifies 10 kinds of challenges that cause notifications to miscommunicate with developers. Our resulting notification communication theory reveals that many challenges span multiple tools and multiple levels of developer experience. Our results suggest that, for example, future tools that model developer experience could improve communication and help developers build more accurate mental models.

Evaluating how static analysis tools can reduce code review effort

Peer code reviews are important for giving and receiving peer feedback, but the code review process is time consuming. Static analysis tools can help reduce reviewer effort by catching common mistakes prior to peer code review. Ideally, contributors would use static analysis tools prior to pull request submission so common mistakes could be addressed first, before invoking the reviewer. To explore the potential efficiency gains for peer reviewers, we explore the overlap between reviewer comments on pull requests and warnings from the PMD static analysis tool. In an empirical study of 274 comments from 92 pull requests on GitHub, we observed that PMD overlapped with nearly 16% of the reviewer comments, indicating a time benefit to the reviewer if static analyzers would have been used prior to pull request submission. Using the non-overlapping set of comments, we identify four additional rules that, if implemented, could further reduce reviewer effort.

Adapting program analysis tool notifications to the individual developer

There are a variety of tools available for developers to use in their IDEs. Research suggests, however, developers may not use these tools due to difficulty interpreting the output. My research explores the possibility of creating frameworks that enable more individualized program analysis tool notifications for increased usability. I propose creating models that represent what developers know about programming concepts using exisitng developer data. These models could then be used to inform tools on notifications the developer may or may not understand.

Enhancing tools' intelligence for improved program analysis tool usability

Program analysis tools can help developers produce high quality code by automating time-consuming tasks such as error-finding. Research has shown, however, that these tools are often not used by developers. Results from studies I have conducted provide insights into the difficulties programmers may encounter when using program analysis tools, leading to lower productivity and desire to use them. Based on these findings, my dissertation will research improving program analysis tools by enhancing tool intelligence with a programmer model that adapts notifications to programmers based on their experience with the concepts relevant to the notification.