David Hovemeyer

Finding more null pointer bugs, but not too many

In the summer of 2006, the FindBugs project was challenged to improve the null pointer analysis in FindBugs so that we could find more null pointer bugs. In particular, we were challenged to try to do as well as a publicly available analysis by Reasoning, Inc on version 4.1.24 of Apache Tomcat. Reasonings report is a result of running their own static analysis tool and using manual auditing to remove false positives. Reasoning reported a total of 9 null pointer warnings in Tomcat 4.1.24, of which only 2 were reported by FindBugs 1.0. While we wanted to improve the analysis in FindBugs, we wanted to retain our current low level of false positives. As of result of the work presented in this paper, FindBugs now reports 4 of the 9 warnings in Tomcat, shows that one of the warnings reported by Reasoning is a false positive, and classifies the remaining 4 as being dependent on the feasibility of a particular path, which cannot be easier ascertained by a local examination of the source code. Moreover, we found 24 additional null pointer bugs in Tomcat that had been missed by Reasoning, and overall doubled the number of null pointer bugs found by FindBugs while improving the quality and significance of reported defects.

Evaluating and tuning a static analysis to find null pointer bugs

Using static analysis to detect memory access errors, such as null pointer dereferences, is not a new problem. However, much of the previous work has used rather sophisticated analysis techniques in order to detect such errors.In this paper we show that simple analysis techniques can be used to identify many such software defects, both in production code and in student code. In order to make our analysis both simple and effective, we use a non-standard analysis which is neither complete nor sound. However, we find that it is effective at finding an interesting class of software defects.We describe the basic analysis we perform, as well as the additional errors we can detect using techniques such as annotations and inter-procedural analysis.In studies of both production software and student projects, we find false positive rates of around 20% or less. In the student code base, we find that our static analysis techniques are able to pinpoint 50% to 80% of the defects leading to a null pointer exception at runtime.

Tracking defect warnings across versions

Various static analysis tools will analyze a software artifact in order to identify potential defects, such as misused APIs, race conditions and deadlocks, and security vulnerabilities. For a number of reasons, it is important to be able to track the occurrence of each potential defect over multiple versions of a software artifact understudy: in other words, to determine when warnings reported in multiple versions of the software all correspond the same underlying issue. One motivation for this capability is to remember decisions about code that has been reviewed and found to be safe despite the occurrence of a warning. Another motivation is constructing warning deltas between versions, showing which warnings are new, which have persisted,and which have disappeared. This allows reviewers to focus their efforts on inspecting new warnings. Finally, tracking warnings through a series of software versions reveals where potential defects are introduced and fixed, and how long they persist, exposing interesting trends and patterns.We will discuss two different techniques we have implemented in FindBugs (a static analysis tool to find bugs in Java programs) for tracking defects across versions, discuss their relative merits and how they can be incorporated into the software development process, and discuss the results of tracking defect warnings across Suns Java runtime library.

Improving your software using static analysis to find bugs

FindBugs looks for bugs in Java programs. It is based on the concept of bug patterns. A bug pattern is a code idiom that is often an error. Bug patterns arise for a variety of reasons, such as difficult language features, misunderstood API semantics, misunderstood invariants when code is modified during maintenance, garden variety mistakes: typos, use of the wrong boolean operator and simple mistakes such as typos.FindBugs uses static analysis to inspect Java bytecode for occurrences of bug patterns. We have found that FindBugs finds real errors in most Java software. Because its analysis is sometimes imprecise, FindBugs can report false warnings, which are warnings that do not indicate true errors. In practice, the rate of false warnings reported by FindBugs is generally lower than 50%, often much lower.

Atomic Instructions in Java

Atomic instructions atomically access and update one or more memory locations. Because they do not incur the overhead of lock acquisition or suspend the executing thread during contention, they may allow higher levels of concurrency on multiprocessors than lock-based synchronization. Wait-free data structures are an important application of atomic instructions, and extend these performance benefits to higher level abstractions such as queues. In type-unsafe languages such as C, atomic instructions can be expressed in terms of operations on memory addresses. However, type-safe languages such as Java do not allow manipulation of arbitrary memory locations. Adding support for atomic instructions to Java is an interesting but important challenge.In this paper we consider several ways to support atomic instructions in Java. Each technique has advantages and disadvantages. We propose idiom recognition as the technique we feel has the best combination of expressiveness and simplicity. We describe techniques for recognizing instances of atomic operation idioms in the compiler of a Java Virtual Machine, and converting such instances into code utilizing atomic machine instructions. In addition, we describe a runtime technique which ensures that the semantics of multithreaded Java [11] are preserved when atomic instructions and blocking synchronization are used in the same program. Finally, we present benchmark results showing that for concurrent queues, a wait-free algorithm implemented using atomic compare-and-swap instructions yields better scalability on a large multiprocessor than a queue implemented with lock-based synchronization.

Running on the bare metal with GeekOS

Undergraduate operating systems courses are generally taught using one of two approaches: abstract or concrete. In the abstract approach, students learn the concepts underlying operating systems theory, and perhaps apply them using user-level threads in a host operating system. In the concrete approach, students apply concepts by working on a real operating system kernel. In the purest manifestation of the concrete approach, students implement operating system projects that run on real hardware.GeekOS is an instructional operating system kernel which runs on real hardware. It provides the minimum functionality needed to schedule threads and control essential devices on an x86 PC. On this foundation, we have developed projects in which students build processes, semaphores, a multilevel feedback scheduler, paged virtual memory, a filesystem, and inter-process communication. We use the Bochs emulator for ease of development and debugging. While this approach (tiny kernel run on an emulator) is not new, we believe GeekOS goes further towards the goal of combining realism and simplicity than previous systems have.

Analyzing Student Work Patterns Using Programming Exercise Data

Web-based programming exercises are a useful way for students to practice and master essential concepts and techniques presented in introductory programming courses. Although these systems are used fairly widely, we have a limited understanding of how students use these systems, and what can be learned from the data collected by these systems. In this paper, we perform a preliminary exploratory analysis of data collected by the CloudCoder programming exercise system from five introductory courses taught in two programming languages across three colleges and universities. We explore a number of interesting correlations in the data that confirm existing hypotheses. Finally, and perhaps most importantly, we demonstrate the effectiveness and future potential of systems like CloudCoder to help us study novice programmers.

The Marmoset project: an automated snapshot, submission, and testing system

Marmoset is a framework for storing and testing student submissions to programming assignments. It gives students a limited ability to release test their code using an instructors private suite of test cases. This encourages them to start early and implement their own test cases. It also provides facilities for instructors to manage the grading process and gives researchers access to fine-grained snapshots of the student development process.

CloudCoder: building a community for creating, assigning, evaluating and sharing programming exercises (abstract only)

Automatically-tested online programming exercises can be useful in introductory programming courses as self-tests to accompany readings, for in-class assessment, for skills development, and to provide additional practice for students who need it. CloudCoder (http://cloudcoder.org) is an effort to build a community based on an open-source programming exercise system (currently supporting C, Java, and Python) tightly integrated with a repository of freely-redistributable programming exercises written and used by members of the community. The goal of the project is to make programming exercises easy and free to incorporate into any programming course.

Status report on JSR-305: annotations for software defect detection

Java Specification Request 305 defines a set of annotations that can understood by multiple static analysis tools. Rather than push the bleeding edge of static analysis, this JSR represents an attempt to satisfy different static analysis tool vendors and address the engineering issues required to make these annotations widely useful.