Ciera Jaspan

Statically checking API protocol conformance with mined multi-object specifications

Programmers using an API often must follow protocols that specify when it is legal to call particular methods. Several techniques have been proposed to find violations of such protocols based on mined specifications. However, existing techniques either focus on single-object protocols or on particular kinds of bugs, such as missing method calls. There is no practical technique to find multi-object protocol bugs without a priori known specifications. In this paper, we combine a dynamic analysis that infers multi-object protocols and a static checker of API usage constraints into a fully automatic protocol conformance checker. The combined system statically detects illegal uses of an API without human-written specifications. Our approach finds 41 bugs and code smells in mature, real-world Java programs with a true positive rate of 51%. Furthermore, we show that the analysis reveals bugs not found by state of the art approaches.

Checking framework interactions with relationships

Many constraints between object oriented frameworks and plugins can be defined by specifications that describe how multiple objects relate and change their relationships over time. A tool can statically check a plugins conformance to these specifications in a cost-effective manner and can add benefit or even replace custom tools for plugin development.

Checking Framework Interactions with Relationships

Software frameworks impose constraints on how plugins may interact with them. Many of these constraints involve multiple objects, are temporal, and depend on runtime values. Additionally, they are difficult to specify because they are often extrinsic and may break behavioral subtyping. This work introduces relationships as an abstraction for specifying framework constraints in FUSION (Framework Usage SpecificatIONs), and it presents a formal description and implementation of a static analysis to find constraint violations in plugin code. We define three variants of this analysis: one is sound, one is complete, and a pragmatic variant that balances these tradeoffs. We prove soundness and completeness for the appropriate variants, and we show that the pragmatic variant can effectively check constraints from real-world programs.

Understanding the value of program analysis tools

It is difficult to determine the cost effectiveness of program analysis tools because we cannot evaluate them in the same environment where we will be using the tool. Tool evaluations are usually run on mature, stable code after it has passed developer testing. However, program analysis tools are usually run on unstable code, and some tools are meant to run right after compilation. Naturally, the results of the evaluation are not comparable to the true contribution of the tool. This leaves program analysis tool evaluations being very subjective and usually dependent on the evaluators intuition. While we could not solve this problem, we suggest techniques to make the evaluations more objective. We started by making enforcement-based customizations of the tool to be evaluated. When we evaluate a tool, we used a comparative evaluation technique to make the ROI analysis more objective. We also show how to use coverage models to select several tools when they each find different kinds of issues. Finally, we suggest that the tool vendors include features that assist us with a continuous evaluation of the tool as it runs in our software process.

Checking semantic usage of frameworks

Software frameworks are difficult for plugin developers to use, even when they are well designed and documented. Some of these difficulties stem from the many constraints that frameworks impose on plugin code. These constraints might restrict operations from being called on certain objects, or they might restrict how long an object is available. Additionally, the constraints are relative to the current context of the plugin, and they can involve multiple, interacting framework objects. This paper proposes a lightweight specification system and analysis to check plugins from a semantic perspective, rather than a purely structural view.

Are object protocols burdensome?: an empirical study of developer forums

Object protocols are a commonly studied research problem, but there is little known about their usability in practice. In particular, there is little research to show that object protocols cause difficulty for developers. In this work, we use community forums to find empirical evidence that object protocols are burdensome for developers. We analyzed 427 threads from the Spring and ASP.NET forums and discovered that 69 were on a protocol violation. We found that violations of protocols result in unusual runtime behavior rather than exceptions in 45% of our threads, that questions took an average of 62 hours to resolve, and that even though 54% of questions were repeated violations of similar protocols, the manifestation of the violation at runtime was different enough that developers could not search for similar questions.

Error Reporting Logic

When a system fails to meet its specification, it can be difficult to find the source of the error and determine how to fix it. In this paper, we introduce error reporting logic (ERL), an algorithm and tool that produces succinct explanations for why a target system violates a specification expressed in first order predicate logic. ERL analyzes the specification to determine which parts contributed to the failure, and it displays an error message specific to those parts. Additionally, ERL uses a heuristic to determine which object in the target system is responsible for the error. Results from a small user study suggest that the combination of a more focused error message and a responsible object for the error helps users to find the failure in the system more effectively. The study also yielded insights into how the users find and fix errors that may guide future research.

Checking framework plugins

Software frameworks contain many internal constraints which plugins cannot break. These constraints are relative to the context of the plugin, and they can involve multiple framework objects. This work creates a lightweight mechanism specifying semantic constraints on the framework and checking plugins against this specification.

Verifying configuration files

Configuration files are commonly used to create applications by gluing together new or existing components. However, components may be missing, misconfigured, or connected improperly, resulting in exceptions and unusual runtime behavior. This research contributes a mechanism to statically verify configuration files alongside the component code.

Retrieving relationships from declarative files

Software frameworks frequently require developers to interact with them using both OO-based languages and declarative files. The frameworks also require that these declarative files are consistent with the OO code, but it is currently difficult to check this in a generic way. Relationships have shown to be useful for checking constraints within OO code, and as a declarative file is essentially a static list of relationships, they may also be useful for checking constraints within declarative files. However, retrieving relationships from these files has many potential problems. In this paper, we show why it would be useful to retrieve relationships from declarative files, and we explore the hurdles that a solution will have to overcome.