Brian Burg

An analysis of the dynamic behavior of JavaScript programs

The JavaScript programming language is widely used for web programming and, increasingly, for general purpose computing. As such, improving the correctness, security and performance of JavaScript applications has been the driving force for research in type systems, static analysis and compiler techniques for this language. Many of these techniques aim to reign in some of the most dynamic features of the language, yet little seems to be known about how programmers actually utilize the language or these features. In this paper we perform an empirical study of the dynamic behavior of a corpus of widely-used JavaScript programs, and analyze how and why the dynamic features are used. We report on the degree of dynamism that is exhibited by these JavaScript programs and compare that with assumptions commonly made in the literature and accepted industry benchmark suites.

Interactive record/replay for web application debugging

During debugging, a developer must repeatedly and manually reproduce faulty behavior in order to inspect different facets of the programs execution. Existing tools for reproducing such behaviors prevent the use of debugging aids such as breakpoints and logging, and are not designed for interactive, random-access exploration of recorded behavior. This paper presents Timelapse, a tool for quickly recording, reproducing, and debugging interactive behaviors in web applications. Developers can use Timelapse to browse, visualize, and seek within recorded program executions while simultaneously using familiar debugging tools such as breakpoints and logging. Testers and end-users can use Timelapse to demonstrate failures in situ and share recorded behaviors with developers, improving bug report quality by obviating the need for detailed reproduction steps. Timelapse is built on Dolos, a novel record/replay infrastructure that ensures deterministic execution by capturing and reusing program inputs both from the user and from external sources such as the network. Dolos introduces negligible overhead and does not interfere with breakpoints and logging. In a small user evaluation, participants used Timelapse to accelerate existing reproduction activities, but were not significantly faster or more successful in completing the larger tasks at hand. Together, the Dolos infrastructure and Timelapse developer tool support systematic bug reporting and debugging practices.

The Eval That Men Do

Transforming text into executable code with a function such as JavaScript’s eval endows programmers with the ability to extend applications, at any time, and in almost any way they choose. But, this expressive power comes at a price: reasoning about the dynamic behavior of programs that use this feature becomes challenging. Any ahead-of-time analysis, to remain sound, is forced to make pessimistic assumptions about the impact of dynamically created code. This pessimism affects the optimizations that can be applied to programs and significantly limits the kinds of errors that can be caught statically and the security guarantees that can be enforced. A better understanding of how eval is used could lead to increased performance and security. This paper presents a large-scale study of the use of eval in JavaScript-based web applications. We have recorded the behavior of 337 MB of strings given as arguments to 550,358 calls to the eval function exercised in over 10,000 web sites. We provide statistics on the nature and content of strings used in eval expressions, as well as their provenance and data obtained by observing their dynamic behavior.

On the use of delta debugging to reduce recordings and facilitate debugging of web applications

Recording the sequence of events that lead to a failure of a web application can be an effective aid for debugging. Nevertheless, a recording of an event sequence may include many events that are not related to a failure, and this may render debugging more difficult. To address this problem, we have adapted Delta Debugging to function on recordings of web applications, in a manner that lets it identify and discard portions of those recordings that do not influence the occurrence of a failure. We present the results of three empirical studies that show that (1) recording reduction can achieve significant reductions in recording size and replay time on actual web applications obtained from developer forums, (2) reduced recordings do in fact help programmers locate faults significantly more efficiently as, and no less effectively than non-reduced recordings, and (3) recording reduction produces even greater reductions on larger, more complex applications.

Explaining Visual Changes in Web Interfaces

Web developers often want to repurpose interactive behaviors from third-party web pages, but struggle to locate the specific source code that implements the behavior. This task is challenging because developers must find and connect all of the non-local interactions between event-based JavaScript code, declarative CSS styles, and web page content that combine to express the behavior. The Scry tool embodies a new approach to locating the code that implements interactive behaviors. A developer selects a page element; whenever the element changes, Scry captures the rendering engines inputs (DOM, CSS) and outputs (screenshot) for the element. For any two captured element states, Scry can compute how the states differ and which lines of JavaScript code were responsible. Using Scry, a developer can locate an interactive behaviors implementation by picking two output states; Scry indicates the JavaScript code directly responsible for their differences.

1st international workshop on live programming (LIVE 2013)

Live programming is an idea espoused by programming environments from the earliest days of computing (such as Lisp machines and SmallTalk) but have since lain dormant. Recently, the prevalence of asynchronous feedback in programming languages such as Javascript and advances in visualizations and user interfaces have lead to a resurgence of live programming in online education communities (such as Khan Academy) and in experimental IDEs (such as LightTable). The LIVE 2013 workshop includes 11 papers describing visions, implementations, mashups, and new directions of live programming environments. The participants include both practitioners of live coding and researchers in programming languages and software engineering. Finally, several demos curated on the live workshop page are presented.

Facilitating debugging of web applications through recording reduction

Recording the sequence of events that lead to a failure of a web application can be an effective aid for debugging. Users can send a recording that results in a failure to a web application’s developer. The developer can then replay the recording, reproduce the failure, and find the fault(s) that cause it. Developers can do the same thing when faced with faults encountered in web applications in-house. A recording of an event sequence, however, may include many events that are not related to a failure, and this may render debugging more difficult. To address this problem, we have adapted Delta Debugging to function on recordings of web applications, in a manner that lets it identify and discard portions of those recordings that do not influence the occurrence of a failure, The resulting recording reduction technique can enable developers to localize faults based on reduced recordings instead of larger unreduced recordings, potentially reducing the amount of time and effort required to locate faults. We present the results of four empirical studies of our approach, in which we apply it to recordings created by Selenium IDE. In our first study we applied our technique to 30 faulty web applications obtained from developer forums, and showed that our technique could achieve significant reductions in recording size and replay time on these applications. In our second study we explored whether programmers could benefit from the use of reduced recordings when attempting to locate faults, and showed that our technique did increase their efficiency and effectiveness. In our third study we explored the scalability of our approach by applying it to substantially larger, more complex applications, and found that the approach worked even better on these larger applications than on the first set of smaller ones studied. In our fourth study we considered whether programmers working with two of these larger applications, who had more direct experience with the applications and the use of recordings and debugging could benefit from our technique. We found that the technique improved their efficiency and effectiveness, and the degree of improvement was even larger than that observed in our second study. Overall, these results suggest that recording reduction may be useful as means for helping programmers debug web applications.

Answering program understanding questions on demand with task-specific runtime information

When debugging or otherwise understanding software, developers frequently ask questions such as “how did this function get called?” and “where did this value come from?”. Runtime data is a valuable yet underused resource for answering these questions. There are at least four major barriers to answering questions with runtime data: 1) Deciding what runtime data should be collected 2) Deciding how to collect relevant runtime data 3) Aggregating and summarizing collected data 4) Interpreting the results to make conclusions.

RETRACTED ARTICLE: Facilitating debugging of web applications through recording reduction: A family of empirical studies

Recording the sequence of events that lead to a failure of a web application can be an effective aid for debugging. Users can send a recording that results in a failure to a web application’s developer. The developer can then replay the recording, reproduce the failure, and find the fault(s) that cause it. Developers can do the same thing when faced with faults encountered in web applications in-house. A recording of an event sequence, however, may include many events that are not related to a failure, and this may render debugging more difficult. To address this problem, we have adapted Delta Debugging to function on recordings of web applications, in a manner that lets it identify and discard portions of those recordings that do not influence the occurrence of a failure, The resulting recording reduction technique can enable developers to localize faults based on reduced recordings instead of larger unreduced recordings, potentially reducing the amount of time and effort required to locate faults. We present the results of four empirical studies of our approach, in which we apply it to recordings created by Selenium IDE. In our first study we applied our technique to 30 faulty web applications obtained from developer forums, and showed that our technique could achieve significant reductions in recording size and replay time on these applications. In our second study we explored whether programmers could benefit from the use of reduced recordings when attempting to locate faults, and showed that our technique did increase their efficiency and effectiveness. In our third study we explored the scalability of our approach by applying it to substantially larger, more complex applications, and found that the approach worked even better on these larger applications than on the first set of smaller ones studied. In our fourth study we considered whether programmers working with two of these larger applications, who had more direct experience with the applications and the use of recordings and debugging could benefit from our technique. We found that the technique improved their efficiency and effectiveness, and the degree of improvement was even larger than that observed in our second study. Overall, these results suggest that recording reduction may be useful as means for helping programmers debug web applications.

RETRACTED ARTICLE: Facilitating debugging of web applications through recording reduction

Recording the sequence of events that lead to a failure of a web application can be an effective aid for debugging. Users can send a recording that results in a failure to a web application’s developer. The developer can then replay the recording, reproduce the failure, and find the fault(s) that cause it. Developers can do the same thing when faced with faults encountered in web applications in-house. A recording of an event sequence, however, may include many events that are not related to a failure, and this may render debugging more difficult. To address this problem, we have adapted Delta Debugging to function on recordings of web applications, in a manner that lets it identify and discard portions of those recordings that do not influence the occurrence of a failure, The resulting recording reduction technique can enable developers to localize faults based on reduced recordings instead of larger unreduced recordings, potentially reducing the amount of time and effort required to locate faults. We present the results of four empirical studies of our approach, in which we apply it to recordings created by Selenium IDE. In our first study we applied our technique to 30 faulty web applications obtained from developer forums, and showed that our technique could achieve significant reductions in recording size and replay time on these applications. In our second study we explored whether programmers could benefit from the use of reduced recordings when attempting to locate faults, and showed that our technique did increase their efficiency and effectiveness. In our third study we explored the scalability of our approach by applying it to substantially larger, more complex applications, and found that the approach worked even better on these larger applications than on the first set of smaller ones studied. In our fourth study we considered whether programmers working with two of these larger applications, who had more direct experience with the applications and the use of recordings and debugging could benefit from our technique. We found that the technique improved their efficiency and effectiveness, and the degree of improvement was even larger than that observed in our second study. Overall, these results suggest that recording reduction may be useful as means for helping programmers debug web applications.