Shay Artzi

Finding bugs in dynamic web applications

Web script crashes and malformed dynamically-generated Web pages are common errors, and they seriously impact usability of Web applications. Current tools for Web-page validation cannot handle the dynamically-generated pages that are ubiquitous on todays Internet. In this work, we apply a dynamic test generation technique, based on combined concrete and symbolic execution, to the domain of dynamic Web applications. The technique generates tests automatically, uses the tests to detect failures, and minimizes the conditions on the inputs exposing each failure, so that the resulting bug reports are small and useful in finding and fixing the underlying faults. Our tool Apollo implements the technique for PHP. Apollo generates test inputs for the Web application, monitors the application for crashes, and validates that the output conforms to the HTML specification. This paper presents Apollos algorithms and implementation, and an experimental evaluation that revealed 214 faults in 4 PHP Web applications.

A framework for automated testing of javascript web applications

Current practice in testing JavaScript web applications requires manual construction of test cases, which is difficult and tedious. We present a framework for feedback-directed automated test generation for JavaScript in which execution is monitored to collect information that directs the test generator towards inputs that yield increased coverage. We implemented several instantiations of the framework, corresponding to variations on feedback-directed random testing, in a tool called Artemis. Experiments on a suite of JavaScript applications demonstrate that a simple instantiation of the framework that uses event handler registrations as feedback information produces surprisingly good coverage if enough tests are generated. By also using coverage information and read-write sets as feedback information, a slightly better level of coverage can be achieved, and sometimes with many fewer tests. The generated tests can be used for detecting HTML validity problems and other programming errors.

Finding Bugs in Web Applications Using Dynamic Test Generation and Explicit-State Model Checking

Web script crashes and malformed dynamically generated webpages are common errors, and they seriously impact the usability of Web applications. Current tools for webpage validation cannot handle the dynamically generated pages that are ubiquitous on todays Internet. We present a dynamic test generation technique for the domain of dynamic Web applications. The technique utilizes both combined concrete and symbolic execution and explicit-state model checking. The technique generates tests automatically, runs the tests capturing logical constraints on inputs, and minimizes the conditions on the inputs to failing tests so that the resulting bug reports are small and useful in finding and fixing the underlying faults. Our tool Apollo implements the technique for the PHP programming language. Apollo generates test inputs for a Web application, monitors the application for crashes, and validates that the output conforms to the HTML specification. This paper presents Apollos algorithms and implementation, and an experimental evaluation that revealed 673 faults in six PHP Web applications.

In vivo and in vitro tracking of erosion in biodegradable materials using non-invasive fluorescence imaging

The design of erodible biomaterials relies on the ability to program the in vivo retention time, which necessitates real-time monitoring of erosion. However, in vivo performance cannot always be predicted by traditional determination of in vitro erosion, and standard methods sacrifice samples or animals, preventing sequential measures of the same specimen. We harnessed non-invasive fluorescence imaging to sequentially follow in vivo material-mass loss to model the degradation of materials hydrolytically (PEG:dextran hydrogel) and enzymatically (collagen). Hydrogel erosion rates in vivo and in vitro correlated, enabling the prediction of in vivo erosion of new material formulations from in vitro data. Collagen in vivo erosion was used to infer physiologic in vitro conditions that mimic erosive in vivo environments. This approach enables rapid in vitro screening of materials, and can be extended to simultaneously determine drug release and material erosion from a drug-eluting scaffold, or cell viability and material fate in tissue-engineering formulations.

ReCrash: Making Software Failures Reproducible by Preserving Object States

It is very hard to fix a software failure without being able to reproduce it. However, reproducing a failure is often difficult and time-consuming. This paper proposes a novel technique, ReCrash, that generates multiple unit tests that reproduce a given program failure. During every execution of the target program, ReCrash stores partial copies of method arguments in memory. If the program fails (e.g., crashes), ReCrash uses the saved information to create unit tests reproducing the failure. We present ReCrashJ, an implementation of ReCrash for Java. ReCrashJ reproduced real crashes from Javac, SVNKit, Eclipsec, and BST. ReCrashJ is efficient, incurring 13%---64% performance overhead. If this overhead is unacceptable, then ReCrashJ has another mode that has negligible overhead until a crash occurs and 0%---1.7% overhead until the crash occurs for a second time, at which point the test cases are generated.

Automated repair of HTML generation errors in PHP applications using string constraint solving

PHP web applications routinely generate invalid HTML. Modern browsers silently correct HTML errors, but sometimes malformed pages render inconsistently, cause browser crashes, or expose security vulnerabilities. Fixing errors in generated pages is usually straightforward, but repairing the generating PHP program can be much harder. We observe that malformed HTML is often produced by incorrect constant prints, i.e., statements that print string literals, and present two tools for automatically repairing such HTML generation errors. PHPQuickFix repairs simple bugs by statically analyzing individual prints. PHPRepair handles more general repairs using a dynamic approach. Based on a test suite, the property that all tests should produce their expected output is encoded as a string constraint over variables representing constant prints. Solving this constraint describes how constant prints must be modified to make all tests pass. Both tools were implemented as an Eclipse plugin and evaluated on PHP programs containing hundreds of HTML generation errors, most of which our tools were able to repair automatically.

Object and reference immutability using Java generics

A compiler-checked immutability guarantee provides useful documentation, facilitates reasoning, and enables optimizations. This paper presents Immutability Generic Java (IGJ), a novel language extension that expresses immutability without changing Javas syntax by building upon Javas generics and annotation mechanisms. In IGJ, each class has one additional type parameter that is Immutable, Mutable, or ReadOnly. IGJ guarantees both reference immutability (only mutable references can mutate an object) and object immutability (an immutable reference points to an immutable object). IGJ is the first proposal for enforcing object immutability within Javas syntax and type system, and its reference immutability is more expressive than previous work. IGJ also permits covariant changes of type parameters in a type-safe manner, e.g., a readonly list of integers is a subtype of a readonly list of numbers. IGJ extends Javas type system with a few simple rules. We formalize this type system and prove it sound. Our IGJ compiler works by type-erasure and generates byte-code that can be executed on any JVM without runtime penalty.

Practical fault localization for dynamic web applications

We leverage combined concrete and symbolic execution and several fault-localization techniques to create a uniquely powerful tool for localizing faults in PHP applications. The tool automatically generates tests that expose failures, and then automatically localizes the faults responsible for those failures, thus overcoming the limitation of previous fault-localization techniques that a test suite be available upfront. The fault-localization techniques we employ combine variations on the Tarantula algorithm with a technique based on maintaining a mapping between statements and the fragments of output they produce. We implemented these techniques in a tool called Apollo, and evaluated them by localizing 75 randomly selected faults that were exposed by automatically generated tests in four PHP applications. Our findings indicate that, using our best technique, 87.7% of the faults under consideration are localized to within 1% of all executed statements, which constitutes an almost five-fold improvement over the Tarantula algorithm.

F4F: taint analysis of framework-based web applications

This paper presents F4F (Framework For Frameworks), a system for effective taint analysis of framework-based web applications. Most modern web applications utilize one or more web frameworks, which provide useful abstractions for common functionality. Due to extensive use of reflective language constructs in framework implementations, existing static taint analyses are often ineffective when applied to framework-based applications. While previous work has included ad hoc support for certain framework constructs, adding support for a large number of frameworks in this manner does not scale from an engineering standpoint. F4F employs an initial analysis pass in which both application code and configuration files are processed to generate a specification of framework-related behaviors. A taint analysis engine can leverage these specifications to perform a much deeper, more precise analysis of framework-based applications. Our specification language has only a small number of simple but powerful constructs, easing analysis engine integration. With this architecture, new frameworks can be handled with no changes to the core analysis engine, yielding significant engineering benefits. We implemented specification generators for several web frameworks and added F4F support to a state-of-the-art taint-analysis engine. In an experimental evaluation, the taint analysis enhanced with F4F discovered 525 new issues across nine benchmarks, a harmonic mean of 2.10X more issues per benchmark. Furthermore, manual inspection of a subset of the new issues showed that many were exploitable or reflected bad security practice.

HAMPI: a string solver for testing, analysis and vulnerability detection

Many automatic testing, analysis, and verification techniques for programs can effectively be reduced to a constraint-generation phase followed by a constraint-solving phase. This separation of concerns often leads to more effective and maintainable software reliability tools. The increasing efficiency of offthe-shelf constraint solvers makes this approach even more compelling. However, there are few effective and sufficiently expressive off-the-shelf solvers for string constraints generated by analysis of string-manipulating programs, and hence researchers end up implementing their own ad-hoc solvers. Thus, there is a clear need for an effective and expressive string-constraint solver that can be easily integrated into a variety of applications. To fulfill this need, we designed and implemented Hampi, an efficient and easy-to-use string solver. Users of the Hampi string solver specify constraints using membership predicate over regular expressions, context-free grammars, and equality/dis-equality between string terms. These terms are constructed out of string constants, bounded string variables, and typical string operations such as concatenation and substring extraction. Hampi takes such a constraint as input and decides whether it is satisfiable or not. If an input constraint is satisfiable, Hampi generates a satsfying assignment for the string variables that occur in it. We demonstrate Hampis expressiveness and efficiency by applying it to program analysis and automated testing: We used Hampi in static and dynamic analyses for finding SQL injection vulnerabilities in Web applications with hundreds of thousands of lines of code.We also used Hampi in the context of automated bug finding in C programs using dynamic systematic testing (also known as concolic testing). Hampis source code, documentation, and experimental data are available at http://people.csail.mit.edu/akiezun/hampi