Yudi Zheng

DiSL: a domain-specific language for bytecode instrumentation

Many dynamic analysis tools for programs written in managed languages such as Java rely on bytecode instrumentation. Tool development is often tedious because of the use of low-level bytecode manipulation libraries. While aspect-oriented programming (AOP) offers high-level abstractions to concisely express certain dynamic analyses, the join point model of mainstream AOP languages such as AspectJ is not well suited for many analysis tasks and the code generated by weavers in support of certain language features incurs high overhead. In this paper we introduce DiSL (domain-specific language for instrumentation), a new language especially designed for dynamic program analysis. DiSL offers an open join point model where any region of bytecodes can be a shadow, synthetic local variables for efficient data passing, efficient access to comprehensive static and dynamic context information, and weave-time execution of user-defined static analysis code. We demonstrate the benefits of DiSL with a case study, recasting an existing dynamic analysis tool originally implemented in AspectJ. We show that the DiSL version offers better code coverage, incurs significantly less overhead, and eases the integration of new analysis features that could not be expressed in AspectJ.

Turbo DiSL: partial evaluation for high-level bytecode instrumentation

Bytecode instrumentation is a key technique for the implementation of dynamic program analysis tools such as profilers and debuggers. Traditionally, bytecode instrumentation has been supported by low-level bytecode engineering libraries that are difficult to use. Recently, the domain-specific aspect language DiSL has been proposed to provide high-level abstractions for the rapid development of efficient bytecode instrumentations. While DiSL supports user-defined expressions that are evaluated at weave-time, the DiSL programming model requires these expressions to be implemented in separate classes, thus increasing code size and impairing code readability and maintenance. In addition, the DiSL weaver may produce a significant amount of dead code, which may impair some optimizations performed by the runtime. In this paper we introduce Turbo, a novel partial evaluator for DiSL, which processes the generated instrumentation code, performs constant propagation, conditional reduction, and pattern-based code simplification, and executes pure methods at weave-time. With Turbo, it is often unnecessary to wrap expressions for evaluation at weave-time in separate classes, thus simplifying the programming model. We present Turbos partial evaluation algorithm and illustrate its benefits with several case studies. We evaluate the impact of Turbo on weave-time performance and on runtime performance of the instrumented application.

ShadowVM: robust and comprehensive dynamic program analysis for the java platform

Dynamic analysis tools are often implemented using instrumentation, particularly on managed runtimes including the Java Virtual Machine (JVM). Performing instrumentation robustly is especially complex on such runtimes: existing frameworks offer limited coverage and poor isolation, while previous work has shown that apparently innocuous instrumentation can cause deadlocks or crashes in the observed application. This paper describes ShadowVM, a system for instrumentation-based dynamic analyses on the JVM which combines a number of techniques to greatly improve both isolation and coverage. These centre on the offload of analysis to a separate process; we believe our design is the first system to enable genuinely full bytecode coverage on the JVM. We describe a working implementation, and use a case study to demonstrate its improved coverage and to evaluate its runtime overhead.

A programming model and framework for comprehensive dynamic analysis on Android

The multi-process architecture of Android applications combined with the lack of suitable APIs make dynamic program analysis (DPA) on Android challenging and unduly difficult. Existing analysis tools and frameworks are tailored mainly to the needs of security-related analyses and are not flexible enough to support the development of generic DPA tools. In this paper we present a framework that, besides providing the fundamental support for the development of DPA tools for Android, enables development of cross-platform analyses that can be applied to applications targeting the Android and Java platforms. The framework provides a convenient high-level programming model, flexible instrumentation support, and strong isolation of the base program from the analysis. To boost developer productivity, the framework retains Java as the main development language, while seamless integration with the platform overcomes the recurring obstacles hindering development of DPA tools for Android. We evaluate the framework on two diverse case studies, demonstrating key concepts, the flexibility of the framework, and analysis portability.

A comprehensive toolchain for workload characterization across JVM languages

The Java Virtual Machine (JVM) today hosts implementations of numerous languages. To achieve high performance, JVM implementations rely on heuristics in choosing compiler optimizations and adapting garbage collection behavior. Historically, these heuristics have been tuned to suit the dynamics of Java programs only. This leads to unnecessarily poor performance in case of non-Java languages, which often exhibit systematic differences in workload behavior. Dynamic metrics characterizing the workload help to identify and quantify useful optimizations, but so far, no cohesive suite of metrics has adequately covered properties that vary systematically between Java and non-Java workloads. We present a suite of such metrics, justifying our choice with reference to a range of guest languages. These metrics are implemented on a common portable infrastructure which ensures ease of deployment and customization.

An Automation-Assisted Empirical Study on Lock Usage for Concurrent Programs

Nowadays concurrent programs are becoming more and more important with the development of hardware and network technologies. However, it is not easy for programmers to write reliable concurrent programs. Concurrency characteristics such as thread-interleaving make it difficult to debug or maintain concurrent programs. Although there are lots of research work on concurrency such as multi-thread testing tools, concurrent program verification and data race detection, all of them leave open problems. For instance, some are not scalable enough for large real world applications and some may report false warnings. Since locks are widely used to protect shared memory, it is beneficial for both programmers and tool designers in all fields to have a good understanding of common lock usage patterns in real world concurrent programs. This paper reports an empirical study on lock usage in concurrent programs. It is based on our automatic lock analysis tool called LUPA. The study analyzes how lock is used in concurrent programs and how lock usage changes throughout the product environment. In this study, four representative concurrent programs (Apache httpd, Mysql, Aget, Pbzip2) are selected, of which both lock manifestation and lock usage pattern in different versions are studied. This study reveals some interesting findings including but not limited to: (1) about 80.5% of the lock related functions acquire only one lock, (2) simple lock patterns account for 54.5% of all lock usage in real world applications, (3) only 12 out of 527 detected patterns belong to condition lock pattern which may lead to vulnerabilities easily, (4) only 0.65% of the functions are lock related. Additionally, a potential bug caused by problematic locking pattern is found.

Java Bytecode Instrumentation Made Easy: The DiSL Framework for Dynamic Program Analysis

Many software development tools (e.g., profilers, debuggers, testing tools) and frameworks (e.g., aspect weavers) are based on bytecode instrumentation techniques. While there are many low-level bytecode manipulation libraries that support the development of such tools and frameworks, they typically provide only low-level abstractions and require detailed knowledge of the Java Virtual Machine. In addition, they often lack the necessary infrastructure for load-time instrumentation with complete bytecode coverage to ensure that each method with a bytecode representation gets instrumented. In this paper we give an introduction to DiSL, a domain-specific aspect language and framework for bytecode instrumentation that reconciles high expressiveness of the language, high level of abstraction, and efficiency of the generated code. We illustrate the strengths of DiSL with a concrete analysis as a case study. The DiSL framework is open-source and has been successfully used in several research projects.

Enabling modularity and re-use in dynamic program analysis tools for the java virtual machine

Dynamic program analysis tools based on code instrumentation serve many important software engineering tasks such as profiling, debugging, testing, program comprehension, and reverse engineering. Unfortunately, constructing new analysis tools is unduly difficult, because existing frameworks offer little or no support to the programmer beyond the incidental task of instrumentation. We observe that existing dynamic analysis tools re-address recurring requirements in their essential task: maintaining state which captures some property of the analysed program. This paper presents a general architecture for dynamic program analysis tools which treats the maintenance of analysis state in a modular fashion, consisting of mappers decomposing input events spatially, and updaters aggregating them over time. We show that this architecture captures the requirements of a wide variety of existing analysis tools.

Dynamic optimization of bytecode instrumentation

Accuracy, completeness, and performance are all major concerns in the context of dynamic program analysis. Emphasizing one of these factors may compromise the other factors. For example, improving completeness of an analysis may seriously impair performance. In this paper, we present an analysis model and a framework that enables reducing analysis overhead at runtime through adaptive instrumentation of the base program. Our approach targets analyses implemented with code instrumentation techniques on the Java platform. Overhead reduction is achieved by removing instrumentation from code locations that are considered unimportant for the analysis results, thereby avoiding execution of analysis code for those locations. For some analyses, our approach preserves result accuracy and completeness. For other analyses, accuracy and completeness may be traded for a major performance improvement. In this paper, we explore accuracy, completeness, and performance of our approach with two concrete analyses as case studies.

DiSL: an extensible language for efficient and comprehensive dynamic program analysis

Dynamic program analysis tools support numerous software engineering tasks, including profiling, debugging, and reverse engineering. Prevailing techniques for building dynamic analysis tools are based on low-level abstractions that make tool development tedious, error-prone, and expensive. To simplify the development of dynamic analysis tools, some researchers promoted the use of aspect-oriented programming (AOP). However, as mainstream AOP languages have not been designed to meet the requirements of dynamic analysis, the success of using AOP in this context remains limited. For example, in AspectJ, join points that are important for dynamic program analysis (e.g., the execution of bytecodes or basic blocks of code) are missing, access to reflective dynamic join~point information is expensive, data passing between woven advice in local variables is not supported, and the mixing of low-level bytecode instrumentation and high-level AOP code is not foreseen. In this talk, we present DiSL [1], a new domain-specific aspect language for bytecode instrumentation. DiSL uses Java annotation syntax such that standard Java compilers can be used for compiling DiSL code. The language features an open join point model, novel constructs inspired by weave-time evaluation of conditional join~points and by staged execution, and access to custom static and dynamic context information. Moreover, the DiSL weaver guarantees complete bytecode coverage. We have implemented several dynamic analysis tools in DiSL, including profilers for the inter- and intra-procedural control flow, debuggers, dynamic metrics collectors integrated in the Eclipse IDE to augment the static source views with dynamic information, and tools for workload characterization. These tools are concise and perform equally well as implementations using low-level techniques. DiSL has also been conceived as an intermediate language for future domain-specific analysis languages, as well as for AOP languages.