Matthias Grimmer

Practical partial evaluation for high-performance dynamic language runtimes

Most high-performance dynamic language virtual machines duplicate language semantics in the interpreter, compiler, and runtime system. This violates the principle to not repeat yourself. In contrast, we define languages solely by writing an interpreter. The interpreter performs specializations, e.g., augments the interpreted program with type information and profiling information. Compiled code is derived automatically using partial evaluation while incorporating these specializations. This makes partial evaluation practical in the context of dynamic languages: It reduces the size of the compiled code while still compiling all parts of an operation that are relevant for a particular program. When a speculation fails, execution transfers back to the interpreter, the program re-specializes in the interpreter, and later partial evaluation again transforms the new state of the interpreter to compiled code. We evaluate our approach by comparing our implementations of JavaScript, Ruby, and R with best-in-class specialized production implementations. Our general-purpose compilation system is competitive with production systems even when they have been heavily optimized for the one language they support. For our set of benchmarks, our speedup relative to the V8 JavaScript VM is 0.83x, relative to JRuby is 3.8x, and relative to GNU R is 5x.

Dynamically composing languages in a modular way: supporting C extensions for dynamic languages

Many dynamic languages such as Ruby, Python and Perl offer some kind of functionality for writing parts of applications in a lower-level language such as C. These C extension modules are usually written against the API of an interpreter, which provides access to the higher-level languages internal data structures. Alternative implementations of the high-level languages often do not support such C extensions because implementing the same API as in the original implementations is complicated and limits performance. In this paper we describe a novel approach for modular composition of languages that allows dynamic languages to support C extensions through interpretation. We propose a flexible and reusable cross-language mechanism that allows composing multiple language interpreters, which run on the same VM and share the same form of intermediate representation - in this case abstract syntax trees. This mechanism allows us to efficiently exchange runtime data across different interpreters and also enables the dynamic compiler of the host VM to inline and optimize programs across multiple language boundaries. We evaluate our approach by composing a Ruby interpreter with a C interpreter. We run existing Ruby C extensions and show how our system executes combined Ruby and C modules on average over 3x faster than the conventional implementation of Ruby with native C extensions, and on average over 20x faster than an existing alternate Ruby implementation on the JVM (JRuby) calling compiled C extensions through a bridge interface. We demonstrate that cross-language inlining, which is not possible with native code, is performance-critical by showing how speedup is reduced by around 50% when it is disabled.

High-performance cross-language interoperability in a multi-language runtime

Programmers combine different programming languages because it allows them to use the most suitable language for a given problem, to gradually migrate existing projects from one language to another, or to reuse existing source code. However, existing cross-language mechanisms suffer from complex interfaces, insufficient flexibility, or poor performance. We present the TruffleVM, a multi-language runtime that allows composing different language implementations in a seamless way. It reduces the amount of required boiler-plate code to a minimum by allowing programmers to access foreign functions or objects by using the notation of the host language. We compose language implementations that translate source code to an intermediate representation (IR), which is executed on top of a shared runtime system. Language implementations use language-independent messages that the runtime resolves at their first execution by transforming them to efficient foreign-language-specific operations. The TruffleVM avoids conversion or marshaling of foreign objects at the language boundary and allows the dynamic compiler to perform its optimizations across language boundaries, which guarantees high performance. This paper presents an implementation of our ideas based on the Truffle system and its guest language implementations JavaScript, Ruby, and C.

Bringing low-level languages to the JVM: efficient execution of LLVM IR on Truffle

Although the Java platform has been used as a multi-language platform, most of the low-level languages (such as C, Fortran, and C++) cannot be executed efficiently on the JVM. We propose Sulong, a system that can execute LLVM-based languages on the JVM. By targeting LLVM IR, Sulong is able to execute C, Fortran, and other languages that can be compiled to LLVM IR. Sulong combines LLVM’s static optimizations with dynamic compilation to reach a peak performance that is near to the performance achievable with static compilers. For C benchmarks, Sulong’s peak runtime performance is on average 1.39× slower (0.79× to 2.45×) compared to the performance of executables compiled by Clang O3. For Fortran benchmarks, Sulong is 2.63× slower (1.43× to 4.96×) than the performance of executables compiled by GCC O3. This low overhead makes Sulong an alternative to Java’s native function interfaces. More importantly, it also allows other JVM language implementations to use Sulong for implementing their native interfaces.

An efficient native function interface for Java

We present an efficient and dynamic approach for calling native functions from within Java. Traditionally, programmers use the Java Native Interface (JNI) to call such functions. This paper introduces a new mechanism which we tailored specifically towards calling native functions from Java. We call it the Graal Native Function Interface (GNFI). It is faster than JNI in all relevant cases and more flexible because it avoids the JNI boiler-plate code. GNFI enables the user to directly invoke native code from Java applications. We describe how GNFI creates call stubs for native functions that a just-in-time (JIT) compiler can optimize and how we embed these stubs into Java code. We introduce different approaches for calling native functions from within compiled and interpreted Java code. In particular, we describe how our approach embeds the call stubs into a Java application so that the JIT-compiled code consists of a direct call to a native function. We evaluate the call overhead of GNFI. The measurements demonstrate a significant performance advantage of GNFI compared to JNI and the Java Native Access (JNA). Also, we evaluate our approach against JNI and JNA on a jblas matrix multiplication benchmark. The evaluation shows that GNFI outperforms JNI and JNA in compiled and interpreted mode by a factor of 1.9 in the best case.

Memory-safe Execution of C on a Java VM

In low-level languages such as C, spatial and temporal safety errors (e.g. buffer overflows or dangling pointer dereferences) are hard to find and can cause security vulnerabilities. Modern high-level languages such as Java avoid these problems by running programs on a virtual machine that provides automated memory management. In this paper we show how we can safely execute C code on top of a modern runtime (e.g., a Java Virtual Machine) by allocating all data on the managed heap. We reuse the memory management of the runtime, hence, we can ensure spatial and temporal safety with little effort. Nevertheless, we retain all characteristics that are typical for unsafe languages (such as pointer arithmetic, pointers into objects, or arbitrary type casts).We discuss how our approach complies with the C99 standard. Compared to an optimized unsafe execution of a C program (compiled with the GNU C compiler and all optimizations enabled) our approach has overhead of 15% on average (peak-performance).

An efficient approach for accessing C data structures from JavaScript

We present a novel approach for allowing JavaScript applications to access C data structures without performance overhead or additional boiler plate code. Dynamic languages such as JavaScript do not have a fixed memory layout for run-time data nor do they allow low-level memory accesses, which makes interoperability with languages such as C hard. Our approach allows JavaScript applications to transparently use C pointers as receivers of object accesses or array accesses, thus the programmer can access C data structures as if they were JavaScript objects. We describe how we modified an existing JavaScript interpreter so that it generates different access operations at run time depending on whether the receiver is a JavaScript object or a C pointer. We evaluated our prototype using benchmarks that measure array access performance. The evaluation of our JavaScript interpreter shows that using C arrays from within JavaScript is on average 27% faster than using regular JavaScript arrays and 19% faster than using Typed Arrays.

Sulong, and Thanks for All the Bugs: Finding Errors in C Programs by Abstracting from the Native Execution Model

In C, memory errors, such as buffer overflows, are among the most dangerous software errors; as we show, they are still on the rise. Current dynamic bug-finding tools that try to detect such errors are based on the low-level execution model of the underlying machine. They insert additional checks in an ad-hoc fashion, which makes them prone to omitting checks for corner cases. To address this, we devised a novel approach to finding bugs during the execution of a program. At the core of this approach is an interpreter written in a high-level language that performs automatic checks (such as bounds, NULL, and type checks). By mapping data structures in C to those of the high-level language, accesses are automatically checked and bugs discovered. We have implemented this approach and show that our tool (called Safe Sulong) can find bugs that state-of-the-art tools overlook, such as out-of-bounds accesses to the main function arguments.

Introspection for C and its Applications to Library Robustness

Context: In C, low-level errors, such as buffer overflow and use-after-free, are a major problem, as they cause security vulnerabilities and hard-to-find bugs. C lacks automatic checks, and programmers cannot apply defensive programming techniques because objects (e.g., arrays or structs) lack run-time information about bounds, lifetime, and types. Inquiry: Current approaches to tackling low-level errors include dynamic tools, such as bounds or type checkers, that check for certain actions during program execution. If they detect an error, they typically abort execution. Although they track run-time information as part of their runtimes, they do not expose this information to programmers. Approach: We devised an introspection interface that allows C programmers to access run-time information and to query object bounds, object lifetimes, object types, and information about variadic arguments. This enables library writers to check for invalid input or program states and thus, for example, to implement custom error handling that maintains system availability and does not terminate on benign errors. As we assume that introspection is used together with a dynamic tool that implements automatic checks, errors that are not handled in the application logic continue to cause the dynamic tool to abort execution. Knowledge: Using the introspection interface, we implemented a more robust, source-compatible version of the C standard library that validates parameters to its functions. The library functions react to otherwise undefined behavior; for example, they can detect lurking flaws, handle unterminated strings, check format string arguments, and set errno when they detect benign usage errors. Grounding: Existing dynamic tools maintain run-time information that can be used to implement the introspection interface, and we demonstrate its implementation in Safe Sulong, an interpreter and dynamic bug-finding tool for C that runs on a Java Virtual Machine and can thus easily expose relevant run-time information. Importance: Using introspection in user code is a novel approach to tackling the long-standing problem of low-level errors in C. As new approaches are lowering the performance overhead of run-time information maintenance, the usage of dynamic runtimes for C could become more common, which could ultimately facilitate a more widespread implementation of such an introspection interface.

Trace-based Register Allocation in a JIT Compiler

State-of-the-art dynamic compilers often use global approaches, like Linear Scan or Graph Coloring, for register allocation. These algorithms consider the complete compilation unit for allocation, which increases the complexity of the implementation (e.g., support for lifetime holes in Linear Scan) and potentially also affects compilation time. We propose a novel non-global algorithm, which splits a compilation unit into traces based on profiling feedback and subsequently performs register allocation within each trace individually. Traces reduce the problem size to a single linear code segment, which simplifies the problem a register allocator needs to solve. Additionally, we can apply different register allocation algorithms to each trace. We show that this non-global approach can achieve results competitive to global register allocation. We present an implementation of Trace Register Allocation based on the Graal VM and show an evaluation for common Java benchmarks. We demonstrate that performance of this non-global approach is within 3% (on AMD64) and 1% (on SPARC) of global Linear Scan register allocation.