Wim De Pauw

Visualizing the Execution of Java Programs

Jinsight is a tool for exploring a program’s run-time behavior visually. It is helpful for performance analysis, debugging, and any task in which you need to better understand what your Java program is really doing.

Modeling Object-Oriented Program Execution

This paper describes a way of organizing information about an object-oriented programs execution. The context is our system for visualizing that execution. The critical constraints are completeness, compactness, and efficient retrieval. We describe our design and how it meets these constraints.

Visualizing Reference Patterns for Solving Memory Leaks in Java

Many Java programmers believe they do not have to worry about memory management because of automatic garbage collection. In fact, many Java programs run out of memory unexpectedly after performing a number of operations. A memory leak in Java is caused when an object that is no longer needed cannot be reclaimed because another object is still referring to it. Memory leaks can be difficult to solve, since the complexity of most programs prevents us from manually verifying the validity of every reference. In this paper we show a new methodology for finding the causes of memory leaks. We have identified a basic memory leak scenario which fits many important cases. In this scenario, we allow the programmer to identify a period of time in which temporary objects are expected to be created and released. Using this information we are able to identify objects that persist beyond this period and the references which are holding on to them. Scaling this methodology to real-world systems brings additional challenges. We propose a novel combination of visual syntax and reference pattern extraction to manage this additional complexity. We also describe how these techniques can be applied to a wider class of memory problems, including the exploration of large data structures. These techniques have been implemented and have been proven successful on large projects.

Execution patterns for visualizing web services

Web Services are well on their way to becoming the Lingua Franca for distributed computing. Although tools for building and monitoring web services applications are more powerful and easier to use than ever, they do not yet fully address the horizontal complexity of mature applications built as large nets of interconnected web services. We present a pattern-based visualization that enables business owners, application designers, programmers, and operations staff to quickly understand the behavior of complex web services applications. We describe a novel pattern extraction algorithm that captures important trends from web services execution traces. We demonstrate a new way to visualize these patterns that shows the behavior of web services applications at different levels of abstraction. Finally, we explain how this can help developers with performance analysis by showing both the averages and variations in the data contained in each pattern.

Zinsight: a visual and analytic environment for exploring large event traces

Information in event traces from software systems can help developers with performance analysis, debuggingand troubleshooting. However, the volume of data contained in these traces can make such tasks a challenge. In this paper we propose a new tool, Zinsight, to visualize event traces from complex systems. Our contribution is a novel combination of visualizations and pattern extraction techniques, enabling user exploration, analysis and understanding of traces containing millions of events. Three complimentary views help the user answer different questions. First, the Event Flow view shows the trace in its entirety or in detail. The user sees visual patterns representing phases of processing and the relative order of events. Second, the Event Statistics view quantifies events, and presents distributions and averages enabling the user to identify outlier behavior. Third, the Sequence Context view extracts patterns of interest from the trace and represents them along with frequency and performance data in succinct execution flow diagrams. The user can navigate from patterns to their constituent instance sequences and even back to individual events in the other views. Questions can be answered and hypotheses tested using the most natural view for the task.

Visual debugging for stream processing applications

Stream processing is a new computing paradigm that enables continuous and fast analysis of massive volumes of streaming data. Debugging streaming applications is not trivial, since they are typically distributed across multiple nodes and handle large amounts of data. Traditional debugging techniques like breakpoints often rely on a stop-the-world approach, which may be useful for debugging single node applications, but insufficient for streaming applications. We propose a new visual and analytic environment to support debugging, performance analysis, and troubleshooting for stream processing applications. Our environment provides several visualization methods to study, characterize, and summarize the flow of tuples between stream processing operators. The user can interactively indicate points in the streaming application from where tuples will be traced and visualized as they flow through different operators, without stopping the application. To substantiate our discussion, we also discuss several of these features in the context of a financial engineering application.

Streamsight: a visualization tool for large-scale streaming applications

Stream processing is becoming a new and important computing paradigm. Innovative streaming applications are being developed in areas ranging from scientific applications (e.g., environment monitoring), to business intelligence (e.g., fraud detection and trend analysis), to financial markets (e.g., algorithmic trading strategies). Developing, understanding, debugging, and optimizing streaming applications is non-trivial because of the adaptive and dynamic nature of these applications. The sheer complexity and the distributed character of a large number of cooperating components hosted on a distributed environment further complicate matters. In this paper we describe Streamsight, a new visualization tool built to examine, monitor, and help understand the dynamic behavior of streaming applications. Previously developed stream processing visualization tools focus solely on composition of dataflow graphs. Streamsights novelty hinges on a wide range of capabilities, including the ability to manage the dynamics of large and evolving topologies comprising multiple streaming applications with thousands of nodes and interconnections. From rendering live performance counters using different perspectives to allowing recordings and replays of the execution process, Streamsight provides the mechanisms that permit a better understanding of the evolving and adaptive behavior of streaming applications. These capabilities are used for debugging purposes, for performance optimization, and management of resources, including capacity planning. More than 50 developers, both inside and outside IBM, have been using Streamsight.

Visual and algorithmic tooling for system trace analysis: a case study

Despite advances in the application of automated statistical and machine learning techniques to system log and trace data there will always be a need for human analysis of machine traces, because trace information on unstable systems may be incomplete, or incorrect. In addition, false positives from automated analysis will not likely disappear, and remediation measures and candidate fix tests will need to be evaluated. We present Zinsight, a visual and analytic tool that supports performance analysts and debugging, using large event traces to understand complex systems. This tool enables analysts to quickly create and manipulate high-level structural representations linked with statistical analysis derived from the underlying event trace data. The original raw trace is annotated with module names and a domain specific database is incorporated to relate software functions to module names. Navigable sequence context graph views present automatically extracted execution flow patterns from arbitrarily definable sets of events and are linked to frequency, distribution, and response time views. The goal is to reduce the cognitive and computational load on the analyst while providing answers to the most natural questions in a problem determination session. We present a case study of the tool in use on field problems from the recently shipped (late 2008) IBM z10 mainframe. As a result of the industry trend toward higher parallelism and memory latency, many issues were encountered with legacy code. The tool was applied successfully to diagnose these problems.

Visualizing large-scale streaming applications

Stream processing is a new and important computing paradigm. Innovative streaming applications are being developed in areas ranging from scientific applications (for example, environment monitoring), to business intelligence (for example, fraud detection and trend analysis), to financial markets (for example, algorithmic trading systems). In this paper we describe Streamsight, a new visualization tool built to examine, monitor and help understand the dynamic behavior of streaming applications. Streamsight can handle the complex, distributed and large-scale nature of stream processing applications by using hierarchical graphs, multi-perspective visualizations, and de-cluttering strategies. To address the dynamic and adaptive nature of these applications, Streamsight also provides real-time visualization as well as the capability to record and replay. All these features are used for debugging, for performance optimization, and for management of resources, including capacity planning. More than 100 developers, both inside and outside IBM, have been using Streamsight to help design and implement large-scale stream processing applications.

Visualizing Object-Oriented Programs with Jinsight

Jinsight is a tool that lets you visualize and explore many aspects of your Java program’s run-time behavior. It is helpful for performance analysis, debugging, and any task in which you need to better understand what your code is really doing. Jinsight was developed at IBM’s Research Division, and is available free as a technology preview.