Lucas Mello Schnorr

On the validity of flow-level tcp network models for grid and cloud simulations

Researchers in the area of grid/cloud computing perform many of their experiments using simulations that must capture network behavior. In this context, packet-level simulations, which are widely used to study network protocols, are too costly given the typical large scales of simulated systems and applications. An alternative is to implement network simulations with less costly flow-level models. Several flow-level models have been proposed and implemented in grid/cloud simulators. Surprisingly, published validations of these models, if any, consist of verifications for only a few simple cases. Consequently, even when they have been used to obtain published results, the ability of these simulators to produce scientifically meaningful results is in doubt. This work evaluates these state-of-the-art flow-level network models of TCP communication via comparison to packet-level simulation. While it is straightforward to show cases in which previously proposed models lead to good results, instead we follow the critical method, which places model refutation at the center of the scientific activity, and we systematically seek cases that lead to invalid results. Careful analysis of these cases reveals fundamental flaws and also suggests improvements. One contribution of this work is that these improvements lead to a new model that, while far from being perfect, improves upon all previously proposed models in the context of simulation of grids or clouds. A more important contribution, perhaps, is provided by the pitfalls and unexpected behaviors encountered in this work, leading to a number of enlightening lessons. In particular, this work shows that model validation cannot be achieved solely by exhibiting (possibly many) “good cases.” Confidence in the quality of a model can only be strengthened through an invalidation approach that attempts to prove the model wrong.

Triva: Interactive 3D visualization for performance analysis of parallel applications

The successful execution of parallel applications in grid infrastructures depends directly on a performance analysis that takes into account the grid characteristics, such as the network topology and resources location. This paper presents Triva, a software analysis tool that implements a novel technique to visualize the behavior of parallel applications. The proposed technique explores 3D graphics to show the application behavior together with a description of the resources, highlighting communication patterns, the network topology and a visual representation of a logical organization of the resources. We have used a real grid infrastructure to execute and trace applications composed of thousands of processes.

Detection and analysis of resource usage anomalies in large distributed systems through multi-scale visualization

Understanding the behavior of large scale distributed systems is generally extremely difficult as it requires to observe a very large number of components over very large time.

Impact of Parallel Workloads on NoC Architecture Design

Due to the multi-core processors, the importance of parallel workloads has increased considerably. However, many-core chips demand new interconnection strategies, since traditional crossbars or buses, common for current multi-core processors, have problems related to wires and scalability. For this reason, Networks-on-Chip (NoCs) have been developed in order to support the performance and parallelism focused on several workloads. Although a Network-on-Chip is a good option, most designs consist of a large number of routers. These routers are responsible for forwarding packets, and consequently, for supporting message-passing workloads. In this context, the NoC performance is a problem. Therefore, the main goal of this paper is to evaluate the impact of well-known parallel workloads on NoC architecture design. In order to achieve high performance, the results point out to parallel workloads with small packets and cluster-based NoCs with circuit switching and adaptable topologies.

Towards Visualization Scalability through Time Intervals and Hierarchical Organization of Monitoring Data

Highly distributed systems such as Grids are used today to the execution of large-scale parallel applications. The behavior analysis of these applications is not trivial. The complexity appears because of the event correlation among processes, external influences like time-sharing mechanisms and saturation of network links, and also the amount of data that registers the application behavior. Almost all visualization tools to analysis of parallel applications offer a space-time representation of the application behavior. This paper presents a novel technique that combines traces from grid applications with a treemap visualization of the data. With this combination, we dynamically create an annotated hierarchical structure that represents the application behavior for the selected time interval. The experiments in the grid show that we can readily use our technique to the analysis of large-scale parallel applications with thousands of processes.

JRastro: a trace agent for debugging multithreaded and distributed Java programs

Program tracing is one of the most used techniques to debug parallel and distributed programs. In this technique, events are recorded in trace files during the execution of the program for post mortem visualization of its behavior. We describe JRastro, a trace agent capable of tracing Java programs. The agent was designed to cover three key features: to be transparent to the application developer, to use unmodified Java virtual machines and to observe remote method invocations. By integrating these three features, JRastro differentiates itself from similar tools. Unfortunately, for a complete and clean implementation of RMI visualization, additional support on the Java monitoring system is needed.

Evaluating trace aggregation for performance visualization of large distributed systems

Performance analysis through visualization techniques usually suffers semantic limitations due to the size of parallel applications. Most performance visualization tools rely on data aggregation to work at scale, without any attempt to evaluate the loss of information caused by such aggregations. This paper proposes a technique to evaluate the quality of aggregated representations - using measures from information theory - and to optimize such measures in order to build consistent multiresolution representations of large execution traces.

A hierarchical aggregation model to achieve visualization scalability in the analysis of parallel applications

The analysis of large-scale parallel applications today has several issues, such as the observation and identification of unusual behavior of processes, expected state of the application, and so on. Performance visualization tools offer a wide spectrum of techniques to visually analyze the monitoring data collected from these applications. The problem is that most of the techniques were not conceived to deal with a high number of processes, in large-scale scenarios. A common example for that is the space-time view, largely used in the performance visualization area, but limited on how much data can be analyzed at the same time. The work presented in this article addresses the problem of visualization scalability in the analysis of parallel applications, through a combination of a temporal integration technique, an aggregation model and treemap representations. Results show that our approach can be used to analyze applications composed of several thousands of processes in large-scale and dynamic scenarios.

3D approach to the visualization of parallel applications and Grid monitoring information

Parallel computing is increasingly used to provide more performance to applications that need tremendous computational power. The main characteristics of distributed parallel machines are heterogeneity, dynamism and size. They influence directly the way the application and platform monitoring tasks are performed, especially when analyzing a large quantity of information collected in a topologically complex machine. This paper describes our efforts to provide parallel programmers and Grid users a new way to visualize monitoring data. Using graphics in three dimensions and information visualization techniques, we aim at bringing rich topological information to the rendered scene. It results in an immersible and human readable representation of complex monitoring data, suited to Grid environments. We first review known techniques in information visualization context, especially those that address the case of hierarchical information, and we discuss about their use in our context. Then, we propose a new 3D approach that combines the classical space-time visualization of application traces with the representation of the applicationpsilas communication pattern. Finally, we present experimental results obtained through the visualization of parallel applications in our prototype.

A spatiotemporal data aggregation technique for performance analysis of large-scale execution traces

Analysts commonly use execution traces collected at runtime to understand the behavior of an application running on distributed and parallel systems. These traces are inspected post mortem using various visualization techniques that, however, do not scale properly for a large number of events. This issue, mainly due to human perception limitations, is also the result of bounded screen resolutions preventing the proper drawing of many graphical objects. This paper proposes a new visualization technique overcoming such limitations by providing a concise overview of the trace behavior as the result of a spatiotemporal data aggregation process. The experimental results show that this approach can help the quick and accurate detection of anomalies in traces containing up to two hundred million events.