Andrea Santoro

Nonblocking checkpointing for optimistic parallel simulation: description and an implementation

Describes a nonblocking checkpointing mode in support of optimistic parallel discrete event simulation. This mode allows real concurrency in the execution of state saving and other simulation specific operations (e.g, event list update, event execution) with the aim of removing the cost of recording state information from the completion time of the parallel simulation application. We present an implementation of a C library supporting nonblocking checkpointing on a myrinet based cluster, which demonstrates the practical viability of this checkpointing mode on standard off-the-shelf hardware. By the results of an empirical study on classical parameterized synthetic benchmarks, we show that, except for the case of minimal state granularity applications, nonblocking checkpointing allows improvement of the speed of the parallel execution, as compared to commonly adopted, optimized checkpointing methods based on the classical blocking mode. A performance study for the case of a personal communication system (PCS) simulation is additionally reported to point out the benefits from nonblocking checkpointing for a real world application.

Offloading Data Distribution Management to Network Processors in HLA-Based Distributed Simulations

The high-level architecture (HLA) standard developed by the Department of Defense in the United States is a key technology to perform distributed simulation. Inside the HLA framework, many different simulators (termed federates) may be interconnected to create a single more complex simulator (federation). Data distribution management (DDM) is an optional subset of services that controls which federates should receive notification of state modifications made by other federates. A simple DDM implementation will usually generate much more traffic than needed, whereas a complex one might introduce too much overhead. In this work, we describe an approach to DDM that delegates a portion of the DDM computation to a processor on the network card in order to provide more CPU time for other federate and Runtime Infrastructure (RTI) computations while still being able to exploit the benefits of a complex DDM implementation to reduce the amount of information exchange.

Approximate Analytical Models for Networked Servers Subject to MMPP Arrival Processes

Input characterization to describe the flow of incoming traffic in network systems, such as the GRID and the WWW, is often performed by using Markov modulated Poisson processes (MMPP). Therefore, to enact capacity planning and quality-of-service (QoS) oriented design, the model of the hosts that receive the incoming traffic is often described as a MMPP/M/1 queue. The drawback of this model is that no closed form for its solution has been derived. This means that evaluating even the simplest output statistics of the model, such as the average response times of the queue, is a computationally intensive task and its usage in the above contexts is often unadvisable. In this paper we discuss the possibility to approximate the behavior of a MMPP/M/1 queue with a computational effective analytical approximation, thus saving the large amount of calculations required to evaluate the same data by other means. The employed method consists in approximating the MMPP/M/1 queue as a weighted superposition of different M/M/1 queues. The analysis is validated by comparing the results of a discrete event simulator with those obtained from the proposed approximations, in the context of a real case study involving a GRID networked server.

Tuning of the Checkpointing and Communication Library for optimistic simulation on Myrinet based NOWs

Recently a Checkpointing and Communication Library (CCL) for optimistic simulation on Myrinet based network of workstations (NOWs) has been presented. CCL offloads checkpoint operations from the CPU by charging them to a programmable DMA engine on the Myrinet network card. CCL includes also functionalities for freezing the simulation application on demand, which can be used for data consistency maintenance (for example when a state buffer needs to be accessed for further modifications while a DMA based checkpoint operation involving it is still in progress). Programming the DMA to perform a checkpoint operation by transferring large data blocks in a single burst allows the latency of any checkpoint operation to be kept low. This reduces the probability for application freezing to really occur On the other hand, transferring large data blocks in a single burst might cause negative interference on communication since that DMA (and other circuitry) cannot be used for communication functionalities until the currently executed data transfer is not yet completed. In this paper we present a detailed identification of the effects of the burst length, from which we outline a set of relevant phenomena to take into account in order to determine a compile time suited value for the burst length itself. We also report measures quantifying these phenomena for the case of a PC cluster. Actually, the data indicate that communication functionalities do not suffer from the use of non-minimal burst lengths for checkpoint operations, thus pointing out how, if well tuned, CCL provides highly effective, CPU off-loaded, checkpointing functionalities.

A version of MASM portable across different UNIX systems and different hardware architectures

Magic state manager (MASM) is recently developed software architecture for completely transparent checkpointing/recovery in support of optimistic synchronization in the high level architecture. In the original design, MASM relies on: (i) user level machine dependent modules; (ii) patches for specific versions of the LINUX kernel; and (iii) static linking of specific application libraries, all of them required for performing ad-hoc, low level memory management operations associated with optimistic synchronization requirements. In this paper, we propose a complete re-engineering of this software architecture which allows all those memory management tasks to be carried out through user level, machine independent modules, with the additional advantage of avoiding the need for static linking of specific application libraries, thus achieving portability of MASM across different UNIX systems and different computer architectures.

Transparent State Management for Optimistic Synchronization in the High Level Architecture

In this article, the authors present the design and implementation of a software architecture—namely, MAgic State Manager (MASM)—to be employed within a runtime infrastructure (RTI) in support of High Level Architecture (HLA) federations. MASM allows performing checkpointing/recovery of the federate state in a way completely transparent to the federate itself, thus providing the possibility of demanding to the RTI any task related to state management in optimistic synchronization. Different from existing proposals, through this approach, the federate programmer is required neither to supply modules for state management within the federate code nor to explicitly interface the federate code with existing, third-party checkpointing/recovery libraries. Hence, the federate programmer is completely relieved from the burden of facing state management issues. One major application of this proposal is the possibility to employ optimistic synchronization, even in case of federates originally designed for the conservative approach. This can provide a way of improving the simulation system performance in specific scenarios (e.g., in case of poor or zero lookahead within the federation). The authors elaborate on this issue by discussing on how to integrate MASM within the RTI to achieve such a synchronization objective. Some experimental results demonstrating limited runtime overhead introduced by MASM are also reported for two case studies—namely, an interconnection network simulation and a personal communication system simulation.

Semi-asynchronous checkpointing for optimistic simulation on a Myrinet based NOW

Great effort has been devoted to the design of optimized checkpointing strategies for optimistic parallel discrete event simulators. On the other hand, there is less work being done in the direction of improving the execution mode of any single checkpoint operation. Specifically, checkpoint operations are typically charged to the CPU, thus leading to freezing of the simulation application while checkpointing is in progress, i.e. the execution mode of the checkpointing protocol is typically synchronous. In this paper, we focus on improvements of the execution mode and present a software architecture, designed for Myrinet-based networks of workstations (NOWs), to avoid application freezing during any checkpoint operation, thus moving the execution itself towards an asynchronous mode. This is done by charging checkpoint operations to a hardware component that is distinct from the CPU, namely a DMA (direct memory access) engine. On the other hand, totally asynchronous checkpointing could suffer from data inconsistency whenever the content of a state buffer is accessed for further modifications while a checkpoint operation involving it has not yet completed. To avoid this, the architecture includes functionalities for resynchronization on demand. We have used these functionalities to implement an execution mode of the checkpointing protocol that we refer to as semi-asynchronous. By the results of an experimental study, we argue that the semi-asynchronous mode can be an effective solution to almost completely remove the delay associated with any checkpoint operation from the completion time of the simulation.

Transparent Optimistic Synchronization in HLA via a Time-Management Converter

In this paper we present the design and implementation of a Time Management Converter (TiMaC) for HLA based simulation systems. TiMaC is a layer interposed in between the federate and the underlying RTI in order to map the conservative Time Management interface onto the optimistic one. In this way, TiMaC transparently supports optimistic execution for federates originally designed for the conservative approach, which is achieved without the need for developing any ad-hoc RTI system. TiMaC relies on a recently proposed software architecture for transparent treatment of checkpointing/recovery of the federate state, namely Magic State Manager (MASM), and implements a set of additional facilities required to support all the tasks associated with the mapping of conservative onto optimistic Time Management interfaces. The implementation has been tailored to the Georgia Tech B-RTI package, although the underlying design principles would allow it to be integrated with any RTI system. We also report an experimental study demonstrating the viability and effectiveness of our proposal in allowing conservative federates to be supported with highly increased run-time effectiveness in general contexts for what concerns the features of the underlying computing systems (e.g. LAN vsWAN based systems.

Short note: Modeling and optimization of non-blocking checkpointing for optimistic simulation on myrinet clusters

Checkpointing-and-Communication Library (CCL) is a recently developed software which implements CPU offloaded, non-blocking checkpointing functionalities in support of optimistic parallel simulation on myrinet clusters. This is achieved by exploiting data transfer capabilities provided by a programmable DMA engine on board of myrinet network cards. Re-synchronization between CPU and DMA activities must sometimes be employed for several reasons, such as the maintenance of data consistency, thus adding overhead to (otherwise CPU cost-free) non-blocking checkpoint operations. In this paper we present a detailed cost model for non-blocking checkpointing and derive a performance effective re-synchronization semantic which we call minimum cost re-synchronization. With this semantic, an occurrence of re-synchronization either commits an on-going DMA based checkpoint operation (causing suspension of CPU activities) or aborts the operation (with possible increase in the expected rollback cost due to a reduced amount of committed checkpoints) on the basis of a minimum overhead expectation evaluated through the cost model.

SOFTWARE SUPPORTS FOR EVENT PREEMPTIVE ROLLBACK IN OPTIMISTIC PARALLEL SIMULATION ON MYRINET CLUSTERS

Optimistic synchronization protocols for parallel discrete event simulation employ rollback techniques to ensure causally consistent execution of simulation events. Although event preemptive rollback (i.e. rollback based on timely event execution interruption upon the arrival of a message revealing a causality inconsistency) is recognized as an approach for increasing the performance and tackling run-time anomalies of this type of synchronization, the lack of adequate functionalities at the level of general purpose communication layers typically prevents any effective implementation of event preemptive rollback operations. In this paper we present the design and implementation of a communication layer for Myrinet based clusters, which efficiently supports event preemptive rollback operations. Beyond standard low latency message delivery funcbionalities, this layer also embeds functionalities for allowing the overlying simulation application to efficiently track whether a message will actually produce causality inconsistency of the currently executed simulation event upon its receipt at the application level. Exploiting these functionalities, awareness of the inconsistency precedes the message receipt at the application level, thus allowing timely event execution interruption for activating rollback procedures. We also report experimental results demonstrating the effectiveness of our solution for a Personal Communication System (PCS) simulation application.