Dhrubajyoti Goswami

From Design Patterns to Parallel Architectural Skeletons

The concept of design patterns has been extensively studied and applied in the context of object-oriented software design. Similar ideas are being explored in other areas of computing as well. Over the past several years, researchers have been experimenting with the feasibility of employing design-patterns related concepts in the parallel computing domain. In the past, several pattern-based systems have been developed with the intention to facilitate faster parallel application development through the use of preimplemented and reusable components that are based on frequently used parallel computing design patterns. However, most of these systems face several serious limitations such as limited flexibility, zero extensibility, and the ad hoc nature of their components. Lack of flexibility in a parallel programming system limits a programmer to using only the high-level components provided by the system. Lack of extensibility here refers to the fact that most of the existing pattern-based parallel programming systems come with a set of prebuilt patterns integrated into the system. However, the system provides no obvious way of increasing the repertoire of patterns when need arises. Also, most of these systems do not offer any generic view of a parallel computing pattern, a fact which may be at the root of several of their shortcomings. This research proposes a generic (i.e., pattern- and application-independent) model for realizing and using parallel design patterns. The term “parallel architectural skeleton” is used to represent the set of generic attributes associated with a pattern. The Parallel Architectural Skeleton Model (PASM) is based on the message-passing paradigm, which makes it suitable for a LAN of workstations and PCs. The model is flexible as it allows the intermixing of high-level patterns with low-level message-passing primitives. An object-oriented and library-based implementation of the model has been completed using C++and MPI, without necessitating any language extension. The generic model and the library-based implementation allow new patterns to be defined and included into the system. The skeleton-library serves as a framework for the systematic, hierarchical development of network-oriented parallel applications.

Granularity-Driven Dynamic Predicate Slicing Algorithms for Message Passing Systems

Program Slicing is a well-known decomposition technique that transforms a large program into a smaller one that contains only statements relevant to the computation of a selected function. In this paper, we present two novel predicate-based dynamic slicing algorithms for message passing programs. Unlike more traditional slicing criteria that focus only on parts of the program that influence a variable of interest at a specific position in the program, a predicate focuses on those parts of the program that influence the predicate. The dynamic predicate slices capture some global requirements or suspected error properties of a distributed program and computes all statements that are relevant. The presented algorithms differ from each other in their computational approaches (forward versus backward) and in the granularity of information they provide. A proof of correctness of these algorithms is provided. Through the introduction of dominant states and dominant events, critical statement executions are identified that change the value of the global predicate. Under this formulation, optimizing dynamic predicate slicing becomes a meaningful goal as well. Finally, we present how predicate slices can be applied to support comprehension tasks for analyzing and maintaining distributed programs.

Functionality distribution for parallel rendering

Handling very large datasets has been a key problem addressed in real-time distributed rendering research. With the advent of the programmable graphics processing unit (GPU), it is now possible and even profitable to move many application-specific computations to be carried out by the GPU. It has been shown that modern GPUs outperform the standard PC-platform CPUs on a broad class of computations by over a factor of seven. Given the low costs and high processing speeds of GPUs, there is a trend towards using clusters of CPU/GPU systems. Configuring and programming these clusters for efficient distribution of data and computations is a major challenge. What are the computations that can be offloaded from the CPU to a GPU? The answer to this question is not simple as it depends on the following four factors: GPUs processing capacity, GPUs internal bandwidth, GPU-CPU communication bandwidth and the external network bandwidth. All these factors are subject to change with every generation of hardware. But additions and alternatives to the traditional data-parallel architectures are now needed to exploit the full capability of such clusters using functional parallelism. In this paper, we present a number of architectural configurations that could be adapted on such clusters. Specifically, we demonstrate use of one such architecture: application of a GPU-based pipelined architecture to our work on real-time processing and rendering of large-point datasets, which demands complex computations. We have also introduced a list of application and system parameters that are necessary to determine an optimal distribution of computation on the GPUs of a graphics cluster.

Quasi-atomic recovery for distributed agents

Distributed multi-agent systems are usually large-scale, involving a large number of agents and messages. Existing checkpoint and recovery strategies are not quite favorable to such systems due to either global recovery spread or runtime logging overhead associated with these strategies. This paper presents our work on the design of correct and efficient checkpoint and recovery strategies for distributed agent systems. The initial part of the paper introduces a formal model to capture the correctness of recovery that is applicable in general, including those used by existing techniques such as deterministic as well as non-deterministic, and single as well as simultaneous recoveries. In particular, notions of atomic and quasi-atomic recovery blocks are introduced to capture the subset of events nullified in a single recovery. It is proved that the correctness of multiple recoveries is guaranteed if a recovery technique ensures well-ordering of corresponding recovery blocks. The rest of the paper utilizes the features of agent communication protocols towards the design of a simple and efficient checkpoint protocol. In particular, agents interact with each other via well-defined agent communication protocols. Agent protocol sessions are group-based and all message interactions are localized inside such groups. A group checkpoint strategy is proposed that uses these features of locality and well-structured-ness for the purpose of both reducing runtime overhead and minimizing recovery spread. The resulted protocol creates strong and asynchronous group checkpoints, i.e., without any explicit kernel message or runtime message logging. An accompanying recovery protocol uses a notion of a protocol dependency graph to identify the minimal quasi-atomic recovery block corresponding to single or simultaneous agent crashes. Correctness of the recovery protocol is proved under the formal model. The paper concludes with a discussion on the significance and contrast of our research with other related works, followed by future research directions.

Predicate-based dynamic slicing of message passing programs

Program slicing is a well-known decomposition technique that transforms a large program into a smaller one that contains only statements relevant to the computation of a selected function. We present a novel predicate-based dynamic slicing algorithm for message passing programs. Unlike the more traditional slicing criteria that focus only on the parts of the program that influence a variable of interest at a specific position in the program, a predicate focuses on those parts of the program that influence the predicate. The dynamic predicate slice captures some global requirements or suspected error properties of a distributed program and computes all statements that are relevant. We present an algorithm and a sample computation to illustrate how the predicate slice can be computed. Additionally, we introduce a predicate trace to classify the relevance of statement executions based on the predicate slice. A compressed predicate trace can be used to reveal those instances of statement execution that turn the global predicate true, among others.

Building parallel applications using design patterns

Parallel application design and development is a major area of interest in the domain of high-performance scientific and industrial computing. In fact, parallel computing is becoming an integral part in several major application domains such as space, medicine, cancer and genetic research, graphics and animation, image processing, to name a few. With the advent of fast interconnecting networks of workstations and PCs, it is now becoming increasingly possible to develop high-performance parallel applications using the combined computing powers of these networked resources, at no extra cost. Contrast this to the situation until the early 90s, where parallel computing was mostly confined only to special-purpose parallel computers that were unaffordable by small research institutions. Nowadays, high-speed networks and fast general-purpose computers are aiding in the mainstream adoption of parallel computing at a much more affordable cost.

Detecting Atomicity Errors in Message Passing Programs

A distributed application can be viewed as a collection of processes that execute a number of atomic actions. Atomicity is the basis for reasoning about the correctness of a program. Atomicity errors in a run typically indicate the presence of program errors. This paper formalizes the notion of atomicity of an action in a message passing program based on a weak-order relation among atoms. An atom can be a single statement or a sequence of statements in a program. Knowing the atoms, the atomicity of a run can be monitored and checked. Serialization of conflicting atoms is another generic correctness requirement. When atoms affect a common property, such as in sharing resources or maintaining a common constraint, they must be serialized in a run. This paper presents two efficient algorithms for dynamically detecting atomicity and serialization errors, accompanied with their proof of correctness.In this paper, we present a Distributed Geography-based Fault Tolerant topology control algorithm (DGFT) for static large-scale wireless sensor networks. We introduce the scale-free characteristic of complex networks into the topology of large- scale wireless sensor networks to obtain robustness and time efficiency. DGFT enables wireless nodes to define the topology by neighbor relationship, based on certain initialized weights whose distribution follows a negative power law. We prove that the topology constructed under DGFT is strongly connected and bidirectional. Simulation studies show that the resulting topology has good network performance in terms of transmission delay and robustness.

A model for designing and implementing parallel applications using extensible architectural skeletons

With the advent of hardware technologies, high-performance parallel computers and commodity clusters are becoming affordable. However, complexity of parallel application development remains one of the major obstacles towards the mainstream adoption of parallel computing. As one of the solution techniques, researchers are actively investigating the pattern-based approaches to parallel programming. As re-usable components, patterns are intended to ease the design and development phases of a parallel applications. While using patterns, a developer supplies the application specific code-components whereas the underlying environment generates most of the code for parallelization. PAS (Parallel Architectural Skeleton) is one such pattern-based parallel programming model and tool, which defines the architectural aspects of parallel computational patterns. Like many other pattern-based models and tools, the PAS model was hampered by its lack of extensibility, i.e., lacking of support for the systematic addition of new skeletons to an existing skeleton repository. Lack of extensibility significantly reduces the flexibility and hence the usability of a particular approach. SuperPAS is an extension of PAS that defines a model for systematically designing and implementing PAS skeletons by a skeleton designer. The newly implemented skeletons can subsequently be used by an application developer. SuperPAS model is realized through a Skeleton Description Language (SDL), which assists both a skeleton designer and an application developer. The paper discusses the SuperPAS model through examples that use the SDL. The paper also discusses some of the recent usability and performance studies, which demonstrate that SuperPAS is a practical and usable parallel programming model and tool.

Using Object-Oriented Techniques for Realizing Parallel Architectural Skeletons

The concept of design patterns has recently emerged as a new paradigm in the context of object-oriented design methodology. Similar ideas are being explored in other areas of computing. In the parallel computing domain, design patterns describe recurring parallel computing problems and their solution strategies. Starting with the late 1980’s, several pattern-based systems have been built for facilitating parallel application development. However, most of these systems use patterns in ad hoc manners, thus lacking a generic or standard model for using and intermixing different patterns. This substantially hampers the usability of such systems. Lack of flexibility and extensibility are some of the other major concerns associated with most of these systems. In this paper, we propose a generic (i.e., pattern- and application-independent) model for realizing and using parallel design patterns. The term architectural skeleton is used to represent the application independent, re-usable set of attributes associated with a pattern. The model can provide most of the functionalities of low level message passing libraries, such as PVM or MPI, plus the benefits of the patterns. This results in tremendous flexibility to the user. It turns out that the model is an ideal candidate for object-oriented style of design and implementation. It is currently implemented as a C++ template-library without requiring any language extension. The generic model, together with the object-oriented and library-based approach, facilitates extensibility.

Using Atoms to Simplify Distributed Programs Checking

The execution of a distributed program generates a large state space which needs to be checked in testing and debugging. This state space can be reduced by using atoms corresponding to code blocks before performing the checking of the required program properties. This paper presents our results in using atoms which are known at program design time for this purpose. We consider the impact of incomplete or incorrect knowledge of atoms on the validity of checking if a run is indeed atomic with respect to the identified atoms and if it satisfies the required program properties.