Steve MacDonald

From patterns to frameworks to parallel programs

Object-oriented programming, design patterns, and frameworks are abstraction techniques that have been used to reduce the complexity of sequential programming. This paper describes our approach of applying these three techniques to the more difficult parallel programming domain. The Parallel Design Patterns (PDP) process, the basis of the CO2P3S parallel programming system, combines these techniques in a layered development model. The result is a new approach to parallel programming that addresses correctness and openness in a unique way. At the topmost developmem layer, a customized framework is generated from a design pattern specification of the parallel structure of the program. This framework encapsulates all of the structural details of the pattern, including communication and synchronization, to prevent programmer errors and ensure correctness. Lower layers are used only for performance tuning to make the code as efficient as necessary. This paper describes CO2P3S, based on the PDP process, and demonstrates it using an example application. We also provide results from a usability study of CO2P3S.

Generating parallel programs from the wavefront design pattern

Object-oriented programming, design patterns, and frameworks are common techniques that have been used to reduce the complexity of sequential programming. We have applied these techniques to the more difficult domain of parallel programming. This paper describes CO2P3S, a pattern-based parallel programming system that generates parallel programs from parallel design patterns. We demonstrate CO2P3S by applying a new design pattern called the Wavefront pattern to three problems. We show that it is quick and easy to use CO2P3S to generate structurally correct parallel programs with good speed-ups on shared-memory computers.

Using generative design patterns to generate parallel code for a distributed memory environment

A design pattern is a mechanism for encapsulating the knowledge of experienced designers into a re-usable artifact. Parallel design patterns reflect commonly occurring parallel communication and synchronization structures. Our tools, CO2P3S (Correct Object-Oriented Pattern-based Parallel Programming System) and MetaCO2P3S, use generative design patterns. A programmer selects the parallel design patterns that are appropriate for an application, and then adapts the patterns for that specific application by selecting from a small set of code-configuration options. CO2P3S then generates a custom framework for the application that includes all of the structural code necessary for the application to run in parallel. The programmer is only required to write simple code that launches the application and to fill in some application-specific sequential hook routines. We use generative design patterns to take an application specification (parallel design patterns + sequential user code) and use it to generate parallel application code that achieves good performance in shared memory and distributed memory environments. Although our implementations are for Java, the approach we describe is tool and language independent. This paper describes generalizing CO2P3S to generate distributed-memory parallel solutions.

Generative design patterns

A design pattern encapsulates the knowledge of object-oriented designers into re-usable artifacts. A design pattern is a descriptive device that fosters software design re-use. There are several reasons why design patterns are not used as generative constructs that support code re-use. The first reason is that design patterns describe a set of solutions to a family of related design problems and it is difficult to generate a single body of code that adequately solves each problem in the family. A second reason is that it is difficult to construct and edit generative design patterns. A third major impediment is the lack of a tool-independent representation. A common representation could lead to a shared repository to make more patterns available. We describe a new approach to generative design patterns that solves these three difficult problems. We illustrate this approach using tools called CO/sub 2/P/sub 2/S and Meta-CO/sub 2/P/sub 2/S but our approach is tool-independent.

Rethinking the pipeline as object-oriented states with transformations

The pipeline is a simple and intuitive structure to speed up many problems. Novice parallel programmers are usually taught this structure early on. However, expert parallel programmers typically eschew using the pipeline in coarse-grained applications because it has three serious problems that make it difficult to implement efficiently. First, processors are idle when the pipeline is not full. Second, load balancing is crucial to obtaining good speedup. Third, it is difficult to incrementally incorporate more processors into an existing pipeline. Instead, experts recast the problem as a master/slave structure which does not suffer from these problems. This paper details a transformation that allows programs written in a pipeline style to execute using the master/slave structure. Parallel programmers can benefit from both the intuitive simplicity of the pipeline and the efficient execution of a master/slave structure. This is demonstrated by performance results from two applications.

Pattern-based parallel programming

The advantages of pattern-based programming have been well-documented in the sequential programming literature. However patterns have yet to make their way into mainstream parallel computing, even though several research tools support them. There are two critical shortcomings of pattern (or template) based systems for parallel programming: lack of extensibility and performance. This paper describes our approach for addressing these problems in the CO/sub 2/P/sub 3/S parallel programming system. CO/sub 2/P/sub 3/S supports multiple levels of abstraction, allowing the user to design an application with high-level patterns, but move to lower levels of abstraction for performance tuning. Patterns are implemented as parameterized templates, allowing the user the ability to customize the pattern to meet their needs. CO/sub 2/P/sub 3/S generates code that is specific to the pattern/parameter combination selected by the user. The MetaCO/sub 2/P/sub 3/S tool addresses extensibility by giving users the ability to design and add new pattern templates to CO/sub 2/P/sub 3/S. Since the pattern templates are stored in a system-independent format, they are suitable for storing in a repository to be shared throughout the user community.

Towards a better collaboration of static and dynamic analyses for testing concurrent programs

Testing concurrent programs remains a difficult task due to the non-deterministic nature of concurrent executions. Many approaches have been proposed to combine static and dynamic analysis to reduce the complexity of uncovering potential concurrency bugs. However, the existing collaboration schemes only provide a limited mechanism for exchanging relevant information between the two analyses. For example, alias information only flows from the static analysis module to the dynamic analysis module at the beginning of the dynamic analysis. Therefore, we cannot fully exploit the advantages of each type of analysis. Motivated by this observation, in this paper we present a new testing technique which enables a tighter collaboration between static analysis and dynamic analysis. In this collaboration scheme, static analysis and dynamic analysis interact iteratively throughout the whole testing process. Static analysis uses coarse-grained analysis to guide the dynamic analysis to concentrate on the relevant search space, while dynamic analysis collects concrete runtime information during the guided exploration. The runtime information provided by the dynamic analysis helps the static analysis to refine its coarse-grained analysis and provides better guidance on dynamic analysis. Currently, our implementation consists of a static analysis module based on Soot and a dynamic analysis module based on JPF (Java PathFinder).

Generating Parallel Program Frameworks from Parallel Design Patterns

Object-oriented programming, design patterns, and frameworks are abstraction techniques that have been used to reduce the complexity of sequential programming. The CO2P3S parallel programming system provides a layered development process that applies these three techniques to the more difficult domain of parallel programming. The system generates correct frameworks from pattern template specifications at the highest layer and provides performance tuning opportunities at lower layers. Each of these features is a solution to a major problem with current parallel programming systems. This paper describes CO2P3S and its highest level of abstraction using an example program to demonstrate the programming model and one of the supported pattern templates. Our results show that a programmer using the system can quickly generate a correct parallel structure. Further, applications built using these structures provide good speedups for a small amount of development effort.

Testing concurrent programs using value schedules

Concurrent programs are difficult to debug and verify because of the nondeterministic nature of concurrent executions. A particular concurrency-related bug may only show up under certain rarely-executed thread interleavings. Therefore, commonly used debugging methodologies, such as inserting print statements, are no longer sufficient for uncovering concurrency-related bugs. However, many existing bug detection methods, such as dynamic analysis and model checking, have a very high computational cost. In this paper, we introduce a new technique for uncovering concurrency-related bugs from multithreaded Java programs. Our technique uncovers concurrency-related bugs by generating and testing read-write assignment sequences, referred to as value schedules, of a multithreaded Java program. Our value-schedule-based technique distinguishes itself in its ability to avoid exploring superfluous program state space caused by speculative permutation on transitions. Therefore, our technique can achieve a higher degree of POR (Partial Order Reduction) than existing methods. We demonstrate our technique using some programs, with an implementation built using an explicit state model checker called JPF

Deferring design pattern decisions and automating structural pattern changes using a design-pattern-based programming system

In the design phase of software development, the designer must make many fundamental design decisions concerning the architecture of the system. Incorrect decisions are relatively easy and inexpensive to fix if caught during the design process, but the difficulty and cost rise significantly if problems are not found until after coding begins. Unfortunately, it is not always possible to find incorrect design decisions during the design phase. To reduce the cost of expensive corrections, it would be useful to have the ability to defer some design decisions as long as possible, even into the coding stage. Failing that, tool support for automating design changes would give more freedom to revisit and change these decisions when needed. This article shows how a design-pattern-based programming system based on generative design patterns can support the deferral of design decisions where possible, and automate changes where necessary. A generative design pattern is a parameterized pattern form that is capable of generating code for different versions of the underlying design pattern. We demonstrate these ideas in the context of a parallel application written with the CO2P3S pattern-based parallel programming system. We show that CO2P3S can defer the choice of execution architecture (shared-memory or distributed-memory), and can automate several changes to the application structure that would normally be daunting to tackle late in the development cycle. Although we have done this work with a pattern-based parallel programming system, it can be generalized to other domains.