Jon MacLaren

EnLIGHTened Computing: An architecture for co-allocating network, compute, and other grid resources for high-end applications

Many emerging high performance applications require distributed infrastructure that is significantly more powerful and flexible than traditional grids. Such applications require the optimization, close integration, and control of all grid resources, including networks. The EnLIGHTened (ENL) computing project has designed an architectural framework that allows grid applications to dynamically request (in-advance or on-demand) any type of grid resource: computers, storage, instruments, and deterministic, high-bandwidth network paths, including lightpaths. Based on application requirements, the ENL middleware communicates with grid resource managers and, when availability is verified, co-allocates all the necessary resources. ENLpsilas domain network manager controls all network resource allocations to dynamically setup and delete dedicated circuits using generalized multiprotocol label switching (GMPLS) control plane signaling. In order to make optimal brokering decisions, the ENL middleware uses near-real-time performance information about grid resources. A prototype of this architectural framework on a national-scale testbed implementation has been used to demonstrate a small number of applications. Based on this, a set of changes for the middleware have been laid out and are being implemented.

Distributed and collaborative visualization of large data sets using high-speed networks

We describe an architecture for distributed collaborative visualization that integrates video conferencing, distributed data management and grid technologies as well as tangible interaction devices for visualization. High-speed, low-latency optical networks support high-quality collaborative interaction and remote visualization of large data.

HARC: the highly-available resource co-allocator

HARC--the Highly-Available Resource Co-allocator--is an open-source system for reserving multiple resources in a coordinated fashion. HARC can handle different types of resource, and has been used to reserve time on supercomputers across a US-wide testbed, together with dedicated lightpaths connecting the machines. At HARCs core are a distributed set of processes called Acceptors, which provide a coallocation service. HARC functions normally provided a majority of the Acceptors are working; this replication gives HARC its high availability. The Paxos Commit protocol ensures that consistency across all Acceptors is maintained. This paper gives an overview of HARC, and explains both how it works and how it is used. We show that HARCs design makes it easy for the community to contribute new components for co-allocating different types of resource, while the stability of the overall system is maintained.

Architecture of a Community Infrastructure for Predicting and Analyzing Coastal Inundation

The Southeastern Universities Research Association (SURA) has advanced the SURA Coastal Ocean Observing and Prediction (SCOOP) program as a multi-institution collaboration to design and prototype a modular, distributed system for real-time prediction and visualization of the coastal impacts from extreme atmospheric events, including hurricane inundation and waves. The SCOOP program vision is a community “cyberinfrastructure” that enables advances in the science of environmental prediction and coastal hazard planning. The system architecture is a coordinated and distributed network of interoperable, modularized components that include numerical models, information catalogs, distributed archives, computing resources, and network infrastructure. The components are linked over the Internet by standardized web-service interfaces in a service-oriented architecture (SOA). The design philosophy allows geographically disparate partnering institutions to provide complementary data-provider and integration services. The overall system enables coordinated sharing of resources, tools, and ideas among a virtual community of coastal and computer scientists. The distributed design builds on the notion that standards enable innovation, and seeks to leverage successes of the World Wide Web by creating an environment that nurtures interaction between the research community, the private sector, and government agencies working together on behalf of the nation.

Shelter from the storm: building a safe archive in a hostile world

The storing of data and configuration files related to scientific experiments is vital if those experiments are to remain reproducible, or if the data is to be shared easily. The prescence of historical (observed) data is also important in order to assist in model evaluation and development. This paper describes the design and implementation process for a data archive, which was required for a coastal modelling project. The construction of the archive is described in detail, from its design through to deployment and testing. As we will show, the archive has been designed to tolerate failures in its communications with external services, and also to ensure that no information is lost if the archive itself fails, i.e. upon restarting, the archive will still be in exactly the same state.

Large scale computational science on federated international grids: The role of switched optical networks

The provision of high performance compute and data resources on a grid has often been the primary concern of grid resource providers, with the network links used to connect them only a secondary matter. Certain large scale distributed scientific simulations, especially ones which involve cross-site runs or interactive visualisation and steering capabilities, often require high quality of service, high bandwidth, low latency network interconnects between resources. In this paper, we describe three applications which require access to such network infrastructure, together with the middleware and policies needed to make them possible.

An application portal for collaborative coastal modeling

We describe the background, architecture and implementation of a user portal for the SCOOP coastal ocean observing and modeling community. SCOOP is engaged in the real‐time prediction of severe weather events, including tropical storms and hurricanes, and provides operational information including wind, storm surge and resulting inundation, which are important for emergency management. The SCOOP portal, built with the GridSphere Framework, currently integrates customized Grid portlet components for data access, job submission, resource management and notification. Copyright

Visualizing Katrina - merging computer simulations with observations

Hurricane Katrina has had a devastating impact on the US Gulf Coast, and her effects will be felt for many years. Forecasts of such events, coupled with timely response, can greatly reduce casualties and save billions of dollars. We show how visualizations from storm surge and atmospheric simulations were used to understand the predictions of how strong, where, and when flooding would occur in the hours leading up to Katrinas landfall. Sophisticated surface, flow and volume visualization techniques show these simulation results interleaved with actual observations, including satellite cloud images, GIS aerial maps and LIDAR showing the 3D terrain of New Orleans. The sheer size and complexity of the data in this application also motivated research in efficient data access mechanisms and rendering algorithms. Our goals were to use the resulting animation as a vehicle for raising awareness in the general populace to the true impact of the event, to create a scientifically accurate representation of the storm and its effects, and to develop a workflow to create similar visualizations for future and simulated hurricanes. Screenings of the animation have been well received, both by the general public and by scientists in the field.

The SURA Coastal Ocean Observing and Prediction Program (SCOOP) Service-Oriented Architecture

The Southeastern Universities Research Association (SURA) Coastal Ocean Observing and Prediction Program (SCOOP) is a multi-institution collaboration whose partners are working to implement a modular, distributed system for real-time prediction and visualization of the impacts of extreme atmospheric events, including storm surge and wind-driven waves. SCOOP Program partners are developing an interoperable network of modularized components (numerical models, information catalogs, distributed archives, computing resources and network infrastructure) linked by standardized interfaces. This service-oriented architecture (SOA) is emerging as a prototype open access, distributed virtual laboratory for oceanographic research and coastal applications. The SOA approach allows data integration from multiple platforms and enables the exchange of resources, tools, and ideas among a virtual community. The SOA framework consists of five layers: (1) a user interface; (2) an application and tools layer; (3) a management layer; (4) a resource access layer; and (5) physical resources all linked by cross-cutting services. The SOA layer components support several different use cases because they can be configured into a variety of workflows