Robert Schuler

Wide area data replication for scientific collaborations

Scientific applications require sophisticated data management capabilities. We present the design and implementation of a data replication service (DRS), one of a planned set of higher-level data management services for Grids. The capabilities of the DRS are based on the publication capability of the lightweight data replicator (LDR) system developed for the LIGO Scientific Collaboration. We describe LIGO publication requirements and LDR functionality. We also describe the design and implementation of the DRS in the Globus Toolkit Version 4.0 environment and present performance results.

The Earth System Grid: Enabling Access to Multimodel Climate Simulation Data

By leveraging current technologies to manage distributed climate data in a unified virtual environment, the Earth System Grid (ESG) project is promoting data sharing between international research centers and diverse users. In transforming these data into a collaborative community resource, ESG is changing the way global climate research is conducted. Since ESGs production beginnings in 2004, its most notable accomplishment was to efficiently store and distribute climate simulation data of some 20 global coupled ocean-atmosphere models to the scores of scientific contributors to the Fourth Assessment Report (AR4) of the Intergovernmental Panel on Climate Change (IPCC); the IPCC collective scientific achievement was recognized by the award of a 2007 Nobel Peace Prize. Other international climate stakeholders such as the North American Regional Climate Change Assessment Program (NARCCAP) and the developers of the Community Climate System Model (CCSM) and of the Climate Science Computational End Station (CC...

Data placement for scientific applications in distributed environments

Scientific applications often perform complex computational analyses that consume and produce large data sets. We are concerned with data placement policies that distribute data in ways that are advantageous for application execution, for example, by placing data sets so that they may be staged into or out of computations efficiently or by replicating them for improved performance and reliability. In particular, we propose to study the relationship between data placement services and workflow management systems. In this paper, we explore the interactions between two services used in large-scale science today. We evaluate the benefits of prestaging data using the Data Replication Service versus using the native data stage-in mechanisms of the Pegasus workflow management system. We use the astronomy application, Montage, for our experiments and modify it to study the effect of input data size on the benefits of data prestaging. As the size of input data sets increases, prestaging using a data placement service can significantly improve the performance of the overall analysis.

The Globus Replica Location Service: Design and Experience

Distributed computing systems employ replication to improve overall system robustness, scalability, and performance. A replica location service (RLS) offers a mechanism to maintain and provide information about physical locations of replicas. This paper defines a design framework for RLSs that supports a variety of deployment options. We describe the RLS implementation that is distributed with the Globus toolkit and is in production use in several grid deployments. Features of our modular implementation include the use of soft-state protocols to populate a distributed index and Bloom filter compression to reduce overheads for distribution of index information. Our performance evaluation demonstrates that the RLS implementation scales well for individual servers with millions of entries and up to 100 clients. We describe the characteristics of existing RLS deployments and discuss how RLS has been integrated with higher-level data management services.

Conserved and Divergent Features of Human and Mouse Kidney Organogenesis

Human kidney function is underpinned by approximately 1,000,000 nephrons, although the number varies substantially, and low nephron number is linked to disease. Human kidney development initiates around 4 weeks of gestation and ends around 34-37 weeks of gestation. Over this period, a reiterative inductive process establishes the nephron complement. Studies have provided insightful anatomic descriptions of human kidney development, but the limited histologic views are not readily accessible to a broad audience. In this first paper in a series providing comprehensive insight into human kidney formation, we examined human kidney development in 135 anonymously donated human kidney specimens. We documented kidney development at a macroscopic and cellular level through histologic analysis, RNA in situ hybridization, immunofluorescence studies, and transcriptional profiling, contrasting human development (4-23 weeks) with mouse development at selected stages (embryonic day 15.5 and postnatal day 2). The high-resolution histologic interactive atlas of human kidney organogenesis generated can be viewed at the GUDMAP database (www.gudmap.org) together with three-dimensional reconstructions of key components of the data herein. At the anatomic level, human and mouse kidney development differ in timing, scale, and global features such as lobe formation and progenitor niche organization. The data also highlight differences in molecular and cellular features, including the expression and cellular distribution of anchor gene markers used to identify key cell types in mouse kidney studies. These data will facilitate and inform in vitro efforts to generate human kidney structures and comparative functional analyses across mammalian species.

Data management and analysis for the Earth System Grid

The international climate community is expected to generate hundreds of petabytes of simulation data within the next five to seven years. This data must be accessed and analyzed by thousands of analysts worldwide in order to provide accurate and timely estimates of the likely impact of climate change on physical, biological, and human systems. Climate change is thus not only a scientific challenge of the first order but also a major technological challenge. In order to address this technological challenge, the Earth System Grid Center for Enabling Technologies (ESG-CET) has been established within the U.S. Department of Energys Scientific Discovery through Advanced Computing (SciDAC)-2 program, with support from the offices of Advanced Scientific Computing Research and Biological and Environmental Research. ESG-CETs mission is to provide climate researchers worldwide with access to the data, information, models, analysis tools, and computational capabilities required to make sense of enormous climate simulation datasets. Its specific goals are to (1) make data more useful to climate researchers by developing Grid technology that enhances data usability; (2) meet specific distributed database, data access, and data movement needs of national and international climate projects; (3) provide a universal and secure web-based data access portal for broad multi-model data collections; and (4) provide a wide-range of Grid-enabled climate data analysis tools and diagnostic methods to international climate centers and U.S. government agencies. Building on the successes of the previous Earth System Grid (ESG) project, which has enabled thousands of researchers to access tens of terabytes of data from a small number of ESG sites, ESG-CET is working to integrate a far larger number of distributed data providers, high-bandwidth wide-area networks, and remote computers in a highly collaborative problem-solving environment.

A data placement service for petascale applications

We examine the use of policy-driven data placement services to improve the performance of data-intensive, petascale applications in high performance distributed computing environments. In particular, we are interested in using an asynchronous data placement service to stage data in and out of application workflows efficiently as well as to distribute and replicate data according to Virtual Organization policies. We propose a data placement service architecture and describe our implementation of one layer of this architecture, which provides efficient, priority-based bulk data transfers.

Enabling Distributed Petascale Science

Petascale science is an end-to-end endeavour, involving not only the creation of massive datasets at supercomputers or experimental facilities, but the subsequent analysis of that data by a user community that may be distributed across many laboratories and universities. The new SciDAC Center for Enabling Distributed Petascale Science (CEDPS) is developing tools to support this end-to-end process. These tools include data placement services for the reliable, high-performance, secure, and policy-driven placement of data within a distributed science environment; tools and techniques for the construction, operation, and provisioning of scalable science services; and tools for the detection and diagnosis of failures in end-to-end data placement and distributed application hosting configurations. In each area, we build on a strong base of existing technology and have made useful progress in the first year of the project. For example, we have recently achieved order-of-magnitude improvements in transfer times (for lots of small files) and implemented asynchronous data staging capabilities; demonstrated dynamic deployment of complex application stacks for the STAR experiment; and designed and deployed end-to-end troubleshooting services. We look forward to working with SciDAC application and technology projects to realize the promise of petascale science.

Building a Global Federation System for Climate Change Research: The Earth System Grid Center for Enabling Technologies (ESG-CET)

The recent release of the Intergovernmental Panel on Climate Change (IPCC) 4th Assessment Report (AR4) has generated significant media attention. Much has been said about the U.S. role in this report, which included significant support from the Department of Energy through the Scientific Discovery through Advanced Computing (SciDAC) and other Department of Energy (DOE) programs for climate model development and the production execution of simulations. The SciDAC-supported Earth System Grid Center for Enabling Technologies (ESG-CET) also played a major role in the IPCC AR4: all of the simulation data that went into the report was made available to climate scientists worldwide exclusively via the ESG-CET. At the same time as the IPCC AR4 database was being developed, the National Center for Atmospheric Research (NCAR), a leading U.S. climate science laboratory and a ESG participant, began publishing model runs from the Community Climate System Model (CCSM), and its predecessor the Parallel Coupled Model (PCM) through ESG. In aggregate, ESG-CET provides seamless access to over 180 terabytes of distributed climate simulation data to over 6,000 registered users worldwide, who have taken delivery of more than 250 terabytes from the archive. Not only does this represent a substantial advance in scientific knowledge, it is also a major step forward in how we conduct the research process on a global scale. Moving forward, the next IPCC assessment report, AR5, will demand multi-site metadata federation for data discovery and cross-domain identity management for single sign- on of users in a more diverse federation enterprise environment. Towards this aim, ESG is leading the effort in the climate community towards standardization of material for the global federation of metadata, security, and data services required to standardize, analyze, and access data worldwide.

Data Centric Discovery with a Data-Oriented Architecture

Increasingly, scientific discovery is driven by the analysis, manipulation, organization, annotation, sharing, and reuse of high-value scientific data. While great attention has been given to the specifics of analyzing and mining data, we find that there are almost no tools nor systematic infrastructure to facilitate the process of discovery from data. We argue that a more systematic perspective is required, and in particular, propose a data-centric approach in which discovery stands on a foundation of data and data collections, rather than on fleeting transformations and operations. To address the challenges of data-centric discovery, we introduce a Data-Oriented Architecture and contrast it with the prevalent Service-Oriented Architecture. We describe an instance of the Data-Oriented Architecture and describe how it has been used in a variety of use cases.