Ben Kobler

Architecture and design of storage and data management for the NASA Earth observing system Data and Information System (EOSDIS)

Mission to Planet Earth (MTPE) is a long-term NASA research mission to study the processes leading to global climate change. The EOS Data and Information System (EOSDIS) is the component within MTPE that will provide the Earth science community with easy, affordable, and reliable access to Earth science data. EOSDIS is a distributed system, with major facilities at eight Distributed Active Archive Centers (DAACs) located throughout the United States. At the DAACs the Science Data Processing Segment (SDPS) will receive, process, archive, and manage all data. It is estimated that several hundred gigaflops of processing power will be required to process and archive the several terabytes of new data that will be generated and distributed daily. Thousands of science users and perhaps several hundred thousand nonscience users will access the system.

NASA Earth Observing System Data Information System (EOSDIS)

EOSDIS is being developed by the National Aeronautics and Space Administration (NASA) to be a comprehensive data and information system providing the Earth science research community with easy, affordable, and reliable access to EOS and other appropriate Earth science data. EOSDIS will archive approximately 2 TBytes of data per day over a 15-year period. The system will be geographically distributed and provide data access to scientists worldwide.<<ETX>>

EOSDIS petabyte archives: tenth anniversary

One of the worlds largest scientific data systems, NASAs Earth observing system data and information system (EOSDIS) has stored over three petabytes of earth science data in a geographically distributed mass storage system. Design for this system began in the early 1990s and included a presentation of the design of the mass storage system at this conference in 1995. Many changes have occurred in the ten years since that presentation, much of it performed while the system was operational. In its first operational year (2000), the EOSDIS system had increased NASAs collection of earth science data holdings eight-fold. Today, EOSDIS collects over 7,000 gigabytes of data per week, almost 60 times more than the hubble space telescope. This load represents major challenges for ingest into the mass storage system, as well as for timely and balanced data distribution out of the mass storage system. This paper discusses the evolution of the EOSDIS archives focusing primarily on the mass storage system component of the archive. We present the lessons that were learned over the years and some directions that we are taking for the future.

Beyond the storage area network: data intensive computing in a distributed environment

NASA Earth and space science applications are currently utilizing geographically distributed computational platforms. These codes typically require the most compute cycles and generate the largest amount of data over any other applications currently supported by NASA. Furthermore, with the development of a leadership class SGI system at NASA Ames (Project Columbia), NASA has created an agency wide computational resource. This resource is heavily employed by Earth and space science users resulting in large amounts of data. The management of this data in a distributed environment requires a significant amount of effort from the users. This paper defines the approach taken to create an enabling infrastructure to help users easily access and move data across distributed computational resources. Specifically, this paper discusses the approach taken to create a wide area storage area network (SAN) using the SGI CXFS file system over standard TCP/IP. In addition, an emerging technology test bed initiative is described to show how NASA is creating an environment to continually evaluate new technology for data intensive computing.

Considerations and performance evaluations of shared storage area networks at NASA Goddard Space Flight Center

The NASA Goddard Space Flight Center (GSFC) in Greenbelt Maryland is exploring advanced storage architectures for retaining and distributing its large data holdings. As a research vehicle, a multi-building Storage Area Network (SAN) was deployed at GSFC in early 2002. The initial objective was to demonstrate the feasibility and advantages of fibre channel-connected, centralized storage as it applies to a campus installation. The secondary objective was to illustrate the advantages of a heterogeneous SAN shared file system that would allow a single instance of data to be globally shared amongst multiple SAN-connected clients on different platforms. The GSFC SAN has since been extended to include off-campus connections for evaluating the Internet Protocol (IP) as an option for connecting a broader, more geographically dispersed user base. The focus of this paper is the series of tests conducted to characterize distance data sharing using both native FC and IP based technologies. These experiments include both standard I/O benchmarks as well as representative GSFC applications.

Early Experiences in Managing Inter-Site Storage Area Networks Using Secure Web Services

The NASA Goddard Space Flight Center and the University of Maryland Institute for advanced computer studies have deployed a pilot system for managing distributed IP-based storage area networks of dynamically allocated SAN extensions in the advanced virtual engine test cell (AVETEC) data intensive computing environment (DICE). The system implements the framework for managing inter-site storage area networks using grid and web services technologies presented at MSST06. It includes several new components including a basic registry and a java-based command line interface. The test-bed provides some early experiences with the reliability, usability, and security of the system, as well as some performance testing of the web services.