Jeanne Behnke

Evolution of the Earth Observing System (EOS) Data and Information System (EOSDIS)

The earth observing system (EOS) data and information system (EOSDIS) has been serving a broad user community since 1994. Most of NASAs Earth science data are currently being archived, managed and distributed by EOSDIS. Also, EOSDIS commands and controls EOS spacecraft and instruments, captures data from the instruments and processes them into a set of standard products. As of March 2006, the archives of EOSDIS held over 4.3 petabytes of data from over 90 instruments and over 2000 distinct science products. The distribution of data to end users amounts to approximately 2 TB a day. The community receiving data from EOSDIS is on the order of 200,000 distinct users from a diverse set of organizations and scientific disciplines. While EOSDIS is effectively managing a large amount of data and successfully serving a broad user community, it is a system whose design and development originated more than 10 years ago during which many advances have occurred in information technology. Although there has been an on-going process of technology infusion, incremental improvements in processing and performance, and new functionality added in areas of user access, distribution, and archive management over the years, the underlying design has remained essentially the same. During this time frame, data volumes have grown dramatically and the science community has gained considerable experience in processing and analyzing their data. More recently, through examination of current operations and a series of lessons learned, there has been a desire to re-examine current operations for significant improvements in a variety of areas. The overall objectives of the EOSDIS evolution are to: increase end-to-end data system efficiency and autonomy while decreasing operations costs, increase data interoperability and usability by the science research, application, and modeling communities, improve data access and processing, and ensure safe stewardship.

EOSDIS petabyte archives: tenth anniversary

One of the worlds largest scientific data systems, NASAs Earth observing system data and information system (EOSDIS) has stored over three petabytes of earth science data in a geographically distributed mass storage system. Design for this system began in the early 1990s and included a presentation of the design of the mass storage system at this conference in 1995. Many changes have occurred in the ten years since that presentation, much of it performed while the system was operational. In its first operational year (2000), the EOSDIS system had increased NASAs collection of earth science data holdings eight-fold. Today, EOSDIS collects over 7,000 gigabytes of data per week, almost 60 times more than the hubble space telescope. This load represents major challenges for ingest into the mass storage system, as well as for timely and balanced data distribution out of the mass storage system. This paper discusses the evolution of the EOSDIS archives focusing primarily on the mass storage system component of the archive. We present the lessons that were learned over the years and some directions that we are taking for the future.

NASA workshop on issues in the application of data mining to scientific data

In this paper, we describe the NASA sponsored workshop on Issues in the Application of Data Mining to Scientific Data. The workshop was held at the University of Alabama in Huntsville on October 19-21, 1999. The full text of the report can be found in PDF and MSWord format at the following website: h..ttp://www.cs.uah.edu/NASA....Minin~z/

Evolving a ten year old data system

The Earth Observing System (EOS) Data and Information System (EOSDIS) is a comprehensive distributed system designed to support NASAs Earth Science missions. Designed in the early 1990s, EOSDIS has been archiving, managing, and distributing Earth science data since 1994. Over the life of EOSDIS an on-going process of technology updates and improvements in user access, distribution mechanisms, and archive management has attempted to keep the system current. However, data volumes have grown rapidly and the science community has gained experience and capability in processing and analyzing their data. The result is a growing desire to re-examine the current operations for gains and improvements in a variety of areas. The objectives of the evolution of EOSDIS are to: increase end-to-end data system efficiency while decreasing operations costs, increase data interoperability and usability by the science research, application, and modeling communities, improve data access and processing, and ensure safe stewardship

Requirements engineering in science data processing systems

Science data processing is a component of most research and development programs nationally and internationally and is a prime focus of work performed at NASA s Goddard Space Flight Center in Greenbelt, Maryland. Requirements engineering is probably the most important phase in the software development lifecycle and there are many ways requirements engineering is performed by different projects. Comparisons and contrasts can be made about how science products are specified, created and managed by different projects. Identification of these characteristics leads to a better understanding of how requirements engineering is performed and how it can be improved.