Bruce Beaumont

Science data processing for the Advanced Microwave Scanning Radiometer-Earth observing system

The National Aeronautics and Space Administration established the framework for the Science Investigator-led Processing Systems (SIPS) to enable the Earth science data products to be generated by personnel directly associated with the instrument science team and knowledgeable of the science algorithms. One of the first instantiations implemented for NASA was the Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E) SIPS. The AMSR-E SIPS is a decentralized, geographically distributed ground data processing system composed of two primary components located in California and Alabama. Initial science data processing in the U.S. is conducted at Remote Sensing Systems (RSS) in Santa Rosa, California. RSS ingests antenna temperature orbit data sets from the Japanese Aerospace Exploration Agency and converts them to calibrated, resampled, geolocated brightness temperatures. The brightness temperatures are sent to the Global Hydrology and Climate Center in Huntsville, Alabama, which generates the geophysical science data products (e.g., water vapor, sea surface temperature, sea ice extent, etc.) suitable for climate research and applications usage. These science products are subsequently sent to the National Snow and Ice Data Center Distributed Active Archive Center in Boulder, Colorado for archival and dissemination to the at-large science community. This paper describes the organization, coordination and production techniques employed by the AMSR-E SIPS in implementing, automating and operating the distributed data processing system.

Introducing Provenance Capture into a Legacy Data System

Accurate provenance information facilitates improved understanding of Earth science data and scientific reproducibility and can serve as an indicator of data quality. Provenance capture is an integral part of many modern workflow systems but may not have been considered in the design of legacy data production systems. Furthermore, in addition to data lineage, it is also important to capture contextual information needed for understanding how a data set was produced. This paper describes our experience in retrofitting a legacy data system to support capture, storage, and dissemination of provenance. Data inputs and transformations are logged automatically, while broader context information describing science algorithms and ancillary files is manually compiled. Provenance and context information are integrated for interactive user access and embedded into data files as XML documents compliant with the “Lineage” specification for geographic metadata defined by the International Organization for Standardization in the ISO 19115-2 standard. Lessons learned from this approach can inform others who need to incorporate provenance into a data system after the fact.

The use of NASA LANCE imagery and data for Near real-time applications

Advances in satellite technologies, computing and web mapping have led to a huge increase in the number of people accessing satellite data, or satellite-derived information. Satellite images are routinely used in media reports, virtual globes and interactive maps. The increased exposure, and familiarization, of the general public to satellite data and products is leading to greater expectations about what data should be available and how they should be packaged. To meet these expectations, NASAs Land Atmosphere Near-real time Capability for EOS (LANCE) has refined the way in which users can browse, filter and retrieve satellite imagery for a particular area of interest. The developments on LANCE, part of the NASA Earth Data website, are largely user-driven based on interviews and interactions with end users. This paper describes the tools available at LANCE, key application areas supported and examples of how LANCE data are being used. All of the LANCE tools can be accessed through http://earthdata.nasa.gov/lance.

SCOOP Data Management: A Standards-based Distributed System for Coastal Data and Modeling

The Southeastern Universities Research Association (SURA) coastal ocean observing and prediction (SCOOP) program is a SURA Coastal Research initiative that is deploying cutting edge information technology to advance the science of environmental prediction and hazard planning for our nations coasts. SCOOP is a distributed program, incorporating heterogeneous data, software and hardware; thus the use of standards to enable interoperability is key to SCOOPs success. Standards activities range from internal coordination among SCOOP partners to participation in national standards efforts. As the lead partner in the SCOOP program for both data management and data translation, the University of Alabama in Huntsville (UAH) is developing a suite of advanced technologies to provide core data and information management services for scientific data, including the SCOOP Catalog and a suite of standards-based web services providing Catalog access. Currently under development is a web service that will export information on SCOOP data collections in a schema compliant with the Federal Geographic Data Committees Content Standard for Digital Geospatial Metadata. SCOOP is also a participant in the OpenlOOS Interoperability Demonstration, which leverages open geospatial consortium (OGQ standards such as the Web map service (WMS) and Web Feature Service (WFS) protocols to display near real time coastal observations together with water level, wave, and surge forecasts. SCOOP partners are also active participants in several data and metadata standards efforts, including the national ocean sciences data management and communications metadata studies and the marine metadata interoperability project. Continued close cooperation between the IT and coastal science modeling communities is producing positive results toward a real-time modeling environment that will benefit coastal stakeholders through better predictive capabilities.

Flexible processing at the AMSR-E SIPS

Processing requirements at NASAs AMSR-E SIPS * have evolved considerably throughout the life span of the mission, as the SIPS has moved from mission testing and early experimental data processing to full operations supporting both forward processing and reprocessing. Working in close collaboration with the University of Alabama in Huntsvilles Information Technology and Systems Center (ITSC), the SIPS has developed a robust and flexible hardware and software framework that supports a variety of data streams and processing requirements. Today the SIPS maintains four operational processing environments in addition to development and integration test areas: • Routine operations environment for forward processing of AMSR-E data upon acquisition, • Late processing environment to handle occasional swaths of data that arrive after gridded products have been created for their time period, • Reprocessing environment for large-scale reprocessing efforts of individual data products or the entire product suite, • A special processing environment to manage special requests by the AMSR-E science team, such as evaluation of new algorithms with seasonal data. In addition, ITSC personnel have integrated science data subsetting and browse imagery generation into the SIPS processing flow, enabling the scientists to quickly and easily target parameters of interest. While similar in structure, each of these environments and their target data streams have unique requirements that would normally necessitate customized software. This paper will describe the evolution of the SIPS, the resulting processing and distribution framework, and plans to meet future requirements.