Kathryn Regner

Using sensor web protocols for environmental data acquisition and management

Abstract Standard interfaces for data and information access facilitate data management and usability by minimizing the effort required to acquire, catalog and integrate data from a variety of sources. The authors have prototyped several data management and analysis applications using Sensor Web Enablement Services, a suite of service protocols being developed by the Open Geospatial Consortium specifically for handling sensor data in near-real time. This paper provides a brief overview of some of the service protocols and describes how they are used in various sensor web projects involving near-real-time management of sensor data.

Using Sensor Web Processes and Protocols to Assimilate Satellite Data into a Forecast Model

Working closely with atmospheric scientists at the Marshall Space Flight Center, researchers at the University of Alabama in Huntsville are applying Sensor Web Enablement (SWE) technologies to the real world problem of efficiently assimilating NASA satellite data into weather forecast models in near real time. By implementing SWE protocols and services into our Data Assimilation System we expect to realize a processing framework that is distributed, interoperable and plug-and-play, thereby increasing access to scientific products in a more efficient, autonomous, and affordable way.

Science data processing for the Advanced Microwave Scanning Radiometer-Earth observing system

The National Aeronautics and Space Administration established the framework for the Science Investigator-led Processing Systems (SIPS) to enable the Earth science data products to be generated by personnel directly associated with the instrument science team and knowledgeable of the science algorithms. One of the first instantiations implemented for NASA was the Advanced Microwave Scanning Radiometer for the Earth Observing System (AMSR-E) SIPS. The AMSR-E SIPS is a decentralized, geographically distributed ground data processing system composed of two primary components located in California and Alabama. Initial science data processing in the U.S. is conducted at Remote Sensing Systems (RSS) in Santa Rosa, California. RSS ingests antenna temperature orbit data sets from the Japanese Aerospace Exploration Agency and converts them to calibrated, resampled, geolocated brightness temperatures. The brightness temperatures are sent to the Global Hydrology and Climate Center in Huntsville, Alabama, which generates the geophysical science data products (e.g., water vapor, sea surface temperature, sea ice extent, etc.) suitable for climate research and applications usage. These science products are subsequently sent to the National Snow and Ice Data Center Distributed Active Archive Center in Boulder, Colorado for archival and dissemination to the at-large science community. This paper describes the organization, coordination and production techniques employed by the AMSR-E SIPS in implementing, automating and operating the distributed data processing system.

Introducing Provenance Capture into a Legacy Data System

Accurate provenance information facilitates improved understanding of Earth science data and scientific reproducibility and can serve as an indicator of data quality. Provenance capture is an integral part of many modern workflow systems but may not have been considered in the design of legacy data production systems. Furthermore, in addition to data lineage, it is also important to capture contextual information needed for understanding how a data set was produced. This paper describes our experience in retrofitting a legacy data system to support capture, storage, and dissemination of provenance. Data inputs and transformations are logged automatically, while broader context information describing science algorithms and ancillary files is manually compiled. Provenance and context information are integrated for interactive user access and embedded into data files as XML documents compliant with the “Lineage” specification for geographic metadata defined by the International Organization for Standardization in the ISO 19115-2 standard. Lessons learned from this approach can inform others who need to incorporate provenance into a data system after the fact.

The use of NASA LANCE imagery and data for Near real-time applications

Advances in satellite technologies, computing and web mapping have led to a huge increase in the number of people accessing satellite data, or satellite-derived information. Satellite images are routinely used in media reports, virtual globes and interactive maps. The increased exposure, and familiarization, of the general public to satellite data and products is leading to greater expectations about what data should be available and how they should be packaged. To meet these expectations, NASAs Land Atmosphere Near-real time Capability for EOS (LANCE) has refined the way in which users can browse, filter and retrieve satellite imagery for a particular area of interest. The developments on LANCE, part of the NASA Earth Data website, are largely user-driven based on interviews and interactions with end users. This paper describes the tools available at LANCE, key application areas supported and examples of how LANCE data are being used. All of the LANCE tools can be accessed through http://earthdata.nasa.gov/lance.

Flexible processing at the AMSR-E SIPS

Processing requirements at NASAs AMSR-E SIPS * have evolved considerably throughout the life span of the mission, as the SIPS has moved from mission testing and early experimental data processing to full operations supporting both forward processing and reprocessing. Working in close collaboration with the University of Alabama in Huntsvilles Information Technology and Systems Center (ITSC), the SIPS has developed a robust and flexible hardware and software framework that supports a variety of data streams and processing requirements. Today the SIPS maintains four operational processing environments in addition to development and integration test areas: • Routine operations environment for forward processing of AMSR-E data upon acquisition, • Late processing environment to handle occasional swaths of data that arrive after gridded products have been created for their time period, • Reprocessing environment for large-scale reprocessing efforts of individual data products or the entire product suite, • A special processing environment to manage special requests by the AMSR-E science team, such as evaluation of new algorithms with seasonal data. In addition, ITSC personnel have integrated science data subsetting and browse imagery generation into the SIPS processing flow, enabling the scientists to quickly and easily target parameters of interest. While similar in structure, each of these environments and their target data streams have unique requirements that would normally necessitate customized software. This paper will describe the evolution of the SIPS, the resulting processing and distribution framework, and plans to meet future requirements.

Product Delivery Record Server Data Transfer at the AMSR-E SIPS

Issues associated with failed data transfers can be complicated and inconvenient, not only for end users expecting timely products, but also for the data managers who are responsible for recovering from failures. Working in close collaboration with engineers at the University of Alabama in Huntsvilles Information Technology and Systems Center, the AMSR-E SIPS has evolved a local implementation of the product delivery record server from a simple ECS-compliant data exchange system to a smart automated message handling system that also executes routine operational error recovery procedures. While automated data exchange mechanisms have been the norm for quite some time, the AMSR-E SIPS has deployed unique technologies in our local implementation of the PDRS by integrating it with and automating some routine data management functions, such as process scheduling, restaging of certain classes of failed data transfers, and statistics reporting. This paper discusses the evolution of the PDRS design and architecture, highlights the unique features of this flexible processing system that has contributed to reduced operating costs, and describes our plans to meet future requirements.