Christopher Lynnes

Access, Visualization, and Interoperability of Air Quality Remote Sensing Data Sets via the Giovanni Online Tool

This paper describes the air quality data products and services available through Giovanni, a web based tool for access, visualization, and analysis of satellite remote sensing products, and also model output and surface observations relevant to global air quality. Available datasets include total column aerosol measurements from numerous satellite instruments, column NO2 and SO2, vertical aerosol products from CALIPSO, surface PM2.5 concentrations over the continental U.S, and speciated model Aerosol Optical Depth. Giovanni was designed to make satellite and ground-based data easier to use; it does not require separate access to or downloading of data sets, making the visualizations and analysis services accessible to both the novice and the experienced user. Giovanni air quality data products are provided on a common grid and can also be obtained in KMZ format for Google Earth visualization. This feature allows collocation of datasets to aid in analysis of pollution events and to facilitate satellite/monitor comparisons and aerosol intercomparison studies in a fraction of the time compared to traditional methods. Giovanni also supports multiple interoperability protocols which permit data sharing with other online tools, in order to enhance access to the datasets for improved air quality decision making. The Giovanni team is currently actively involved in several data networking initiatives with service oriented tools at other institutions such as DataFed.

Evolution of Information Management at the GSFC Earth Sciences (GES) Data and Information Services Center (DISC): 2006–2007

Increasingly sophisticated National Aeronautics and Space Administration (NASA) Earth science missions have driven their associated data and data management systems from providing simple point-to-point archiving and retrieval to performing user-responsive distributed multisensor information extraction. To fully maximize the use of remote-sensor-generated Earth science data, NASA recognized the need for data systems that provide data access and manipulation capabilities responsive to research brought forth by advancing scientific analysis and the need to maximize the use and usability of the data. The decision by NASA to purposely evolve the Earth Observing System Data and Information System (EOSDIS) at the Goddard Space Flight Center (GSFC) Earth Sciences (GES) Data and Information Services Center (DISC) and other information management facilities was timely and appropriate. The GES DISC evolution was focused on replacing the EOSDIS Core System (ECS) by reusing the in-house developed disk-based Simple, Scalable, Script-based Science Product Archive (S4PA) data management system and migrating data to the disk archives. Transition was completed in December 2007.

Global bias adjustment for MODIS aerosol optical thickness using neural network

Abstract Large uncertainties in calculating radiative forcings from aerosols due to their location, loading, and types pose a great challenge to global climate modeling. Trying to improve retrievals in a statistical manner normally requires detailed knowledge of uncertainty statistics and bias due to possible error sources such as different measurement viewing geometries, instrument calibration, and dynamically changing atmospheric and earth surface conditions. However, a priori estimates of these error sources are not, in general, available. The use of a neural network (NN) approach to compensate for biases and systematic errors of aerosol optical thickness (AOT) from the Moderate Resolution Imaging Spectrometer (MODIS) operational retrieval algorithm is explored. By utilizing the NN as an estimator, we can compensate against unknown sources of errors, nonlinearity in the data sets, and the presence of non-normal distributions. In this study, the highly accurate ground-based Aerosol Robotic Network (AERONET) measurements are used as the ground truth (GT). Our results show that the adjusted AOT with NN has decreased root mean square errors, improved correlations with GT data by 4% to 6%, and increased the number of NN-adjusted data falling within the published expected error envelope by ∼ 10 % .

Mirador: A Simple Fast Search Interface for Global Remote Sensing Data Sets

A major challenge for remote sensing researchers is searching and acquiring relevant data files for their research projects based on content, space, and time constraints. Several structured query (SQ) and hierarchical navigation (HN) search interfaces have been developed to satisfy this requirement. However, the popularity of free-text (FT) search in the general domain led the Goddard Earth Sciences Data and Information Services Center to develop an FT search interface named Mirador that supports space-time queries, including a gazetteer and geophysical event gazetteer. In order to compensate for a slightly reduced search precision relative to SQ and HN methods, Mirador uses several search optimizations to return results quickly, enabling iterative search strategies.

Big Data Challenges in Climate Science: Improving the next-generation cyberinfrastructure

The knowledge we gain from research in climate science depends on the generation, dissemination, and analysis of high-quality data. This work comprises technical practice as well as social practice, both of which are distinguished by their massive scale and global reach. As a result, the amount of data involved in climate research is growing at an unprecedented rate. Some examples of the types of activities that increasingly require an improved cyberinfrastructure for dealing with large amounts of critical scientific data are climate model intercomparison (CMIP) experiments; the integration of observational data and climate reanalysis data with climate model outputs, as seen in the Observations for Model Intercomparison Projects (Obs4MIPs), Analysis for Model Intercomparison Projects (Ana4MIPs), and Collaborative Reanalysis Technical Environment-Intercomparison Project (CREATE-IP) activities; and the collaborative work of the Intergovernmental Panel on Climate Change (IPCC). This article provides an overview of some of climate sciences big data problems and the technical solutions being developed to advance data publication, climate analytics as a service, and interoperability within the Earth System Grid Federation (ESGF), which is the primary cyberinfrastructure currently supporting global climate research activities.

A quality screening service for remote sensing data

NASA provides a wide variety of Earth-observing satellite data products to a diverse community. These data are annotated with quality information in a variety of ways, with the result that many users struggle to understand how to properly account for quality when dealing with satellite data. To address this issue, a Data Quality Screening Service (DQSS) is being implemented for a number of datasets. The DQSS will enable users to obtain data files in which low-quality pixels have been filtered out, based either on quality criteria recommended by the science team or on the users particular quality criteria. The objective is to increase proper utilization of this critical quality data in science data analysis of satellite data products.

Ensuring Long-Term Access to Remotely Sensed Data With Layout Maps

The Hierarchical Data Format (HDF) has been a data format standard in National Aeronautic and Space Administration (NASA)s Earth Observing System Data and Information System since the 1990s. Its rich structure, platform independence, full-featured application programming interface (API), and internal compression make it very useful for archiving science data and utilizing them with a rich set of software tools. However, a key drawback for long-term archiving is the complex internal byte layout of HDF files, requiring one to use the API to access HDF data. This makes the long-term readability of HDF data for a given version dependent on long-term allocation of resources to support that version. Much of the data from NASAs Earth Observing System have been archived in HDF Version 4 (HDF4) format. To address the long-term archival issues for these data, a collaborative study between The HDF Group and NASAs Earth Science Data Centers (ESDCs) is underway. One of the first activities was an assessment of the range of HDF4-formatted data held by NASA to determine the capabilities inherent in the HDF format that were used in practice and for use in estimating the effort for full implementation across NASAs ESDCs. Based on the results of this assessment, methods for producing a map of the layout of the HDF4 files held by NASA were prototyped using a markup-language-based HDF tool. The resulting maps allow a separate program to read the file without recourse to the HDF API. To verify this, two independent tools based solely on the map files were developed and tested.

Interoperability Middleware between Geoscience and Geospatial Catalog Protocols

OGC protocols have been widely accepted by the geospatial/land science communities, We call this community geospatial community. Others communities such as atmosphere, ocean, and modeling science adopt the other set of data access protocols. We name these protocols as geoscience protocols. Although these two protocols have been used wildly, but unfortunately, there is no interoperability between them. For bridging this cross-protocol and cross-community gap, CSISS have developed a catalog middleware to mediate client/server interactions between OGC catalog clients and THREDDS servers. A prototype system has been implemented to demonstrate the concept and approach.

A Simple, Scalable, Script-Based Science Processor

In 1999, the impending launch of the Terra satellite, combined with concerns about the ability of commercial software to process its high data volumes, led the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) to develop a contingency science processing system. Severe time and money constraints forced the GES DAAC towards a minimalist architecture that eventually becomes the Simple, Scalable, Script-based Science Processor (S4P). It was named Simple for its architecture and small size (a few thousand lines of code), Scalable for its ability to scale up to heavy processing loads or down to modest automation tasks, Script-based for its reliance on the Perl scripting language for its infrastructure, and Science Processor for its genesis in running scientific algorithms.

Simple, Scalable, Script-Based Science Processor (S4P)

The development and deployment of data processing systems to process Earth Observing System (EOS) data has proven to be costly and prone to technical and schedule risk. Integration of science algorithms into a robust operational system has been difficult. The core processing system, based on commercial tools, has demonstrated limitations at the rates needed to produce the several terabytes per day for EOS, primarily due to job management overhead. This has motivated an evolution in the EOS Data Information System toward a more distributed one incorporating Science Investigator-Led Processing Systems (SIPS). As part of this evolution, the Goddard Earth Sciences Distributed Active Archive Center (GES DAAC) has developed a simplified processing system to accommodate the increased load expected with the advent of reprocessing and launch of a second satellite. This system, the Simple, Scalable, Script-Based Science Processor (S4P) could also serve as a resource for future SIPS.