Xiaoyang Sean Wang

Information technology implementation for a distributed data system serving Earth scientists: seasonal to interannual ESIP

We address the implementation of a distributed data system designed to serve Earth system scientists. A consortium led by George Mason University has been funded by NASAs Working Prototype Earth Science Information Partner (WP-ESIP) program to develop, implement, and operate a distributed data and information system. The system will address the research needs of seasonal to interannual scientists whose research focus includes phenomena such as El Nino, monsoons and associated climate studies. The system implementation involves several institutions using a multitiered client-server architecture. Specifically the consortium involves an information system of three physical sites, GMU, the Center for Ocean-Land-Atmosphere Studies (COLA) and the Goddard Distributed Active Archive Center, distributing tasks in the areas of user services, access to data, archiving, and other aspects enabled by a low-cost, scalable information technology implementation. The project can serve as a model for a larger WP-ESIP Federation to assist in the overall data information system associated with future large Earth Observing System data sets and their distribution. The consortium has developed innovative information technology techniques such as content based browsing, data mining and associated component working prototypes; analysis tools particularly GrADS developed by COLA, the preferred analysis tool of the working seasonal to interannual communities; and a Java front-end query engine working prototype.

Managing scientific metadata using XML

We present our XML-based Distributed Metadata Server (Dimes) - which comprises a flexible metadata model, search software, and a Web-based interface - to support multilevel metadata access, and introduce two prototype systems. Our Scientific Data and Information Super Server (SDISS), which is based on Dimes and GDS, solves accurate data-search and outdated data-link problems by integrating metadata with the data systems. On the implementation front, we combine independent components and open-source technologies into a coherent system to dramatically extend system capabilities. Obviously, our approach can be applied to other scientific communities, such as bioinformatics and space science.

Optimizing statistical queries by exploiting orthogonality and interval properties of grouping relations

A statistical query first manipulates source category data to build a target category in the form of a grouping relation and then performs statistical functions on the associated measurement data. In this paper, the attributes in a grouping relation are partitioned into pair-wise disjoint sets, each called a dimension. A grouping relation is said to be orthogonal if it is equal to the cross product of the projections of itself on all the dimensions. Orthogonality is useful in searching for and using pre-computed summaries on other categories. However, a grouping relation is sometimes not orthogonal, but rather k-partially orthogonal (i.e., the union of k orthogonal ones). It is shown that it is NP-complete to decide if a grouping relation is k-partially orthogonal. The paper then gives an algorithm to derive partial orthogonality. Also investigated in this paper are interval properties of grouping relations useful for optimizing statistical queries. An algorithm is described to derive interval properties.

A pyramid data model for supporting content-based browsing and knowledge discovery

Remote sensing from space can provide global and continuous observations. The associated measurement data need to be stored and studied to understand the Earth system processes. The ability of interactive content-based browsing, i.e., browsing or searching the content to narrow-down the interesting portions of data sets prior to actually accessing or ordering full data sets, is highly desirable for any Earth science data information system. However the large volumes of archived and future Earth science remote sensing data are clearly a serious challenge for an interactive browsing process. In this paper a pyramid data model is introduced to support interactive content-based browsing and knowledge discovery for a wide variety of Earth science remote sensing data sets. By using multi-level precomputation and robust nonparametric approximation procedures, the interactive browsing performance can be enhanced greatly. An initial implementation and testing of this data model has been carried out through our research prototype system, Virtual Domain Application Data Center (VDADC). Future implementations are planned for our Seasonal to Interannual Earth Science Information Partner (SIESIP) project.

Value Range Queries on Earth Science Data via Histogram Clustering

Remote sensing data as well as ground-based and model output data about the Earth system can be very large in volume. On the other hand, in order to use the data efficiently, scientists need to search for data based on not only metadata but also actual data values. To answer value range queries by scanning very large volumes of data is obviously unrealistic. This article studies a clustering technique on histograms of data values on predefined cells to index the cells. Through this index system, the so-called statistical range queries can be answered quickly and approximately together with an accuracy assessment. Examples of using this technique for Earth science data sets are given in this article.

Using triangle inequality to efficiently process continuous queries on high-dimensional streaming time series

In many applications, it is important to quickly find, from a database of patterns, the nearest neighbors of high-dimensional query points that come into the system in a streaming form. Treating each query point as a separate one is inefficient. Consecutive query points are often neighbors in the high-dimensional space, and intermediate results in the processing of one query should help the processing of the next. This paper extends the KD tree with triangle inequality to deal with high-dimensional streaming time series. More specifically, the distances calculated for earlier query points (to patterns) are used to filter out patterns that are not possible to be the nearest neighbor of the current one. Experiments show that this extension works well.

Remote data access via the SIESIP distributed information system

Illustrates a distributed system that provides online searching, analysis and ordering capabilities for distributed Earth science data. The system is under development by a consortium led by George Mason University in a project called Seasonal-to-Interannual Earth Science Information Partners (SIESIP) as a part of a federation of information partners funded by NASA. The integrated system is composed of data, a database management system (DBMS), communication protocols, data analysis tools and a user interface. Through a Web-based Java GUI, users can search the DBMS for metadata information, conduct content-based searches, perform some initial analyses and issue an order for the selected data.

The seasonal to interannual Earth Science Information Partner: a distributed data and information broker

A consortium of three main sites consisting of the Center for Earth Observing and Space Research (CEOSR) at George Mason University (GMU), the Center for Ocean-Land-Atmosphere Studies (COLA) and the NASA Goddard Space Flight Center Distributed Active Archive Center (GDAAC) has been formed. This consortium was recently funded by NASAs Earth Science Information Partner program to provide data and information products serving the needs of seasonal to interannual (S-I) scientists and other related science communities. Specifically, CEOSR will provide information technology and a distributed system architecture as wall as specific data products to be accessed by the scientific communities, COLA will provide scientific direction and a widely-used tool in the S-I research communities and GDAAC data management and archiving. Along with several other contributors, our consortium will create an interdisciplinary source of data for S-I researchers to expand the usage and usefulness of NASA data and serve as broker on behalf of the S-I researchers by providing them enhanced access to NASAs data. The consortium under guidance by an advisory board of experts will select the most appropriate data from the DAAC, NOAA, COLA and the University of Delaware for S-I research applications. Metadata and summary statistics will be extracted and stored in databases at distributed nodes, while the data will be stored on an advanced array of disks. There will be a JAVA-based WWW user interface featuring three advances: (1) content-based searching of the summary metadata (2) exploratory analysis of on-line data and (3) phenomenon-based searching. The consortium will be responsive to the changing needs of its science users as well as in tune with significant information technology advances.