James Frew

Lineage retrieval for scientific data processing: a survey

Scientific research relies as much on the dissemination and exchange of data sets as on the publication of conclusions. Accurately tracking the lineage (origin and subsequent processing history) of scientific data sets is thus imperative for the complete documentation of scientific work. Researchers are effectively prevented from determining, preserving, or providing the lineage of the computational data products they use and create, however, because of the lack of a definitive model for lineage retrieval and a poor fit between current data management tools and scientific software. Based on a comprehensive survey of lineage research and previous prototypes, we present a metamodel to help identify and assess the basic components of systems that provide lineage retrieval for scientific data products.

Rapid calculation of terrain parameters for radiation modeling from digital elevation data

Digital elevation models are widely used to calculate terrain parameters to determine incoming solar and longwave radiation for use in surface climate models, interpretation of remote-sensing data, and parameters in hydrologic models. Because of the large number of points in an elevation grid, fast algorithms are useful to save computation time. A description is given of rapid methods for calculating slope and azimuth, solar illumination angle, horizons, and view factors for radiation from sky and terrain. Calculation time is reduced by fast algorithms and lookup tables. >

The SEQUOIA 2000 storage benchmark

This paper presents a benchmark that concisely captures the data base requirements of a collection of Earth Scientists working in the SEQUOIA 2000 project on various aspects of global change research. This benchmark has the novel characteristic that it uses real data sets and real queries that are representative of Earth Science tasks. Because it appears that Earth Science problems are typical of the problems of engineering and scientific DBMS users, we claim that this benchmark represents the needs of this more general community. Also included in the paper are benchmark results for three example DBMSs: GRASS, IPW and POSTGRES.

Earth System Science Workbench: a data management infrastructure for earth science products

The Earth System Science Workbench (ESSW) is a non-intrusive data management infrastructure for researchers who are also data publishers. An implementation of ESSW to track the processing of locally received satellite imagery is presented, demonstrating the Workbenchs transparent and robust support for archiving and publishing data products. ESSW features a Lab Notebook metadata service, an ND-WORM (No Duplicate-Write Once Read Many) storage service, and Web user interface tools. The Lab Notebook logs processes (experiments) and their relationships via a custom API to XML documents stored in a relational database. The ND-WORM provides a managed storage archive for the Lab Notebook by keeping unique file digests and name-space meta-data, also in a relational database. ESSW Notebook tools allow project searching and ordering, and file and meta-data management.

Alexandria digital library: user evaluation studies and system design

The Alexandria Digital Library (ADL) is one of the six digital library projects funded by NSF, DARPA, and NASA. ADLs collection and services focus on information containing georeferences: maps, images, data sets, text, and other information sources with links to geographic locations. During this study period, three different user interfaces were developed and tested by user groups. User feedback was collected through various formal and informal approaches and the results fed back into the design and implementation cycle. This article describes the evolution of the ADL system and the effect of user evaluation on that evolution. ADL is an ongoing project; user feedback and evaluation plans for the remainder of the project are described.

Alexandria Digital Library

The goal of the Alexandria Digital Library Project is to develop a distributed system that provides a comprehensive range of library services for collections of spatially indexed and graphical information. While such collections include digitized maps and images as important special components, the Alexandria Digital Library will involve a very wide range of graphical materials and will include textual materials. Users of the Alexandria Digital Library will range from school children to academic researchers to members of the general public. They will be able to retrieve materials from the library on the basis of information content as well by reference to spatial location.

Collection metadata solutions for digital library applications

Within a digital library, collections may range from an ad hoc set of objects that serve a temporary purpose to established library collections intended to persist through time. The objects in these collections vary widely, from library and data center holdings to pointers to real‐world objects, such as geographic places, and the various metadata schemas that describe them. The key to integrated use of such a variety of collections in a digital library is collection metadata that represents the inherent and contextual characteristics of a collection. The Alexandria Digital Library (ADL) Project has designed and implemented collection metadata for several purposes: in XML form, the collection metadata “registers” the collection with the user interface client; in HTML form, it is used for user documentation; eventually, it will be used to describe the collection to network search agents; and it is used for internal collection management, including mapping the object metadata attributes to the common search parameters of the system.

Volunteered geographic information production as a spatial process

Wikipedia is a free encyclopedia that anyone can edit and a popular example of user-generated content that includes volunteered geographic information (VGI). In this article, we present three main contributions: (1) a spatial data model and collection methods to study VGI in systems that may not explicitly support geographic data; (2) quantitative methods for measuring distance between online authors and articles; and (3) empirically calibrated results from a gravity model of the role of distance in VGI production. To model spatial processes of VGI contributors, we use an invariant exponential gravity model based on article and author proximity. We define a proximity metric called a ‘signature distance’ as a weighted average distance between an article and each of its authors, and we estimate the location of 2.8 million anonymous authors through IP geolocation. Our study collects empirical data directly from 21 language-specific Wikipedia databases, spanning 7 years of contributions (2001–2008) to nearly 1 million geotagged articles. We find empirical evidence that the spatial processes of anonymous contributors fit an exponential distance decay model. Our results are consistent with the prior results on information diffusion as a spatial process, but run counter to theories that a globalized Internet neutralizes distance as a determinant of social behaviors.

The ADEPT digital library architecture

The Alexandria Digital Earth ProtoType (ADEPT) architecture is a framework for building distributed digital libraries of georeferenced information. An ADEPT system comprises one or more autonomous libraries, each of which provides a uniform interface to one or more collections, each of which manages metadata for one or more items. The primary standard on which the architecture is based is the ADEPT bucket framework, which defines uniform client-level metadata query services that are compatible with heterogeneous underlying collections. ADEPT functionality strikes a balance between the simplicity of Web document delivery and the richness of Z39.50. The current ADEPT implementation runs as servlet-based middleware and supports collections housed in arbitrary relational databases.

Composing lineage metadata with XML for custom satellite-derived data products

As peer-to-peer dissemination of custom data products evolves among Earth science research groups, investigators and data managers must consider how to compose appropriate metadata for their research computing activities. Because workflows may span multiple groups, it is critical that lineage (provenance) metadata also be assembled to document and preserve the origins and processing history of constituent data products and transformations for future data consumers. To demonstrate methods for composing lineage metadata for custom processing, we introduce our terminology for workflow and employ a case study for the creation of satellite-derived ocean color data products. Our example contributes to a general metadata model for workflow that incorporates lineage. We then discuss metadata requirements for remote sensing-related data products. We propose two techniques for composing lineage metadata, both based on accessory XML metadata documents that are paired with the data products and versioned data transformations they describe. The first technique, implemented as a prototype, features a dedicated lineage server that introduces the indirection and flexibility necessary for Web-based lineage navigation. The second, more promising technique involves defining a simple Resource Description Framework (RDF) vocabulary for lineage metadata, and using extant RDF/XML tools for query and navigation. These methods provide guidelines for composing lineage metadata that are applicable to other domains.