Kai Lin

Managing scientific data: From data integration to scientific workflows*

Scientists are confronted with significant datamanagement problems due to the large volume and high complexity of scientific data. In particular, the latter makes data integration a difficult technical challenge. In this paper, we describe our work on semantic mediation and scientific workflows, and discuss how these technologies address integration challenges in scientific data management. We first give an overview of the main data-integration problems that arise from heterogeneity in the syntax, structure, and semantics of data. Starting from a traditional mediator approach, we show how semantic extensions can facilitate data integration in complex, multipleworlds scenarios, where data sources cover different but related scientific domains. Such scenarios are not amenable to conventional schema-integration approaches. The core idea of semantic mediation is to augment database mediators and query evaluation algorithms with appropriate knowledge-representation techniques to exploit information from shared ontologies. Semantic mediation relies on semantic data registration, which associates existing data with semantic information from an ontology. The Kepler scientific workflow system addresses the problem of synthesizing, from existing tools and applications, reusable workflow components and analytical pipelines to automate scientific analyses. After presenting core features and example workflows in Kepler, we present a framework for adding semantic information to scientific workflows. The resulting system is aware of semantically plausible connections between workflow components as well as between data sources and workflow components. This information can be used by the scientist during workflow design, and by the workflow engineer for creating data transformation steps between semantically compatible but structurally incompatible analytical steps. ∗Work supported by NSF/ITR 0225673 (GEON), NSF/ITR 0225676 (SEEK), NIH/NCRR 1R24 RR019701-01 Biomedical Informatics Research Network (BIRN-CC), and DOE SciDAC DE-FC02-01ER25486 (SDM) †San Diego Supercomputer Center, University of California, San Diego, {ludaesch,lin,bowers,efrat,baru}@sdsc.edu ‡Natural Resources of Canada, brodaric@nrcan.gc.ca

On integrating scientific resources through semantic registration

In many data-centric scientific applications it is common to register datasets and computational services with a federation registry (also commonly called a catalog, directory, or repository). For example, the scientific data-handling system under development in the SEEK project must consider various dataset registries, including: MCAT, for access to SRB-registered datasets Metacat, for KNB-registered datasets DiGIR, for UDDI-registered data and Xanthoria, an XML-based data registry. A challenge for SEEK, and similar efforts such as GEON is to provide uniform access to registries and registered resources, based on emerging Web and grid standards.

The GEON portal: accelerating knowledge discovery in the geosciences

Geoscience studies produce data from various observations, experiments, and simulations at an enormous rate. With proliferation of applications and data formats, the geoscience research community faces many challenges in effectively managing and sharing resources and in efficiently integrating and analyzing the data. In this paper, we discuss how this challenge is being addressed by the GEON Portal, a Web based distributed resource management system that provides integrated access to data and tools needed for knowledge discovery in the geosciences. Unlike previous data management efforts that were either data-driven or application-driven, the GEON Portal provides facilities for efficient sharing, discovery and integration of both data and services that use geoscience data. We identify the challenges involved in managing geoscientific resources and provide solutions that exploit the syntactic, semantic, temporal and spatial metadata associated with the resources. One of our goals is to provide some insight into the challenges involved in providing a comprehensive scientific data management solution based on our experiences with geoscientific data.

A Web service composition and deployment framework for scientific workflows

The article presents the Web services framework in the Kepler scientific workflow system and illustrates them with a real-world example.

GEONGrid portal: design and implementations

We have developed the GEONGrid system for coordinating and managing naturally distributed computing, data, and cluster resources on the cyberinfrastructure. Recently, since the use of Grid technology is still very complex for researchers and scientists, the area of Grid Portals has made excellent progress. The Grid portal system is an emerging open Grid computing environment that promises to provide users with uniform seamless access to remote computing and data resources by providing an easy to use interface to cover over the complexity of more sophisticated Grid technologies. In this paper, we present our initial efforts in the design and implementation of service components in the GEONGrid portal. These service components may be implemented as Web services that follow the conventions of service‐oriented architecture design. In this approach, service components are self‐contained, have a well‐defined programming interface defined in WSDL, and communicate using SOAP messaging. In building a GEONGrid portal, we also use a component‐based user interface design. Portlets provide the desired component model for user interfaces in the same way as Web services. Using this approach, which allows Grid portals to be built out of reusable components, has the obvious advantages of reusability and modularity. Copyright

The GEON service-oriented architecture for Earth Science applications

Abstract The Geosciences Network (GEON) project has been developing cyberinfrastructure for data sharing in the Earth Science community based on a service-oriented architecture. The layered architecture consists of Core, Middleware, and Applications services. Core services provide system-level functions (e.g. user authentication), Middleware services provide generic capabilities (e.g. catalog search), and Application services provide functions that users directly interact with, including applications that are specific to Earth Sciences. The GEON ‘service stack’ includes a standardized set of these services and the corresponding software modules. The GEON Portal provides Web-based access to these services via a set of portlets. This service-oriented approach has enabled GEON to expand to new partner sites and leverage GEON services for other projects. To facilitate interoperation in a distributed geoinformatics environment, GEON is focusing on standards for distributed search across federated catalogs.

Cyberinfrastructure for Observatory and Monitoring Networks: A Case Study from the TEAM Network

Environmental-monitoring and observatory networks currently operating or under development at the national, regional, and global scales have the potential to provide an unprecedented understanding of our natural environment and the threats that endanger it. The breadth of these networks, as well as advances in technology (e.g., from mobile devices to in situ sensors and multidimensional satellite sensor data), will result in larger volumes of data and more complex data sets than ever before. All of these networks require robust cyberinfrastructure to support their varying mission, governance, operational, and scientific objectives. In this article, we use the Tropical Ecology Assessment and Monitoring (TEAM) Network as a fully functional environmental-monitoring network case study to highlight the key cyberinfrastructure components and services that support the network. We provide valuable lessons from our experience building the TEAM Network cyberinfrastructure and suggest future improvements that have broad applicability for other observatory and monitoring networks.

TEAM Network: Building Web-based Data Access and Analysis Environments for Ecosystem Services

A team of ecologists, computer scientists, and engineers from Conservation International (CI) and San Diego Supercomputer Center at the University of California San Diego (SDSC, UCSD) has been collaborating over the past 3 years to develop cyberinfrastructure for the TEAM (Tropical Ecology, Assessment and Monitoring) Network of tropical forest field sites. The TEAM project provides real-time data to understand how tropical forest ecosystems are being impacted by global climate change and land cover change and to improve conservation decisions. A major objective of this project is to provide information and services on tropical forest data disseminated by TEAM sites within countries participating in the TEAM network. The cyberinfrastructure provides a pervasive computational ecosystem, integrating grid computing infrastructure with highperformance backend resources, data warehouses, sophisticated client applications, new instruments, and embedded sensors, thus enabling a new paradigm for monitoring, understanding, and managing ecological and environmental systems. This paper presents the TEAM data management, access, and analysis system that provides end-to-end solutions for sensor-based data and field observations collected at TEAM sites. Specifically, two major applications are presented, viz., the “Data Query and Download Application (DQA)” application, which allows users to navigate and download diverse TEAM datasets such as Tree and Liana Biodiversity, Terrestrial Vertebrate, Climate and Forest Carbon data using a Google Maps based interface; and, the “Forest Carbon Calculator (FCC)” application, which calculates tree biomass using equations for forests with different precipitation regimes, thereby predicting relationships between tree biomass, tree diameter and wood density to estimate the amount of above ground carbon in the forests.

Mediating among GeoSciML resources

Abstract Integration of data across multiple independently developed data sources can be challenging due to a variety of heterogeneities that exist across such systems. Data mediation technologies provide approaches for overcoming these heterogeneities. Standards such as Geoscience Markup Language can address some of the heterogeneity issues by providing schema standards which sources can adhere to. This article addresses the issue of semantic heterogeneity across information resources by using domain ontologies and registering schema elements and data values to such ontologies. Registering data to ontologies provides a powerful search and data integration capability across disparate geoscience information resources.

Firemap: A Dynamic Data-Driven Predictive Wildfire Modeling and Visualization Environment

Abstract Wildfires are destructive fires over the wildland that can wipe out large areas of vegetation and infrastructure. Such fires are hard to control and manage as they can change directions almost instantly, driven by changing environmental conditions. Effective response to such events requires the ability to monitor and predict the behavior of the fire as fast as they change. The WIFIRE project builds an end-to-end cyberinfrastructure for real-time and data-driven simulation, prediction, and visualization of wildfire behavior. One goal of WIFIRE is to provide the tools to predict a more accurate rate of a spreading wildfire. To this end, WIFIRE has developed interfaces for ingesting and visualizing high-density sensor networks to improve fire and weather predictions, and has created a data model for wildfire resources including sensed and archived data, sensors, satellites, cameras, modeling tools, workflows, and social information including Twitter feeds for wildfire research and response. This paper presents WIFIRE’s Firemap web platform to make these geospatial data and products accessible. Through a web browser, Firemap enables geospatial information visualization and a unified access to geospatial workflows using Kepler. Using GIS capabilities combined with scalable big data integration and processing, Firemap enables simple execution of the model with options for running ensembles by taking the information uncertainty into account. The results are easily viewable, sharable, repeatable, and can be animated as a time series.