Tara D. Gibson

An Extensible, Scalable Architecture for Managing Bioinformatics Data and Analyses

Systems biology research demands the availability of tools and technologies that span a comprehensive range of computational capabilities, including data management, transfer, processing, integration, and interpretation. To address these needs, we have created the bioinformatics resource manager (BRM), a scalable, flexible, and easy to use tool for biologists to undertake complex analyses. This paper describes the underlying software architecture of the BRM that integrates multiple commodity platforms to provide a highly extensible and scalable software infrastructure for bioinformatics. The architecture integrates a J2EE 3-tier application with an archival experimental data management system, the GAGGLE framework for desktop tool integration, and the MeDICi integration framework for high-throughput data analysis workflows. This architecture facilitates a systems biology software solution that enables the entire spectrum of scientific activities, from experimental data access to high throughput processing and analysis of data for biologists and experimental scientists.

GridOPTICS(TM) A Novel Software Framework for Integrating Power Grid Data Storage, Management and Analysis

This paper describes the architecture and design of GridOPTICSTM, a novel software framework for integrating a collection of software tools developed by NPNNLs Future Power Grid Initiative (FPGI) into a coherent, powerful operations and planning tool for the power grid of the future. GridOPTICSTM enables plug-and-play of various analysis, modeling and visualization software tools to improve the efficiency and reliability of power grid. To bridge the data access for different control purposes, GridOPTICSTM provides a scalable, lightweight event processing layer that hides the complexity of data collection, storage, delivery and management. A significant challenge is the requirement to access large amount of data in real time. We address this challenge though a scalable system architecture that balances system performance and ease of integration. The initial prototype of GridOPTICSTM was demonstrated with several use cases from PNNLs FPGI and show that our system can provide real time data access to a diverse set of applications with easy to use APIs.

Power Grid Data Analysis with R and Hadoop

In this chapter, we use the R and Hadoop Integrated Programming Environment (RHIPE) as a flexible, scalable environment for analyzing multiterabyte data sets being produced by a phasor measurement unit sensor network on the electrical power grid. RHIPE enables exploratory data analysis on the entire data set, allowing us to develop both data cleaning and event classification methods that reflect event characteristics as represented by the actual data instead of relying on theoretical models. We describe several of the data cleaning filters that we have developed as well as one approach we have used for event detection. To ensure the generality of this chapter, we focus on the techniques we are using for our data analysis and example code that demonstrates how these techniques are used within the RHIPE package, instead of the domain-specific details of the data or events that we are extracting.

Provenance Store Evaluation

Requirements for the provenance store and access API are developed. Existing RDF stores and APIs are evaluated against the requirements and performance benchmarks. The team’s conclusion is to use MySQL as a database backend, with a possible move to Oracle in the near-term future. Both Jena and Sesame’s APIs will be supported, but new code will use the Jena API

An Integrated Security Framework for GOSS Power Grid Analytics Platform

In power grid operations, security is an essential component for any middleware platform. Security protects data against unwanted access as well as cyber attacks. GridOpticsTM Software System (GOSS) is an open source power grid analytics platform that facilitates ease of access between applications and data sources and promotes development of advanced analytical applications. GOSS contains an API that abstracts many of the difficulties in connecting to various heterogeneous data sources. A number of applications and data sources have already been implemented to demonstrate functionality and ease of use. A security framework has been implemented which leverages widely accepted, robust Java TM security tools in a way such that they can be interchanged as needed. This framework supports the complex fine-grained, access control rules identified for the diverse data sources already in GOSS. Performance and reliability are also important considerations in any power grid architecture. An evaluation is done to determine the overhead cost caused by security within GOSS and ensure minimal impact to performance.

A Multi-Tier Provenance Model for Global Climate Research

Global climate researchers rely upon many forms of sensor data and analytical methods to help profile subtle changes in climate conditions. The U.S. Department of Energy Atmospheric Radiation Measurement (ARM) program provides researchers with curated Value Added Products (VAPs) resulting from continuous instrumentation streams, data fusion, and analytical profiling. The ARM operational staff and software development teams (data producers) rely upon a number of techniques to ensure strict quality control (QC) and quality assurance (QA) standards are maintained. Climate researchers (data consumers) are highly interested in obtaining as much provenance evidence as possible to establish data trustworthiness. Currently all the evidence is not easily attainable or identifiable without significant efforts to extract and piece together information from configuration files, log files, codes, or status information on the ARM website. Our objective is to identify a provenance model that serves the needs of both the VAP producers and consumers. This paper shares our initial results – a comprehensive multi-tier provenance model. We describe how both ARM operations staff and the climate research community can greatly benefit from this approach to more effectively assess and quantify the data historical record.

Large-scale exploratory analysis, cleaning, and modeling for event detection in real-world power systems data

In this paper, we present an approach to large-scale data analysis, Divide and Recombine (D&R), and describe a hardware and software implementation that supports this approach. We then illustrate the use of D&R on large-scale power systems sensor data to perform initial exploration, discover multiple data integrity issues, build and validate algorithms to filter bad data, and construct statistical event detection algorithms. This paper also reports on experiences using a non-traditional Hadoop distributed computing setup on top of a HPC computing cluster.