Craig E. Tull

On-demand grid application tuning and debugging with the NetLogger activation service

Typical grid computing scenarios involve many distributed hardware and software components. The more components that are involved, the more likely it is that one of them may fail. In order for grid computing to succeed, there must be a simple mechanism to determine which component failed and why. Instrumentation of all grid applications and middleware is an important part of the solution to this problem. However, it must be possible to control and adapt the amount of instrumentation data produced in order to not be flooded by this data. We describe a scalable, high-performance instrumentation activation mechanism that addresses this problem.

High-Speed Distributed Data Handling for On-Line Instrumentation Systems

The advent (and promise) of shared, widely available, high-speed networks provides the potential for new approaches to the collection, organization, storage, and analysis of high-speed and high-volume data streams from high data-rate, on-line instruments. We have worked in this area for several years, have identified and addressed a variety of problems associated with this scenario, and have evolved an architecture, implementations, and a monitoring methodology that have been successful in addressing several different application areas. We describe a distributed, wide area network-based architecture that deals with data streams that originate from online instruments. Such instruments and imaging systems are a staple of modern scientific, health care, and intelligence environments. Our work provides an approach for reliable, distributed real-time analysis, cataloguing, and archiving of the data streams through the integration and distributed management of a high-speed distributed cache, distributed high performance applications, and tertiary storage systems.

Workflow management for real-time analysis of lightsource experiments

The Advanced lightsource (ALS) is a X-ray synchrotron facility at Lawrence Berkeley National Laboratory. The ALS generates terabytes of raw and derived data each day and serves 1,000s of researchers each year. Only a subset of the data is analyzed due to barriers in terms of processing that small science teams are ill-equipped to surmount. In this paper, we discuss the development and application of a computational framework, termed SPOT, fed with synchrotron data, powered by storage, networking and compute resources at NERSC and ESnet. We describe issues and recommendations for an end-to-end analysis workflow for ALS data. After one year of operation, the collection contains over 90,000 datasets (550 TB) from 85 users across three beamlines. For 16 months, beamline data taken has been promptly and automatically analyzed and annotated with metadata, allowing users to focus on analysis, conclusions and experiments.

High performance data management and analysis for tomography

The Advanced Light Source (ALS) is a third-generation synchrotron X-ray source that operates as a user facility with more than 40 beamlines hosting over 2000 users per year from around the world. Users of the Hard X-ray Micro-Tomography Beamline (8.3.2) often collect more than 1 Terabyte of raw data per day that in turn generates additional Terabytes of processed data. The data rate continues to increase rapidly due to faster detectors and new sample automation capabilities. We will present the development and deployment of a computational pipeline, fed by data from the ALS, and powered by the storage, networking, and computing resources of the local National Energy Research Scientific Computing Center (NERSC) and the Energy Sciences Network (ESNET). After one year of operation, the system contained 70,000 datasets and 350 TB of data from 85 users. All datasets now collected at the Hard X-ray Tomography Beamline are automatically reconstructed using parameters set by users and/or that are automatically detected from the data acquisition control system. Results are presented to users for visualization through a secure web portal. Users can then download their data or launch a (currently limited but) growing number of operations based on the data-such as filtering, segmentation, and simulation. The massive computational resources of NERSC are thus made available on a level that is easily accessible to the full range of micro-tomography users.

Real-time data-intensive computing

Today users visit synchrotrons as sources of understanding and discovery—not as sources of just light, and not as sources of data. To achieve this, the synchrotron facilities frequently provide not just light but often the entire end station and increasingly, advanced computational facilities that can reduce terabytes of data into a form that can reveal a new key insight. The Advanced Light Source (ALS) has partnered with high performance computing, fast networking, and applied mathematics groups to create a “super-facility”, giving users simultaneous access to the experimental, computational, and algorithmic resources to make this possible. This combination forms an efficient closed loop, where data—despite its high rate and volume—is transferred and processed immediately and automatically on appropriate computing resources, and results are extracted, visualized, and presented to users or to the experimental control system, both to provide immediate insight and to guide decisions about subsequent experim...

Spade: Decentralized orchestration of data movement and warehousing for physics experiments

The paper describes the implementation and experiences gathered from the Spade project which is a decentralized mechanism for data management that has been used in High Energy and Light Source physics experiments. The background and motivations that shaped the Spade project are covered, followed by an outline of the current design of the deployed application. The operational scenarios that have been developed from the use of Spade in a number of difference contexts are enumerated, after which there is a short discussion on how the implementation of the application has progressed. The paper finishes with some highlight of Spades performance and a conclusion about what has been learnt from this project.

Information Technology/Large-Scale Data Handling

The experience of light source users has been transformed in recent years by large increases in flux and brightness, revolutionary new optics and detectors, and automation and advanced sample environments. Beamlines are producing data at rates and volumes that challenge the capabilities of even the most experienced user groups. Meanwhile, the community of synchrotron users continues to grow in size and diversity: researchers come from physics, material science, energy and battery research, geology, biology, chemistry, art history, and more. Almost every natural science domain is being advanced through the techniques employed at these facilities, but a significant fraction of these researchers are first-time or infrequent users of a particular beamline. The combination of an expanding base of new users and increased beamline capabilities is leading to an increase in the amount of “dark data” that is not analyzed fully (or, in some cases, at all).

ALS User Meeting Highlights Accomplishments and Challenges

The 2014 ALS User Meeting, held in Berkeley, California, from October 6-8, 2014, launched with a welcome from UEC Chair Peter Nico and remarks from Associate Lab Director for Energy and Environmental Sciences, Don DePaolo. ALS Director Roger Falcone then addressed the gathering, noting that while the past year has been tough in terms of funding for the ALS, there were also many scientific achievements to highlight and a record number of users with an impressive number of publications.