Luciano Piccoli

Lattice QCD Workflows: A Case Study

This paper discusses the application of existing workflow management systems to a real world science application (LQCD). Typical workflows and execution environment used in production are described. Requirements for the LQCD production system are discussed. The workflow management systems Askalon and Swift were tested by implementing the LQCD workflows and evaluated against the requirements. We report our findings and future work.

High precision SC cavity alignment measurements with higher order modes

Experiments at the FLASH linac at DESY have demonstrated that the higher order modes (HOMs) induced in superconducting cavities can be used to provide a variety of beam and cavity diagnostics. The centers of the cavities can be determined from the beam orbit which produces minimum power in the dipole HOM modes. The phase and amplitude of the dipole modes can be used as a high resolution beam position monitor. For most superconducting accelerators, the existing HOM couplers provide the necessary signals, and the downmix and digitizing electronics are straightforward, similar to those for a conventional BPM.

LQCD workflow execution framework: Models, provenance and fault-tolerance

Large computing clusters used for scientific processing suffer from systemic failures when operated over long continuous periods for executing workflows. Diagnosing job problems and faults leading to eventual failures in this complex environment is difficult, specifically when the success of an entire workflow might be affected by a single job failure. In this paper, we introduce a model-based, hierarchical, reliable execution framework that encompass workflow specification, data provenance, execution tracking and online monitoring of each workflow task, also referred to as participants. The sequence of participants is described in an abstract parameterized view, which is translated into a concrete data dependency based sequence of participants with defined arguments. As participants belonging to a workflow are mapped onto machines and executed, periodic and on-demand monitoring of vital health parameters on allocated nodes is enabled according to pre-specified rules. These rules specify conditions that must be true pre-execution, during execution and post-execution. Monitoring information for each participant is propagated upwards through the reflex and healing architecture, which consists of a hierarchical network of decentralized fault management entities, called reflex engines. They are instantiated as state machines or timed automatons that change state and initiate reflexive mitigation action(s) upon occurrence of certain faults. We describe how this cluster reliability framework is combined with the workflow execution framework using formal rules and actions specified within a structure of first order predicate logic that enables a dynamic management design that reduces manual administrative workload, and increases cluster-productivity.

The NOνA Far Detector Data Acquisition System

The NOνA experiment is a long-baseline neutrino experiment designed to make measurements to determine the neutrino mass hierarchy, neutrino mixing parameters and CP violation in the neutrino sector. In order to make these measurements the NOνA collaboration has designed a highly distributed, synchronized, continuous digitization and readout system that is able to acquire and correlate data from the Fermilab accelerator complex (NuMI), the NOνA near detector at the Fermilab site and the NOνA far detector which is located 810 km away at Ash River, MN. This system has unique properties that let it fully exploit the physics capabilities of the NOνA detector. The design of the NOνA DAQ system and its capabilities are discussed in this paper.

The Main Injector Beam Position Monitor Front-End Software

The front‐end software developed for the Main Injector (MI) BPM upgrade is described. The software is responsible for controlling a VME crate equipped with a Motorola PowerPC board running the VxWorks operating system, a custom‐made timing board, and up to 10 commercial digitizer boards. The complete MI BPM system is composed of 7 independent units, each collecting data from 19 to 38 BPM pickups. The MI BPM system uses several components already employed on the successful upgrade of another Fermilab machine, the Tevatron, The front‐end software framework developed for the Tevatron BPM upgrade is the base for building the MI front‐end software. The framework is implemented in C++ as a generic component library (GBPM) that provides an event‐driven data acquisition environment. Functionality of GBPM is extended to meet MI BPM requirements, such as the ability to handle and manage data from multiple cycles; perform readout of the digitizer boards without disrupting or missing subsequent cycles; transition bet...

The new Tevatron beam position monitor front-end software

The Tevatron is a proton anti-proton accelerator collider operating at the Fermi National Accelerator Laboratory. The machine is currently delivering beam for the CDF and DO experiments, which expect increasing luminosity until the conclusion of Run II, planned for 2009. The laboratory defined a plan for achieving higher luminosity, and one of the tasks is the upgrade of the accelerators beam position monitor (BPM). The Tevatron was built during the early eighties and some of its control systems, including the BPMs, are still the original ones. This paper describes the front-end software of the Tevatron BPM upgrade, from the requirements to the implementation, and the underlying hardware setup. The front-end software designed is presented, emphasizing its modularity and reusability, allowing it to be applied to other Fermilab machines

A System for Exchanging Control and Status Messages in the NOvA Data Acquisition

In preparation for NOvA, a future neutrino experiment at Fermilab, we are developing a system for passing control and status messages in the data acquisition system. The DAQ system will consist of applications running on approximately 450 nodes. The message passing system will use a publish-subscribe model and will provide support for sending messages and receiving the associated replies. Additional features of the system include a layered architecture with custom APIs tailored to the needs of a DAQ system, the use of an open source messaging system for handling the reliable delivery of messages, the ability to send broadcasts to groups of applications, and APIs in Java, C++, and Python. Our choice for the open source system to deliver messages is EPICS. We will discuss the architecture of the system, our experience with EPICS, and preliminary test results.

Investigations of the wideband spectrum of higher order modes measured on tesla-style cavities at the FLASH LINAC

Higher Order Modes (HOMs) excited by the passage of the beam through an accelerating cavity depend on the properties of both the cavity and the beam. It is possible, therefore, to draw conclusions on the inner geometry of the cavities based on observations of the properties of the HOM spectrum. A data acquisition system based on two 20 GS/s, 6 GHz scopes has been set up at the FLASH facility, DESY, in order to measure a significant fraction of the HOM spectrum predicted to be generated by the TESLA cavities used for the acceleration of its beam. The HOMs from a particular cavity at FLASH were measured under a range of known beam conditions. The dipole modes have been identified in the data. 3D simulations of different manufacturing errors have been made, and it has been shown that these simulations can predict the measured modes.