B. Green

The base-line DataFlow system of the ATLAS trigger and DAQ

The base-line design and implementation of the ATLAS DAQ DataFlow system is described. The main components of the DataFlow system, their interactions, bandwidths, and rates are discussed and performance measurements on a 10% scale prototype for the final ATLAS TDAQ DataFlow system are presented. This prototype is a combination of custom design components and of multithreaded software applications implemented in C++ and running in a Linux environment on commercially available PCs interconnected by a fully switched gigabit Ethernet network.

Performance of the final Event Builder for the ATLAS Experiment

Event data from proton-proton collisions at the LHC will be selected by the ATLAS experiment in a three level trigger system, which reduces the initial bunch crossing rate of 40 MHz at its first two trigger levels (LVL1+LVL2) to ~3 kHz. At this rate the Event-Builder collects the data from all read-out system PCs (ROSs) and provides fully assembled events to the the event-filter (EF), which is the third level trigger, to achieve a further rate reduction to ~ 200 Hz for permanent storage. The event-builder is based on a farm of O(100) PCs, interconnected via gigabit Ethernet to O(150) ROSs. These PCs run Linux and multi-threaded software applications implemented in C++. All the ROSs and one third of the event-builder PCs are already installed and commissioned. We report on performance tests on this initial system, which show promising results to reach the final data throughput required for the ATLAS experiment.

The ATLAS ROBIN

The ATLAS readout subsystem is the main interface between ~ 1600 detector front-end readout links and the higher-level trigger farms. To handle the high event rate (up to 100 kHz) and bandwidth (up to 160 MB/s per link) the readout PCs are equipped with four ROBIN (readout buffer input) cards. Each ROBIN attaches to three optical links, provides local event buffering for approximately 300 ms and communicates with the higher-level trigger system for data and delete requests. According to the ATLAS baseline architecture this communication runs via the PCI bus of the host PC. In addition, each ROBIN provides a private Gigabit Ethernet port which can be used for the same purpose. Operational monitoring is performed via PCI. This paper presents a summary of the ROBIN hardware and software together with measurements results obtained from various test setups.

The C-RORC PCIe card and its application in the ALICE and ATLAS experiments

The ALICE and ATLAS DAQ systems read out detector data via point-to-point serial links into custom hardware modules, the ALICE RORC and ATLAS ROBIN. To meet the increase in operational requirements both experiments are replacing their respective modules with a new common module, the C-RORC. This card, developed by ALICE, implements a PCIe Gen 2 x8 interface and interfaces to twelve optical links via three QSFP transceivers. This paper presents the design of the C-RORC, its performance and its application in the ALICE and ATLAS experiments.

Development of a modular and scalable data acquisition system for calorimeters at a linear collider

A data acquisition (DAQ) system has been developed which will read out and control calorimeters serving as prototype systems for a future detector at an electron-positron linear collider. This is a modular, flexible and scalable DAQ system in which the hardware and signals are standards-based, using FPGAs and serial links. The idea of a backplaneless system was also pursued with a commercial development board housed in a PC and a chain of concentrator cards between it and the detector forming the basis of the system. As well as describing the concept and performance of the system, its merits and disadvantages are discussed.

Performance of the Final Event Builder for the ATLAS Experiment

Event data from proton-proton collisions at the LHC will be selected by the ATLAS experiment by a three level trigger system, which reduces the initial bunch crossing rate of 40 MHz at its first two trigger levels (LVL1+LVL2) to ~3 kHz. At this rate the Event-Builder collects the data from all Read-Out system PCs (ROSs) and provides fully assembled events to the the Event-Filter (EF), which is the third level trigger, to achieve a further rate reduction to ~200 Hz for permanent storage. The Event-Builder is based on a farm of 0 (100) PCs, interconnected via Gigabit Ethernet to 0 (150) ROSs. These PCs run Linux and multi-threaded software applications implemented in C++. All the ROSs and one third of the Event-Builder PCs are already installed and commissioned. Performance measurements have been exercised on this initial system, which show promising results that the required final data rates and bandwidth for the ATLAS event builder are in reach.

Performance of the ATLAS DAQ DataFlow system

The baseline DAQ architecture of the ATLAS Experiment at LHC is introduced and its present implementation and the performance of the DAQ components as measured in a laboratory environment are summarized. It will be shown that the discrete event simulation model of the DAQ system, tuned using these measurements, does predict the behaviour of the prototype configurations well, after which, predictions for the final ATLAS system are presented. With the currently available hardware and software, a system using ~140 ROSs with 3GHz single cpu, ~100 SFIs with dual 2.4 GHz cpu and ~500 L2PUs with dual 3.06 GHz cpu can achieve the dataflow for 100 kHz Level 1 rate, with 97% reduction at Level 2 and 3 kHz event building rate. ATLAS DATAFLOW SYSTEM The 40 MHz collision rate at the LHC produces about 25 interactions per bunch crossing, resulting in terabytes of data per second, which has to be handled by the detector electronics and the trigger and DAQ system [1]. A Level1 (L1) trigger system based on custom electronics will reduce the event rate to 75 kHz (upgradeable to 100 kHz – this paper uses the more demanding 100 kHz). The ________________________________________ #. Also affiliated with University of California at Irvine, Irvine, USA *. On leave from Henryk Niewodniczanski Institute of Nucl. Physics, Cracow +. Presently at CERN, Geneva, Switzerland 91 DAQ system is responsible for: the readout of the detector specific electronics via 1630 point to point read-out links (ROL) hosted by Readout Subsystems (ROS), the collection and provision of “Region of Interest data” (ROI) to the Level2 (L2) trigger, the building of events accepted by the L2 trigger and their subsequent input to the Event Filter (EF) system where they are subject to further selection criteria. The DAQ also provides the functionality for the configuration, control, information exchange and monitoring of the whole ATLAS detector readout [2]. The applications in the DAQ software dealing with the flow of event and monitoring data as well as the trigger information are called “DataFlow” applications. The DataFlow applications up to the EF input and their interactions are shown in Figure 1. Figure 1 ATLAS DAQ-DataFlow applications and their interactions (up to the EventFilter) SFI L2PU L2SV DFM pROS ROS ROI data

A second level data buffer with LHC performance

Abstract For the ATLAS experiment at the Large Hadron Collider (LHC) buffers are required which will accept data at rates in excess of 100 Mbytes/s and distribute selected data to the second and third level trigger systems. A design for a buffer using a digital signal processor as the memory manager is described and the performance of a prototype module is given. Tests of the system in a beam line will be used to develop and check discrete event simulation models of the buffers and a second level trigger system.

The evolution of the region of interest builder for the ATLAS experiment at CERN

The ATLAS detector uses a real time selective triggering system to reduce the high interaction rate from 40 MHz to its data storage capacity of 1 kHz. A hardware first level (L1) trigger limits the rate to 100 kHz and a software high level trigger (HLT) selects events for offline analysis. The HLT uses the Regions of Interest (RoIs) identified by L1 and provided by the Region of Interest Builder (RoIB). The current RoIB is a custom VMEbus based system that operated reliably since the first run of the LHC . Since the LHC will reach higher luminosity and ATLAS will increase the complexity and number of L1 triggers, it is desirable to have a more flexible and more operationally maintainable RoIB in the future. In this regard, the functionality of the multi-card VMEbus based RoIB is being migrated to a PC based RoIB with a PCI-Express card. Testing has produced a system that achieved the targeted rate of 100 kHz.

The ATLAS Level-2 Trigger Pilot Project

The Level-2 Trigger Pilot Project of ATLAS, one of the two general purpose LHC experiments, is part of the on-going programme to develop the ATLAS High Level Triggers (HLT). The Level-2 Trigger will receive events at up to 100 kHz, which has to be reduced to a rate suitable for full event-building of the order of 1 kHz. To reduce the data collection bandwidth and processing power required for the challenging Level-2 task it is planned to use Region of Interest guidance (from Level-1) and sequential processing. The Pilot Project included the construction and use of testbeds of up to 48 processing nodes, development of optimised components and computer simulations of a full system. It has shown how the required performance can be achieved, using largely commodity components and operating systems, and validated an architecture for the Level-2 system. This paper describes the principal achievements and conclusions of this project.