Harald Joerg Stelzer

The ATLAS data quality defect database system

The ATLAS experiment at the Large Hadron Collider has implemented a new system for recording information on detector status and data quality, and for transmitting this information to users performing physics analysis. This system revolves around the concept of “defects,” which are well-defined, fine-grained, unambiguous occurrences affecting the quality of recorded data. The motivation, implementation, and operation of this system is described.

Run Control Communication for the Upgrade of the ATLAS Muon-to-Central-Trigger-Processor Interface (MUCTPI)

The Muon-to-Central-Trigger-Processor Interface (MUCTPI) of the ATLAS experiment at the Large Hadron Collider (LHC) at CERN will be upgraded to an ATCA blade system for Run 3, starting in 2021. The new design requires development of new communication models for control, configuration and monitoring. A System-on-Chip (SoC) with a programmable logic part and a processor part will be used for communication to the run control system and to the MUCTPI processing FPGAs. Different approaches have been compared. First, we tried an available UDP-based implementation in firmware for the programmable logic. Although this approach works as expected, it does not provide any flexibility to extend the functionality to more complex operations, e.g. for serial protocols. Second, we used a SoC processor with an embedded Linux operating system and an application-specific software written in C++ using a TCP remote-procedure-call approach. The software is built and maintained using the framework of the Yocto Project. This approach was successfully used to test and validate the MUCTPI prototype. A third approach investigated is the option of porting the ATLAS run control software directly to an embedded Linux instance. THE ATLAS EXPERIMENT AT THE LHC Figure 1: The Trigger and Data Acquisition System of ATLAS. The ATLAS experiment [1] is a general-purpose experiment at the Large Hadron Collider (LHC) at CERN. It observes proton-proton collisions at a centre-of-mass energy of 13 TeV. With about 25 interactions in every bunch crossing (BC) every 25 ns, there are 10 interactions per second potentially producing interesting physics. The trigger system selects those events which are interesting to physics and which can be recorded to permanent storage at a reasonable rate. The ATLAS trigger system (see Figure 1) consists of a Level-1 trigger, based on custom electronics and firmware, which reduces the event rate to a maximum of 100 kHz, and a high-level trigger system based on commercial-off-the-shelf computers, network components, and software which reduces the event rate to around 1 kHz. THE LEVEL-1 TRIGGER SYSTEM The Level-1 trigger system (see Figure 2) uses reducedgranularity information from the calorimeters and dedicated muon trigger detectors. The trigger information is based on multiplicities and topologies of trigger candidate objects. The muon trigger is based on Resistive Plate Chambers (RPC) in the barrel region and Thin-Gap Chambers (TGC) in the end-cap region. The Muon-to-CentralTrigger-Processor Interface (MUCTPI) [2] combines the muon candidate counts from the RPC and TGC taking into account double counting of single muons that are detected by more than one chamber due to geometrical overlap of the chambers and the trajectory of the muon in the magnetic field. It sends the results to the Central Trigger Processor (CTP) which combines all trigger object multiplicities from the calorimeter trigger and from the MUCTPI, as well as the topology flags from the Topological Processor to make the final Level-1 decision based on rules described in a trigger menu. The CTP then sends the Level-1 decision back to the detector front-end electronics. Figure 2: The Level-1 Trigger System of ATLAS. ___________________________________________ † Ralf.Spiwoks@cern.ch THE UPGRADE OF THE MUCTPI The MUCTPI upgrade is part of the overall upgrade of ATLAS on the road to High-Luminosity LHC and is in line with the development of the New Small Wheel [3] of the muon trigger system to be installed during the shutdown of 2019 and 2020. The new MUCTPI will use optical links replacing bulky electrical cables. Those links will allow the muon trigger detectors to send more muon candidates with more precise information. The MUCTPI will provide improved overlap handling and full-precision information of the muon candidates to the Topological Processor. The new MUCTPI (see Figure 3) will be built as a single ATCA blade, compared to 18 VME modules in the current system. It will receive 208 optical links and will use two state-of-the-art FPGAs (Xilinx Virtex Ultrascale+) for the overlap handling, counting of muon candidates, and sending candidates to the Topological Processor. A third FPGA (Xilinx Kintex Ultrascale) will provide the total count of muon candidates to the CTP and readout data to the data acquisition system. A System-on-Chip (SoC, Xilinx Zynq 7000) [4] will act as a control processor and integrate the MUCTPI into the ATLAS run control (RC) system. THE RUN CONTROL OF THE MUCTPI The RC system of the ATLAS experiment provides control of the MUCTPI, e.g. start and stop commands; loading of configuration data, e.g. overlap lookup table (LUT) data and collection of monitoring data, e.g. counter values. Due to the new technology (ATCA), new forms of communication between the MUCTPI and the RC system had to be investigated. A SoC with a processor part and a programmable logic part will be used for the communication with the RC system and the processing FPGAs of the MUCTPI (see Figure 4). The processor system runs embedded Linux and the programmable logic provides the communication with the processing FPGAs using Xilinx chip-2-chip links, the configuration of the FPGAs using the Xilinx slave serial protocol, and the configuration and monitoring of the MUCTPI hardware with its power, clock and optical modules using serial buses like I2C and SPI. Three models of communication with the RC system have been investigated and will be discussed below. The software for the operating system, a kernel module for accessing memory, including the use of DMA (for Model #2 and #3), the port of the RC software (for Model #3), and all the applications are built and maintained using the framework of the Yocto Project [5] from the Linux Foundation. Recipes for all packages have been developed, in order to fetch, configure, and compile the software, and create all images necessary to boot the processor system.

The ATLAS Muon-to-Central-Trigger-Processor-Interface (MUCTPI) Upgrade

The Muon-to-Central-Trigger-Processor Interface is part of the Level-1 trigger system of the ATLAS experiment at the Large Hadron Collider at CERN. The upgrade of the Muon-to-Central Trigger Processor Interface will be described. It will use optical input and provide full precision region-of-interest information for muon candidates to the topological trigger processor of the Level-1 trigger system. The new Muon-to-Central-Trigger-Processor Interface will be implemented as an ATCA blade receiving 208 optical serial links from the ATLAS muon trigger detectors. Two high-end processing FPGAs will eliminate double counting of identical muon candidates in overlapping regions and send candidate information to the topological trigger and multiplicities to a third FPGA which will combine the candidate information, send muon multiplicities to the Central Trigger Processor and provide readout data to the ATLAS data acquisition system. A System-on-Chip module will provide communication with the ATLAS run control system for control, configuration and monitoring of the new Muon-to-Central-Trigger-Processor Interface.