Gunter Quast

The data acquisition system of the OPAL detector at LEP

Abstract This report describes the 1991 implementation of the data acquisition system of the OPAL detector at LEP including the additional services and infrastructure necessary for its correct and reliable operation. The various tasks in this “on-line” environment are distributed amongst many VME subsystems, workstations and minicomputers which communicate over general purpose local area networks and special purpose buses. The tasks include data acquisition, control, monitoring, calibration and event reconstruction. The modularity of both hardware and software facilitates the upgrading of the system to meet new requirements.

The PAX toolkit and its applications at Tevatron and LHC

At the CHEP03 conference, we launched the Physics Analysis eXpert (PAX), a C++ toolkit released for the use in advanced high energy physics (HEP) analyses. This toolkit allows to define a level of abstraction beyond detector reconstruction by providing a general, persistent container model for HEP events. Physics objects such as particles, vertices and collisions can easily be stored, accessed and manipulated. Bookkeeping of relations between these objects (like decay trees, vertex and collision separation, etc.) including deep copies is fully provided by the relation management. Event container and associated objects represent a uniform interface for algorithms and facilitate the parallel development and evaluation of different physics interpretations of individual events. So-called analysis factories, which actively identify and distinguish different physics processes and study systematic uncertainties, can easily be realized with the PAX toolkit. PAX is officially released to experiments at Tevatron and LHC. Being explored by a growing user community, it is applied in a number of complex physics analyses, two of which are presented here. We report the successful application in studies of tt~ production at the Tevatron and Higgs searches in the channel tt~H at the LHC and give a short outlook on further developments.

Virtualizing a batch queuing system at a university grid center

Computing clusters of High Energy Physics institutes at universities are often shared between different user groups, having their own requirements concerning the computing infrastructure. Those requirements can lead to incompatibilities between the needed operating systems, software packages, access policies or different grid middlewares. Some of the above incompatibilities can be solved by providing different portal machines for each group. Incompatibilities at the level of the shared worker nodes of the cluster are, however, difficult to overcome. In this paper, an approach to overcome this incompatibility using the virtualization technique Xen is presented. Each physical worker node hosts different virtual machines, acting as virtual worker node for every group supported at the site. Ways to integrate this into an existing batch queue are shown. The performance of different programs used in High Energy Physics on native and virtual machines are also presented.

A low-cost AFM setup with an interferometer for undergraduates and secondary-school students

Atomic force microscopy (AFM) is an important tool in nanotechnology. This method makes it possible to observe nanoscopic surfaces beyond the resolution of light microscopy. In order to provide undergraduate and secondary-school students with insights into this world, we have developed a very robust lowcost AFM setup with a Fabry–Perot interferometer as a detecting device. This setup is designed to be operated almost completely manually and its simplicity gives access to a profound understanding of the working principle. Our AFM is operated in a constant height mode, i.e. the topography of the sample surface is represented directly by the deflection of the cantilever. Thus, the measuring procedure can be understood even by secondary-school students; furthermore, it is the method with the lowest cost, totalling not more than 10–15 k Euros. Nevertheless, we are able to examine a large variety of sample topographies such as CD and DVD surfaces, IC structures, blood cells, butterfly wings or moth eyes. Furthermore, force–distance curves can be recorded and the tensile moduli of some materials can be evaluated. We present our setup in detail and describe its working principles. In addition, we show various experiments which have already been performed by students. (Some figures may appear in colour only in the online journal)

New Applications of PAX in Physics Analyses at Hadron Colliders

The PAX (Physics Analysis eXpert) toolkit assists physicists in the analysis and interpretation step of a particle physics research project. Its aim is to provide a new level of abstraction beyond detector reconstruction which facilitates code reuse and unification. PAX makes use of fourvector arithmetics and offers sophisticated relation management and memory management functionality. This paper gives an overview of the toolkit and reports about its application in the hadron collider experiments CDF and CMS.

Application of Virtualisation Techniques at a University Grid Center

Computing clusters at universities are often shared between many different user groups, having their own requirements concerning computing infrastructure, e.g. the operating system (OS), software packages, access policies and different grid middlewares. The integration of local computing and storage resources into the grid of a specific user group is realised by portal machines, providing different services. In the current implementation of the Worldwide LHC Grid and the CDF grid middleware, these services require dedicated operating systems and should be installed on different machines for stability reasons. Virtualisation of these services on one high-capacity machine eases maintenance of the systems and reduces the hardware overhead especially for smaller clusters. This approach of hardware consolidation is not restricted to the grid infrastructure and may therefore be applied to other even simpler services. This paper describes the process of hardware consolidation at a typical university grid centre. A discussion of different virtualisation techniques is also included and benchmarks show that Xen is an optimal candidate.

Parallel track reconstruction in CMS using the cellular automaton approach

The Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider (LHC) is a general-purpose particle detector and comprises the largest silicon-based tracking system built to date with 75 million individual readout channels. The precise reconstruction of particle tracks from this tremendous amount of input channels is a compute-intensive task. The foreseen LHC beam parameters for the next data taking period, starting in 2015, will result in an increase in the number of simultaneous proton-proton interactions and hence the number of particle tracks per event. Due to the stagnating clock frequencies of individual CPU cores, new approaches to particle track reconstruction need to be evaluated in order to cope with this computational challenge. Track finding methods that are based on cellular automata (CA) offer a fast and parallelizable alternative to the well-established Kalman filter-based algorithms. We present a new cellular automaton based track reconstruction, which copes with the complex detector geometry of CMS. We detail the specific design choices made to allow for a high-performance computation on GPU and CPU devices using the OpenCL framework. We conclude by evaluating the physics performance, as well as the computational properties of our implementation on various hardware platforms and show that a significant speedup can be attained by using GPU architectures while achieving a reasonable physics performance at the same time.

Dynamic Extensions of Batch Systems with Cloud Resources

Compute clusters use Portable Batch Systems (PBS) to distribute workload among individual cluster machines. To extend standard batch systems to Cloud infrastructures, a new service monitors the number of queued jobs and keeps track of the price of available resources. This meta-scheduler dynamically adapts the number of Cloud worker nodes according to the requirement profile. Two different worker node topologies are presented and tested on the Amazon EC2 Cloud service.

Integration of virtualized worker nodes in standard batch systems

Current experiments in HEP only use a limited number of operating system flavours. Their software might only be validated on one single OS platform. Resource providers might have other operating systems of choice for the installation of the batch infrastructure. This is especially the case if a cluster is shared with other communities, or communities that have stricter security requirements. One solution would be to statically divide the cluster into separated sub-clusters. In such a scenario, no opportunistic distribution of the load can be achieved, resulting in a poor overall utilization efficiency. Another approach is to make the batch system aware of virtualization, and to provide each community with its favoured operating system in a virtual machine. Here, the scheduler has full flexibility, resulting in a better overall efficiency of the resources. In our contribution, we present a lightweight concept for the integration of virtual worker nodes into standard batch systems. The virtual machines are started on the worker nodes just before jobs are executed there. No meta-scheduling is introduced. We demonstrate two prototype implementations, one based on the Sun Grid Engine (SGE), the other using Maui/Torque as a batch system. Both solutions support local job as well as Grid job submission. The hypervisors currently used are Xen and KVM, a port to another system is easily envisageable. To better handle different virtual machines on the physical host, the management solution VmImageManager is developed. We will present first experience from running the two prototype implementations. In a last part, we will show the potential future use of this lightweight concept when integrated into high-level (i.e. Grid) work-flows.

Site specific monitoring of multiple information systems ? the HappyFace Project

An efficient administration of computing centres requires sophisticated tools for the monitoring of the local infrastructure. Sharing such resources in a grid infrastructure, like the Worldwide LHC Computing Grid (WLCG), goes ahead with a large number of external monitoring systems, offering information on the status of the services and user jobs at a grid site. This huge flood of information from many different sources retards the identification of problems and complicates the local administration. In addition, the web interfaces for the access to the site specific information are often very slow and uncomfortable to use. A meta-monitoring system which automatically queries the different relevant monitoring systems could provide a fast and comfortable access to all important information for the local administration. It becomes also feasible to easily correlate information from different sources and provides an easy access also for non-expert users. In this paper, we describe the HappyFace Project, a modular software framework for such purpose. It queries existing monitoring sources and processes the results to provide a single point of entrance for information on a grid site and its specific services.