C. Timmer

The Jefferson Lab trigger supervisor system

We discuss the design and performance of a trigger Supervisor System for use in nuclear physics experiments at Jefferson Lab. We also discuss the enhanced features of a new Trigger Supervisor Module now under construction.

CODA performance in the real world

The most ambitious implementation of the Jefferson Lab data acquisition system (CODA) to date is for the CLAS spectrometer in Experimental Hall B. CLAS has over 40,000 instrumented channels and uses up to 30 front-end (FASTBUS/VME) crates in the DAQ subsystem. During the initial experiments we found that performance of the fully instrumented DAQ system did not scale as expected based on single point to point benchmarks. Over the past year we have been able to study various performance bottlenecks in the CLAS DAQ system including front-end real time performance, switched 100BaseT Ethernet data transport, and online data distribution and recording. Performance tuning was necessary for components on both real time (VxWorks) and UNIX (Solaris) operating systems. In addition, a new efficient Event Transfer System (ET) was developed to provide faster online monitoring while having minimal impact on data throughput to storage. We discuss these issues and efforts to overcome the real world problems associated with running a high performance DAQ system on a variety of commercial hardware and software.

Jefferson Lab Data Acquisition Run Control System

A general overview of the Jefferson Lab data acquisition run control system is presented. This run control system is designed to operate the configuration, control, and monitoring of all Jefferson Lab experiments. It controls data-taking activities by coordinating the operation of DAQ sub-systems, online software components and third-party software such as external slow control systems. The main, unique feature which sets this system apart from conventional systems is its incorporation of intelligent agent concepts. Intelligent agents are autonomous programs which interact with each other through certain protocols on a peer-to-peer level. In this case, the protocols and standards used come from the domain-independent Foundation for Intelligent Physical Agents (FIPA), and the implementation used is the Java Agent Development Framework (JADE). A lightweight, XML/RDF-based language was developed to standardize the description of the run control system for configuration purposes.

cMsg - a publish/subscribe package for real-time and online control systems

cMsg is a messaging system designed for use in realtime data acquisition and online controls systems, and includes support for C/C++ and Java clients. It is highly customizable and extensible, and includes a full-featured asynchronous publish/subscribe component as well as a synchronous peer-to-peer component. We first describe the publish/subscribe and peer-to-peer messaging paradigms, discuss their use in real-time and online systems, and describe the power and flexibility of the publish/subscribe paradigm. Next we describe the philosophy of the cMsg implementation and present some details and benchmarks. Finally, since the core of cMsg is written in pure Java, we discuss the suitability of Java for use in real-time and online systems

Evaluation of JAVA 2 and CORBA for platform independent control and monitoring of the next generation of the CODA data acquisition system

The CODA data acquisition system was developed as a common data acquisition environment for experiments at Jefferson lab. The control and monitoring system was developed in EIFFEL and later ported to C++. The original design ran on ULTRIX and has been ported to HP-UX, Solaris, IRIX and Linux. The porting process is tedious and error prone. The code is difficult to maintain and the principal authors have left the group. With this in mind we are evaluating JAVA as a portable, and stable, language with which to write the next generation run control system. In parallel we are evaluating CORBA as the communication mechanism. The recent release of Java 2 with integrated support for CORBA has made this route even more interesting. This paper presents the results of our evaluation of tools and technologies, experience gained while writing a prototype of the preliminary design for the control system.

Real-time performance of commercial Intel-based VME Controllers for the CODA data acquisition system

We have evaluated the performance of several Intel-based VME Controllers for the use in the data acquisition systems (DAQs) at Jefferson Lab. For the 12 GeV upgrade, PPC-based VME controllers running vxWorks will be replaced with those that are Intel-based running Linux. Their task will be facilitated by the use of FPGAs on the VME modules to perform trigger logic and communicating trigger information over serial and fiber connections through the DAQ. The need for a hard realtime operating system on the VME controller is removed from the equation as the readout of the digitized data from the VME modules (using VME-2eSST) is done in a threaded environment with multiple CPU cores while digitization takes place in the buffered, pipelined system. In this paper we briefly discuss a bench setup for evaluating a VME Controller and its kernel and user space environment. We present results from baseline testing of various models from different vendors using different Linux kernels, including results from a kernel compiled with the CONFIG_PREEMPT_RT patch.

cMsg - A general purpose, publish-subscribe, interprocess communication implementation and framework

cMsg is software used to send and receive messages in the Jefferson Lab online and runcontrol systems. It was created to replace the several IPC software packages in use with a single API. cMsg is asynchronous in nature, running a callback for each message received. However, it also includes synchronous routines for convenience. On the framework level, cMsg is a thin API layer in Java, C, or C++ that can be used to wrap most message-based interprocess communication protocols. The top layer of cMsg uses this same API and multiplexes user calls to one of many such wrapped protocols (or domains) based on a URL-like string which we call a Uniform Domain Locator or UDL. One such domain is a complete implementation of a publish-subscribe messaging system using network communications and written in Java (user APIs in C and C++ too). This domain is built in a way which allows it to be used as a proxy server to other domains (protocols). Performance is excellent allowing the system not only to be used for messaging but also as a data distribution system

AFECS. multi-agent framework for experiment control systems

AFECS is a pure Java based software framework for designing and implementing distributed control systems. AFECS creates a control system environment as a collection of software agents behaving as finite state machines. These agents can represent real entities, such as hardware devices, software tasks, or control subsystems. A special control oriented ontology language (COOL), based on RDFS (Resource Definition Framework Schema) is provided for control system description as well as for agent communication. AFECS agents can be distributed over a variety of platforms. Agents communicate with their associated physical components using range of communication protocols, including tcl-DP, cMsg (publish-subscribe communication system developed at Jefferson Lab), SNMP (simple network management protocol), EPICS channel access protocol and JDBC.

The ET system - high speed event transfer and distribution via shared memory and networks

The event transfer (ET) system manages and distributes high volume data as it is being produced in real time. After data creation and insertion into the system, other processes can retrieve the data based on selection criteria, possibly modify the data, then return the data to the system. A chain of processes can thus process the data sequentially. The ET system has fast, local operation based on shared memory and POSIX threads and mutexes. Event transfer may also occur transparently over the network to remote users. The ET system has been in use for several years and is easy to use as well as extremely robust. It is implemented in C on Solaris and Linux platforms with a client library for vxWorks and has been ported to Java (the Java version does not use shared memory).

EVIO — A lightweight object-oriented I/O package

The EVIO package implements I/O between disk and an in-memory hierarchical object tree. Tree nodes are either container nodes, holding other nodes, or leaf nodes, containing any amount of data of a single type (int64_t,..., int8_t, float, double, and string). Trees can easily be constructed using factory methods and stream operators; queried using STL-like algorithms, function object, etc; or modified using tree or node methods. The tree logically maps to XML, and utilities exist to transform between binary and ASCII XML formats. The binary representation is compact, I/O is simple and efficient, and endian conversions are handled automatically. EVIO will be used by the next generation of experiments at Jefferson Lab1.