Extending RECAST for Truth-Level Reinterpretations
EExtending RECAST For Truth-Level Reinterpretations
Alex Schuy , Lukas Heinrich , Kyle Cranmer and Shih-Chieh Hsu University of Washington, Seattle, USA European Laboratory for Particle Physics, CERN New York University, New York, USA
Abstract:
RECAST is an analysis reinterpretation framework; since analyses are often sensitive toa range of models, RECAST can be used to constrain the plethora of theoretical models without thesignificant investment required for a new analysis. However, experiment-specific full simulation isstill computationally expensive. Thus, to facilitate rapid exploration, RECAST has been extendedto truth-level reinterpretations, interfacing with existing systems such as RIVET.
Talk presented at the 2019 Meeting of the Division of Particles and Fields of the American PhysicalSociety (DPF2019), July 29–August 2, 2019, Northeastern University, Boston, C1907293.
At the LHC, searches for extensions of the standard model (SM) of particle physics, so-called ’be-yond the standard model’ (BSM) theories, are undertaken using a complex workflow incorporatingevent simulation, selection, and statistical analysis. In general, each search requires a new set ofevent selections, in order to focus on a region of kinematic phase space in which there is a sig-nificant, detectable difference between the predictions of the BSM and SM theories. Developmentof these event selections, along with a corresponding method of rigorous statistical analysis, cantake years to complete. In an effort to reduce this cost, work has been done in recent years todevelop the capability to preserve and reinterpret existing analyses in a software framework knownas RECAST [1].
Work on RECAST originated in the ATLAS collaboration at CERN. RECAST was designed tosolve two concerns that ATLAS had:1. The number of BSM models that need to be considered is growing faster than the availablepersonnel resources for analysis design.2. Use of the software related to an analysis often requires expert knowledge only present on theoriginal team, which has contributed to a reproducibility crisis.For the first item, it was realized that analyses often utilize signatures that make them sensitiveto a wide range of models, more than are initially explored. Thus, if search workflows are writtenin a modular fashion that is easily usable by future collaborators, then progress can be made onmore searches more quickly by reusing existing analyses. Fortunately, eliminating the need forexpert knowledge by making standardized modular search workflows also solves the reproducibilityproblem, so we see that we can solve both issues with the same system.1 a r X i v : . [ phy s i c s . d a t a - a n ] O c t a)(b) Figure 1: An example of a typical search workflow (a) and recast workflow (b) in an ATLAS searchfor a BSM model. 2igure 2: Illustration of the recasting of an analysis originally designed for model A for a new modelB. Note that the data (black dots) and background distributions (purple curves) are identical: onlythe signal distribution changes (red or green curve). See [2].Figure 1(a) shows a typical search workflow. In order to use the collision data from the detectorto discriminate between the BSM and SM theories, we need to know what we should expect to seein each case. Since the details of physics processes with a detector are highly complex, an exactdistribution is impossible to predict. Instead, many Monte Carlo simulations are performed toobtain an approximate distribution describing our predictions. The simulation tool-chain is a setof four steps that are highly standardized at ATLAS:1. Generation – Generation of physics processes using Feynman diagrams based on the theory.2. Simulation – Simulation of the particles as they move through the detector.3. Digitization – Simulation of the digital readings that the detector will record.4. Reconstruction – Reconstruction of the final particles based on the detector’s digital readings.The last step must also be performed on real data, as of course the raw data from the detectoris a set of digital readings. Once this is complete, both the simulated data and real data must bepassed through a set of event selections, which focus on a particular region of phase space in whichwe expect there to be large differences between the BSM and SM theories. Finally, the selecteddata can be passed to statistical software that determines whether there is sufficient evidence fordiscovery, and if not, which regions of the new model’s parameter space can likely be eliminated.In contrast, Figure 1(b) shows a recast workflow. Note that since we are reusing an existinganalysis, we can just use stored collision and background data. All we need to calculate is theprediction of our new signal model. Figure 2 shows the distribution of the final observable in thetwo cases. Note again that the distribution of the background and collision data are equivalent ineach case; only the signal changes.In order to preserve an analysis, three components are necessary:1. Software – what framework(s) does the analysis use and what are the dependencies?2. Commands – what needs to be done to use the framework(s) for each stage of the analysis?3. Workflow – how are the analysis stages connected?The first step is accomplished using Docker images, an industry-standard containerization tool.The latter two steps are specified using a Yadage, a workflow language. More details, examples, anddiscussion on how to integrate with GitLab CI are available at https://recast-docs.web.cern.ch/.Such analysis preservation was recently made a requirement for ATLAS.
While the collaboration-specific reinterpretation is very useful, there are a few reasons for alternativeapproaches:1. ’Full simulation’ (modelling detector effects) is computationally expensive and difficult to use.2. To determine which regions of phase space would be interesting for a full reinterpretation.3. To explore other quantities, such as signal theory uncertainties.Thus, there was the idea to perform so-called ’truth-level’ reinterpretations using recast. Inthis case, the analysis is performed without modelling the particle’s behavior as it interacts withthe detector. In this case, we can eliminate the simulation, digitization and reconstruction steps,leaving only three steps: generation, selection and statistics.Figure 3 illustrates these three steps. Over the years, many tools have been developed thatimplement one or several of the steps in different ways. Therefore, we are now developing the’recast catalogue’ that implements standardized sub-workflows for each of these steps, as well asdescribing how to combine them using common interfaces into a single workflow. The end goal isto facilitate easier truth-level reinterpretation and unify different types of reinterpretation underthe same framework. This should also allow for much easier comparison of results across differenttools, which can serve as a valuable consistency check, particularly given the complexity of theoperations that these tools perform. 4igure 3: A cartoon illustration of the recast catalogue. For each of the three steps, there are manycollaboration-external tools that have been developed. Thus, it is necessary to define a cataloguethat describes how the various tools can be put together in a standardized fashion.5
Future work
There are three ideas that we would like to incorporate into the recast framework moving forward:1. REANA backend integration2. Web interface3. Smart grid selectionOur first goal is integration with the reana backend. Once we have determined an appropriateworkflow for either a full or fast reinterpretation, we need to evaluate the workflow and retrieveresults. Reana is a cloud-backed computational workflow platform that supports submission of non-interactive batch workflows written in a workflow language such as Yadage. As such, it’s a perfect’backend’ computational tool for recast that can enable us to provide faster and more accurateresults in greater quantity.Once we integrate reana, we’d also like to create a unifying web interface for all recast work.Presently, recasting is accomplished through command-line tools, but as part of our goal of makingresults more accessible to non-experts such as theorists, we want to create a user-friendly webinterface. A prototype was created in the past, but work needs to be done to update it to modernstandards.Finally, we’d like to add ’smart grid selection’. Each BSM model typically has some set ofparameters that must be specified in order to fully specify the predictions of the model. Whenperforming a search, we evaluate the model on some set of points in that parameter space, andthen extrapolate between those points. The set of points we evaluate is known as the ’grid’. Thetypical approach is to use a uniform grid over a region of parameter space that is thought to be ofinterest. However, this often involves a lot of wasted computation, as many of the points are farfrom the contour that describes the bounds on our model.Other approaches besides uniform sampling have been developed. In particular, the idea is toimplement an active-learning alternative to uniform grid selection that selects points in an iterativefashion, using each evaluation to inform our Bayesian prior [3]. See Figure 4 for an illustration.Typically, such a sequential approach would suffer from being much slower than a uniformgrid, where you can exploit parallelization by calculating all points simultaneously. However, thecomputation of each point is already highly parallelizable. In practice, we only have finite com-puting resources, so we therefore expect that by shifting parallelization from inter- to intra-pointcomputations, we can reduce both total computation and wall-clock time, as shown in Figure 5.6igure 4: A cartoon illustrating smart grid selection [3]. In a typical uniform grid selection, pointsfar from the contour (gray) are uninformative and result in wasted computation. With smartselection, the search tends to focus on points near the contour (orange).7igure 5: A cartoon illustrating the distribution of finite computing resources for the evaluationof points in uniform grid selection (left) and smart grid selection (right) [3]. Each set of coloredblocks corresponds to a single point. As can be seen, in the smart case, even though each pointmust be calculated sequentially, high utilization is still achieved by parallelizing the computationfor each point. 8
Conclusion
RECAST is an analysis preservation and reinterpretation tool that has been used for full reinterpre-tations that account for detector specifics. Now, recast is being extended to fast reinterpretationsthat utilize truth-level information that is detector-agnostic through development of a catalogue.This should be useful both for theorists interested in quickly putting limits on their model and formore sophisticated users and developers who are interested in comparing the consistency of thevarious tools, which should ultimately spur further development. In the future, we have severalother ideas planned, including reana integration, a web interface, and smart grid selection and wehope that further contributions are forthcoming.
AcknowledgementsReferences [1] K. Cranmer and I. Yavin, RECAST: Extending the Impact of Existing Analyses, (2010), arXiv:1010.2506, url: http://dx.doi.org/10.1007/JHEP04(2011)038.[2] L. Heinrich, Towards a unified interface and archive for reinterpration tools, ReinterpretationWorkshop, Fermilab, October 2017.https://indico.cern.ch/event/639314/contributions/2726367/[3] L. Heinrich, G. Louppe, K. Cranmer, Excursion Set Estimation using Sequential EntropyReduction for Efficient Searches for New Physics at the LHC, ACAT 2019.https://indico.cern.ch/event/708041/contributions/3269754/[4] A. Buckley, J. Butterworth, L. Lonnblad, D. Grellscheid, H. Hoeth, J. Monk, H. Schulzand F. Siegert, Comput. Phys. Commun.184