Erasmia Lois

Expert elicitation approach for performing ATHEANA quantification

Abstract An expert elicitation approach has been developed to estimate probabilities for unsafe human actions (UAs) based on error-forcing contexts (EFCs) identified through the ATHEANA (A Technique for Human Event Analysis) search process. The expert elicitation approach integrates the knowledge of informed analysts to quantify UAs and treat uncertainty (‘quantification-including-uncertainty’). The analysis focuses on (a) the probabilistic risk assessment (PRA) sequence EFCs for which the UAs are being assessed, (b) the knowledge and experience of analysts (who should include trainers, operations staff, and PRA/human reliability analysis experts), and (c) facilitated translation of information into probabilities useful for PRA purposes. Rather than simply asking the analysts their opinion about failure probabilities, the approach emphasizes asking the analysts what experience and information they have that is relevant to the probability of failure. The facilitator then leads the group in combining the different kinds of information into a consensus probability distribution. This paper describes the expert elicitation process, presents its technical basis, and discusses the controls that are exercised to use it appropriately. The paper also points out the strengths and weaknesses of the approach and how it can be improved. Specifically, it describes how generalized contextually anchored probabilities (GCAPs) can be developed to serve as reference points for estimates of the likelihood of UAs and their distributions.

Issues in Benchmarking Human Reliability Analysis Methods: A Literature Review

There is a diversity of human reliability analysis (HRA) methods available for use in assessing human performance within probabilistic risk assessments (PRA). Due to the significant differences in the methods, including the scope, approach, and underlying models, there is a need for an empirical comparison investigating the validity and reliability of the methods. To accomplish this empirical comparison, a benchmarking study comparing and evaluating HRA methods in assessing operator performance in simulator experiments is currently underway. In order to account for as many effects as possible in the construction of this benchmarking study, a literature review was conducted, reviewing past benchmarking studies in the areas of psychology and risk assessment. A number of lessons learned through these studies is presented in order to aid in the design of future HRA benchmarking endeavors.

The Use of Empirical Data Sources in HRA

Abstract This paper presents a review of available information related to human performance to support Human Reliability Analysis (HRA) performed for nuclear power plants (NPPs). A number of data sources are identified as potentially useful. These include NPP licensee event reports, augmented inspection team reports, operator requalification data, results from the literature in experimental psychology, and the Aviation Safety Reporting System. The paper discusses how utilizing such information improves our capability to model and quantify human performance. In particular, the paper discusses how information related to performance shaping factors can be extracted from empirical data to determine their size effect, their relative effects, as well as their interactions. The paper concludes that appropriate use of existing sources can help addressing some of the important issues we are currently facing in HRA.

Capturing control room simulator data with the HERA System

The Human Event Repository and Analysis (HERA) system has been developed as a tool for classifying and recording human performance data extracted from primary data sources. This paper reviews the process of extracting data from simulator studies for use in HERA. Simulator studies pose unique data collection challenges, both in types and quality of data measures, but such studies are ideally suited to gather operator performance data, including the full spectrum of performance shaping factors used in a HERA analysis. This paper provides suggestions for obtaining relevant human performance data for a HERA analysis from a control room simulator study and for inputting those data in a format suitable for HERA.

An empirical study of HRA methods - overall design and issues

A diversity of Human Reliability Analysis (HRA) methods are currently available to treat human performance in Probabilistic Risk Assessments (PRAs). This range of methods reflects traditional concerns with human-machine interfaces and with the basic feasibility of actions in PRA scenarios as well as the more recent attention paid to Errors of Commission and decision- making performance. Given the differences in the scope of the methods and their underlying models, there is a substantial interest in assessing HRA methods and ultimately in validating the approaches and models underlying these methods. A significant step in this direction is an international evaluation study of HRA methods, based on comparing the observed performance in simulator experiments with the outcomes predicted in HRA analyses. Its aim is to develop an empirically- based understanding of the performance, strengths, and weaknesses of the methods. This paper presents the overall methodology for this initial assessment study.

Human reliability analysis (HRA) in the context of HRA testing with empirical data

Given the significant differences in the scope, approach, and underlying models of a relatively wide range of existing HRA methods, there has been a growing interest on the part of HRA method developers and users to empirically test the various methods. To this end, there is an ongoing international effort to begin this process by testing the application of HRA methods to nuclear power plant operating crew performance in the HAMMLAB simulators at the Halden Reactor Project in Norway. Initial efforts in designing and implementing these studies have identified a number of issues associated with structuring the studies in order to allow an adequate and appropriate test of the different methods. This paper focuses on issues associated with applying HRA methods in the context of an empirical study, particularly when a research simulator is used for data collection. Example issues include: determining the scope of the analysis when the methods themselves differ in the scope of the HRA processes they address, accounting for differences between the methods in how they use simulator exercises to support the analysis, addressing the impact of experimental controls on application of methods, and given the low probability of human failure events typically modelled in nuclear power plant probabilistic risk/safety assessments (PRAs/PSAs), the need for analysts to present their results in a somewhat different format than they usually do. These types of issues related to applying HRA methods in the context of empirical studies are discussed and resolutions are proposed.

An overview of the evolution of human reliability analysis in the context of probabilistic risk assessment.

Since the Reactor Safety Study in the early 1970s, human reliability analysis (HRA) has been evolving towards a better ability to account for the factors and conditions that can lead humans to take unsafe actions and thereby provide better estimates of the likelihood of human error for probabilistic risk assessments (PRAs). The purpose of this paper is to provide an overview of recent reviews of operational events and advances in the behavioral sciences that have impacted the evolution of HRA methods and contributed to improvements. The paper discusses the importance of human errors in complex human-technical systems, examines why humans contribute to accidents and unsafe conditions, and discusses how lessons learned over the years have changed the perspective and approach for modeling human behavior in PRAs of complicated domains such as nuclear power plants. It is argued that it has become increasingly more important to understand and model the more cognitive aspects of human performance and to address the broader range of factors that have been shown to influence human performance in complex domains. The paper concludes by addressing the current ability of HRA to adequately predict human failure events and their likelihood.

International HRA Empirical Study, Overall Methodology and HAMMLAB Results

The International HRA Empirical Study addresses the need for assessing HRA (Human Reliability Analysis) methods in light of human performance data. The study is based on a comparison of observed performance in HAMMLAB simulator trials with the outcomes predicted in HRA analyses. The project goal is to develop an empirically-based understanding of the performance, strengths, and weaknesses of a number of different HRA methods. This chapter presents the overall methodology for the initial assessment study (the pilot study), provides an overview of the HAMMLAB results and presents insights from the initial assessment.

Using Information from Operating Experience to Inform Human Reliability Analysis

This paper reports on efforts being sponsored by the U.S. NRC and performed by INEEL to develop a technical basis and perform work to extract information from sources for use in HRA. The objectives of this work are to: 1) develop a method for conducting risk-informed event analysis of human performance information that stems from operating experience at nuclear power plants and for compiling and documenting the results in a structured manner; 2) provide information from these analyses for use in risk-informed and performance-based regulatory activities; 3) create methods for information extraction and a repository for this information that, likewise, support HRA methods and their applications.

Human Performance Research and Its Uses to Inform Human Reliability Analysis

The field of human reliability analysis (HRA) plays an important role in probabilistic risk assessments (PRAs) performed for commercial nuclear power plants. The international community recognizes the need for using information about human performance from relevant settings to improve HRA method capability and to evaluate the human events modeled in a PRA. This chapter discusses the needs for HRA research, presents an experimental paradigm for research, and suggests ways that capabilities at the OECD Halden Reactor Project can play an important role in addressing HRA needs. The expertise that Halden has gained from many years of studying human performance is uniquely suited to address emerging opportunities for HRA research. Its initial efforts have already proven important and a long term plan of collaborative research is encouraged.