Barry Kirwan

The validation of three human reliability quantification techniques — THERP, HEART and JHEDI: Part 1 — technique descriptions and validation issues

This is the first of a set of three papers reviewing the validity of three Human Reliability Assessment (HRA) techniques used in the UK to predict human performance in high risk industries. The techniques are used to determine the risks inherent in such industries due to human error, and also the benefit in terms of risk reduction by having human operators in the system. These techniques culminate in a quantitative human error probability for each error or failure identified, predicting how often errors or failed performance will occur. The major question with such an approach is whether such probabilities are accurate. This key validation question is answered in the second paper, which reports the results of a large and independent validation experiment. Once a technique achieves some degree of predictive validity, the next question becomes whether such a technique can consistently produce valid and accurate results. This is called the reliability of the technique, and is dependent upon its consistency of usage by different assessors. Consistency can only be analysed by investigating the detailed usage of the techniques, and such an analysis is reported in the third paper in this series. The advantage of such an analysis is that it can also lead to specification of practical guidance for practitioners, and may lead to derivation of ways to improve the reliability and consistency of usage of specific HRA techniques. This first paper introduces the three techniques themselves, and reviews the major validation criteria and issues which should be considered when trying to determine if such techniques work.

Modelling of safety management systems

Abstract Management systems for safety and environment and audits for assessing them have been a major research topic of the last few years. This attention has been fuelled by increasing emphasis in European directives on auditable safety management systems (SMS) 1 and by the increasing interest in their certification following the principles of the ISO 9000 series of standards. This paper reviews briefly the literature on this topic and presents a framework within which the total activity of an SMS can be presented using a consistent descriptive language. The framework can be used to describe and evaluate an SMS or to assess the completeness of audit tools designed for SMS evaluation. It can also be used as a didaictic framework for safety practitioners and managers and as a tool for accident analysis. The framework combines the following principles: • safety management seen as a set of problem solving activities at different levels of abstraction in all phases of the system life cycle; • safety related tasks are modelled using the Structured Analysis and Design Technique (SADT). This shows the inputs, resources and criteria/constraints necessary to produce the required outputs; • risks are modelled as deviations from normal or desired processes. The framework emphasises the dynamics of safety management as a process. It aims to provide an abstract ordering of the field which can clarify and specify research and policy needs for the future. It also provides a clear definition of safety culture.

Development of a Hazard and Operability-based method for identifying safety management vulnerabilities in high risk systems

In recent years there has been widespread acknowledgement of the significance of managerial and organisational failures in the causation of accidents. The activities and processes involved in managing safety have come under increasing scrutiny via the development of approaches for safety management and safety culture assessment. This paper argues, however, that current approaches do not fully or explicitly address how safety management systems will actually fail in practice. Therefore, the Safety Culture Hazard and Operability (SCHAZOP) approach is described as a means by which specific safety management vulnerabilities, and targeted resolution of such vulnerabilities can be identified. The development of the method, examples of its application, and the way forward with this methodology are discussed.

Human error identification techniques for risk assessment of high risk systems--Part 1: Review and evaluation of techniques

This is the first in a two-part series of papers dealing with the area of assessing human errors in high risk complex systems. This first paper outlines thirty-eight approaches of error identification, categorising them into types of error identification approach. The paper then reviews these techniques with respect to a broad range of criteria. Viable and non-viable techniques are identified. Trends and research needs are also noted. The second paper proposes a framework or tool-kit approach to Human Error Identification, and presents a prototype methodology to show what such a framework approach would look like in practice, for the nuclear power domain.

Human error identification in human reliability assessment. Part 2: Detailed comparison of techniques.

This is the second part of a two-part review of human error identification (HEI) approaches in human reliability assessment (HRA). Part 1 reviewed the probabilistic risk assessment (PRA) context in which HRA occurs, and then detailed 12 HEI techniques which have evolved in the field of HRA. Part 2 attempts to compare the way these techniques perform against a range of criteria relevant to HEI theoretical and empirical validity, and practical usefulness in applied HRA. It is hoped that these comparisons will help assessors in the selection of techniques for practical applications. The comparisons also point to research and development needs in the area of applied HEI.

The validation of three Human Reliability Quantification techniques — THERP, HEART and JHEDI: Part II — Results of validation exercise

This is the second of three papers dealing with the validation of three Human Reliability Assessment (HRA) techniques. The first paper introduced the need for validation, the techniques themselves and pertinent validation issues. This second paper details the results of the validation study carried out on the Human Reliability Quantification techniques THERP, HEART and JHEDI. The validation study used 30 real Human Error Probabilities (HEPs) and 30 active Human Reliability Assessment (HRA) assessors, 10 per technique. The results were that 23 of the assessors showed a significant correlation between their estimates and the real HEPs, supporting the predictive accuracy of the techniques. Overall precision showed 72% (60-87%) of all HEPs to be within a factor of 10 of the true HEPs, with 38% of all estimates being within a factor of three of the true values. Techniques also tended to be pessimistic rather than optimistic, when they were imprecise. These results lend support to the empirical validity of these three approaches.

Human error identification techniques for risk assessment of high risk systems. Part 2 : towards a framework approach

This is the second paper in a series of two, reviewing human error identification approaches for risk assessment of high risk socio-technical systems. The previous paper identified and reviewed thirty-eight techniques. One of the closing comments was that no single technique sufficed for all the practitioners needs, and that a potential way forward was to utilise a number of techniques in a toolkit fashion, or to develop a new framework-based or toolkit-based approach. This paper therefore describes in detail a framework approach developed for the UK nuclear power and reprocessing industry, showing the practical implementation of such a system. It then considers some of the advantages and disadvantages of such framework approaches. The paper also discusses some of the relationships between error identification and Ergonomics.

Human reliability data requirements

For many years there has been increasing concern about the effects of human error in complex system safety and reliability. This concern has been increased owing to accidents such as Chernobyl, Bhopal, Herald of Free Enterprise, Three Mile Island and the Kegworth air disaster. In the vast majority of these accidents, human error has played a critical role in the events precipitating the accident. Such accidents can in theory be predicted and prevented by risk assessment, in particular assessing the human contribution to risk. However, the collection of human‐error data has proved a difficult field for the past 30 years, and yet industry would benefit from the existence of a robust human‐error database. Provides therefore a brief historical resume of past human‐error databases, and discusses data collection and the inherent problems associated with data‐collection schemes. Goes on to outline a human‐error database currently being developed at Birmingham University, and presents information on regularly qua...

The development of a nuclear chemical plant human reliability management approach: HRMS and JHEDI

Abstract In the late 1980s, amidst the qualitative and quantitative validation of certain Human Reliability Assessment (HRA) techniques, there was a desire for a new technique specifically for a nuclear reprocessing plant being designed. The technique was to have the following attributes: it should be data-based rather than involving pure expert judgement; it was to be flexible, so that it would allow both relatively rapid screening and more detailed assessment; and it was to have sensitivity analysis possibilities, so that Human Factors design-related parameters, albeit at a gross level, could be brought into the risk assessment equation. The techniques and literature were surveyed, and it was decided that no one technique fulfilled these requirements, and so a new approach was developed. Two techniques were devised, the Human Reliability Management System (HRMS), and the Justification of Human Error Data Information (JHEDI) technique, the latter being essentially a quicker screening version of the former. Both techniques carry out task analysis, error analysis, and Performance Shaping Factor-based quantification, but JHEDI involves less detailed assessment than HRMS. Additionally, HRMS can be utilised to determine error reduction mechanisms, based on the way the Performance Shaping Factors are contributing to the assessed error probabilities. Both techniques are fully computerised and assessments are highly documentable and auditable, which was seen as a useful feature both by the company developing the techniques, and by the regulatory authorities assessing the final output risk assessments into which these two techniques fed data. This paper focuses in particular on the quantification process used by these techniques. The quantification approach for both techniques was principally one of extrapolation from real data to the desired Human Error Probability (HEP), based on a comparison between Performance Shaping Factor (PSF) profiles for the real, and the to-be-assessed, task or scenario. The extrapolation process therefore required a set of PSF profiles for a number of real and representative data-based scenarios, and empirically-derived rules to extrapolate to new but similar scenarios. Using existing nuclear chemical plant human error and other data, a PSF profiling and extrapolation system was developed, which could assess most HEPs required for nuclear chemical risk assessments. The two techniques were then employed in a major risk assessment, with HRMS being utilised for approximately twenty high risk scenarios, and JHEDI being used to calculate well over five hundred HEPs for a large range of tasks and scenarios, by a number of assessors.

Collection of offshore human error probability data

Abstract Accidents such as Piper Alpha have increased concern about the effects of human errors in complex systems. Such accidents can in theory be predicted and prevented by risk assessment, and in particular human reliability assessment (HRA), but HRA ideally requires qualitative and quantitative human error data. A research initiative at the University of Birmingham led to the development of CORE-DATA, a Computerised Human Error Data Base. This system currently contains a reasonably large number of human error data points, collected from a variety of mainly nuclear-power related sources. This article outlines a recent offshore data collection study, concerned with collecting lifeboat evacuation data. Data collection methods are outlined and a selection of human error probabilities generated as a result of the study are provided. These data give insights into the type of errors and human failure rates that could be utilised to support offshore risk analyses.