Curtis Smith

Global Sensitivity Measures from Given Data

Simulation models support managers in the solution of complex problems. International agencies recommend uncertainty and global sensitivity methods as best practice in the audit, validation and application of scientific codes. However, numerical complexity, especially in the presence of a high number of factors, induces analysts to employ less informative but numerically cheaper methods. This work introduces a design for estimating global sensitivity indices from given data (including simulation input–output data), at the minimum computational cost. We address the problem starting with a statistic based on the L1-norm. A formal definition of the estimators is provided and corresponding consistency theorems are proved. The determination of confidence intervals through a bias-reducing bootstrap estimator is investigated. The strategy is applied in the identification of the key drivers of uncertainty for the complex computer code developed at the National Aeronautics and Space Administration (NASA) assessing the risk of lunar space missions. We also introduce a symmetry result that enables the estimation of global sensitivity measures to datasets produced outside a conventional input–output functional framework.

Bayesian Inference in Probabilistic Risk Assessment -- The Current State of the Art

Markov chain Monte Carlo (MCMC) approaches to sampling directly from the joint posterior distribution of aleatory model parameters have led to tremendous advances in Bayesian inference capability in a wide variety of fields, including probabilistic risk analysis. The advent of freely available software coupled with inexpensive computing power has catalyzed this advance. This paper examines where the risk assessment community is with respect to implementing modern computational-based Bayesian approaches to inference. Through a series of examples in different topical areas, it introduces salient concepts and illustrates the practical application of Bayesian inference via MCMC sampling to a variety of important problems.

Construction of event-tree/fault-tree models from a Markov approach to dynamic system reliability

While the event-tree (ET)/fault-tree (FT) methodology is the most popular approach to probability risk assessment (PRA), concerns have been raised in the literature regarding its potential limitations in the reliability modeling of dynamic systems. Markov reliability models have the ability to capture the statistical dependencies between failure events that can arise in complex dynamic systems. A methodology is presented that combines Markov modeling with the cell-to-cell mapping technique (CCMT) to construct dynamic ETs/FTs and addresses the concerns with the traditional ET/FT methodology. The approach is demonstrated using a simple water level control system. It is also shown how the generated ETs/FTs can be incorporated into an existing PRA so that only the (sub)systems requiring dynamic methods need to be analyzed using this approach while still leveraging the static model of the rest of the system.

A Study of Interactions in the Risk Assessment of Complex Engineering Systems: An Application to Space PSA

Risk managers are often confronted with the evaluation of operational policies in which two or more system components are simultaneously affected by a change. In these instances, the decision-making process should be informed by the relevance of interactions. However, because of system and model complexity, a rigorous study for determining whether and how interactions quantitatively impact operational choices has not been developed yet. In light of the central role played by the multilinearity of the decision support models, we investigate the presence of interactions in multilinear functions first. We identify interactions that can be a priori excluded from the analysis. We introduce sensitivity measures that apportion the model output change to individual factors and interaction contributions in an exact fashion. The sensitivity measures are linked to graphical representation methods as tornado diagrams and Pareto charts, and a systematic way of inferring managerial insights is presented. We then specialize the findings to reliability and probabilistic safety assessment (PSA) problems. We set forth a procedure for determining the magnitude of changes that make interactions relevant in the analysis. Quantitative results are discussed by application to a PSA model developed at NASA to support decision making in space mission planning and design. Numerical findings show that suboptimal decisions concerning the components on which to focus managerial attention can be made, if the decision-making process is not informed by the relevance of interactions.

Calculating conditional core damage probabilities for nuclear power plant operations

Abstract A part of managing nuclear power plant operations is the control of plant risk over time as components are taken out of service or plant upsets are caused by initiating events. Unfortunately, measuring risk over time proves to be challenging, even with modern probabilistic risk analyses (PRAs) and PRA tools. In general, the process of measuring the operational risk would satisfy three desires: (1) the measurement would provide the risk magnitude for a particular event or over a period of time; (2) the risk results could be summed for a period of time to obtain a cumulative risk profile; and (3) the measurement process would be tractable while still using the current modeling techniques and tools. This paper demonstrates the calculation of the conditional core damage probability (CCDP) for the two cases of component outages and initiating events. In addition, two potential complications were identified that must be addressed when performing a CCDP calculation. The first complication, determining the appropriate nonrecovery probabilities to be applied to an inoperable component or initiating event, addresses the possibility of the plant operators preventing damage to the plant from their actions. The second complication, adjusting common-cause probabilities specific to the plant configuration, accounts for the fact that the PRA common-cause probabilities built into the model are applicable only during nominal conditions. The examples presented in the paper illustrate the potential under-estimation in CCDP when modifications to common-cause probabilities are ignored. These underestimation errors ranged from a factor of two to over a factor of six underestimation in CCDP.

A Bayesian method for using simulator data to enhance human error probabilities assigned by existing HRA methods

In the past several years, several international organizations have begun to collect data on human performance in nuclear power plant simulators. The data collected provide a valuable opportunity to improve human reliability analysis (HRA), but these improvements will not be realized without implementation of Bayesian methods. Bayesian methods are widely used to incorporate sparse data into models in many parts of probabilistic risk assessment (PRA), but Bayesian methods have not been adopted by the HRA community. In this paper, we provide a Bayesian methodology to formally use simulator data to refine the human error probabilities (HEPs) assigned by existing HRA methods. We demonstrate the methodology with a case study, wherein we use simulator data from the Halden Reactor Project to update the probability assignments from the SPAR-H method. The case study demonstrates the ability to use performance data, even sparse data, to improve existing HRA methods. Furthermore, this paper also serves as a demonstration of the value of Bayesian methods to improve the technical basis of HRA.

Key Attributes of the SAPHIRE Risk and Reliability Analysis Software for Risk-Informed Probabilistic Applications

Abstract The Idaho National Laboratory is a primary developer of probabilistic risk and reliability analysis (PRRA) tools, dating back over 35 years. Evolving from mainframe-based software, the current state-of-the-practice has led to the creation of the SAPHIRE software. Currently, agencies such as the Nuclear Regulatory Commission, the National Aeronautics and Aerospace Agency, the Department of Energy, and the Department of Defense use version 7 of the SAPHIRE software for many of their risk-informed activities. In order to better understand and appreciate the power of software as part of risk-informed applications, we need to recall that our current analysis methods and solution methods have built upon pioneering work done 30–40 years ago. We contrast this work with the current capabilities in the SAPHIRE analysis package. As part of this discussion, we provide information for both the typical features and special analysis capabilities, which are available. We also present the application and results typically found with state-of-the-practice PRRA models. By providing both a high-level and detailed look at the SAPHIRE software, we give a snapshot in time for the current use of software tools in a risk-informed decision arena.

Calculating and addressing uncertainty for risk-based allowable outage times

Abstract This article examines the calculation and treatment of uncertainty in risk-based allowable outage times (AOTs) for operational control at nuclear power plants, where an AOT is defined as the time that a component or system is permitted to be out of service. The US Nuclear Regulatory Commission (NRC) has explored the possibility of using a nuclear power plants probabilistic risk assessment results to determine component or system AOTs. The analysis and results from previous work prepared for the NRC on determining risk-based AOTs are presented. As part of the discussion, the article examines the inherent uncertainty in calculating risk-based AOTs and presents the difficulties in calculating these risk-based AOTs. It is noted that care should be taken when dealing with uncertainty analysis results where a time-interval is the outcome of the analysis. In addition, potential improvements in the mechanism of calculating risk-based AOTs are suggested.

Analyzing simulation-based PRA data through traditional and topological clustering: A BWR station blackout case study

Dynamic probabilistic risk assessment (DPRA) methodologies couple system simulator codes (e.g., RELAP and MELCOR) with simulation controller codes (e.g., RAVEN and ADAPT). Whereas system simulator codes model system dynamics deterministically, simulation controller codes introduce both deterministic (e.g., system control logic and operating procedures) and stochastic (e.g., component failures and parameter uncertainties) elements into the simulation. Typically, a DPRA is performed by sampling values of a set of parameters and simulating the system behavior for that specific set of parameter values. For complex systems, a major challenge in using DPRA methodologies is to analyze the large number of scenarios generated, where clustering techniques are typically employed to better organize and interpret the data. In this paper, we focus on the analysis of two nuclear simulation datasets that are part of the risk-informed safety margin characterization (RISMC) boiling water reactor (BWR) station blackout (SBO) case study. We provide the domain experts a software tool that encodes traditional and topological clustering techniques within an interactive analysis and visualization environment, for understanding the structures of such high-dimensional nuclear simulation datasets. We demonstrate through our case study that both types of clustering techniques complement each other for enhanced structural understanding of the data.

Composite multilinearity, epistemic uncertainty and risk achievement worth

Risk achievement worth is one of the most widely utilized importance measures. RAW is defined as the ratio of the risk metric value attained when a component has failed over the base case value of the risk metric. Traditionally, both the numerator and denominator are point estimates. Relevant literature has shown that inclusion of epistemic uncertainty (i) induces notable variability in the point estimate ranking and (ii) causes the expected value of the risk metric to differ from its nominal value. We investigate the conditions under which the equality of the nominal and expected values of a reliability risk metric holds. We then study how the presence of epistemic uncertainty affects RAW and the associated ranking. We propose an extension of RAW (called ERAW) which allows one to obtain a ranking robust to epistemic uncertainty. We discuss the properties of ERAW and the conditions under which it coincides with RAW. We apply our findings to a probabilistic risk assessment model developed for the safety analysis of NASA lunar space missions.