Kathleen V. Diegert

Error and uncertainty in modeling and simulation

Abstract This article develops a general framework for identifying error and uncertainty in computational simulations that deal with the numerical solution of a set of partial differential equations (PDEs). A comprehensive, new view of the general phases of modeling and simulation is proposed, consisting of the following phases: conceptual modeling of the physical system, mathematical modeling of the conceptual model, discretization and algorithm selection for the mathematical model, computer programming of the discrete model, numerical solution of the computer program model, and representation of the numerical solution. Our view incorporates the modeling and simulation phases that are recognized in the systems engineering and operations research communities, but it adds phases that are specific to the numerical solution of PDEs. In each of these phases, general sources of uncertainty, both aleatory and epistemic, and error are identified. Our general framework is applicable to any numerical discretization procedure for solving ODEs or PDEs. To demonstrate this framework, we describe a system-level example: the flight of an unguided, rocket-boosted, aircraft-launched missile. This example is discussed in detail at each of the six phases of modeling and simulation. Two alternative models of the flight dynamics are considered, along with aleatory uncertainty of the initial mass of the missile and epistemic uncertainty in the thrust of the rocket motor. We also investigate the interaction of modeling uncertainties and numerical integration error in the solution of the ordinary differential equations for the flight dynamics.

System-based component-test plans and operating characteristics: binomial data

Component-test plans are often designed by allocating system reliability among the systems components, then choosing individual component plans suitable for demonstrating achievcment of each components reliability goal. This approach does not consider how much information relative to the system reliability goal is provided by the ensemble of component tests. We consider the notion of system reliability operating characteristic (OC) curves, based on the component tests, and illustrate their use in designing or evaluating an overall test program. By specifying OC values (akin to producers and consumers risks), optimum, system-ortented component-test plans can be derived. These ideas are illustrated for a series system, and for a simple series-parallel system, with binomial data.

Methodology for characterizing modeling and discretization uncertainties in computational simulation

This research effort focuses on methodology for quantifying the effects of model uncertainty and discretization error on computational modeling and simulation. The work is directed towards developing methodologies which treat model form assumptions within an overall framework for uncertainty quantification, for the purpose of developing estimates of total prediction uncertainty. The present effort consists of work in three areas: framework development for sources of uncertainty and error in the modeling and simulation process which impact model structure; model uncertainty assessment and propagation through Bayesian inference methods; and discretization error estimation within the context of non-deterministic analysis.

Quantifying reliability uncertainty : a proof of concept.

This paper develops Classical and Bayesian methods for quantifying the uncertainty in reliability for a system of mixed series and parallel components for which both go/no-go and variables data are available. Classical methods focus on uncertainty due to sampling error. Bayesian methods can explore both sampling error and other knowledge-based uncertainties. To date, the reliability community has focused on qualitative statements about uncertainty because there was no consensus on how to quantify them. This paper provides a proof of concept that workable, meaningful quantification methods can be constructed. In addition, the application of the methods demonstrated that the results from the two fundamentally different approaches can be quite comparable. In both approaches, results are sensitive to the details of how one handles components for which no failures have been seen in relatively few tests.

Bayesian methods for estimating the reliability in complex hierarchical networks (interim report).

Current work on the Integrated Stockpile Evaluation (ISE) project is evidence of Sandias commitment to maintaining the integrity of the nuclear weapons stockpile. In this report, we undertake a key element in that process: development of an analytical framework for determining the reliability of the stockpile in a realistic environment of time-variance, inherent uncertainty, and sparse available information. This framework is probabilistic in nature and is founded on a novel combination of classical and computational Bayesian analysis, Bayesian networks, and polynomial chaos expansions. We note that, while the focus of the effort is stockpile-related, it is applicable to any reasonably-structured hierarchical system, including systems with feedback.

Using statistical methods to assess a surveillance program

ABSTRACT Three metrics have been been developed to assess the National Nuclear Security Agency (NNSA) Surveillance Program against its objectives of detecting defects, determining margins and validating predictions. The surveillance metrics use statistical methods and are probabilities or confidences that produce quantitive assessments on a 0 to 1 scale—from no confidence that a given data stream achieves its surveillance program objectives to complete confidence that the data stream fulfills the objectives. These metrics may be compared and rolled up to support NNSA Surveillance Program management decisions.