Anca M. Hanea

Hybrid Method for Quantifying and Analyzing Bayesian Belief Nets

Bayesian belief nets (BBNs) have become a popular tool for specifying high-dimensional probabilistic models. Commercial tools with an advanced graphical user interface that support BBNs construction and inference are available. Thus, building and working with BBNs is very efficient as long as one is not forced to quantify complex BBNs. A high assessment burden of discrete BBNs is often caused by the discretization of continuous variables. Until recently, continuous BBNs were restricted to the joint normal distribution. We present the ‘copula–vine’ approach to continuous BBNs. This approach is quite general and allows traceable and defendable quantification methods, but it comes at a price: these BBNs must be evaluated by Monte Carlo simulation. Updating such a BBN requires re-sampling the whole structure. The advantages of fast updating algorithms for discrete BBNs are decisive. A hybrid method advanced here samples the continuous BBN once, and then discretizes this so as to enable fast updating. This combines the reduced assessment burden and modelling flexibility of the continuous BBNs with the fast updating algorithms of discrete BBNs. Sampling large complex structures only once can still involve time consuming numerical calculations. Therefore a new sampling protocol based on normal vines is developed. Normal vines are used to realize the dependence structure specified via (conditional) rank correlations on the continuous BBN. We will emphasize the advantages of this method by means of examples. Copyright

Mining and visualising ordinal data with non-parametric continuous BBNs

Data mining is the process of extracting and analysing information from large databases. Graphical models are a suitable framework for probabilistic modelling. A Bayesian Belief Net (BBN) is a probabilistic graphical model, which represents joint distributions in an intuitive and efficient way. It encodes the probability density (or mass) function of a set of variables by specifying a number of conditional independence statements in the form of a directed acyclic graph. Specifying the structure of the model is one of the most important design choices in graphical modelling. Notwithstanding their potential, there are only a limited number of applications of graphical models on very complex and large databases. A method for mining ordinal multivariate data using non-parametric BBNs is presented. The main advantage of this method is that it can handle a large number of continuous variables, without making any assumptions about their marginal distributions, in a very fast manner. Once the BBN is learned from data, it can be further used for prediction. This approach allows for rapid conditionalisation, which is a very important feature of a BBN from a users standpoint.

Non-parametric Bayesian networks: Improving theory and reviewing applications

Applications in various domains often lead to high dimensional dependence modelling. A Bayesian network (BN) is a probabilistic graphical model that provides an elegant way of expressing the joint distribution of a large number of interrelated variables. BNs have been successfully used to represent uncertain knowledge in a variety of fields. The majority of applications use discrete BNs, i.e. BNs whose nodes represent discrete variables. Integrating continuous variables in BNs is an area fraught with difficulty. Several methods that handle discrete-continuous BNs have been proposed in the literature. This paper concentrates only on one method called non-parametric BNs (NPBNs). NPBNs were introduced in 2004 and they have been or are currently being used in at least twelve professional applications. This paper provides a short introduction to NPBNs, a couple of theoretical advances, and an overview of applications. The aim of the paper is twofold: one is to present the latest improvements of the theory underlying NPBNs, and the other is to complement the existing overviews of BNs applications with the NPNBs applications. The latter opens the opportunity to discuss some difficulties that applications pose to the theoretical framework and in this way offers some NPBN modelling guidance to practitioners.

A practical guide to structured expert elicitation using the IDEA protocol

Summary Expert judgment informs a variety of important applications in conservation and natural resource management, including threatened species management, environmental impact assessment, and structured decision-making. However, expert judgments can be prone to contextual biases. Structured elicitation protocols mitigate these biases and improve the accuracy and transparency of the resulting judgments. Despite this, the elicitation of expert judgment within conservation and natural resource management remains largely informal. We suggest this may be attributed to financial and practical constraints which are not addressed by many existing structured elicitation protocols. In this paper, we advocate that structured elicitation protocols must be adopted when expert judgments are used to inform science. In order to motivate a wider adoption of structured elicitation protocols, we outline the IDEA protocol. The protocol improves the accuracy of expert judgments and includes several key steps which may be familiar to many conservation researchers, such as the four-step elicitation, and a modified Delphi procedure (“Investigate”, “Discuss”, “Estimate” and “Aggregate”). It can also incorporate remote elicitation, making structured expert judgment accessible on a modest budget. The IDEA protocol has recently been outlined in the scientific literature, however, a detailed description has been missing. This paper fills that important gap by clearly outlining each of the steps required to prepare for and undertake an elicitation. Whilst this paper focuses on the need for the IDEA protocol within conservation and natural resource management, the protocol (and the advice contained in this paper), is applicable to a broad range of scientific domains, as evidenced by its application to biosecurity, engineering, and political forecasting. By clearly outlining the IDEA protocol, we hope that structured protocols will be more widely understood and adopted, resulting in improved judgments and increased transparency when expert judgment is required. This article is protected by copyright. All rights reserved.

Classical meets modern in the IDEA protocol for structured expert judgement

Expert judgement is pervasive in all forms of risk analysis, yet the development of tools to deal with such judgements in a repeatable and transparent fashion is relatively recent. This work outlines new findings related to an approach to expert elicitation termed the IDEA protocol. IDEA combines psychologically robust interactions among experts with mathematical aggregation of individual estimates. In particular, this research explores whether communication among experts adversely effects the reliability of group estimates. Using data from estimates of the outcomes of geopolitical events, we find that loss of independence is relatively modest and it is compensated by improvements in group accuracy.

Expert judgement for dependence in probabilistic modelling: A systematic literature review and future research directions

Many applications in decision making under uncertainty and probabilistic risk assessment require the assessment of multiple, dependent uncertain quantities, so that in addition to marginal distributions, interdependence needs to be modelled in order to properly understand the overall risk. Nevertheless, relevant historical data on dependence information are often not available or simply too costly to obtain. In this case, the only sensible option is to elicit this uncertainty through the use of expert judgements. In expert judgement studies, a structured approach to eliciting variables of interest is desirable so that their assessment is methodologically robust. One of the key decisions during the elicitation process is the form in which the uncertainties are elicited. This choice is subject to various, potentially conflicting, desiderata related to e.g. modelling convenience, coherence between elicitation parameters and the model, combining judgements, and the assessment burden for the experts. While extensive and systematic guidance to address these considerations exists for single variable uncertainty elicitation, for higher dimensions very little such guidance is available. Therefore, this paper offers a systematic review of the current literature on eliciting dependence. The literature on the elicitation of dependence parameters such as correlations is presented alongside commonly used dependence models and experience from case studies. From this, guidance about the strategy for dependence assessment is given and gaps in the existing research are identified to determine future directions for structured methods to elicit dependence.

Non-parametric Bayesian networks for parameter estimation in reservoir simulation: a graphical take on the ensemble Kalman filter (part I)

Reservoir simulation models are used both in the development of new fields and in developed fields where production forecasts are needed for investment decisions. When simulating a reservoir, one must account for the physical and chemical processes taking place in the subsurface. Rock and fluid properties are crucial when describing the flow in porous media. In this paper, the authors are concerned with estimating the permeability field of a reservoir. The problem of estimating model parameters such as permeability is often referred to as a history-matching problem in reservoir engineering. Currently, one of the most widely used methodologies which address the history-matching problem is the ensemble Kalman filter (EnKF). EnKF is a Monte Carlo implementation of the Bayesian update problem. Nevertheless, the EnKF methodology has certain limitations that encourage the search for an alternative method.For this reason, a new approach based on graphical models is proposed and studied. In particular, the graphical model chosen for this purpose is a dynamic non-parametric Bayesian network (NPBN). This is the first attempt to approach a history-matching problem in reservoir simulation using a NPBN-based method. A two-phase, two-dimensional flow model was implemented for a synthetic reservoir simulation exercise, and initial results are shown. The methods’ performances are evaluated and compared. This paper features a completely novel approach to history matching and constitutes only the first part (part I) of a more detailed investigation. For these reasons (novelty and incompleteness), many questions are left open and a number of recommendations are formulated, to be investigated in part II of the same paper.

IDEA for Uncertainty Quantification

It is generally agreed that an elicitation protocol for quantifying uncertainty will always benefit from the involvement of more than one domain expert. The two key mechanisms by which judgements may be pooled across experts are through striving for consensus, via behavioural aggregation, where experts share and discuss information, and via mathematical methods, where judgements are combined using a mechanistic rule. Mixed approaches combine elements of both deliberative (behavioural) and mechanical (mathematical) styles of aggregation.

Bayesian networks for identifying incorrect probabilistic intuitions in a climate trend uncertainty quantification context

Abstract Probabilistic thinking can often be unintuitive. This is the case even for simple problems, let alone the more complex ones arising in climate modelling, where disparate information sources need to be combined. The physical models, the natural variability of systems, the measurement errors and their dependence upon the observational period length should be modelled together in order to understand the intricacies of the underlying processes. We use Bayesian networks (BNs) to connect all the above-mentioned pieces in a climate trend uncertainty quantification framework. Inference in such models allows us to observe some seemingly nonsensical outcomes. We argue that they must be pondered rather than discarded until we understand how they arise. We would like to stress that the main focus of this paper is the use of BNs in complex probabilistic settings rather than the application itself.

Eliciting Multivariate Uncertainty from Experts : Considerations and Approaches Along the Expert Judgement Process

A useful definition of a problem is that it is a situation where there is a current state, and a desired state, and they are not the same. Most people are familiar with this sort of situation and many day to day problems can be dealt with by largely subconscious or automatic processes (the coffee is too bitter, so I add sugar, the water is too cold, so I turn the tap). But some problems (I want to take up a new hobby, perhaps a new sport, a new language, or a new instrument) require reflection: I have to reflect what goals i want to achieve and whether the actions I have at my disposal will help me achieve the. In such cases I have to build a mental model of my problem to organize my thoughts and help me choose wisely. Other problems, even more complex, involve the significant others in my life (where should we go on holiday?; should we move to a new city, or new country, to take that new job?): in these cases, the model I build should be a shared one, so as to ensure that all those involved in the problem understand what they are getting into. At a higher level still, society has to take important decisions about responses to threats to our environmental and economic wellbeing and security: in a democracy these decisions should take account of the news of the public in some organised fashion.In decision and risk analysis problems, modelling uncertainty probabilistically provides key insights and information for decision makers. A common challenge is that uncertainties are typically not isolated but interlinked which introduces complex (and often unexpected) effects on the model output. Therefore, dependence needs to be taken into account and modelled appropriately if simplifying assumptions, such as independence, are not sensible. Similar to the case of univariate uncertainty, which is described elsewhere in this book, relevant historical data to quantify a (dependence) model are often lacking or too costly to obtain. This may be true even when data on a model’s univariate quantities, such as marginal probabilities, are available. Then, specifying dependence between the uncertain variables through expert judgement is the only sensible option. A structured and formal process to the elicitation is essential for ensuring methodological robustness. This chapter addresses the main elements of structured expert judgement processes for dependence elicitation. We introduce the processes’ common elements, typically used for eliciting univariate quantities, and present the differences that need to be considered at each of the process’ steps for multivariate uncertainty. Further, we review findings from the behavioural judgement and decision making literature on potential cognitive fallacies that can occur when assessing dependence as mitigating biases is a main objective of formal expert judgement processes. Given a practical focus, we reflect on case studies in addition to theoretical findings. Thus, this chapter serves as guidance for facilitators and analysts using expert judgement.Our methodology is based on the premise that expertise does not reside in the stochastic characterisation of the unknown quantity of interest, but rather upon other features of the problem to which an expert can relate her experience. By mapping the quantity of interest to an expert’s experience we can use available empirical data about associated events to support the quantification of uncertainty. Our rationale contrasts with other approaches to elicit subjective probability which ask an expert to map, according to her belief, the outcome of an unknown quantity of interest to the outcome of a lottery for which the randomness is understood and quantifiable. Typically, such a mapping represents the indifference of an expert on making a bet between the quantity of interest and the outcome of the lottery. Instead, we propose to construct a prior distribution with empirical data that is consistent with the subjective judgement of an expert. We develop a general methodology, grounded in the theory of empirical Bayes inference. We motivate the need for such an approach and illustrate its application through industry examples. We articulate our general steps and show how these translate to selected practical contexts. We examine the benefits, as well as the limitations, of our proposed methodology to indicate when it might, or might not be, appropriate.