Rafael Rumí

Review: Bayesian networks in environmental modelling

Bayesian networks (BNs), also known as Bayesian belief networks or Bayes nets, are a kind of probabilistic graphical model that has become very popular to practitioners mainly due to the powerful probability theory involved, which makes them able to deal with a wide range of problems. The goal of this review is to show how BNs are being used in environmental modelling. We are interested in the application of BNs, from January 1990 to December 2010, in the areas of the ISI Web of Knowledge related to Environmental Sciences. It is noted that only the 4.2% of the papers have been published under this item. The different steps that configure modelling via BNs have been revised: aim of the model, data pre-processing, model learning, validation and software. Our literature review indicates that BNs have barely been used for Environmental Science and their potential is, as yet, largely unexploited.

Mixtures of Truncated Exponentials in Hybrid Bayesian Networks

In this paper we propose the use of mixtures of truncated exponential (MTE) distributions in hybrid Bayesian networks. We study the properties of the MTE distribution and show how exact probability propagation can be carried out by means of a local computation algorithm. One feature of this model is that no restriction is made about the order among the variables either discrete or continuous. Computations are performed over a representation of probabilistic potentials based on probability trees, expanded to allow discrete and continuous variables simultaneously. Finally, a Markov chain Monte Carlo algorithm is described with the aim of dealing with complex networks.

INFERENCE IN HYBRID BAYESIAN NETWORKS

Abstract Since the 1980s, Bayesian networks (BNs) have become increasingly popular for building statistical models of complex systems. This is particularly true for boolean systems, where BNs often prove to be a more efficient modelling framework than traditional reliability techniques (like fault trees and reliability block diagrams). However, limitations in the BNs’ calculation engine have prevented BNs from becoming equally popular for domains containing mixtures of both discrete and continuous variables (the so-called hybrid domains). In this paper we focus on these difficulties, and summarize some of the last decades research on inference in hybrid Bayesian networks. The discussions are linked to an example model for estimating human reliability.

Approximating probability density functions in hybrid Bayesian networks with mixtures of truncated exponentials

Mixtures of truncated exponentials (MTE) potentials are an alternative to discretization and Monte Carlo methods for solving hybrid Bayesian networks. Any probability density function (PDF) can be approximated by an MTE potential, which can always be marginalized in closed form. This allows propagation to be done exactly using the Shenoy-Shafer architecture for computing marginals, with no restrictions on the construction of a join tree. This paper presents MTE potentials that approximate standard PDF’s and applications of these potentials for solving inference problems in hybrid Bayesian networks. These approximations will extend the types of inference problems that can be modelled with Bayesian networks, as demonstrated using three examples.

Mixtures of truncated basis functions

In this paper we propose a framework, called mixtures of truncated basis functions (MoTBFs), for representing general hybrid Bayesian networks. The proposed framework generalizes both the mixture of truncated exponentials (MTEs) framework and the Mixture of Polynomials (MoPs) framework. Similar to MTEs and MoPs, MoTBFs are defined so that the potentials are closed under combination and marginalization, which ensures that inference in MoTBF networks can be performed efficiently using the Shafer-Shenoy architecture. Based on a generalized Fourier series approximation, we devise a method for efficiently approximating an arbitrary density function using the MoTBF framework. The translation method is more flexible than existing MTE or MoP-based methods, and it supports an online/anytime tradeoff between the accuracy and the complexity of the approximation. Experimental results show that the approximations obtained are either comparable or significantly better than the approximations obtained using existing methods.

Learning hybrid Bayesian networks using mixtures of truncated exponentials

In this paper we introduce an algorithm for learning hybrid Bayesian networks from data. The result of the algorithm is a network where the conditional distribution for each variable is a mixture of truncated exponentials (MTE), so that no restrictions on the network topology are imposed. The structure of the network is obtained by searching over the space of candidate networks using optimisation methods. The conditional densities are estimated by means of Gaussian kernel densities that afterwards are approximated by MTEs, so that the resulting network is appropriate for using standard algorithms for probabilistic reasoning. The behaviour of the proposed algorithm is tested using a set of real-world and artificially generated databases.

Estimating mixtures of truncated exponentials in hybrid bayesian networks

The MTE (Mixture of Truncated Exponentials) model allows to deal with Bayesian networks containing discrete and continuous variables simultaneously. This model offers an alternative to discretisation, since standard algorithms to compute the posterior probabilities in the network, in principle designed for discrete variables, can be directly applied over MTE models. In this paper, we study the problem of estimating these models from data. We propose an iterative algorithm based on least squares approximation. The performance of the algorithm is tested both with artificial and actual data.

Hybrid Bayesian network classifiers: Application to species distribution models

Bayesian networks are one of the most powerful tools in the design of expert systems located in an uncertainty framework. However, normally their application is determined by the discretization of the continuous variables. In this paper the naive Bayes (NB) and tree augmented naive Bayes (TAN) models are developed. They are based on Mixtures of Truncated Exponentials (MTE) designed to deal with discrete and continuous variables in the same network simultaneously without any restriction. The aim is to characterize the habitat of the spur-thighed tortoise (Testudo graeca graeca), using several continuous environmental variables, and one discrete (binary) variable representing the presence or absence of the tortoise. These models are compared with the full discrete models and the results show a better classification rate for the continuous one. Therefore, the application of continuous models instead of discrete ones avoids loss of statistical information due to the discretization. Moreover, the results of the TAN continuous model show a more spatially accurate distribution of the tortoise. The species is located in the Donana Natural Park, and in semiarid habitats. The proposed continuous models based on MTEs are valid for the study of species predictive distribution modelling.

Approximating Conditional MTE Distributions by Means of Mixed Trees

Mixtures of truncated exponential (MTE) distributions have been shown to be a powerful alternative to discretisation within the framework of Bayesian networks. One of the features of the MTE model is that standard propagation algorithms as Shenoy-Shafer and Lazy propagation can be used. Estimating conditional MTE densities from data is a rather difficult problem since, as far as we know, such densities cannot be expressed in parametric form in the general case. In the univariate case, regression-based estimators have been successfully employed. In this paper, we propose a method to estimate conditional MTE densities using mixed trees, which are graphical structures similar to classification trees. Criteria for selecting the variables during the construction of the tree and for pruning the leaves are defined in terms of the mean square error and entropy-like measures.

Parameter estimation and model selection for mixtures of truncated exponentials

Bayesian networks with mixtures of truncated exponentials (MTEs) support efficient inference algorithms and provide a flexible way of modeling hybrid domains (domains containing both discrete and continuous variables). On the other hand, estimating an MTE from data has turned out to be a difficult task, and most prevalent learning methods treat parameter estimation as a regression problem. The drawback of this approach is that by not directly attempting to find the parameter estimates that maximize the likelihood, there is no principled way of performing subsequent model selection using those parameter estimates. In this paper we describe an estimation method that directly aims at learning the parameters of an MTE potential following a maximum likelihood approach. Empirical results demonstrate that the proposed method yields significantly better likelihood results than existing regression-based methods. We also show how model selection, which in the case of univariate MTEs amounts to partitioning the domain and selecting the number of exponential terms, can be performed using the BIC score.