Marco Valtorta

Construction of Bayesian network structures from data: A brief survey and an efficient algorithm☆

Abstract Previous algorithms for the recovery of Bayesian belief network structures from data have been either highly dependent on conditional independence (CI) tests, or have required on ordering on the nodes to be supplied by the user. We present an algorithm that integrates these two approaches: CI tests are used to generate an ordering on the nodes from the database, which is then used to recover the underlying Bayesian network structure using a non-CI-test-based method. Results of the evaluation of the algorithm on a number of databases (e.g., alarm , led , and soybean ) are presented. We also discuss some algorithm performance issues and open problems.

An algorithm for the construction of Bayesian network structures from data

Previous algorithms for the construction of Bayesian belief network structures from data have been either highly dependent on conditional independence (CI) tests, or have required an ordering on the nodes to be supplied by the user. We present an algorithm that integrates these two approaches - CI tests are used to generate an ordering on the nodes from the database which is then used to recover the underlying Bayesian network structure using a non CI based method. Results of preliminary evaluation of the algorithm on two networks (ALARM and LED) are presented. We also discuss some algorithm performance issues and open problems.

PAID: A Probabilistic Agent-Based Intrusion Detection system

In this paper we describe architecture and implementation of a Probabilistic Agent-Based Intrusion Detection (PAID) system. The PAID system has a cooperative agent architecture. Autonomous agents can perform specific intrusion detection tasks (e.g., identify IP-spoofing attacks) and also collaborate with other agents. The main contributions of our work are the following: our model allows agents to share their beliefs, i.e., the probability distribution of an event occurrence. Agents are capable to perform soft-evidential update, thus providing a continuous scale for intrusion detection. We propose methods for modelling errors and resolving conflicts among beliefs. Finally, we have implemented a proof-of-concept prototype of PAID.

Soft evidential update for probabilistic multiagent systems

We address the problem of updating a probability distribution represented by a Bayesian network upon presentation of soft evidence. Our motivation is the desire to let agents communicate with each other by exchanging beliefs, as in the Agent-Encapsulated Bayesian Network (AEBN) model, and soft evidential update (under several different names) is a problem with a long history. We give methodological guidance to model soft evidence in the form of beliefs (marginals) on single and multiple variables, propositional logical formulae (arbitrary events in the universe of discourse), and even conditional distributions, by introducing observation variables and explaining their use. The extended networks with observation variables fully capture the independence structure of the model, even upon receipt of soft evidence. We provide two algorithms that extend the celebrated junction tree algorithm, process soft evidence, and have different efficiency characteristics. One of the extensions, the big clique algorithm, promises to be more time efficient at the cost of possible space penalties. The other extension requires only minimal modifications to the junction tree at the cost of possibly substantial time penalties. Our results open new avenues of application for graphical probabilistic models.

A Prototypical System for Soft Evidential Update

Autonomous agents that communicate using probabilistic information and use Bayesian networks for knowledge representation need an update mechanism that goes beyond conditioning on the basis of evidence. In a related paper (M. Valtorta, Y.G. Kim, and J. Vomlel, International Journal of Approximate Reasoning, vol. 29, no. 1, pp. 71–106, 2002), we describe this mechanism, which we call soft evidential update, its properties, and algorithms to realize it. Here, we describe an implementation of the most promising such algorithm, the big clique algorithm, together with examples of its use.

MENTOR: A bayesian model for prediction of mental retardation in newborns

Mental retardation (MR) is a diagnosis that is made with extreme caution because of the many uncertainties in its etiology and prognosis. In fact, most physicians will delay the diagnosis for months or years so that substantial evidence is available to rule the diagnosis in or out. MENTOR is a Bayesian Model for the prediction of MR in newborns that provides probabilities for the full range of cognitive outcomes, ranging from MR to superior intelligence. Using the model to confirm clinical judgment could help physicians decide when to proceed with diagnostic tests. The physician and family could discuss the probabilities for MR, borderline, normal, and superior intelligence, given the childs status in infancy and base their decision about additional testing, in part, on this information.

Building Bayesian Network Models in Medicine: The MENTOR Experience

An experiment in Bayesian model building from a large medical dataset for Mental Retardation is discussed in this paper. We give a step by step description of the practical aspects of building a Bayesian Network from a dataset. We enumerate and briefly describe the tools required, address the problem of missing values in big datasets resulting from incomplete clinical findings and elaborate on our solution to the problem. We advance some reasons why imputation is a more desirable approach for model building than some other ad hoc methods suggested in literature. In our experiment, the initial Bayesian Network is learned from a dataset using a machine learning program called CB. The network structure and the conditional probabilities are then modified under the guidance of a domain expert. We present validation results for the unmodified and modified networks and give some suggestions for improvement of the model.

Performance Evaluation of Algorithms for Soft Evidential Update in Bayesian Networks: First Results

In this paper we analyze the performance of three algorithms for soft evidential update, in which a probability distribution represented by a Bayesian network is modified to a new distribution constrained by given marginals, and closest to the original distribution according to cross entropy. The first algorithm is a new and improved version of the big clique algorithm [1] that utilizes lazy propagation [2]. The second and third algorithm [3] are wrapper methods that convert soft evidence to virtual evidence, in which the evidence for a variable consists of a likelihood ratio. Virtual evidential update is supported in existing Bayesian inference engines, such as Hugin. To evaluate the three algorithms, we implemented BRUSE (Bayesian Reasoning Using Soft Evidence), a new Bayesian inference engine, and instrumented it. The resulting statistics are presented and discussed.

Extending Polynomial Chaos to Include Interval Analysis

Polynomial chaos theory (PCT) has been proven to be an efficient and effective way to represent and propagate uncertainty through system models and algorithms in general. In particular, PCT is a computationally efficient way to analyze and solve dynamic models under uncertainty. This paper presents a new way to use a polynomial expansion to incorporate uncertainties that are not expressed in terms of a probability density function (pdf). This paper presents the formalization of the process and some simple applications. The authors show that, within the framework introduced in this paper, it is possible to incorporate interval analysis. The long-term goal of this paper is to support the claim that the proposed framework can extract and represent uncertain behaviors in a form more general than previously used for these engineering problems. The proposed approach is first applied to an algebraic model and then to a differential equation model. The results thus obtained are analyzed in two different perspectives: 1) interpreting the PCT expansion as a fully probabilistic method and 2) in the framework of possibility theory. The conclusions in these two cases are compared and discussed.

Towards a Method for Data Accuracy Assessment Utilizing a Bayesian Network Learning Algorithm

This research develops a data quality algorithm entitled the Accuracy Assessment Algorithm (AAA). This is an extension of research in developing an enhancement to a Bayesian Network (BN) learning algorithm called the Data Quality (DQ) algorithm. This new algorithm is concerned with estimating the accuracy levels of a dataset by assessing the quality of the data with no prior knowledge of the dataset. The AAA and associated metrics were tested using two canonical BNs and one large-scale medical network. The article presents the results regarding the efficacy of the algorithm and the implications for future research and practice.