Wim Wiegerinck

Graphical model inference in optimal control of stochastic multi-agent systems

In this article we consider the issue of optimal control in collaborative multi-agent systems with stochastic dynamics. The agents have a joint task in which they have to reach a number of target states. The dynamics of the agents contains additive control and additive noise, and the autonomous part factorizes over the agents. Full observation of the global state is assumed. The goal is to minimize the accumulated joint cost, which consists of integrated instantaneous costs and a joint end cost. The joint end cost expresses the joint task of the agents. The instantaneous costs are quadratic in the control and factorize over the agents. The optimal control is given as a weighted linear combination of single-agent to single-target controls. The single-agent to single-target controls are expressed in terms of diffusion processes. These controls, when not closed form expressions, are formulated in terms of path integrals, which are calculated approximately by Metropolis-Hastings sampling. The weights in the control are interpreted as marginals of a joint distribution over agent to target assignments. The structure of the latter is represented by a graphical model, and the marginals are obtained by graphical model inference. Exact inference of the graphical model will break down in large systems, and so approximate inference methods are needed. We use naive mean field approximation and belief propagation to approximate the optimal control in systems with linear dynamics. We compare the approximate inference methods with the exact solution, and we show that they can accurately compute the optimal control. Finally, we demonstrate the control method in multi-agent systems with nonlinear dynamics consisting of up to 80 agents that have to reach an equal number of target states.

Central extensions and physics

Abstract In this paper two themes are considered; first of all we consider the question under what circumstances a central extension of the Lie algebra of a given Lie group determines a central extension of this Lie group (and how many different ones). The answer will be that if we give the algebra extension in the form of a left invariant closed 2-form ω on the Lie group, then there exists an associated group extension iff the group of periods of ω is a discrete subgroup of IR and ω admits a momentum mapping for the left action of the group on itself. The second theme concerns the process of pre-quantization; we show that the construction needed to answer the previous question is exactly the same as the construction of prequantum bundles in geometric quantization. Moreover we show that the formalism of prequantization over a symplectic manifold and the formalism of quantum mechanics (where the projective Hilbert spaces replaces the (symplectic) phase space) can be identified (modulo some ≪details≫ concerning infinite dimensions).

A multi-model ensemble method that combines imperfect models through learning

Abstract. In the current multi-model ensemble approach climate model simulations are combined a posteriori. In the method of this study the models in the ensemble exchange information during simulations and learn from historical observations to combine their strengths into a best representation of the observed climate. The method is developed and tested in the context of small chaotic dynamical systems, like the Lorenz 63 system. Imperfect models are created by perturbing the standard parameter values. Three imperfect models are combined into one super-model, through the introduction of connections between the model equations. The connection coefficients are learned from data from the unperturbed model, that is regarded as the truth. The main result of this study is that after learning the super-model is a very good approximation to the truth, much better than each imperfect model separately. These illustrative examples suggest that the super-modeling approach is a promising strategy to improve weather and climate simulations.

Approximate inference for medical diagnosis

Abstract Computer-based diagnostic decision support systems (DSSs) will play an increasingly important role in health care. Due to the inherent probabilistic nature of medical diagnosis, a DSS should preferably be based on a probabilistic model. In particular, Bayesian networks provide a powerful and conceptually transparent formalism for probabilistic modeling. A drawback is that Bayesian networks become intractable for exact computation if a large medical domain is to be modeled in detail. This has obstructed the development of a useful system for internal medicine. Advances in approximation techniques, e.g. using variational methods with tractable structures, have opened new possibilities to deal with the computational problem. However, the only way to assess the usefulness of these methods for a DSS in practice is by actually building such a system and evaluating it by users. In the coming years, we aim to build a DSS for anaemia based on a detailed probabilistic model, and equipped with approximate methods to study the practical feasibility and the usefulness of this approach in medical practice. In this paper, we will sketch how variational techniques with tractable structures can be used in a typical model for medical diagnosis. We provide numerical results on artificial problems. In addition, we describe our approach to develop the Bayesian network for the DSS and show some preliminary results.

Inference in the Promedas Medical Expert System

In the current paper, the Promedas model for internal medicine, developed by our team, is introduced. The model is based on up-to-date medical knowledge and consists of approximately 2000 diagnoses, 1000 findings and 8600 connections between diagnoses and findings, covering a large part of internal medicine. We show that Belief Propagation (BP) can be successfully applied as approximate inference algorithm in the Promedas network. In some cases, however, we find errors that are too large for this application. We apply a recently developed method that improves the BP results by means of a loop expansion scheme. This method, termed Loop Corrected (LC) BP, is able to improve the marginal probabilities significantly, leaving a remaining error which is acceptable for the purpose of medical diagnosis.

Bayesian Networks for Expert Systems: Theory and Practical Applications

Bayesian networks are widely accepted as models for reasoning with uncertainty. In this chapter, we focus on models that are created using domain expertise only. After a short review of Bayesian network models and common Bayesian network modeling approaches, we will discuss in more detail three applications of Bayesian networks.With these applications, we aim to illustrate the modeling power and flexibility of the Bayesian networks, which go beyond the standard textbook applications. The first network is applied in a system for medical diagnostic decision support. A distinguishing feature of this network is the large amount of variables in the model. The second one involves an application for petrophysical decision support to determine the mineral content of a well, based on borehole measurements. This model differs from standard Bayesian networks in terms of its continuous variables and nonlinear relations. Finally, we will discuss an application for victim identification by kinship analysis based on DNA profiles. The distinguishing feature in this application is that Bayesian networks are generated and computed on-the-fly based on case information.

A neural network study of precollicular saccadic averaging

Abstract. Saccadic averaging is the phenomenon that two simultaneously presented retinal inputs result in a saccade with an endpoint located on an intermediate position between the two stimuli. Recordings from neurons in the deeper layers of the superior colliculus have revealed neural correlates of saccade averaging, indicating that it takes place at this level or upstream. Recently, we proposed a neural network for internal feedback in saccades. This neural network model is different from other models in that it suggests the possibility that averaging takes place in a stage upstream of the colliculus. The network consists of output units representing the neural map of the deeper layers of the superior colliculus and hidden layers imitating areas in the posterior parietal cortex. The deeper layers of the superior colliculus represent the motor error of a desired saccade, e.g. an eye movement to a visual target. In this article we show that averaging is an emergent property of the proposed network. When two retinal targets with different intensities are simultaneously presented to the network, the activity in the output layer represents a single motor error with a weighted average value. Our goal is to understand the mechanism of weighted averaging in this neural network. It appears that averaging in the model is caused by the linear dependence of the net input, received by the hidden units, on retinal error, independent of its retinal coding format. For nonnormalized retinal error inputs, also the nonlinearity between the net input and the activity of the hidden units plays a role in the averaging process. The averaging properties of the model are in agreement with physiological experiments if the hypothetical retinal error input map is normalized. The neural network predicts that if this normalization is overruled by electrical stimulation, averaging still takes place. However, in this case – as a consequence of the feedback task – the location of the resulting saccade depends on the initial eye position and the total intensity/current applied at the two locations. This could be a way to verify the neural network model. If the assumptions for the model are valid, a physiological implication of this paper is that averaging of saccades takes place upstream of the superior colliculus.

Stochastic dynamics of learning with momentum in neural networks

We study on-line learning with a momentum term for nonlinear learning rules. Through introduction of auxiliary variables, we show that the learning process can be described by a Markov process. For small learning parameters eta and momentum parameters alpha close to 1, such that gamma = eta /(1- alpha )2 is finite, the time-scales for the evolution of the weights and the auxiliary variables are the same. In this case Van Kampens expansion can be applied in a straightforward manner. We obtain evolution equations for the average network state and the fluctuations around this average. These evolution equations depend (after rescaling a of the time and fluctuations) only on gamma : all combinations ( eta , alpha ) with the same value of gamma give rise to similar behaviour. The case with alpha constant and eta small requires a completely different analysis. There are two different time-scales: a fast time-scale on which the auxiliary variables equilibrate and a slow time-scale for the change of the weights. By projection on the space of slow variables the fast variables can be eliminated. We find that, for small learning parameters eta and finite momentum parameters alpha , learning with momentum is equivalent to learning without a momentum term with a rescaled learning parameter eta = eta /(1- alpha ). Simulations with the nonlinear Oja learning rule confirm the theoretical results.

Dynamically combining climate models to "supermodel" the tropical Pacific

We construct an interactive ensemble of two different climate models to improve simulation of key aspects of tropical Pacific climate. Our so-called supermodel is based on two atmospheric general circulation models (AGCMs) coupled to a single ocean GCM, which is driven by a weighted average of the air-sea fluxes. Optimal weights are determined using a machine learning algorithm to minimize sea surface temperature errors over the tropical Pacific. This coupling strategy synchronizes atmospheric variability in the two AGCMs over the equatorial Pacific, where it improves the representation of ocean-atmosphere interaction and the climate state. In particular, the common double Intertropical Convergence Zone error is suppressed, and the positive Bjerknes feedback improves substantially to match observations well, and the negative heat flux feedback is also much improved. This study supports the concept of supermodeling as a promising multimodel ensemble strategy to improve weather and climate predictions.

BOVINOSE: Pheromone-Based Sensor System for Detecting Estrus in Dairy Cows

The BOVINOSE project (www.bovinose.eu) aims to develop an electronic nose to detect estrus in a dairy cow, and thus to determine the optimal timing of artificial insemination. The physical principle is based on detection of sex pheromones that are secreted by the cow, exclusively during estrus. These pheromones are the natural olfactory signal for the bull that the cow is in heat. This technology aims to help the dairy farmers in the EU, the vast majority being micro-enterprises run as family businesses.