Jan R. J. Nunnink

Accelerated EM-based clustering of large data sets

Motivated by the poor performance (linear complexity) of the EM algorithm in clustering large data sets, and inspired by the successful accelerated versions of related algorithms like k-means, we derive an accelerated variant of the EM algorithm for Gaussian mixtures that: (1) offers speedups that are at least linear in the number of data points, (2) ensures convergence by strictly increasing a lower bound on the data log-likelihood in each learning step, and (3) allows ample freedom in the design of other accelerated variants. We also derive a similar accelerated algorithm for greedy mixture learning, where very satisfactory results are obtained. The core idea is to define a lower bound on the data log-likelihood based on a grouping of data points. The bound is maximized by computing in turn (i) optimal assignments of groups of data points to the mixture components, and (ii) optimal re-estimation of the model parameters based on average sufficient statistics computed over groups of data points. The proposed method naturally generalizes to mixtures of other members of the exponential family. Experimental results show the potential of the proposed method over other state-of-the-art acceleration techniques.

A multi-agent systems approach to distributed bayesian information fusion

This paper introduces design principles for modular Bayesian fusion systems which can (i) cope with large quantities of heterogeneous information and (ii) can adapt to changing constellations of information sources on the fly. The presented approach exploits the locality of relations in causal probabilistic processes, which facilitates decentralized modeling and information fusion. Observed events resulting from stochastic causal processes can be modeled with the help of Bayesian networks, compact and mathematically rigorous probabilistic models. With the help of the theory of Bayesian networks and factor graphs we derive design and organization rules for modular fusion systems which implement exact belief propagation without centralized configuration and fusion control. These rules are applied in distributed perception networks (DPN), a multi-agent systems approach to distributed Bayesian information fusion. While each DPN agent has limited fusion capabilities, multiple DPN agents can autonomously collaborate to form complex modular fusion systems. Such self-organizing systems of agents can adapt to the available information sources at runtime and can infer critical hidden events through interpretation of complex patterns consisting of many heterogeneous observations.

Inference Meta Models: Towards Robust Information Fusion with Bayesian Networks

This paper discusses the properties of Bayesian networks (BNs) in the context of accurate state estimation. We focus on a relevant class of problems where state estimation can be viewed as a classification of possible states based on the fusion of heterogeneous and noisy information. We introduce the inference meta model (IMM), a coarse runtime perspective on the inference processes which facilitates the analysis of the state estimation with BNs. By making coarse and realistic assumptions, we show that such inference can be very robust and has asymptotic properties regarding the fusion accuracy, even if we use models and evidence associated with significant uncertainties. Moreover, the IMM provides guidance for the development of (i) robust fusion systems and (ii) methods for runtime detection of potentially misleading fusion results

A MAS Approach to Fusion of Heterogeneous Information

Distributed perception networks (DPN) are a MAS approach to large scale fusion of heterogeneous and noisy information. DPN agents can establish meaningful information filtering channels between the relevant information sources and the decision makers. Through specification of high level concepts, DPN agent organizations generate distributed Bayesian networks, which provide mappings between the observed symptoms and the hypotheses relevant to the decision making. In addition, DPNs support robust distributed inference as well as decentralized probabilistic resource allocation.