Johan Kwisthout

Most probable explanations in Bayesian networks: Complexity and tractability

One of the key computational problems in Bayesian networks is computing the maximal posterior probability of a set of variables in the network, given an observation of the values of another set of variables. In its most simple form, this problem is known as the MPE-problem. In this paper, we give an overview of the computational complexity of many problem variants, including enumeration variants, parameterized problems, and approximation strategies to the MPE-problem with and without additional (neither observed nor explained) variables. Many of these complexity results appear elsewhere in the literature; other results have not been published yet. The paper aims to provide a fairly exhaustive overview of both the known and new results.

Intentional Communication: Computationally Easy or Difficult?

Human intentional communication is marked by its flexibility and context sensitivity. Hypothesized brain mechanisms can provide convincing and complete explanations of the human capacity for intentional communication only insofar as they can match the computational power required for displaying that capacity. It is thus of importance for cognitive neuroscience to know how computationally complex intentional communication actually is. Though the subject of considerable debate, the computational complexity of communication remains so far unknown. In this paper we defend the position that the computational complexity of communication is not a constant, as some views of communication seem to hold, but rather a function of situational factors. We present a methodology for studying and characterizing the computational complexity of communication under different situational constraints. We illustrate our methodology for a model of the problems solved by receivers and senders during a communicative exchange. This approach opens the way to a principled identification of putative model parameters that control cognitive processes supporting intentional communication.

Joint attention and language evolution

This study investigates how more advanced joint attentional mechanisms, rather than only shared attention between two agents and an object, can be implemented and how they influence the results of language games played by these agents. We present computer simulations with language games showing that adding constructs that mimic the three stages of joint attention identified in childrens early development (checking attention, following attention, and directing attention) substantially increase the performance of agents in these language games. In particular, the rates of improved performance for the individual attentional mechanisms have the same ordering as that of the emergence of these mechanisms in infants’ development. These results suggest that language evolution and joint attentional mechanisms have developed in a co-evolutionary way, and that the evolutionary emergence of the individual attentional mechanisms is ordered just like their developmental emergence.

To be precise, the details don't matter: On predictive processing, precision, and level of detail of predictions

HighlightsWe provide a Predictive Processing formalization based on causal Bayesian networks.We propose six mechanisms for lowering prediction error.We identify crucial conceptual, theoretical open problems in Predictive Processing. Abstract Many theoretical and empirical contributions to the Predictive Processing account emphasize the important role of precision modulation of prediction errors. Recently it has been proposed that the causal models used in human predictive processing are best formally modeled by categorical probability distributions. Crucially, such distributions assume a well‐defined, discrete state space. In this paper we explore the consequences of this formalization. In particular we argue that the level of detail of generative models and predictions modulates prediction error. We show that both increasing the level of detail of the generative models and decreasing the level of detail of the predictions can be suitable mechanisms for lowering prediction errors. Both increase precision, yet come at the price of lowering the amount of information that can be gained by correct predictions. Our theoretical result establishes a key open empirical question to address: How does the brain optimize the trade‐off between high precision and information gain when making its predictions?

Beta- and gamma-band activity reflect predictive coding in the processing of causal events.

In daily life, complex events are perceived in a causal manner, suggesting that the brain relies on predictive processes to model them. Within predictive coding theory, oscillatory beta-band activity has been linked to top-down predictive signals and gamma-band activity to bottom-up prediction errors. However, neurocognitive evidence for predictive coding outside lower-level sensory areas is scarce. We used magnetoencephalography to investigate neural activity during probability-dependent action perception in three areas pivotal for causal inference, superior temporal sulcus, temporoparietal junction and medial prefrontal cortex, using bowling action animations. Within this network, Granger-causal connectivity in the beta-band was found to be strongest for backward top-down connections and gamma for feed-forward bottom-up connections. Moreover, beta-band power in TPJ increased parametrically with the predictability of the action kinematics-outcome sequences. Conversely, gamma-band power in TPJ and MPFC increased with prediction error. These findings suggest that the brain utilizes predictive-coding-like computations for higher-order cognition such as perception of causal events.

The complexity of finding kth most probable explanations in probabilistic networks

In modern decision-support systems, probabilistic networks model uncertainty by a directed acyclic graph quantified by probabilities. Two closely related problems on these networks are the KTH MPE and KTH PARTIAL MAP problems, which both take a network and a positive integer k for their input. In the KTH MPE problem, given a partition of the networks nodes into evidence and explanation nodes and given specific values for the evidence nodes, we ask for the kth most probable combination of values for the explanation nodes. In the KTH PARTIAL MAP problem in addition a number of unobservable intermediate nodes are distinguished; we again ask for the kth most probable explanation. In this paper, we establish the complexity of these problems and show that they are FPPP - and FPPPPP-complete, respectively.

When can predictive brains be truly Bayesian

At present, the hierarchical predic-tive coding framework does not yet make stringent commitments as to the nature of the causal models that the brain can rep-resent. Hence, contrary to suggestions by Clark (in press) , the framework does not yet have the virtue that it effectively implements tractable Bayesian inference. At this point in time three mutually exclusive options remain open: either predictive coding does not implement Bayesian inference, or pre-dictive coding is not tractable, or the theory of hierarchical predictive coding is enriched by specific assumptions about the structure of the brain’s causal models.Assuming that one is committed to the Bayesian Brain Hypothesis, the first two options are out and the third is the only one remaining. Formal analyses expanding on this option are beyond the scope of this commentary (see e.g., Blokpoel et al., 2010; van Rooij et al., 2011), but

Structure approximation of most probable explanations in bayesian networks

Typically, when one discusses approximation algorithms for (NP-hard) problems (like Traveling Salesperson, Vertex Cover, Knapsack), one refers to algorithms that return a solution whose value is (at least ideally) close to optimal; e.g., a tour with almost minimal length, a vertex cover of size just above minimal, or a collection of objects that has close to maximal value. In contrast, one might also be interested in approximation algorithms that return solutions that resemble the optimal solutions, i.e., whose structure is akin to the optimal solution, like a tour that is almost similar to the optimal tour, a vertex cover that differs in only a few vertices from the optimal cover, or a collection that is similar to the optimal collection. In this paper, we discuss structure-approximation of the problem of finding the most probable explanation of observations in Bayesian networks, i.e., finding a joint value assignment that looks like the most probable one, rather than has an almost as high value. We show that it is NP-hard to obtain the value of just a single variable of the most probable explanation. However, when partial orders on the values of the variables are available, we can improve on these results.

Complexity results for enhanced qualitative probabilistic networks

While quantitative probabilistic networks (QPNs) allow experts to state influences between nodes in the network as influence signs, rather than conditional probabilities, inference in these networks often leads to ambiguous results due to unresolved trade-offs in the network. Various enhancements have been proposed that incorporate a notion of strength of the influence, such as enhanced and rich enhanced operators. Although inference in standard (i.e. not enhanced) QPNs can be done in time polynomial to the length of the input, the computational complexity of inference in these enhanced networks has not been determined yet. In this paper, we introduce relaxation schemes to relate these enhancements to the more general case, where continuous influence intervals are used. We show that inference in networks with continuous influence intervals is NP-hard, and remains NP-hard when the intervals are discretised and the interval [-1,1] is divided into blocks with length of 14. We discuss membership of NP and show how these general complexity results may be used to determine the complexity of specific enhancements to QPNs. Furthermore, this might give more insight in the particular properties of feasible and infeasible approaches to enhance QPNs.

Ignorance is bliss: A complexity perspective on adapting reactive architectures

We study the computational complexity of adapting a reactive architecture to meet task constraints. This computational problem has application in a wide variety of fields, including cognitive and evolutionary robotics and cognitive neuroscience. We show that—even for a rather simple world and a simple task—adapting a reactive architecture to perform a given task in the given world is NP-hard. This result implies that adapting reactive architectures is computationally intractable regardless the nature of the adaptation process (e.g., engineering, development, evolution, learning, etc.) unless very special conditions apply. In order to find such special conditions for tractability, we have performed parameterized complexity analyses. One of our main findings is that architectures with limited sensory and perceptual abilities are efficiently adaptable.