Cameron E. Freer

Noncomputable Conditional Distributions

We study the computability of conditional probability, a fundamental notion in probability theory and Bayesian statistics. In the elementary discrete setting, a ratio of probabilities defines conditional probability. In more general settings, conditional probability is defined axiomatically, and the search for more constructive definitions is the subject of a rich literature in probability theory and statistics. However, we show that in general one cannot compute conditional probabilities. Specifically, we construct a pair of computable random variables (X, Y) in the unit interval whose conditional distribution P[Y|X] encodes the halting problem. Nevertheless, probabilistic inference has proven remarkably successful in practice, even in infinite-dimensional continuous settings. We prove several results giving general conditions under which conditional distributions are computable. In the discrete or dominated setting, under suitable computability hypotheses, conditional distributions are computable. Likewise, conditioning is a computable operation in the presence of certain additional structure, such as independent absolutely continuous noise.

Invariant measures concentrated on countable structures

Let L be a countable language. We say that a countable infinite L-structure M admits an invariant measure when there is a probability measure on the space of L-structures with the same underlying set as M that is invariant under permutations of that set, and that assigns measure one to the isomorphism class of M. We show that M admits an invariant measure if and only if it has trivial definable closure, i.e., the pointwise stabilizer in Aut(M) of an arbitrary finite tuple of M fixes no additional points. When M is a Fraisse limit in a relational language, this amounts to requiring that the age of M have strong amalgamation. Our results give rise to new instances of structures that admit invariant measures and structures that do not.

Computable de Finetti measures

Abstract We prove a computable version of the de Finetti theorem on exchangeable sequences of real random variables. As a consequence, exchangeable stochastic processes expressed in probabilistic functional programming languages can be automatically rewritten as procedures that do not modify non-local state. Along the way, we prove that a distribution on the unit interval is computable if and only if its moments are uniformly computable.

Algorithmic Aspects of Lipschitz Functions

We characterize the variation functions of computable Lipschitz functions. We show that a realz is computably random if and only if every computable Lipschitz function is differentiable at z. Beyond these principal results, we show that a real z is Schnorr random if and only if every Lipschitz function with L1-computable derivative is differentiable at z.

Computable Exchangeable Sequences Have Computable de Finetti Measures

We prove a uniformly computable version of de Finettis theorem on exchangeable sequences of real random variables. In the process, we develop machinery for computably recovering a distribution from its sequence of moments, which suffices to prove the theorem in the case of (almost surely) continuous directing random measures. In the general case, we give a proof inspired by a randomized algorithm which succeeds with probability one. Finally, we show how, as a consequence of the main theorem, exchangeable stochastic processes in probabilistic functional programming languages can be rewritten as procedures that do not use mutation.

On computability and disintegration

We show that the disintegration operator on a complete separable metric space along a projection map, restricted to measures for which there is a unique continuous disintegration, is strongly Weihrauch equivalent to the limit operator Lim. When a measure does not have a unique continuous disintegration, we may still obtain a disintegration when some basis of continuity sets has the Vitali covering property with respect to the measure; the disintegration, however, may depend on the choice of sets. We show that, when the basis is computable, the resulting disintegration is strongly Weihrauch reducible to Lim, and further exhibit a single distribution realizing this upper bound.

Invariant measures via inverse limits of finite structures

Building on recent results regarding symmetric probabilistic constructions of countable structures, we provide a method for constructing probability measures, concentrated on certain classes of countably infinite structures, that are invariant under all permutations of the underlying set that fix all constants. These measures are constructed from inverse limits of measures on certain finite structures. We use this construction to obtain invariant probability measures concentrated on the classes of countable models of certain first-order theories, including measures that do not assign positive measure to the isomorphism class of any single model. We also characterize those transitive Borel G-spaces admitting a G-invariant probability measure, when G is an arbitrary countable product of symmetric groups on a countable set.

Graph Turing Machines

We consider graph Turing machines, a model of parallel computation on a graph, which provides a natural generalization of several standard computational models, including ordinary Turing machines and cellular automata. In this extended abstract, we give bounds on the computational strength of functions that graph Turing machines can compute. We also begin the study of the relationship between the computational power of a graph Turing machine and structural properties of its underlying graph.

A classification of orbits admitting a unique invariant measure

Abstract We consider the space of countable structures with fixed underlying set in a given countable language. We show that the number of ergodic probability measures on this space that are S ∞ -invariant and concentrated on a single isomorphism class must be zero, or one, or continuum. Further, such an isomorphism class admits a unique S ∞ -invariant probability measure precisely when the structure is highly homogeneous; by a result of Peter J. Cameron, these are the structures that are interdefinable with one of the five reducts of the rational linear order ( Q , ) .

Priors on exchangeable directed graphs

Directed graphs occur throughout statistical modeling of networks, and exchangeability is a natural assumption when the ordering of vertices does not matter. There is a deep structural theory for exchangeable undirected graphs, which extends to the directed case via measurable objects known as digraphons. Using digraphons, we first show how to construct models for exchangeable directed graphs, including special cases such as tournaments, linear orderings, directed acyclic graphs, and partial orderings. We then show how to construct priors on digraphons via the infinite relational digraphon model (di-IRM), a new Bayesian nonparametric block model for exchangeable directed graphs, and demonstrate inference on synthetic data.