Computable Randomness is Inherently Imprecise
aa r X i v : . [ m a t h . P R ] M a y C OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Computable Randomness is Inherently Imprecise
Gert de Cooman
GERT . DECOOMAN @ UGENT . BE Jasper De Bock
JASPER . DEBOCK @ UGENT . BE Ghent University, IDLabTechnologiepark – Zwijnaarde 914, 9052 Zwijnaarde, Belgium
Abstract
We use the martingale-theoretic approach of game-theoretic probability to incorporate imprecisioninto the study of randomness. In particular, we define a notion of computable randomness asso-ciated with interval, rather than precise, forecasting systems, and study its properties. The richermathematical structure that thus arises lets us better understand and place existing results for theprecise limit. When we focus on constant interval forecasts, we find that every infinite sequence ofzeroes and ones has an associated filter of intervals with respect to which it is computably random.It may happen that none of these intervals is precise, which justifies the title of this paper. Weillustrate this by showing that computable randomness associated with non-stationary precise fore-casting systems can be captured by a stationary interval forecast, which must then be less precise:a gain in model simplicity is thus paid for by a loss in precision.
Keywords: computable randomness; imprecise probabilities; game-theoretic probability; intervalforecast; supermartingale; computability.
1. Introduction
This paper documents the first steps in our attempt to incorporate indecision and imprecision intothe study of randomness. Consider a infinite sequence ω = ( z , . . . , z n , . . . ) of zeroes and ones; whendo we call it random ? There are many notions of randomness, and many of them have a numberof equivalent definitions (Ambos-Spies and Kucera, 2000; Bienvenu et al., 2009). We focus here on computable randomness , mainly because its focus on computability—rather than, say, the weakerlower semicomputability—has allowed us in this first attempt to keep the mathematical nitpickingat arm’s length. Randomness of a sequence ω is typically associated with a probability measureon the sample space of all infinite sequences, or—what is equivalent—with a forecasting system γ that associates with each finite sequence of outcomes ( x , . . . , x n ) the (conditional) expectation γ ( x , . . . , x n ) for the next (as yet unknown) outcome X n + . The sequence ω is then called comput-ably random when it passes a (countable) number of computable tests of randomness, where thecollection of randomness tests depends of the forecasting system γ . An alternative but equivalentdefinition, going back to Ville (1939), sees each forecast γ ( x , . . . , x n ) as a fair price for—and there-fore a commitment to bet on—the as yet unknown next outcome X n + . The sequence ω is thencomputably random when there is no computable strategy for getting infinitely rich by exploitingthe bets made available by the forecasting system γ along the sequence, without borrowing. Tech-nically speaking, all computable non-negative supermartingales should remain bounded on ω , andthe forecasting system γ determines what a supermartingale is.It is this last, martingale-theoretic approach which seems to lend itself most easily to allowingfor imprecision in the forecasts, and therefore in the definition of randomness. As we explain in Sec-tions 2 and 3, an ‘imprecise’ forecasting system γ associates with each finite sequence of outcomes E C OOMAN AND D E B OCK ( x , . . . , x n ) a (conditional) expectation interval γ ( x , . . . , x n ) for the next (as yet unknown) outcome X n + , whose lower bound represents a supremum acceptable buying price, and whose upper bounda infimum acceptable selling price for X n + . This idea rests firmly on the common ground betweenWalley’s (1991) theory of coherent lower previsions and Shafer and Vovk’s (2001) game-theoreticapproach to probability that we have established in recent years, through our research on impre-cise stochastic processes (De Cooman and Hermans, 2008; De Cooman et al., 2016). This allowsus to associate supermartingales with an imprecise forecasting system, and therefore in Section 5to extend the existing notion of computable randomness to allow for interval, rather than precise,forecasts—we discuss computability in Section 4. We show in Section 6 that our approach allowsus to extend some of Dawid’s (1982) well-known work on calibration, as well as an interesting‘limiting frequencies’ or computable stochasticity result.We believe the discussion becomes really interesting in Section 7, where we look at stationaryinterval forecasts to extend the classical account of randomness. That classical account typicallyconsiders a forecasting system with stationary expectation forecast / —corresponding to flippinga fair coin. As we have by now come to expect from our experience with imprecise probabilitymodels, a much more interesting mathematical picture appears when allowing for interval forecaststhan the rather simple case of precise forecasts would lead us to suspect. In the precise case, a givensequence may not be (computably) random for any stationary forecast, but in the imprecise casethere is always a set filter of intervals that a given sequence is computably random for. Furthermore,as we show in Section 8, this filter may not have a smallest element, and even when it does, thissmallest element may be a non-vanishing interval: randomness may be inherently imprecise.
2. A single interval forecast
The dynamics of making a single forecast can be made very clear by considering a simple game,with three players, namely Forecaster, Sceptic and Reality.
Game: single forecast of an outcome X In a first step, Forecaster specifies an interval bound I = [ p , p ] for the expectation of an as yetunknown outcome X in { , } —or equivalently, for the probability that X =
1. We interpret this interval forecast I as a commitment, on the part of Forecaster, to adopt p as a supremum buyingprice and p as a infimum selling price for the gamble (with reward function) X . This is taken to meanthat the second player, Sceptic , can now in a second step take Forecaster up on any (combination)of the following commitments:(i) for any p ∈ [ , ] such that p ≤ p , and any α ≥ α [ X − p ] ,leading to an uncertain reward − α [ X − p ] for Sceptic; (ii) for any q ∈ [ , ] such that q ≥ p , and any β ≥ β [ q − X ] ,leading to an uncertain reward − β [ q − X ] for Sceptic.Finally, in a third step, the third player, Reality , determines the value x of X in { , } . (cid:3) Elements x of { , } are called outcomes , and elements p of the real unit interval [ , ] arecalled (precise) forecasts . We denote by C the set of non-empty closed subintervals of the real unitinterval [ , ] . Any element I of C is called an interval forecast . It has a smallest element min I
1. Because we allow p ≤ p rather than p < p , we actually see p as a maximum buying price, rather than a supremumone. We do this because it does not affect the conclusions, but simplifies the mathematics. Similarly for q ≥ p . OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE E p ( f ) = E p ( f ) = f ( ) f ( ) f ( ) ≤ f ( ) f ( ) ≥ f ( ) (a) f ( ) f ( ) f ( ) ≤ f ( ) f ( ) ≥ f ( ) E r ( f ) = (b)Figure 1: Gambles f available to Sceptic when (a) Forecaster announces I ∈ C with p < p ; andwhen (b) Forecaster announces I ∈ C with p = p = : r .and a greatest element max I , so I = [ min I , max I ] . We will use the generic notation I for such aninterval, and p : = min I and p : = max I for its lower and upper bounds, respectively.After Forecaster announces a forecast interval I , what Sceptic can do is essentially to try andincrease his capital by taking a gamble on the outcome X . Any such gamble can be considered as amap f : { , } → R , and can therefore be represented as a vector ( f ( ) , f ( )) in the two-dimensionalvector space R ; see also Figure 1. f ( X ) is then the increase in Sceptic’s capital after the game hasbeen played, as a function of the outcome variable X . Of course, not every gamble f ( X ) on theoutcome X will be available to Sceptic: which gambles he can take is determined by Forecaster’sinterval forecast I . In their most general form, they are given by f ( X ) = − α [ X − p ] − β [ q − X ] ,where α and β are non-negative real numbers, p ≤ p and q ≥ p . If we consider the so-called lowerexpectation (functional) E I associated with an interval forecast I , defined by E I ( f ) = min p ∈ I E p ( f ) = min p ∈ I (cid:2) p f ( ) + ( − p ) f ( ) (cid:3) = ( E p ( f ) if f ( ) ≥ f ( ) E p ( f ) if f ( ) ≤ f ( ) (1)for any gamble f : { , } → R , and similarly, the upper expectation (functional) E I , defined by E I ( f ) = max p ∈ I E p ( f ) = ( E p ( f ) if f ( ) ≥ f ( ) E p ( f ) if f ( ) ≤ f ( ) = − E I ( − f ) , (2)then it is not difficult to see that the cone of gambles f ( X ) that are available to Sceptic after Fore-caster announces an interval forecast I is completely determined by the condition E I ( f ) ≤
0, asdepicted by the blue regions in Figure 1. The functionals E I and E I are easily shown to have thefollowing properties, typical for the more general lower and upper expectation operators defined onmore general gamble spaces (Walley, 1991; Troffaes and De Cooman, 2014): Proposition 1
Consider any forecast interval I ∈ C . Then for all gambles f , g on { , } , µ ∈ R and non-negative λ ∈ R : E C OOMAN AND D E B OCK
C1. min f ≤ E I ( f ) ≤ E I ( f ) ≤ max f ; [bounds]C2. E I ( λ f ) = λ E I ( f ) and E I ( λ f ) = λ E I ( f ) ; [non-negative homogeneity]C3. E I ( f + g ) ≥ E I ( f ) + E I ( g ) and E I ( f + g ) ≤ E I ( f ) + E I ( g ) ; [super/subadditivity]C4. E I ( f + µ ) = E I ( f ) + µ and E I ( f + µ ) = E I ( f ) + µ . [constant additivity]
3. Interval forecasting systems and imprecise probability trees
We now consider a sequence of repeated versions of the forecast game in the previous section, whereat each stage k ∈ N , Forecaster presents an interval forecast I k = [ p k , p k ] for the unknown outcomevariable X k . This effectively allows Sceptic to choose any gamble f k ( X k ) such that E I k ( f ) ≤ x k for X k , resulting in a gain, or increase in capital, f k ( x k ) for Sceptic.We call ( x , x , . . . , x n , . . . ) an outcome sequence , and collect all possible outcome sequences inthe set Ω : = { , } N . We collect the finite outcome sequences ( x , . . . , x n ) in the set Ω ♦ : = { , } ∗ = S n ∈ N { , } n . Finite sequences s in Ω ♦ and infinite sequences ω in Ω are the nodes—called situ-ations —and paths in an event tree with unbounded horizon, part of which is depicted below.000000 001 01010 011 110100 101 11110 111In this repeated game, Forecaster will only provide interval forecasts I k after observing the actualsequence ( x , . . . , x k − ) that Reality has chosen. This is the essence of so-called prequential fore-casting (Dawid, 1982, 1984; Dawid and Vovk, 1999). But for technical reasons, it will be useful toconsider the more involved setting where a forecast I s is specified in each of the possible situations s ∈ Ω ♦ ; see the figure below.000000 001 01010 011 110100 101 11110 111 I (cid:3) I I I I I I Indeed, we can use this idea to generalise the notion of a forecasting system (Vovk and Shen, 2010).
Definition 2 (Forecasting system) A forecasting system is a map γ : Ω ♦ → C , that associates withany situation s in the event tree a forecast γ ( s ) ∈ C . With any forecasting system γ we can associatetwo real-valued maps γ and γ on Ω ♦ , defined by γ ( s ) : = min γ ( s ) and γ ( s ) : = max γ ( s ) for all s ∈ Ω ♦ .A forecasting system γ is called precise if γ = γ . Γ denotes the set C Ω ♦ of all forecasting systems. OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Specifying such a forecasting system requires imagining in advance all moves that Reality couldmake, and devising in advance what forecasts to give in each imaginable situation s . In the precisecase, that is typically what one does when specifying a probability measure on the so-called samplespace Ω —the set Ω of all paths.Since in each situation s the interval forecast I s = γ ( s ) corresponds to a local lower expect-ation E I s , we can use the argumentation in our earlier papers (De Cooman and Hermans, 2008;De Cooman et al., 2016) on stochastic processes to let the forecasting system γ turn the event treeinto a so-called imprecise probability tree , with an associated global lower expectation, and a cor-responding notion of ‘(strictly) almost surely’. In what follows, we briefly recall how to do this; formore context, we also refer to the seminal work by Shafer and Vovk (2001).For any path ω ∈ Ω , the initial sequence that consists of its first n elements is a situation in { , } n that is denoted by ω n . Its n -th element belongs to { , } and is denoted by ω n . As aconvention, we let its 0-th element be the initial situation ω = ω = (cid:3) . We write that s ⊑ t , and saythat the situation s precedes the situation t , when every path that goes through t also goes through s —so s is a precursor of t .A process F is a map defined on Ω ♦ . A real process is a real-valued process: it associates a realnumber F ( s ) ∈ R with every situation s ∈ Ω ♦ . With any real process F , we can always associate aprocess ∆ F , called the process difference . For every situation ( x , . . . , x n ) with n ∈ N , ∆ F ( x , . . . , x n ) is a gamble on { , } defined by ∆ F ( x , . . . , x n )( x n + ) : = F ( x , . . . , x n + ) − F ( x , . . . , x n ) for all x n + ∈ { , } . In the imprecise probability tree associated with a given forecasting system γ , a submartingale M for γ is a real process such that E γ ( x ,..., x n ) ( ∆ M ( x , . . . , x n )) ≥ n ∈ N and ( x , . . . , x n ) ∈ { , } n . A real process M is a supermartingale for γ if − M is a submartingale,meaning that E γ ( x ,..., x n ) ( ∆ M ( x , . . . , x n )) ≤ n ∈ N and ( x , . . . , x n ) ∈ { , } n : all super-martingale differences have non-positive upper expectation, so supermartingales are real processesthat Forecaster expects to decrease. We denote the set of all submartingales for a given forecastingsystem γ by M γ —whether a real process is a submartingale depends of course on the forecasts inthe situations. Similarly, the set M γ : = − M γ is the set of all supermartingales for γ .It is clear from the discussion in Section 2 that the supermartingales are effectively all thepossible capital processes K for a Sceptic who starts with an initial capital K ( (cid:3) ) , and in eachpossible subsequent situation s selects a gamble f s = ∆ K ( s ) that is available there because Fore-caster specifies the interval forecast I s = γ ( s ) and because E I s ( f s ) = E γ ( s ) ( ∆ K ( s )) ≤
0. If Realitychooses outcomes s = ( x , . . . , x n ) , then Sceptic ends up with capital K ( x , . . . , x n ) = K ( (cid:3) ) + ∑ n − k = ∆ K ( x , . . . , x k )( x k + ) . A non-negative supermartingale M is non-negative in all situations,which corresponds to Sceptic never borrowing any money. We call test supermartingale any non-negative supermartingale M that starts with unit capital M ( (cid:3) ) =
1. We collect all test supermartin-gales for γ in the set T γ .In the context of probability trees, we call variable any function defined on the sample space Ω .When this variable is real-valued and bounded, we call it a gamble on Ω . An event A in this contextis a subset of Ω , and its indicator I A is a gamble on Ω assuming the value 1 on A and 0 elsewhere.The following expressions define lower and upper expectations on such gambles g on Ω : E γ ( g ) : = sup n M ( (cid:3) ) : M ∈ M γ and lim sup n → + ∞ M ( ω n ) ≤ g ( ω ) for all ω ∈ Ω o (3) E γ ( g ) : = inf n M ( (cid:3) ) : M ∈ M γ and lim inf n → + ∞ M ( ω n ) ≥ g ( ω ) for all ω ∈ Ω o = − E γ ( g ) . (4) E C OOMAN AND D E B OCK
They satisfy coherence properties similar to those in Proposition 1. We refer to extensive discus-sions elsewhere (De Cooman et al., 2016; Shafer and Vovk, 2001) about why these expressions areinteresting and useful. For our present purposes, it may suffice to mention that for precise fore-casts, they lead to models that coincide with the ones found in measure-theoretic probability theory(Shafer and Vovk, 2001, Chapter 8).
In particular, when all I s = { / } , they coincide with the usualuniform (Lebesgue) expectations on measurable gambles. We call an event A ⊆ Ω null if P γ ( A ) : = E γ ( I A ) =
0, or equivalently P γ ( A c ) : = E γ ( I A c ) = strictly null if there is some test supermartingale T ∈ T γ that converges to + ∞ on A , meaningthat lim n → + ∞ T ( ω n ) = + ∞ for all ω ∈ A . Any strictly null event is null, but null events need notbe strictly null (Vovk and Shafer, 2014; De Cooman et al., 2016). Because it is easily checked that P γ ( /0 ) = P γ ( /0 ) = A c of a (strictly) null event A is never empty. As usual, anyproperty that holds, except perhaps on a (strictly) null event, is said to hold (strictly) almost surely .
4. Basic computability notions
We recall a few notions and results from computability theory that are relevant to the discussion.For a much more extensive treatment, we refer for instance to the books by Pour-El and Richards(1989) and Li and Vitányi (1993).A computable function φ : N → N is a function that can be computed by a Turing machine.All notions of computability that we will need, build on this basic notion. It is clear that it in thisdefinition, we can replace any of the N with any other countable set.We start with the definition of a computable real number. We call a sequence of rational numbers r n computable if there are three computable functions a , b , σ from N to N such that b ( n ) > r n = ( − ) σ ( n ) a ( n ) b ( n ) for all n ∈ N , and we say that it converges effectively to a real number x if thereis some computable function e : N → N such that n ≥ e ( N ) ⇒ | r n − x | ≤ − N for all n , N ∈ N .A real number is then called computable if there is a computable sequence of rational numbers thatconverges effectively to it. Of course, every rational number is a computable real.We also need a notion of computable real processes, or in other words, computable real-valuedmaps F : Ω ♦ → R defined on the set Ω ♦ of all situations. Because there is an obvious comput-able bijection between N and Ω ♦ , whose inverse is also computable, we can in fact identify realprocesses and real sequences, and simply import, mutatis mutandis , the definitions for computablereal sequences common in the literature (Li and Vitányi, 1993, Chapter 0). Indeed, we call a net ofrational numbers r s , n computable if there are three computable functions a , b , s from Ω ♦ × N to N such that b ( s , n ) > r s , n = ( − ) σ ( s , n ) a ( s , n ) b ( s , n ) for all s ∈ Ω ♦ and n ∈ N . We call a real process F : Ω ♦ → R computable if there is a computable net of rational numbers r s , n and a computablefunction e : Ω ♦ × N → N such that n ≥ e ( s , N ) ⇒ | r s , n − F ( s ) | ≤ − N for all s ∈ Ω ♦ and n , N ∈ N .Obviously, it follows from this definition that in particular F ( t ) is a computable real number for any t ∈ Ω ♦ : fix s = t and consider the sequence r t , n that converges to F ( s ) as n → + ∞ . Also, a constantreal process is computable if and only if its constant value is.The following definitions are now obvious. A gamble f on { , } is called computable if bothits values f ( ) and f ( ) are computable real numbers. An interval forecast I = [ p , p ] ∈ C is called computable if both its lower bound p and upper bound p are computable real numbers. A forecastingsystem γ is called computable if the associated real processes γ and γ are. OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
5. Random sequences in an imprecise probability tree
We will now associate a notion of randomness with a forecasting system γ —or in other words, withan imprecise probability tree. In what follows, we will often consider computable test supermartin-gales. These computable test supermartingales for a forecasting system are countable in number,because the computable processes are (Li and Vitányi, 1993; Vovk and Shen, 2010). Definition 3 (Computable randomness)
Consider any forecasting system γ : Ω ♦ → C . We call anoutcome sequence ω computably random for γ if all computable test supermartingales T remainbounded above on ω , meaning that there is some B ∈ R such that T ( ω n ) ≤ B for all n ∈ N , orequivalently, that sup n ∈ N T ( ω n ) < + ∞ . We then also say that the forecasting system γ makes ω computably random . We denote by Γ C ( ω ) : = { γ ∈ Γ : ω is computably random for γ } the set of allforecasting systems for which the outcome sequence ω is computably random. Computable randomness of an outcome sequence means that there is no computable strategy thatstarts with capital 1 and avoids borrowing, and allows Sceptic to increase his capital without boundsby exploiting the bets on these outcomes that are made available to him by Forecaster’s specific-ation of the forecasting system γ . When the forecasting system γ is precise and computable,our notion of computable randomness reduces to the classical notion of computable randomness(Ambos-Spies and Kucera, 2000; Bienvenu et al., 2009).The (computable) vacuous forecasting system γ v assigns the vacuous forecast γ v ( s ) : = [ , ] toall situations s ∈ Ω ♦ . The following proposition implies that no Γ C ( ω ) is empty. Proposition 4
All paths are computably random for the vacuous forecasting system: γ v ∈ Γ C ( ω ) for all ω ∈ Ω . More conservative (or imprecise) forecasting systems have more computably random sequences.
Proposition 5
Let ω be computably random for a forecasting system γ . Then ω is also computablyrandom for any forecasting system γ ∗ such that γ ⊆ γ ∗ , meaning that γ ( s ) ⊆ γ ∗ ( s ) for all s ∈ Ω ♦ .
6. Consistency results
We first show that any Forecaster who specifies a forecasting system is consistent in the sense thathe believes himself to be well calibrated : in the imprecise probability tree generated by his ownforecasts, (strictly) almost all paths will be computably random, so he is sure that Sceptic will notbe able to become infinitely rich at his expense, by exploiting his—Forecaster’s—forecasts. Thisalso generalises the arguments and conclusions in a paper by Dawid (1982).
Theorem 6
Consider any forecasting system γ : Ω ♦ → C . Then (strictly) almost all outcome se-quences are computably random for γ in the imprecise probability tree that corresponds to γ . This result is quite powerful, and it guarantees in particular that:
Corollary 7
For any sequence of interval forecasts ( I , . . . , I n , . . . ) there is a forecasting systemgiven by γ ( x , . . . , x n ) : = I n + for all ( x , . . . , x n ) ∈ { , } n and all n ∈ N , and associated impre-cise probability tree such that (strictly) almost all—and therefore definitely at least one—outcomesequences are computably random for γ in the associated imprecise probability tree. E C OOMAN AND D E B OCK
The following weaker consistency result deals with limits (inferior and superior) of relative fre-quencies, taken with respect to a so-called selection process S : Ω ♦ → { , } . It is a counterpart inour more general context of the notions of computable stochasticity or Church randomness in theprecise case with I = { / } (Ambos-Spies and Kucera, 2000). Theorem 8 (Church randomness)
Let γ : Ω ♦ → C be any computable forecasting system, let ω =( x , . . . , x n , . . . ) ∈ Ω be any outcome sequence that is computably random for γ , and let f be any computable gamble on { , } . If S : Ω ♦ → { , } is any computable selection process such that ∑ nk = S ( x , . . . , x k ) → + ∞ , then also lim inf n → + ∞ ∑ n − k = S ( x , . . . , x k ) (cid:2) f ( x k + ) − E γ ( x ,..., x k ) ( f ) (cid:3) ∑ n − k = S ( x , . . . , x k ) ≥ .
7. Constant interval forecasts
We now introduce a significant simplification. For any interval I ∈ C , we let γ I be the corresponding stationary forecasting system that assigns the same interval forecast I to all nodes: γ I ( s ) : = I for all s ∈ Ω ♦ . In this way, with any outcome sequence ω , we can associate the collection of all intervalforecasts for which the corresponding stationary forecasting system makes ω computably random: C C ( ω ) : = { I ∈ C : γ I ∈ Γ C ( ω ) } = { I ∈ C : γ I makes ω computably random } . As an immediate consequence of Propositions 4 and 5, we find that this set of intervals is non-emptyand increasing.
Proposition 9 (Non-emptiness)
For all ω ∈ Ω , [ , ] ∈ C C ( ω ) , so any sequence of outcomes ω hasat least one stationary forecast that makes it computably random: C C ( ω ) = /0 . Proposition 10 (Increasingness)
Consider any ω ∈ Ω and any I , J ∈ C . If I ∈ C C ( ω ) and I ⊆ J,then also J ∈ C C ( ω ) . Theorem 8 implies the following property. However, quite remarkably, and seemingly in contrastwith Theorem 8, this result does not require any computability assumptions on the (stationary)forecasts.
Corollary 11 (Church randomness)
Consider any outcome sequence ω = ( x , . . . , x n , . . . ) in Ω and any stationary interval forecast I = [ p , p ] ∈ C C ( ω ) that makes ω computably random. Then forany computable selection process S : Ω ♦ → { , } such that ∑ nk = S ( x , . . . , x k ) → + ∞ :p ≤ lim inf n → + ∞ ∑ n − k = S ( x , . . . , x k ) x k + ∑ n − k = S ( x , . . . , x k ) ≤ lim sup n → + ∞ ∑ n − k = S ( x , . . . , x k ) x k + ∑ n − k = S ( x , . . . , x k ) ≤ p . The following proposition can of course be straightforwardly extended to any finite number ofinterval forecasts, and guarantees, together with Proposition 10, that C C ( ω ) is a set filter . Proposition 12
For any ω ∈ Ω and any two interval forecasts I and J: if I ∈ C C ( ω ) and J ∈ C C ( ω ) then I ∩ J = /0 , and I ∩ J ∈ C C ( ω ) . OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
This result also tells us that the collection C C ( ω ) of closed subsets of the compact set [ , ] has thefinite intersection property, and its intersection is therefore a non-empty closed interval: T C C ( ω ) =[ p C ( ω ) , p C ( ω )] . Propositions 10 and 12 guarantee that all intervals [ p C ( ω ) − ε , p C ( ω ) + ε ] in C with ε , ε > C C ( ω ) . But we will see in the next section that this does not generallyhold for ε = ε =
0. For this reason, we now define the following two subsets of [ , ] : L C ( ω ) : = { min I : I ∈ C C ( ω ) } and U C ( ω ) : = { max I : I ∈ C C ( ω ) } . Then Proposition 10 guarantees that L C ( ω ) is a decreasing set, and that U C ( ω ) is increasing. Theyare therefore both subintervals of [ , ] . Obviously, p C ( ω ) = sup L C ( ω ) and p C ( ω ) = inf U C ( ω ) . Onthe one hand clearly L C ( ω ) = [ , p C ( ω )) or L C ( ω ) = [ , p C ( ω )] , and on the other hand U C ( ω ) =( p C ( ω ) , ] or U C ( ω ) = [ p C ( ω ) , ] . Proposition 12 easily allows us to give the following simpledescription of the set C C ( ω ) in terms of L C ( ω ) and U C ( ω ) : I ∈ C C ( ω ) ⇔ (cid:16) min I ∈ L C ( ω ) and max I ∈ U C ( ω ) (cid:17) . A trivial example is given by:
Proposition 13
If the sequence ω is computable with infinitely many zeroes and ones, then C C ( ω ) = { [ , ] } , and therefore L C ( ω ) = { } , U C ( ω ) = { } , p C ( ω ) = and p C ( ω ) = . At the other extreme, there are the sequences ω that are computably random for some precise stationary forecasting system γ { p } , with p ∈ [ , ] . They are amongst the random sequences that havereceived most attention in the literature, thus far. For any such sequence, C C ( ω ) = { I ∈ C : p ∈ I } , L C ( ω ) = [ , p ] and U C ( ω ) = [ p , ] , and therefore also p C ( ω ) = p C ( ω ) = p .We show in the next section that, in between these extremes of total imprecision and maximalprecision, there lies a—to the best of our knowledge—previously uncharted realm of sequences,with similar (and even in some sense ‘larger’) unpredictability than the ones traditionally called‘computably random’, for which L C ( ω ) and U C ( ω ) need not always be closed, and more import-antly, for which 0 < p C ( ω ) < p C ( ω ) <
1. This is what we mean when we claim that ‘computablerandomness is inherently imprecise’.
8. Randomness is inherently imprecise
Our work on imprecise Markov chains (De Cooman et al., 2016) has taught us that in some cases, wecan very efficiently compute tight bounds on expectations in non-stationary precise Markov chains,by replacing them with their stationary imprecise versions. Similarly, in statistical modelling, whenlearning from data sampled from a distribution with a varying (non-stationary) parameter, it seemshard to estimate the exact time sequence of its values. But we may be more successful in learningabout its (stationary) interval range . This idea was also considered earlier by Fierens et al. (2009),when they argued for a frequentist interpretation of imprecise probability models based on non-stationarity.In this section, we exploit this idea, by showing that randomness associated with non-stationaryprecise forecasting systems can be captured by a stationary forecasting system, which must then beless precise: we gain simplicity of representation, but pay for it by losing precision.We begin with a simple example. Consider any p and q in [ , ] with p ≤ q , and any outcomesequence ω = ( x , . . . , x n , . . . ) that is computably random for the forecasting system γ p , q that is E C OOMAN AND D E B OCK defined by γ p , q ( z , . . . , z n ) : = ( p if n is odd q if n is even for all ( z , . . . , z n ) ∈ Ω ♦ .We know from Corollary 7 that there is at least one such outcome sequence. It turns out that thestationary forecasting systems that make such ω computably random have a simple characterisation: Proposition 14
Consider any ω that is computably random for the forecasting system γ p , q . Thenfor all I ∈ C , I ∈ C C ( ω ) ⇔ [ p , q ] ⊆ I. Its proof relies on a very simple argument involving Corollary 11. This result implies in particularalso that L C ( ω ) = [ , p ] , U C ( ω ) = [ q , ] , p C ( ω ) = p and p C ( ω ) = q .Next, we turn to a more complicated example, where we look at sequences that are ‘nearly’computably random for the stationary precise forecast / , but not quite. This example was inspiredby the ideas involving Hellinger-like divergences in a beautiful paper by Vovk (2009).Consider the following sequence { p n } n ∈ N of precise forecasts: p n : = + ( − ) n δ n , with δ n : = e − n + q e n + − n ∈ N , converging to / . Observe that the sequence δ n is decreasing towards its limit 0 and that δ n ∈ ( , / ) and p n ∈ ( , ) , for all n ∈ N . Now consider any outcome sequence ω = ( x , . . . , x n , . . . ) that iscomputably random for the precise forecasting system γ ∼ / that is defined by γ ∼ / ( z , . . . , z n − ) : = p n for all n ∈ N and ( z , . . . , z n − ) ∈ Ω ♦ .We know from Corollary 7 that there is at least one such outcome sequence. It turns out that thestationary forecasting systems that make such ω computably random have a simple characterisation: Proposition 15
Consider any ω that is computably random for the forecasting system γ ∼ / . Thenfor all I ∈ C , I ∈ C C ( ω ) if and only if min I < / and max I > / . This result implies in particular that L C ( ω ) = [ , / ) , U C ( ω ) = ( / , ] and p C ( ω ) = p C ( ω ) = / .
9. Conclusion
Even with the limited number of examples we have been able to examine in this paper, it becomesapparent that incorporating imprecision in the study of randomness allows for much more math-ematical structure to arise, which we would argue lets us better understand and place the existingresults in the precise limit.In our argumentation that ‘randomness is inherently imprecise’, we are well aware that we arerestricting ourselves to stationary forecasts. Our examples in Section 8 all involve sequences thatare computably random for a precise non-stationary forecasting system, but no longer computablyrandom for any stationary precise variant. To make our claim irrefutable, we would have to showthat there are sequences that are computably random for forecasting systems more precise than thevacuous one, but not for any (computable) precise forecasting system. Or in other words, that thereis ‘randomness’ or ‘unpredictability’ that cannot be ‘explained’ by any non-stationary (computable) OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE precise forecasting system. We will keep this challenge foremost in our minds. Nevertheless, theexamples in Section 8 do indicate that it is in some ways possible to replace an ‘explanation’ by acomplex non-stationary precise forecasting model by a(n infinite filter of) more imprecise stationaryone(s).This work may seem promising, but we are well aware that it is only a humble beginning. Wesee many extensions in many directions. First of all, we want to find out if our approach can alsobe used to find interval versions of
Martin-Löf and
Schnorr randomness (Ambos-Spies and Kucera,2000; Bienvenu et al., 2009) with similarly interesting properties and conclusions. Secondly, ourpreliminary exploration suggests that it will be possible to formulate equivalent randomness defin-itions in terms of randomness tests , rather than supermartingales, but this needs to be checked inmuch more detail. Thirdly, the approach we follow here is not prequential: we assume that ourForecaster specifies an entire forecasting system γ , or in other words an interval forecast in all pos-sible situations ( x , . . . , x n ) , rather than only interval forecasts in those situations z , . . . , z n of thesequence ω = ( z , . . . , z n , . . . ) whose potential randomness we are considering. The prequential ap-proach , which we eventually will want to come to, looks at the randomness of a sequence of intervalforecasts and outcomes ( I , z , I , z , . . . , I n , z n , . . . ) , where each I k is an interval forecast for the asyet unknown X k , which is afterwards revealed to be z k , without the need of specifying forecasts insituations that are never reached; see the paper by Vovk and Shen (2010) for an account of how thisworks for precise forecasts. Fourthly, we need to connect our work with earlier approaches to as-sociating imprecision with randomness (Walley and Fine, 1982; Fierens et al., 2009; Fierens, 2009;Gorban, 2016). And finally, and perhaps most importantly, we believe this research could be a veryearly starting point for an approach to statistics that takes imprecise or set-valued parameters moreseriously, when learning from finite amounts of data. Acknowledgments
This research started with discussions between Gert and Philip Dawid about what prequential in-terval forecasting would look like, during a joint stay at Durham University in late 2014. Gert, andJasper who joined in late 2015, wrote an early prequential version of the present paper during ajoint research visit to the Universities of Strathclyde and Durham in May 2016, trying to extend theresults by Volodya Vovk (Vovk, 1987, 2009; Vovk and Shen, 2010) to make them allow for intervalforecasts. In an email exchange, Volodya pointed out a number of difficulties with our approach,which we were able to resolve by letting go of its prequential emphasis, at least for the time being.This was done during research visits of Gert to Jasper at IDSIA in late 2016 and early 2017.We are grateful to Philip and Volodya for their inspiring and helpful comments and guidance.Gert’s research and travel were partly funded through project number G012512N of the ResearchFoundation – Flanders (FWO), Jasper is a Postdoctoral Fellow of the FWO and wishes to acknow-ledge its financial support.
References
K. Ambos-Spies and A. Kucera. Randomness in computability theory.
Contemporary Mathematics ,257:1–14, 2000. E C OOMAN AND D E B OCK
L. Bienvenu, G. Shafer, and A. Shen. On the history of martingales in the study of randomness.
Electronic Journal for History of Probability and Statistics , 5, 2009.A. P. Dawid. The well-calibrated Bayesian.
Journal of The American Statistical Association , 77(379):605–610, 1982.A. P. Dawid. Statistical theory: The prequential approach.
Journal of the Royal Statistical Society,Series A , 147:278–292, 1984.A. P. Dawid and V. G. Vovk. Prequential probability: principles and properties.
Bernoulli , 5:125–162, 1999.G. de Cooman and F. Hermans. Imprecise probability trees: Bridging two theories of impreciseprobability.
Artificial Intelligence , 172(11):1400–1427, 2008. doi: 10.1016/j.artint.2008.03.001.G. de Cooman, J. De Bock, and S. Lopatatzidis. Imprecise stochastic processes in discrete time:global models, imprecise Markov chains, and ergodic theorems.
International Journal of Approx-imate Reasoning , 76:18–46, 2016.P. I. Fierens. An extension of chaotic probability models to real-valued variables.
InternationalJournal of Approximate Reasoning , 50(4):627–641, 2009.P. I. Fierens, L. C. Rego, and T. L. Fine. A frequentist understanding of sets of measures.
Journalof Statistical Planning and Inference , 139(6):1879–1892, 2009.I. I. Gorban.
The Statistical Stability Phenomenon . Springer, 2016.M. Li and P. M. B. Vitányi.
An Introduction to Kolmogorov Complexity and Its Applications .Springer, New York, 1993.M. B. Pour-El and J. I. Richards.
Computability in Analysis and Physics . Springer Verlag, 1989.G. Shafer and V. Vovk.
Probability and Finance: It’s Only a Game!
Wiley, New York, 2001.M. C. M. Troffaes and G. de Cooman.
Lower Previsions . Wiley, 2014.J. Ville.
Étude critique de la notion de collectif . Gauthier-Villars, Paris, 1939.V. Vovk and G. Shafer. Game-theoretic probability. In T. Augustin, F. P. A. Coolen, G. de Cooman,and M. C. M. Troffaes, editors,
Introduction to Imprecise Probabilities . John Wiley & Sons, 2014.V. G. Vovk. On a criterion of randomness.
Doklady Akademii Nauk SSSR , 294(6):1298–1302, 1987.V. G. Vovk. Merging of opinions in game-theoretic probability.
Annals of the Institute of StatisticalMathematics , 61(4):969–993, 2009.V. G. Vovk and A. Shen. Prequential randomness and probability.
Theoretical Computer Science ,411(29-30):2632–2646, 2010.P. Walley.
Statistical Reasoning with Imprecise Probabilities . Chapman and Hall, London, 1991.P. Walley and T. L. Fine. Towards a frequentist theory of upper and lower probability.
Annals ofStatistics , 10:741–761, 1982. OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Appendix A. Proofs and additional material
In this Appendix, we have gathered all proofs, and all additional material necessary for understand-ing the argumentation in these proofs.
A.1 Additional material for Section 3
In the interest of understanding the proofs, we need to pay attention to a particular way of con-structing test supermartingales. We define a multiplier process as a map D from Ω ♦ to non-negative gambles on { , } Given such a multiplier process D , we can construct a non-negative real pro-cess D ⊚ by the recursion equation D ⊚ ( sx ) : = D ⊚ ( s ) D ( s )( x ) for all s ∈ Ω ♦ and x ∈ { , } , with D ⊚ ( (cid:3) ) : =
1. Any multiplier process D that satisfies the additional condition that E γ ( s ) ( D ( s )) ≤ s ∈ Ω ♦ , is called a supermartingale multiplier for the forecasting system γ . It is easy to seethat the non-negative real process D ⊚ is then a test supermartingale for γ : it suffices to check that ∆ D ⊚ ( s ) = D ⊚ ( s )[ D ( s ) − ] , (5)and therefore E γ ( s ) ( ∆ D ⊚ ( s )) = D ⊚ ( s )[ E γ ( s ) ( D ( s )) − ] ≤
0, due to the coherence properties C2and C4 of upper expectation operators.
A.2 Additional material for Section 4
We give a brief survey of those basic notions and results from computability theory that are relevantto the proofs in this appendix. For a much more extensive discussion, we refer, for instance to thebooks by Pour-El and Richards (1989) and Li and Vitányi (1993). The discussion in Section 4 is attimes similar—and even identical—but is overall more limited in scope, as it only deals with aspectsthat are relevant to the main text.A computable function φ : N → N is a function that can be computed by a Turing machine.All further notions of computability that we will need are based on this basic notion. It is clear thatit in this definition, we can replace any of the N with any other countable set.We start with the definition of a computable real number. We call a sequence of rational numbers r n computable if there are three computable functions a , b , σ from N to N such that b ( n ) > r n = ( − ) σ ( n ) a ( n ) b ( n ) for all n ∈ N , and we say that it converges effectively to a real number x if there is some computable function e : N → N such that n ≥ e ( N ) ⇒ | r n − x | ≤ − N for all n , N ∈ N . A real number is then called computable if there is a computable sequence of rational numbers thatconverges effectively to it. Of course, every rational number is a computable real.We also need a notion of computable real processes, or in other words, computable real-valuedmaps F : Ω ♦ → R defined on the set Ω ♦ of all situations. Because there is an obvious comput-able bijection between N and Ω ♦ , whose inverse is also computable, we can in fact identify realprocesses and real sequences, and simply import, mutatis mutandis , the definitions for computablereal sequences common in the literature (Li and Vitányi, 1993, Chapter 0). Indeed, we call a net of E C OOMAN AND D E B OCK rational numbers r s , n computable if there are three computable functions a , b , s from Ω ♦ × N to N such that b ( s , n ) > r s , n = ( − ) σ ( s , n ) a ( s , n ) b ( s , n ) for all s ∈ Ω ♦ and n ∈ N . We call a real process F : Ω ♦ → R computable if there is a computable net of rational numbers r s , n and a computable function e : Ω ♦ × N → N such that n ≥ e ( s , N ) ⇒ | r s , n − F ( s ) | ≤ − N for all s ∈ Ω ♦ and n , N ∈ N . Again, there is no problem with the notions ‘computable net of rational numbers’ or ‘computablefunction’ in this definition, because we can identify Ω ♦ × N with N through a computable bijectionwhose inverse is also computable. Obviously, it follows from this definition that in particular F ( t ) is a computable real number for any t ∈ Ω ♦ : fix s = t and consider the sequence r t , n that convergesto F ( s ) as n → + ∞ . Also, a constant real process is computable if and only if its constant value is.We recall the following standard results (Li and Vitányi, 1993, Chapter 0). Proposition 16
A real process F is computable if and only if there is a computable net of rationalnumbers r s , n such that | r s , n − F ( s ) | ≤ − n for all s ∈ Ω ♦ and n ∈ N . Proof of Proposition 16
We give the proof for the sake of completeness. The ‘if’ part is immedi-ate, so we proceed to the ‘only if’ part. That F is computable means that there is some computablenet of rational numbers r ′ s , n and a computable function e : Ω ♦ × N → N such that n ≥ e ( s , N ) im-plies | r ′ s , n − F ( s ) | ≤ − N for all s ∈ Ω ♦ and N ∈ N . The net of rational numbers r s , n : = r ′ s , e ( s , n ) iscomputable because the function e is computable, and satisfies | r s , n − F ( s ) | = | r ′ s , e ( s , n ) − F ( s ) | ≤ − n . Proposition 17
Consider any computable net x s , n of real numbers and any real process F for whichthere is some computable function e : Ω ♦ × N → N such thatn ≥ e ( s , N ) ⇒ | x s , n − F ( s ) | ≤ − N for all s ∈ Ω ♦ and N ∈ N . Then F is computable.
Also, if F and G are computable real processes, then so are F + G , FG , F / G (provided that G ( s ) = s ∈ Ω ♦ ), max { F , G } , min { F , G } , exp ( F ) , ln F (provided that F ( s ) > s ∈ Ω ♦ ), and F m (provided that F ( s ) ≥ s ∈ Ω ♦ ) for all m ∈ N (Li and Vitányi, 1993, Chapter 0).We also require the notion of a semicomputable real processes. A real process F is lowersemicomputable if it can be approximated from below by a computable net of rational numbers,meaning that there is a computable net of rational numbers r s , n such that(i) r s , n + ≥ r s , n for all s ∈ Ω ♦ and n ∈ N ;(ii) F ( s ) = lim n → + ∞ r s , n for all s ∈ Ω ♦ . F is upper semicomputable if − F is lower semicomputable. The following result is standard, butits proof is illustrative. Proposition 18
A process F is computable if and only is it is both lower and upper semicomputable. OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Proof of Proposition 18
We begin with the ‘if’ part. Assume that F is both lower and uppersemicomputable. This implies that there are two computable nets of rational numbers r s , n and r s , n such that r t , n ↑ F ( t ) and r t , n ↓ F ( t ) for any fixed t ∈ Ω ♦ . Consider the computable nets of rationalnumbers defined by δ s , n : = r s , n − r s , n ≥ r s , n : = ( r s , n + r s , n ) /
2. For any fixed t ∈ Ω ♦ , thesequence δ t , n ↓
0, which implies that for any N ∈ N there is a natural number e ( t , N ) such that δ t , n ≤ − N for all n ≥ e ( t , N ) . It is obvious that the function e : Ω ♦ × N → N is computable, andthat n ≥ e ( s , N ) also implies | F ( s ) − r s , n | ≤ | r s , n − r s , n | = δ s , n ≤ − N , for all s ∈ Ω ♦ and n , N ∈ N .Hence, F is also computable.We continue with the ‘only if’ part. Assume that F is computable, so there is a computable netof rational numbers r s , n and a computable function e : Ω ♦ × N → N such that n ≥ e ( s , N ) implies | r s , n − F ( s ) | ≤ − N for all s ∈ Ω ♦ and n , N ∈ N . We prove that F is lower semicomputable; theproof that F is upper semicomputable is completely similar. Consider the computable net of rationalnumbers r ′ s , n : = r s , e ( s , n + ) − − ( n + ) , then clearly r ′ s , n ≤ F ( s ) and | r ′ s , n − F ( s ) | < − n for all s ∈ Ω ♦ and n ∈ N . This implies that we can always assume without loss of generality from the outsetthat r s , n ≤ F ( s ) and e ( s , N ) = N [a similar idea was used in the proof of Proposition 16]. We nowconstruct a new computable net of rational numbers r s , n from our original net: for any fixed s ∈ Ω ♦ ,start with r s , : = r s , and e ( s , ) =
0; let e ( s , ) be the first k > e ( s , ) such that r s , k ≥ r s , , and let r s , : = r s , e ( s , ) ; let e ( s , ) be the first k > e ( s , ) such that r s , k ≥ r s , , and let r s , : = r s , e ( s , ) ; and soon. The function e : Ω ♦ × N → N is clearly computable, and therefore the net of rational numbers r s , n is computable. Moreover, for any fixed t ∈ Ω ♦ the subsequence r t , n of the sequence r t , n is non-decreasing by construction, and it converges to F ( t ) because the original sequence r t , n does.The following definitions and results are obvious and immediate. A gamble f on { , } is called computable if and only if both its values f ( ) and f ( ) are computable real numbers. An intervalforecast I = [ p , p ] ∈ C is called computable if and only if both its lower bound p and upper bound p are computable real numbers. A forecasting system γ is called computable if the associated realprocesses γ and γ are. Finally, a process difference ∆ F is called (lower/upper semi) computable ifthe real processes ∆ ( s )( ) and ∆ ( s )( ) , s ∈ Ω ♦ are; and similarly for a multiplier process D . Proposition 19
For any computable gamble f on { , } and any computable forecasting system γ ,the real processes E γ ( s ) ( f ) and E γ ( s ) ( f ) , s ∈ Ω ♦ are computable. Proof of Proposition 19
Observe that both the real process min γ ( s ) f ( ) + [ − min γ ( s )] f ( ) , s ∈ Ω ♦ and the real process max γ ( s ) f ( ) + [ − max γ ( s )] f ( ) , s ∈ Ω ♦ are computable. So are,therefore, their maximum process E γ ( s ) ( f ) , s ∈ Ω ♦ and their minimum process E γ ( s ) ( f ) , s ∈ Ω ♦ . Proposition 20
For any I ∈ C , the stationary forecasting system γ I is computable if and only if theinterval I is computable. Proof of Proposition 20
Let p : = min I and p : = max I . Obviously, the constant real processes γ I ( s ) : = p and γ I ( s ) : = p are computable if and only if their constant values p and p are. Proposition 21
For any computable gamble f on { , } and any computable interval forecast I =[ p , p ] ∈ C , the lower and upper expectations E I ( f ) and E I ( f ) are computable real numbers. Proof of Proposition 21
This is an immediate consequence of Propositions 19 and 20. Or,alternatively, observe that both real numbers p f ( ) + ( − p ) f ( ) and p f ( ) + ( − p ) f ( ) are E C OOMAN AND D E B OCK computable. So are, therefore, their maximum E I ( f ) and minimum E I ( f ) .We will also need to use the following basic results. Proposition 22
Consider any real process F, and its process difference ∆ F. Then the followingstatements hold: (i) if F ( (cid:3) ) and ∆ F are lower semicomputable then so is F ; (ii) if F ( (cid:3) ) and ∆ F are upper semicomputable then so is F ; (iii)
F is computable if and only if F ( (cid:3) ) and ∆ F are.
Proof of Proposition 22
We only prove the third statement. The proof for the first and secondstatements are similar to the proof of the ‘if’ part of the third, but simpler.There are a number of ways to prove the third statement, but we will use Proposition 16.For the ‘if’ part, we assume that F ( (cid:3) ) and ∆ F are computable. This implies that there are acomputable sequence of rational numbers r (cid:3) , n and two computable nets of rational numbers r xs , n such that | F ( (cid:3) ) − r (cid:3) , n | ≤ − n and | ∆ F ( s )( x ) − r xs , n | ≤ − n for all s ∈ Ω ♦ , n ∈ N and x ∈ { , } . Wenow define the computable net of rational numbers r s , n as follows: for any s = ( x , . . . , x m ) ∈ Ω ♦ ,where m ∈ N , and any n ∈ N , let r s , n : = r (cid:3) , n + m ∑ k = r x k ( x ,..., x k − ) , n . Then, since also F ( s ) = F ( (cid:3) ) + m ∑ k = ∆ F ( x , . . . , x k − )( x k ) , we see that | F ( s ) − r s , n | ≤ | F ( (cid:3) ) − r (cid:3) , n | + m ∑ k = (cid:12)(cid:12) ∆ F ( x , . . . , x k − )( x k ) − r x k ( x ,..., x k − ) , n (cid:12)(cid:12) ≤ ( m + ) − n , so if we define the (clearly) computable function e by e ( s , N ) : = N + m ≥ N + log ( m + ) , then n ≥ e ( s , N ) implies that | F ( s ) − r s , n | ≤ − N for all s ∈ Ω ♦ and n ∈ N . Hence, F is computable.For the ‘only if’ part, assume that F is computable. Then definitely in particular also its value F ( (cid:3) ) in the initial situation (cid:3) , so it only remains to prove that the process difference ∆ F is com-putable. Consider, to this effect, any x ∈ { , } . It follows from the computability of F that there isa computable net of rational numbers r ′ s , n such that | F ( s ) − r ′ s , n | ≤ − n and | F ( sx ) − r ′ sx , n | ≤ − n andtherefore also r ′ sx , n − r ′ s , n − − ( n − ) ≤ F ( sx ) − F ( s ) ≤ r ′ sx , n − r ′ s , n + − ( n − ) for all s ∈ Ω ♦ and n ∈ N .If we now let r xs , n : = r ′ sx , n + − r ′ s , n + , then this defines a computable net of rational numbers r xs , n thatsatisfies | ∆ F ( s )( x ) − r xs , n | ≤ − n for all s ∈ Ω ♦ and all n ∈ N . This means that ∆ F is computable. Proposition 23
Consider any computable real processes G, G ′ and H. Consider the real processF with computable F ( (cid:3) ) , such that ∆ F ( s ) = I { } G ( s ) + I { } G ′ ( s ) + H ( s ) or ∆ F ( s ) = G ( s ) ∆ H ( s ) .Then F is computable as well. OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Proof of Proposition 23
This is an immediate consequence of Proposition 22, since the condi-tions imply that ∆ F is computable. Proposition 24
Consider any multiplier process D, and the associated real process D ⊚ . Then thefollowing implications hold: (i) if D is lower semicomputable, then so are ∆ D ⊚ and D ⊚ ; (ii) if D is upper semicomputable, then so are ∆ D ⊚ and D ⊚ ; (iii) if D is computable, then so are ∆ D ⊚ and D ⊚ . Proof of Proposition 24
We only give the proof for the first statement. The proof for the secondstatement is completely similar, and the third statement then follows readily from the first and thesecond.Assume that D is lower semicomputable. This implies that there are two computable nets ofrational numbers r xs , n such that r xs , n ↑ D ( s )( x ) , for x ∈ { , } . Since D ( s )( x ) ≥
0, we may assumewithout loss of generality that r xs , n ≥ r s , n as follows: for any s = ( x , . . . , x m ) ∈ Ω ♦ , where m ∈ N , and any n ∈ N , let r s , n : = m − ∏ k = r x k + ( x ,..., x k ) , n ≥ . Then, since also D ⊚ ( s ) = m − ∏ k = D ( x , . . . , x k )( x k + ) , we see that r s , n ↑ D ⊚ ( s ) for all s ∈ Ω ♦ , so D ⊚ is indeed lower semicomputable. Next, we constructtwo computable nets of rational numbers t xs , n as follows: for any s ∈ Ω ♦ and x ∈ { , } , let t xs , n : = r s , n [ r xs , n − ] . Then, since also, by Equation (5), ∆ D ⊚ ( s )( x ) = D ⊚ ( s )[ D ( s )( x ) − ] and r s , n ≥
0, we see that t xs , n ↑ ∆ D ⊚ ( s )( x ) for all s ∈ Ω ♦ and x ∈ { , } , so ∆ D ⊚ is indeed lowersemicomputable. Proposition 25
Consider a multiplier process D and the associated real process D ⊚ . If D ⊚ ispositive and computable, then so is D. Proof of Proposition 25
Since D ⊚ is positive, it follows trivially that D is positive as well.Consider now any x ∈ { , } . Since D ⊚ is computable, it follows from Proposition 22 that ∆ D ⊚ is computable, and therefore, we know that ∆ D ⊚ ( s )( x ) is computable as well. Hence, since D ⊚ iscomputable and positive, we find that D ( s ) = D ⊚ ( s ) + ∆ D ⊚ ( s )( x ) D ⊚ ( s ) is computable. E C OOMAN AND D E B OCK
A.3 Proofs and additional material for Section 5
We denote by T γ C the countable set of all computable test supermartingales for the forecasting sys-tem γ . Proof of Proposition 4
In the imprecise probability tree associated with the vacuous forecastingsystem γ v , a real process is a supermartingale if and only if it is non-increasing. All test super-martingales are therefore bounded above by 1 on any path ω ∈ Ω . Proof of Proposition 5
Since γ ⊆ γ ∗ implies that T γ ∗ C ⊆ T γ C , this follows trivially from Defini-tion 3. Proposition 26
Consider any forecasting system γ . Then for any outcome sequence ω , the follow-ing statements are equivalent: (i) sup n ∈ N T ( ω n ) = + ∞ for some computable non-negative supermartingale T ; (ii) sup n ∈ N T ( ω n ) = + ∞ for some computable test supermartingale T ; (iii) sup n ∈ N T ( ω n ) = + ∞ for some test supermartingale T , with ∆ T computable; (iv) sup n ∈ N T ( ω n ) = + ∞ for some test supermartingale T = D ⊚ , with D computable. Proof of Proposition 26
Proposition 24 implies that (iv) ⇒ (iii) and, since T ( (cid:3) ) = ⇒ (ii). Hence, since (ii) ⇒ (i) holds trivially, itsuffices to prove that (i) ⇒ (iv).So consider any computable non-negative supermartingale T such that sup n ∈ N T ( ω n ) = + ∞ .We will prove that there is a computable supermartingale multiplier D such that sup n ∈ N D ⊚ ( ω n ) =+ ∞ . Let T ′ : = / α ( + T ) , with α : = + T ( (cid:3) ) . Then α is clearly computable, and therefore, T ′ is computable as well. Also, since T is non-negative, we find that T ′ ≥ / α > T ′ ( (cid:3) ) =
1. Furthermore, since T is a supermartingale, T ′ is clearly a supermartingale as well. Finally,since sup n ∈ N T ( ω n ) = + ∞ , we have that sup n ∈ N T ′ ( ω n ) = + ∞ as well. Hence, we find that T ′ is acomputable test supermartingale such that T ′ > n ∈ N T ′ ( ω n ) = + ∞ . Hence, without lossof generality, we may assume that T is a positive test supermartingale. Since T is a positive testsupermartingale, there is a unique supermartingale multiplier D such that T = D ⊚ . Since T ispositive and computable, the computability of D follows from Proposition 25. Proposition 27
Consider a computable precise forecasting system γ . Then for any outcome se-quence ω , the following statements are equivalent: (i) sup n ∈ N T ( ω n ) = + ∞ for some computable non-negative martingale T ; (ii) sup n ∈ N T ( ω n ) = + ∞ for some computable non-negative supermartingale T . Proof of Proposition 27
Since (i) ⇒ (ii) holds trivially, it suffices to prove that (ii) ⇒ (i). Soconsider any computable non-negative supermartingale T ′ such that sup n ∈ N T ′ ( ω n ) = + ∞ . We willprove that there is a computable non-negative martingale T such that sup n ∈ N T ( ω n ) = + ∞ .Since T ′ is a computable non-negative supermartingale such that sup n ∈ N T ′ ( ω n ) = + ∞ , it fol-lows from Proposition 26 that there is a test supermartingale T ′′ , with ∆ T ′′ computable, such thatsup n ∈ N T ′′ ( ω n ) = + ∞ . Now let T be the unique real process such that T ( (cid:3) ) = s ∈ Ω ♦ : ∆ T ( s )( x ) : = ∆ T ′′ ( s )( x ) − E γ ( s ) (cid:0) ∆ T ′′ ( s ) (cid:1) = ∆ T ′′ ( s )( x ) − γ ( s ) ∆ T ′′ ( s )( ) − (cid:0) − γ ( s ) (cid:1) ∆ T ′′ ( s )( ) for all x ∈ { , } . OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Since ∆ T ′′ and γ are computable, ∆ T is clearly computable as well. Therefore, and because T ( (cid:3) ) is rational and therefore also computable, it follows from Proposition 22 that T is comput-able. Furthermore, for any situation s ∈ Ω ♦ , we have that ∆ T ( s ) ≥ ∆ T ′′ ( s ) because by assumption E γ ( s ) ( ∆ T ′′ ( s )) ≤
0, and E γ ( s ) (cid:0) ∆ T ( s ) (cid:1) = E γ ( s ) (cid:0) ∆ T ′′ ( s ) − E γ ( s ) (cid:0) ∆ T ′′ ( s ) (cid:1)(cid:1) = E γ ( s ) (cid:0) ∆ T ′′ ( s ) (cid:1) − E γ ( s ) (cid:0) ∆ T ′′ ( s ) (cid:1) = . Hence, it follows that T is a martingale and that ∆ T ≥ ∆ T ′′ . Since T ( (cid:3) ) = T ′′ ( (cid:3) ) =
1, the latterimplies that T ≥ T ′′ . Since T ′′ is non-negative and sup n ∈ N T ′′ ( ω n ) = + ∞ , this in turn implies that T is non-negative and that sup n ∈ N T ( ω n ) = + ∞ . Since we already know that T is a computablemartingale, this establishes the desired result. A.4 Proofs and additional material for Section 6Proof of Theorem 6
Consider the event A : = { ω ∈ Ω : ω is computably random for γ } . We haveto show that there is a test supermartingale that converges to + ∞ on A c .For every ω in A c , there is some computable test supermartingale T ω that becomes unboundedon ω . Since the T ω , ω ∈ A c are countable in number, we can consider some countable convexcombination T of all of them, with non-zero coefficients. This is again a test supermartingale, thatbecomes unbounded on all ω in A c .We now construct yet another test supermartingale T ′ that converges to + ∞ on all ω where T be-comes unbounded. The argument has by now become standard (Shafer and Vovk, 2001, Lemma 3.1).For any n ∈ N , the real process T ( n ) defined by T ( n ) ( s ) : = ( n if T ( t ) ≥ n for some precursor t ⊑ s of sT ( s ) otherwise for all s ∈ Ω ♦ , is again a test supermartingale. So is therefore the countable convex combination T ′ : = ∑ n ∈ N − n T ( n ) .It is clear that T ′ converges to + ∞ on all paths ω where T becomes unbounded. Proof of Corollary 7
Consider the forecasting system γ defined by γ ( x , . . . , x n ) : = I n + for all ( x , . . . , x n ) ∈ { , } n and all n ∈ N . Then it follows from Theorem 6 that in the imprecise probability tree associated with γ , (strictly)almost all ω are computably random for this γ .In order to state our next set of results, we require some additional notions. Consider a realprocess F : Ω ♦ → R and a selection process S : Ω ♦ → { , } , and use them to define the real process J F K S : Ω ♦ → R as follows: J F K S ( x , . . . , x n ) : = ∑ n − k = S ( x , . . . , x k ) = ∑ n − k = S ( x , . . . , x k )[ F ( x , . . . , x k , x k + ) − F ( x , . . . , x k )] ∑ n − k = S ( x , . . . , x k ) if ∑ n − k = S ( x , . . . , x k ) > , for all n ∈ N and ( x , . . . , x n ) ∈ Ω ♦ . E C OOMAN AND D E B OCK
In particular, fix any gamble f on { , } , and let, for all n ∈ N and ( x , . . . , x n ) ∈ Ω ♦ : M γ f ( x , . . . , x n ) : = n ∑ k = (cid:2) f ( x k ) − E γ ( x ,..., x k − ) ( f ) (cid:3) then on the one hand ∆ M γ f ( x , . . . , x n )( x n + ) : = M γ f ( x , . . . , x n + ) − M γ f ( x , . . . , x n ) = f ( x n + ) − E γ ( x ,..., x n ) ( f ) , (6)so ∆ M γ f ( x , . . . , x n ) = f − E γ ( x ,..., x n ) ( f ) , and therefore, on the other hand E γ ( x ,..., x n ) ( ∆ M γ f ( x , . . . , x n )) = E γ ( x ,..., x n ) ( f ) − E γ ( x ,..., x n ) ( f ) = . (7)We conclude that the real process M γ f is a submartingale, whose differences ∆ M γ f ( x , . . . , x n ) are uni-formly bounded, for instance by k f k v : = max f − min f . Observe by the way that in this particularcase: J M γ f K S ( x , . . . , x n ) : = ∑ n − k = S ( x , . . . , x k ) = ∑ n − k = S ( x , . . . , x k ) (cid:2) f ( x k + ) − E γ ( x ,..., x k ) ( f ) (cid:3) ∑ n − k = S ( x , . . . , x k ) if ∑ n − k = S ( x , . . . , x k ) > . (8) Proposition 28
If the gamble f on { , } and the forecasting system γ are computable, then so isthe submartingale M γ f . Proof of Proposition 28
We use Proposition 23: M γ f ( (cid:3) ) = G be the computable constant real process f ( ) , G ′ the computable constant real process f ( ) , and H the real process defined by H ( s ) = − E γ ( s ) ( f ) for all s ∈ Ω ♦ , computable by Proposi-tion 21.We can now apply our law of large numbers for submartingale differences (De Cooman et al., 2016,Theorem 7) to get to the following result, which generalises Philip Dawid’s well-known consistencyresult for Bayesian Forecasters (Dawid, 1982, Theorem 1), to deal with imprecise assessments: Theorem 29 (The well-calibrated imprecise Bayesian)
Let γ : Ω ♦ → C be any forecasting sys-tem, let S : Ω ♦ → { , } be any selection process, and let f be any gamble on { , } . Then n ∑ k = S ( X , . . . , X k ) → + ∞ ⇒ lim inf n → + ∞ J M γ f K S ( X , . . . , X n ) ≥ (strictly) almost surely, in the imprecise probability tree associated with the forecasting system γ . We repeat the proof here, borrowed from one of our earlier papers (De Cooman et al., 2016, The-orem 7) and suitably adapted to include results on computability, because it will next help us provea related result—Theorem 8—that will turn out to be crucial in establishing the main claim of thispaper. One important step in this proof is, stripped to its bare essentials, based on a surprisinglyelegant and effective idea that goes back to Shafer and Vovk (2001, Lemma 3.3). OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Proof of Theorem 29
Consider the events D : = { ω ∈ Ω : lim n → + ∞ ∑ n − k = S ( ω k ) = + ∞ } and theevent A : = { ω ∈ Ω : lim inf n → + ∞ J M γ f K S ( ω n ) < } . We have to show that there is some test super-martingale T that converges to + ∞ on the set D ∩ A . Let B : = max { , k f k v } >
0, then we infer fromEquation (7) that B is a uniform real bound on ∆ M γ f , meaning that | ∆ M γ f ( s ) | ≤ B for all situations s ∈ Ω ♦ .For any r ∈ N , let A r : = (cid:8) ω ∈ Ω : lim inf n → + ∞ J M γ f K S ( ω n ) < − r (cid:9) , then A = S r ∈ N A r . So fix any r ∈ N and consider any ω ∈ D ∩ A r , then we have in particular thatlim inf n → + ∞ J M γ f K S ( ω n ) < − r , and therefore ( ∀ m ∈ N )( ∃ n m ≥ m ) J M γ f K S ( ω n m ) < − r = − ε , with 0 < ε : = r < B . Consider now the test supermartingale F M of Lemma 30, with in particular M : = M γ f and ξ = ξ r : = ε B = r + B = r + B B < B . We denote it by F ( r ) . It follows from Lemma 30that F ( r ) ( ω n m ) ≥ exp (cid:18) ε B n m − ∑ k = S ( ω k ) (cid:19) = exp (cid:18) r + B n m − ∑ k = S ( ω k ) (cid:19) for all m ∈ N . (9)Consider any real R > m ∈ N . Since also ω ∈ D , we know that lim n → + ∞ ∑ n − k = S ( ω k ) = + ∞ , sothere is some natural number m ′ ≥ m such that exp (cid:0) r + B ∑ m ′ − k = S ( ω k ) (cid:1) > R . Hence it follows fromthe statement in (9) that there is some n m ′ ≥ m ′ ≥ m —whence ∑ n m ′ − k = S ( ω k ) ≥ ∑ m ′ − k = S ( ω k ) —suchthat F ( r ) ( ω n m ′ ) ≥ exp (cid:18) r + B ∑ n m ′ − k = S ( ω k ) (cid:19) ≥ exp (cid:18) r + B ∑ m ′ − k = S ( ω k ) (cid:19) > R , which implies that lim sup n → + ∞ F ( r ) ( ω n ) = + ∞ . Observe that for this test supermartingale, F ( r ) ( x , . . . , x n ) ≤ ( ) n for all n ∈ N and x , . . . , x n ∈ { , } n . Now define the process T γ f : = ∑ r ∈ N w ( r ) F ( r ) as any countable convex combination of the F ( r ) constructed above, with positive weights w ( r ) > T γ f ( x , . . . , x n ) is non-negative, and moreover T γ f ( x , . . . , x n ) ≤ ∑ r ∈ N w ( r ) F ( r ) ≤ ∑ r ∈ N w ( r ) (cid:16) (cid:17) n = (cid:16) (cid:17) n for all n ∈ N and ( x , . . . , x n ) ∈ { , } n . (10)This process is also positive, has T γ f ( (cid:3) ) =
1, and, for any ω ∈ D ∩ A , it follows from the argument-ation above that there is some r ∈ N such that ω ∈ D ∩ A r and thereforelim sup n → + ∞ T γ f ( ω n ) ≥ w ( r ) lim sup n → + ∞ F ( r ) ( ω n ) = + ∞ , (11)so lim sup n → + ∞ T γ f ( ω n ) = + ∞ . E C OOMAN AND D E B OCK
We now prove that T γ f is a supermartingale, and therefore also a test supermartingale. Considerany n ∈ N and any ( x , . . . , x n ) ∈ { , } n , then we have to prove that E γ ( x ,..., x n ) ( − ∆ T γ f ( x , . . . , x n )) ≥
0. Since it follows from the argumentation in the proof of Lemma 30 that − ∆ F ( r ) ( x , . . . , x n ) = r + B F ( r ) ( x , . . . , x n ) ∆ M γ f ( x , . . . , x n ) for all r ∈ N , we see that − ∆ T γ f ( x , . . . , x n ) = − ∑ r ∈ N w ( r ) ∆ F ( r ) = ∆ M γ f ( x , . . . , x n ) ∑ r ∈ N w ( r ) r + B F ( r ) ( x , . . . , x n ) | {z } = : C ( x ,..., x n ) , (12)where C ( x , . . . , x n ) ≥ C ( x , . . . , x n ) = ∑ r ∈ N w ( r ) r + B F ( r ) ( x , . . . , x n ) ≤ L ∑ r ∈ N w ( r ) F ( r ) ( x , . . . , x n ) ≤ L (cid:16) (cid:17) n for some real L >
0. Therefore indeed, using the non-negative homogeneity of lower expectations: E γ ( x ,..., x n ) ( − ∆ T γ f ( x , . . . , x n )) = E γ ( x ,..., x n ) ( C ( x , . . . , x n ) ∆ M γ f ( x , . . . , x n ))= C ( x , . . . , x n ) E γ ( x ,..., x n ) ( ∆ M γ f ( x , . . . , x n )) ≥ , because M γ f is a submartingale; see Equation (7).Since we now know that T γ f is a supermartingale that is furthermore bounded below (by 0) itfollows from our supermartingale convergence theorem (De Cooman et al., 2016) that there is sometest supermartingale that converges to + ∞ on all paths where T γ f does not converge to a real number,and therefore in particular on all paths in D ∩ A . Hence D ∩ A is indeed strictly null. Lemma 30
Consider any real B > and < ξ < B . Let M be any submartingale such that | ∆ M | ≤ B. Let S be any real process that only assumes values in { , } . Then the process F M defined by:F M ( x , . . . , x n ) : = n − ∏ k = (cid:2) − ξ S ( x , . . . , x k ) ∆ M ( x , . . . , x k )( x k + ) (cid:3) for all n ∈ N and ( x , . . . , x n ) ∈ { , } n (13) is a positive supermartingale with F M ( (cid:3) ) = , and therefore in particular a test supermartingale.Moreover, for ξ : = ε B , with < ε < B, we have that J M γ f K S ( x , . . . , x n ) ≤ − ε ⇒ F M ( x , . . . , x n ) ≥ exp (cid:18) ε B n − ∑ k = S ( x , . . . , x k ) (cid:19) for all n ∈ N and ( x , . . . , x n ) ∈ { , } n . Finally, if ξ and M are computable, and S is computable, then F M is computable as well. OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Proof of Lemma 30 F M ( (cid:3) ) = F M is positive, consider any n ∈ N and any ( x , . . . , x n ) ∈ { , } n . Since it follows from 0 < ξ B < | ∆ M | ≤ B and S ∈ { , } that1 − ξ S ( x , . . . , x k ) ∆ M ( x , . . . , x k )( x k + ) ≥ − ξ B > ≤ k ≤ n −
1, we see that indeed: F M ( x , . . . , x n ) = n − ∏ k = (cid:2) − ξ S ( x , . . . , x k ) ∆ M ( x , . . . , x k )( x k + ) (cid:3) > . This also tells us that if we let D M ( s ) : = − ξ S ( s ) ∆ M ( s ) > , (14)then D M is a multiplier process, and F M = D ⊚ M . Moreover, since ξ > S ( s ) ∈ { , } , we inferfrom the coherence and conjugacy properties of lower and upper expectations that E γ ( s ) ( D M ) = E γ ( s ) (cid:0) − ξ ( s ) S ( s ) ∆ M ( s ) (cid:1) = + E γ ( s ) (cid:0) − ξ ( s ) S ( s ) ∆ M ( s ) (cid:1) = − ξ ( s ) S ( s ) E γ ( s ) (cid:0) ∆ M ( s ) (cid:1) ≤ , where the inequality follows from E γ ( s ) ( ∆ M ( s )) ≥
0, because we assumed that M is a submartin-gale. This shows that D M is a supermartingale multiplier, and therefore F M = D ⊚ M is indeed a testsupermartingale.For the second statement, consider any 0 < ε < B and let ξ : = ε B . Then for any n ∈ N and x , . . . , x n ∈ { , } n such that J M γ f K S ( x , . . . , x n ) ≤ − ε , we have for all real K : F M ( x , . . . , x n ) ≥ exp ( K ) ⇔ n − ∏ k = (cid:2) − ξ S ( x , . . . , x k ) ∆ M ( x , . . . , x k )( x k + ) (cid:3) ≥ exp ( K ) ⇔ n − ∑ k = ln (cid:2) − ξ S ( x , . . . , x k ) ∆ M ( x , . . . , x k )( x k + ) (cid:3) ≥ K . (15)Since | ∆ M | ≤ B , S ∈ { , } and 0 < ε < B , we find that − ξ S ( x , . . . , x k ) ∆ M ( x , . . . , x k ) ≥ − ξ B = − ε B > −
12 for 0 ≤ k ≤ n − . As ln ( + x ) ≥ x − x for x > − , this allows us to infer that n − ∑ k = ln (cid:2) − ξ S ( x , . . . , x k ) ∆ M ( x , . . . , x k )( x k + ) (cid:3) ≥ n − ∑ k = (cid:2) − ξ S ( x , . . . , x k ) ∆ M ( x , . . . , x k )( x k + ) − ξ S ( x , . . . , x k ) ( ∆ M ( x , . . . , x k )( x k + )) (cid:3) = − ξ n − ∑ k = S ( x , . . . , x k ) J M γ f K S ( x , . . . , x n ) − ξ n − ∑ k = S ( x , . . . , x k )( ∆ M ( x , . . . , x k )( x k + )) ≥ ξ n − ∑ k = S ( x , . . . , x k ) ε − ξ n − ∑ k = S ( x , . . . , x k ) B = ξ ( ε − ξ B ) n − ∑ k = S ( x , . . . , x k ) = ε B n − ∑ k = S ( x , . . . , x k ) , E C OOMAN AND D E B OCK where the first equality holds because S = S . Now choose K : = ε B ∑ n − k = S ( x , . . . , x k ) in Equa-tion (15).We now prove the last statement, dealing with the computability of F M .Since M is assumed to be computable, so is ∆ M , by Proposition 22. Since also ξ and S are as-sumed to be computable, we infer from Equation (14) that the multiplier process D M is computabletoo. If we now invoke Proposition 24, we find that F M = D ⊚ M is therefore computable as well. Proof of Theorem 8
Assume ex absurdo that the inequality is not satisfied. Then we infer fromthe proof of Theorem 29 that there is some positive test supermartingale T γ f = ∑ r ∈ N w ( r ) F ( r ) that isunbounded above on ω : see Equation (11) there. We will now go back to the details of that proofto show that we can make sure that T γ f = D ⊚ for some computable multiplier process D , therebycontradicting the assumed computable randomness for γ .First of all, recall that for any r ∈ N , the test supermartingale F ( r ) is the test supermartingale F M constructed in Lemma 30, for the particular choices M = M γ f , B = max { , k f k v } and ξ = ξ r = r + B .Since the gamble f is assumed to be computable, so is the real number k f k v = | f ( ) − f ( ) | , andtherefore also the real numbers B and ξ r . We infer from Proposition 28 that the submartingale M γ f iscomputable, because the forecasting system γ and the gamble f are. Since in addition S is assumedto be computable, we infer from Lemma 30 that the test supermartingale F ( r ) is computable.We now consider the version of the test supermartingale T γ f = ∑ r ∈ N w ( r ) F ( r ) corresponding to theparticular choices w ( r ) : = − r , and prove that T γ f is computable. To this effect, we use Proposition 17.Indeed, consider, for each s = ( x , . . . , x m ) ∈ Ω ♦ and n ∈ N , the real number x s , n : = n ∑ r = − r F ( r ) ( s ) , which is computable because all real processes F ( r ) are. Since 0 ≤ F ( r ) ( s ) ≤ (cid:0) (cid:1) m , we get | T γ f ( s ) − x s , n | = + ∞ ∑ r = n + − r F ( r ) ( s ) ≤ (cid:16) (cid:17) m + ∞ ∑ r = n + − r = (cid:16) (cid:17) m − n . If we now let e ( s , N ) : = N + m , then since N + m ≥ N + m log , we see that n ≥ e ( s , N ) implies | T γ f ( s ) − x s , n | ≤ − N for all s ∈ Ω ♦ and n , N ∈ N . Since the function e is clearly computable, we may use Proposition 17 to conclude that T γ f is indeed computable.Now let D be the unique supermartingale multiplier such that T γ f = D ⊚ [the uniqueness followsfrom the fact that T γ f is positive]. Since T γ f is computable, it then follows from Proposition 25 that D is computable as well. A.5 Proofs and additional material for Section 7Proof of Proposition 9
Immediate consequence of Proposition 4, with γ [ , ] = γ v ∈ Γ C ( ω ) . Proof of Proposition 10
This follows from Proposition 5, because I ⊆ J implies γ I ⊆ γ J . OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE
Proof of Corollary 11
First, assume that I is computable. It follows from Proposition 20 that γ I is computable as well. Furthermore, if we let I { } ( x ) : = x for all x ∈ { , } , then I { } and − I { } areclearly computable gambles on { , } . The first and last inequality now follow from Theorem 8,by choosing f = I { } and f = − I { } , respectively, since E I ( I { } ) = p and E I ( − I { } ) = − p . Thesecond inequality is a standard property of limits inferior and superior.If I is not computable, then for any ε >
0, since all rational numbers are computable, there issome computable J = [ q , q ] ∈ C such that p − ε ≤ q ≤ p ≤ p ≤ q ≤ p + ε . Since I ⊆ J , it followsfrom Proposition 10 that also J ∈ C C ( ω ) . Since, moreover, J is computable, it follows from the firstpart of the proof that p − ε ≤ q ≤ lim inf n → + ∞ ∑ n − k = S ( x , . . . , x k ) x k + ∑ n − k = S ( x , . . . , x k ) ≤ lim sup n → + ∞ ∑ n − k = S ( x , . . . , x k ) x k + ∑ n − k = S ( x , . . . , x k ) ≤ q ≤ p + ε . Since ε > Proof of Proposition 12
For the first statement, let I = [ p , p ] and J = [ q , q ] and assume ex ab-surdo that I ∩ J = /0. We may assume without loss of generality that p > q . It then follows fromCorollary 11, with S ( s ) : = s ∈ Ω ♦ , thatlim inf n → + ∞ n n ∑ k = x k ≤ lim sup n → + ∞ n n ∑ k = x k ≤ q < p ≤ lim inf n → + ∞ n n ∑ k = x k , a contradiction.For the second statement, let K : = I ∩ J . We will prove that K ∈ C C ( ω ) . Again, let I = [ p , p ] and J = [ q , q ] . Because of symmetry, we may assume without loss of generality that q ≤ p . Furthermore,due to the first statement, we know that then p ≤ q . If we have that I ⊆ J , then I = I ∩ J and therefore,since I ∈ C C ( ω ) , the result holds trivially. Hence, we may assume without loss of generality that q < p ≤ q < p , which implies that K = I ∩ J = [ p , q ] .Consider any computable test supermartingale T in T γ K C , then we must show that T remainsbounded on ω . We may assume without loss of generality that there is some computable super-martingale multiplier D for γ K such that T = D ⊚ .Now let D I be the map from situations to gambles on { , } , defined by D I ( s )( z ) : = ( min { D ( s )( ) , } if z = { D ( s )( ) , } if z = s ∈ Ω ♦ and z ∈ { , } .Then D I is a supermartingale multiplier for γ I : that it is non-negative follows from the non-negativityof D ; moreover, for any s ∈ Ω ♦ , we have that E I ( D I ( s )) ≤
1. Indeed, to prove the this, we con-sider two cases: D ( s )( ) ≤ D ( s )( ) >
1. If D ( s )( ) ≤
1, then D I ( s ) ≤ E I ( D I ( s )) ≤
1, by C1. The case that D ( s )( ) > D ( s )( ) > D I ( s )( ) < D I ( s )( ) , we find that E I ( D I ( s )) = E p ( D I ( s )) = E K ( D I ( s )) . Furthermore, since by assumption E K ( D ( s )) ≤ D ( s )( ) > D ( s )( ) ≤
1, again by C1.We therefore find that D ( s ) = D I ( s ) . By combining these two findings, it follows that indeed herealso E I ( D I ( s )) = E K ( D I ( s )) = E K ( D ( s )) ≤ . E C OOMAN AND D E B OCK
Since D I is indeed a supermartingale multiplier for γ I , we find that T I : = D ⊚ I is a test supermartingalefor γ I .Furthermore, since D is computable, so is D I , because taking minima and maxima preservescomputability. Hence, T I belongs to T γ I C . Therefore, and because I ∈ C C ( ω ) , it follows that T I ( ω n ) remains bounded as n → + ∞ .Also, if we let D J be a map from situations to gambles on { , } , defined by D J ( s )( z ) : = ( max { D ( s )( ) , } if z = { D ( s )( ) , } if z = s ∈ Ω ♦ and z ∈ { , } , and consider T J : = D ⊚ J , a similar course of reasoning leads us to conclude that T J belongs to T γ J C .Therefore, and because J ∈ C C ( ω ) , it follows that T J ( ω n ) remains bounded as n → + ∞ .Next, we observe that D = D I D J , and therefore also T = D ⊚ = D ⊚ I D ⊚ J = T I T J . And since both T I and T J remain bounded on ω , so, therefore, does T . Proof of Proposition 13
When ω is computable and has infinitely many zeroes and ones, there isa computable selection process S that selects all the ones, with ∑ n − k = S ( x , . . . , x k ) → + ∞ , and an-other computable selection process S = − S that selects all the zeroes, with ∑ n − k = S ( x , . . . , x k ) → + ∞ . For any I ∈ C C ( ω ) , we then infer from Corollary 11 thatmin I ≤ lim inf n → + ∞ ∑ n − k = S ( x , . . . , x k ) x k + ∑ n − k = S ( x , . . . , x k ) = , and similarly max I ≥ lim sup n → + ∞ ∑ n − k = S ( x , . . . , x k ) x k + ∑ n − k = S ( x , . . . , x k ) = , so indeed I = [ , ] . A.6 Proofs and additional material for Section 8Proof of Proposition 14
The converse implication follows at once from Proposition 5 and thefact that for any I ∈ C such that [ p , q ] ⊆ I , the stationary forecasting system γ I is more conservativethan γ p , q , in the sense that γ p , q ⊆ γ I .For the direct implication, assume that I ∈ C C ( ω ) and fix any ε >
0. Since all rational numbersare computable, there are computable intervals [ p , p ] ∈ C and [ q , q ] ∈ C such that p ∈ [ p , p ] ⊆ [ p − ε , p + ε ] and q ∈ [ q , q ] ⊆ [ q − ε , q + ε ] . Consider now the forecasting system γ ε , defined by γ ε ( z , . . . , z n ) : = ( [ p , p ] if n is odd [ q , q ] if n is even for all ( z , . . . , z n ) ∈ Ω ♦ .Then γ ε is clearly computable and, since γ p , q ⊆ γ ε , we know from Proposition 5 that ω is computablyrandom for γ ε . Therefore, we find thatmin I ≤ lim inf n → + ∞ ∑ nk = x k n ≤ lim sup n → + ∞ ∑ nk = x k n ≤ p ≤ p + ε OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE where the first and third inequality follow from Corollary 11 and Theorem 8, respectively, for ap-propriately chosen computable selection processes. Similarly, we also find thatmax I ≥ lim sup n → + ∞ ∑ nk = x k − n ≥ lim inf n → + ∞ ∑ nk = x k − n ≥ q ≥ q − ε . Since ε > I ≤ p and max I ≥ q , and, therefore, that [ p , q ] ⊆ I .That δ n ↓ n ∈ N , that p n ( − p n ) = (cid:16) − δ n (cid:17)(cid:16) + δ n (cid:17) = − δ n (16) = − e − n + (cid:0) e n + − (cid:1) = − e − n + + e − n + = (cid:16) e − n + − (cid:17) , (17)and therefore, since e − n + > , that e ( n + ) √ (cid:16) √ p n + p − p n (cid:17) = e ( n + ) r + p p n ( − p n ) = e ( n + ) q e − n + = . (18) Proof of Proposition 15
We first show that { / } / ∈ C C ( ω ) , so the sequence is not computablyrandom. Consider the multiplier processes D / and D ∼ / , defined for all n ∈ N and ( z , . . . , z n − ) ∈ Ω ♦ by D / ( z , . . . , z n − ) : = e ( n + ) p b p n and D ∼ / ( z , . . . , z n − ) : = e ( n + ) p b p n , where we define, for any p ∈ [ , ] , the corresponding mass function b p by letting b p ( ) : = p and b p ( ) : = − p . We then find thatln (cid:0) D ⊚ / ( x , . . . , x n ) D ⊚ ∼ / ( x , . . . , x n ) (cid:1) = n ∑ k = k + → + ∞ as n → + ∞ . (19)Furthermore, taking into account Equation (18), it is then easy to verify that for all n ∈ N and ( z , . . . , z n − ) ∈ Ω ♦ : E / (cid:0) D / ( z , . . . , z n − ) (cid:1) = e ( n + ) p p n + e ( n + ) p ( − p n ) = E p n (cid:0) D ∼ / ( z , . . . , z n − ) (cid:1) = p n e ( n + ) √ p n + ( − p n ) e ( n + ) p ( − p n ) = . Hence, we find that D / is a supermartingale multiplier for the stationary forecasting system γ { / } and that D ∼ / is a supermartingale multiplier for the forecasting system γ ∼ / . Both of these su-permartingale multipliers are furthermore clearly computable. Since D ∼ / is a computable super-martingale multiplier for the forecasting system γ ∼ / , it follows by assumption that D ⊚ ∼ / ( x , . . . , x n ) remains bounded as n → + ∞ . Therefore, and taking into account Equation (19), we find that D ⊚ / ( x , . . . , x n ) → + ∞ as n → + ∞ . Since D / is a computable supermartingale multiplier for thestationary forecasting system γ { / } , this implies that indeed { / } / ∈ C C ( ω ) . E C OOMAN AND D E B OCK
In a similar way, for all ε >
0, it can also be shown that [ / − ε , / ] / ∈ C C ( ω ) and [ / , / + ε ] / ∈ C C ( ω ) . We sketch the proof for [ / − ε , / ] / ∈ C C ( ω ) . The proof for [ / , / + ε ] / ∈ C C ( ω ) iscompletely analogous. Let the multiplier process D / , − be defined for all n ∈ N and ( z , . . . , z n − ) ∈ Ω ♦ by D / , − ( z , . . . , z n − ) : = ( D / ( z , . . . , z n − ) for n even1 for n odd , and similarly for D ∼ / , − . Thenln (cid:0) D ⊚ / , − ( x , . . . , x n ) D ⊚ ∼ / , − ( x , . . . , x n ) (cid:1) = ⌊ n / ⌋ ∑ k = k + → + ∞ as n → + ∞ . If we now fix any n ∈ N and ( z , . . . , z n − ) ∈ Ω ♦ , then E [ / − ε , / ] (cid:0) D / , − ( z , . . . , z n − ) (cid:1) = ( E / (cid:0) D / ( z , . . . , z n − ) (cid:1) = n even E / ( ) = n odd , because for even n , it follows from b p n ( ) = p n > − p n = b p n ( ) that also D / ( z , . . . , z n − )( ) > D / ( z , . . . , z n − )( ) . Similarly, E p n (cid:0) D ∼ / , − ( z , . . . , z n − ) (cid:1) = ( E p n (cid:0) D ∼ / ( z , . . . , z n − ) (cid:1) = n even E p n ( ) = n odd . This tells us that D / , − is a supermartingale multiplier for the stationary forecasting system γ [ / − ε , / ] ,and that D ∼ / , − is a supermartingale multiplier for the forecasting system γ ∼ / . Since D / , − and D ∼ / , − are clearly computable, the rest of the proof is now similar to that of { / } / ∈ C C ( ω ) .Finally, we show that for any ε , ε ∈ ( , / ] , I ε , ε : = [ / − ε , / + ε ] ∈ C C ( ω ) . Assume exabsurdo that I ε , ε / ∈ C C ( ω ) for some ε , ε ∈ ( , / ] , meaning that there is some computable super-martingale multiplier D ε , ε for the stationary forecasting system I ε , ε such that D ⊚ ε , ε is unboundedon ω = ( x , . . . , x n , . . . ) . Consider any rational number α such that 0 < α ≤ min { ε , ε } and let n α bethe smallest natural number n ∈ N such that δ n ≤ α and therefore p n ∈ I ε , ε , or, using Equations (16)and (17), let n α = (cid:24) − + √ − α − (cid:25) . Then since α is rational and therefore computable, n α is computable as well. We now consider anew multiplier process D , defined by D ( z , . . . , z n − ) : = ( n < n α D ε , ε ( z , . . . , z n − ) if n ≥ n α for all n ∈ N and ( z . . . , z n − ) ∈ Ω ♦ .Since D ε , ε is computable and n α is computable, D is clearly computable. Furthermore, since p n ∈ I ε , ε for n ≥ n α , we find for all n ∈ N and ( z . . . , z n − ) ∈ Ω ♦ that E p n (cid:0) D ( z , . . . , z n − ) (cid:1) ≤ ( E p n ( ) = n < n α E I ε , ε (cid:0) D ε , ε ( z , . . . , z n − ) (cid:1) ≤ n ≥ n α , OMPUTABLE R ANDOMNESS IS I NHERENTLY I MPRECISE which implies that D is a computable supermartingale multiplier for the forecasting system γ ∼ / .By construction, D ⊚ is unbounded on ω , simply because D ⊚ ε , ε is unbounded on ω . This contradictsthe fact that ω is computably random for γ ∼ / ..