Teddy Seidenfeld

Entropy and Uncertainty

This essay is, primarily, a discussion of four results about the principle of maximizing entropy (MAXENT) and its connections with Bayesian theory. Result 1 provides a restricted equivalence between the two where the Bayesian model for MAXENT inference uses an a priori probability that is uniform, and where all MAXENT constraints are limited to 0–1 expectations for simple indicator-variables. The other three results report on an inability to extend the equivalence beyond these specialized constraints. Result 2 establishes a sensitivity of MAXENT inference to the choice of the algebra of possibilities even though all empirical constraints imposed on the MAXENT solution are satisfied in each measure space considered. The resulting MAXENT distribution is not invariant over the choice of measure space. Thus, old and familiar problems with the Laplacean principle of Insufficient Reason also plague MAXENT theory. Result 3 builds upon the findings of Friedman and Shimony (1971,1973) and demonstrates the absence of an exchangeable, Bayesian model for predictive MAXENT distributions when the MAXENT constraints are interpreted according to Jaynes’ (1978) prescription for his (1963) Brandeis Dice problem. Last, Result 4 generalizes the Friedman and Shimony objection to cross-entropy (Kullback-information) shifts subject to a constraint of a new odds-ratio for two disjoint events.

State-Dependent Utilities

Several axiom systems for preference among acts lead to the existence of a unique probability and a state-independent utility such that acts are ranked according to their expected utilities. These axioms have been used as a foundation for Bayesian decision theory and the subjective probability calculus. In this paper, we note that the uniqueness of the probability is relative to the choice of what counts as a constant outcome. Although it is sometimes clear what should be considered constant, there are many cases in which there are several possible choices. Each choice can lead to a different “unique” probability and utility. By focusing attention on state-dependent utilities, we determine conditions under which a truly unique probability and utility can be determined from an agent’s expressed preferences among acts. Suppose that an agent’s preference can be represented in terms of a probability P and a utility U. That is, the agent prefers one act to another if and only if the expected utility of the one act is higher than that of the other. There are many other equivalent representations in terms of probabilities Q, which are mutually absolutely continuous with P, and state-dependent utilities V, which differ from U by possibly different positive affine transformations in each state of nature. An example is described in which two different but equivalent state-independent utility representations exist for the same preference structure. What differs between the two representations is which acts count as constants. The acts involve receiving different amounts of one or the other of two currencies and the states are different exchange rates between the currencies. It is easy to see how it would not be possible for constant amounts of both currencies to simultaneously have constant values across the different states. Savage (Foundations of statistics. John Wiley, New York, 1954, sec. 5.5) discovered a situation in which two seemingly equivalent preference structures are represented by different pairs of probability and utility. Savage attributed the phenomenon to the construction of a “small world”. We show that the small world problem is just another example of two different, but equivalent, representations treating different acts as constants. Finally, we prove a theorem (similar to one of Karni, Decision making under uncertainty. Harvard University Press, Cambridge, 1985) that shows how to elicit a unique state-dependent utility and does not assume that there are prizes with constant value. To do this, we define a new hypothetical kind of act in which both the prize to be awarded and the state of nature are determined by an auxiliary experiment.

The Extent of Non-Conglomerability of Finitely Additive Probabilities

SummaryAn arbitrary finitely additive probability can be decomposed uniquely into a convex combination of a countably additive probability and a purely finitely additive (PFA) one. The coefficient of the PFA probability is an upper bound on the extent to which conglomerability may fail in a finitely additive probability with that decomposition. If the probability is defined on a σ-field, the bound is sharp. Hence, non-conglomerability (or equivalently non-disintegrability) characterizes finitely as opposed to countably additive probability. Nonetheless, there exists a PFA probability which is simultaneously conglomerable over an arbitrary finite set of partitions.Neither conglomerability nor non-conglomerability in a given partition is closed under convex combinations. But the convex combination of PFA ultrafilter probabilities, each of which cannot be made conglomerable in a common margin, is singular with respect to any finitely additive probability that is conglomerable in that margin.

Reasoning to a Foregone Conclusion

Abstract When can a Bayesian select an hypothesis H and design an experiment (or a sequence of experiments) to make certain that, given the experimental outcome(s), the posterior probability of H will be greater than its prior probability? We discuss an elementary result that establishes sufficient conditions under which this reasoning to a foregone conclusion cannot occur. We illustrate how when the sufficient conditions fail, because probability is finitely but not countably additive, it may be that a Bayesian can design an experiment to lead his/her posterior probability into a foregone conclusion. The problem has a decision theoretic version in which a Bayesian might rationally pay not to see the outcome of certain cost-free experiments, which we discuss from several perspectives. Also, we relate this issue in Bayesian hypothesis testing to various concerns about “optional stopping.”

Decision Theory without "Independence" or without "Ordering"

It is a familiar argument that advocates accommodating the paradoxes of decision theory by abandoning the “independence” postulate. After all, if we grant that choice reveals preference, the anomalous choice patterns of the Allais and Ellsberg problems (reviewed in section “Review of the Allais and Ellsberg “Paradoxes”) violate postulate P2 (“sure thing”) of Savage’s (The foundations of statistics. Wiley, New York, 1954) system.

Randomization in a bayesian perspective

“Applying the theory (of personal probability) naively one quickly comes to the conclusion that randomization is without value for statistics. This conclusion does not sound right; and it is not right. Closer examination of the road to this untenable conclusion does lead to new insights into the role and limitations of randomization but does by no means deprive randomization of its important function in statistics.” L.J. Savage (1961)

Decisions Without Ordering

We review the axiomatic foundations of subjective utility theory with a view toward understanding the implications of each axiom. We consider three different approaches, namely, the construction of utilities in the presence of canonical probabilities, the construction of probabilities in the presence of utilities, and the simultaneous construction of both probabilities and utilities. We focus attention on the axioms of independence and weak ordering. The independence axiom is seen to be necessary in order to prevent a form of Dutch Book in sequential problems.

Improper regular conditional distributions

Improper regular conditional distributions (rcds) given a σ-field A have the following anomalous property. For sets A ∈ A, Pr(A |A) is not always equal to the indicator of A. Such a property makes the conditional probability puzzling as a representation of uncertainty. When rcds exist and the σ-field A is countably generated, then almost surely the rcd is proper. We give sufficient conditions for an red to be improper in a maximal sense, and show that these conditions apply to the tail σ-field and the σfield of symmetric events.

Coherent Choice Functions under Uncertainty

We discuss several features of coherent choice functions—where the admissible options in a decision problem are exactly those that maximize expected utility for some probability/utility pair in fixed set S of probability/utility pairs. In this paper we consider, primarily, normal form decision problems under uncertainty—where only the probability component of S is indeterminate and utility for two privileged outcomes is determinate. Coherent choice distinguishes between each pair of sets of probabilities regardless the “shape” or “connectedness” of the sets of probabilities. We axiomatize the theory of choice functions and show these axioms are necessary for coherence. The axioms are sufficient for coherence using a set of probability/almost-state-independent utility pairs. We give sufficient conditions when a choice function satisfying our axioms is represented by a set of probability/state-independent utility pairs with a common utility.

A Conflict between Finite Additivity and Avoiding Dutch Book

For Savage (1954) as for de Finetti (1974), the existence of subjective (personal) probability is a consequence of the normative theory of preference. (De Finetti achieves the reduction of belief to desire with his generalized Dutch-Book argument for Previsions.) Both Savage and de Finetti rebel against legislating countable additivity for subjective probability. They require merely that probability be finitely additive. Simultaneously, they insist that their theories of preference are weak, accommodating all but self-defeating desires. In this paper we dispute these claims by showing that the following three cannot simultaneously hold: (i) Coherent belief is reducible to rational preference, i.e. the generalized Dutch-Book argument fixes standards of coherence. (ii) Finitely additive probability is coherent. (iii) Admissible preference structures may be free of consequences, i.e. they may lack prizes whose values are robust against all contingencies.