Joseph B. Kadane

Accurate Approximations for Posterior Moments and Marginal Densities

Abstract This article describes approximations to the posterior means and variances of positive functions of a real or vector-valued parameter, and to the marginal posterior densities of arbitrary (i.e., not necessarily positive) parameters. These approximations can also be used to compute approximate predictive densities. To apply the proposed method, one only needs to be able to maximize slightly modified likelihood functions and to evaluate the observed information at the maxima. Nevertheless, the resulting approximations are generally as accurate and in some cases more accurate than approximations based on third-order expansions of the likelihood and requiring the evaluation of third derivatives. The approximate marginal posterior densities behave very much like saddle-point approximations for sampling distributions. The principal regularity condition required is that the likelihood times prior be unimodal.

Statistical Methods for Eliciting Probability Distributions

Elicitation is a key task for subjectivist Bayesians. Although skeptics hold that elicitation cannot (or perhaps should not) be done, in practice it brings statisticians closer to their clients and subject-matter expert colleagues. This article reviews the state of the art, reflecting the experience of statisticians informed by the fruits of a long line of psychological research into how people represent uncertain information cognitively and how they respond to questions about that information. In a discussion of the elicitation process, the first issue to address is what it means for an elicitation to be successful; that is, what criteria should be used. Our answer is that a successful elicitation faithfully represents the opinion of the person being elicited. It is not necessarily “true” in some objectivistic sense, and cannot be judged in that way. We see that elicitation as simply part of the process of statistical modeling. Indeed, in a hierarchical model at which point the likelihood ends and the prior begins is ambiguous. Thus the same kinds of judgment that inform statistical modeling in general also inform elicitation of prior distributions. The psychological literature suggests that people are prone to certain heuristics and biases in how they respond to situations involving uncertainty. As a result, some of the ways of asking questions about uncertain quantities are preferable to others, and appear to be more reliable. However, data are lacking on exactly how well the various methods work, because it is unclear, other than by asking using an elicitation method, just what the person believes. Consequently, one is reduced to indirect means of assessing elicitation methods. The tool chest of methods is growing. Historically, the first methods involved choosing hyperparameters using conjugate prior families, at a time when these were the only families for which posterior distributions could be computed. Modern computational methods, such as Markov chain Monte Carlo, have freed elicitation from this constraint. As a result, now both parametric and nonparametric methods are available for low-dimensional problems. High-dimensional problems are probably best thought of as lacking another hierarchical level, which has the effect of reducing the as-yet-unelicited parameter space. Special considerations apply to the elicitation of group opinions. Informal methods, such as Delphi, encourage the participants to discuss the issue in the hope of reaching consensus. Formal methods, such as weighted averages or logarithmic opinion pools, each have mathematical characteristics that are uncomfortable. Finally, there is the question of what a group opinion even means, because it is not necessarily the opinion of any participant.

An overview of robust Bayesian analysis

SummaryRobust Bayesian analysis is the study of the sensitivity of Bayesian answers to uncertain inputs. This paper seeks to provide an overview of the subject, one that is accessible to statisticians outside the field. Recent developments in the area are also reviewed, though with very uneven emphasis.

Interactive Elicitation of Opinion for a Normal Linear Model

Abstract This article describes the mathematical theory underlying an interactive computer program for eliciting the hyperparameters of a subjective conjugate distribution for the multiple linear regression model with the usual normal error structure. Although the methods are heuristic, they are shown to produce hyperparameter estimates satisfying the constraints satisfied by the hyperparameters themselves. An application is given to the problem of predicting the time to fatigue failure of an asphalt-concrete road as a function of several design variables concerning the road.

Fully Exponential Laplace Approximations to Expectations and Variances of Nonpositive Functions

Abstract Tierney and Kadane (1986) presented a simple second-order approximation for posterior expectations of positive functions. They used Laplaces method for asymptotic evaluation of integrals, in which the integrand is written as f(θ)exp(-nh(θ)) and the function h is approximated by a quadratic. The form in which they applied Laplaces method, however, was fully exponential: The integrand was written instead as exp[− nh(θ) + log f(θ)]; this allowed first-order approximations to be used in the numerator and denominator of a ratio of integrals to produce a second-order expansion for the ratio. Other second-order expansions (Hartigan 1965; Johnson 1970; Lindley 1961, 1980; Mosteller and Wallace 1964) require computation of more derivatives of the log-likelihood function. In this article we extend the fully exponential method to apply to expectations and variances of nonpositive functions. To obtain a second-order approximation to an expectation E(g(θ)), we use the fully exponential method to approximate...

Scoring rules and the evaluation of probabilities

SummaryIn Bayesian inference and decision analysis, inferences and predictions are inherently probabilistic in nature. Scoring rules, which involve the computation of a score based on probability forecasts and what actually occurs, can be used to evaluate probabilities and to provide appropriate incentives for “good” probabilities. This paper review scoring rules and some related measures for evaluating probabilities, including decompositions of scoring rules and attributes of “goodness” of probabilites, comparability of scores, and the design of scoring rules for specific inferential and decision-making problems

Separating Probability Elicitation from Utilities

Abstract This article deals with the separation of probability elicitation from utilities. We show that elicited probabilities can be related to utilities not just through the explicit or implicit payoffs related to the elicitation process, but also through other stakes the expert may have in the events of interest. We study three elicitation procedures—lotteries, scoring rules, and promissory notes—and show how the experts utility function and stakes in the events can influence the resulting probabilities. Particularly extreme results are obtained in an example involving a market at equilibrium. The applicability of a no-stakes condition and some implications for probability elicitation are discussed. Let π represent an experts probability for an event A, and let p denote the elicited probability from some elicitation procedure. We determine the value of p that maximizes the experts expected utility. When utility is linear in money, p = π for all of the procedures studied here. Under nonlinear utility...

Using uncleanliness to predict future botnet addresses

The increased use of botnets as an attack tool and the awareness attackers have of blocking lists leads to the question of whether we can effectively predict future bot locations. To that end, we introduce a network quality that we term uncleanliness: an indicator of the propensity for hosts in a network to be compromised by outside parties. We hypothesize that unclean networks will demonstrate two properties: spatial and temporal uncleanliness. Spatial uncleanliness is the tendency for compromised hosts to cluster within unclean networks. Temporal uncleanliness is the tendency for unclean networks to contain compromised hosts for extended periods. We test for these properties by collating data from multiple indicators (spamming, phishing, scanning and botnet IRC log monitoring). We demonstrate evidence for both spatial and temporal uncleanliness. We further show evidence for cross-relationship between the various datasets, showing that botnet activity predicts spamming and scanning, while phishing activity appears to be unrelated to the other indicators.

Optimal Problem-Solving Search: All-or-None Solutions

Abstract Optimal algorithms are derived for satisficing problem-solving search, that is, search where the goal is to reach any solution, no distinction being made among different solutions. This task is quite different from search for best solutions or shortest path solutions. Constraints may be placed on the order in which sites may be searched. This paper treats satisficing searches through partially ordered search spaces where there are multiple alternative goals.

On division of the question

I believe that the mathematical theory of voting may be used to shed light on traditional rules of parliamentary procedure. For example, Blacks [2] theory of voting on issues when the alternatives can be ordered so that each members preferences are single-peaked highlights the importance of the chairmans casting vote in the case of an even number of voters. In this paper I study how another rule, division of the question on amendments, affects the creation of successful packages.