Victor Richmond R. Jose

Scoring Rules, Generalized Entropy, and Utility Maximization

Information measures arise in many disciplines, including forecasting (where scoring rules are used to provide incentives for probability estimation), signal processing (where information gain is measured in physical units of relative entropy), decision analysis (where new information can lead to improved decisions), and finance (where investors optimize portfolios based on their private information and risk preferences). In this paper, we generalize the two most commonly used parametric families of scoring rules and demonstrate their relation to well-known generalized entropies and utility functions, shedding new light on the characteristics of alternative scoring rules as well as duality relationships between utility maximization and entropy minimization. In particular, we show that weighted forms of the pseudospherical and power scoring rules correspond exactly to measures of relative entropy (divergence) with convenient properties, and they also correspond exactly to the solutions of expected utility ...

Evaluating Quantile Assessments

Quantile assessments are commonly encountered in the elicitation of probability distributions in decision analysis, forecasting, and risk analysis. Scoring rules have been developed to provide ex ante incentives for careful and truthful assessments and ex post evaluation measures in the context of probability assessment. We show that these scoring rules designed for probability assessment provide inappropriate incentives if used for quantile assessment. We investigate the properties of a linear family of scoring rules that are intended specifically for quantile assessment (including the assessment of multiple quantiles) and can be related to a realistic decision-making problem. These rules provide proper incentives for quantile assessment and yield higher expected scores for distributions that are more informative in the sense of having less dispersion. We discuss the special case of interval forecasts and a generalization involving transformations, and we briefly mention other possible extensions.

Sensitivity to Distance and Baseline Distributions in Forecast Evaluation

Scoring rules can provide incentives for truthful reporting of probabilities and evaluation measures for the probabilities after the events of interest are observed. Often the space of events is ordered and an evaluation relative to some baseline distribution is desired. Scoring rules typically studied in the literature and used in practice do not take account of any ordering of events, and they evaluate probabilities relative to a default baseline distribution. In this paper, we construct rich families of scoring rules that are strictly proper (thereby encouraging truthful reporting), are sensitive to distance (thereby taking into account ordering of events), and incorporate a baseline distribution relative to which the value of a forecast is measured. In particular, we extend the power and pseudospherical families of scoring rules to allow for sensitivity to distance, with or without a specified baseline distribution.

Tailored Scoring Rules for Probabilities

When scoring rules were first widely used, they were developed as a way to measure the accuracy of probability forecasts ex post. Ex ante, proper scoring rules encourage honestly reported and sharper probabilities, both of which increase the forecasters expected score. Most applications utilize standard off-the-shelf scoring rules. In the spirit of decision analysis, we develop proper scoring rules that are tailored to specific decision-making problems and to the utility functions of particular decision makers. We show how these rules, which are intended for situations where a decision maker consults an expert to assess a probability, not only encourage honest reporting, but also reward sharpness in a way that aligns the interests of the expert and the decision maker. We also illustrate the generation of tailored scoring rules in numerical form, which is useful when analytical expressions for the tailored rules cannot be obtained or are too complex to be helpful in practice. Finally, we show how these numerical scoring rules can be presented to the expert in graphical or tabular form and suggest that this could be desirable even for standard scoring rules.

The role of risk preferences in a multi-target defender-attacker resource allocation game

Abstract This paper studies a sequential defender-attacker game, where the defender allocates defensive resources to multiple potential targets while considering the risk preferences of both attacker and defender. We model and obtain analytical equilibrium results for this problem and study how risk preferences affect a player’s behavior in equilibrium. We find that in the strategic case, when both the attacker and the defender have some target valuation, the strategic attacker’s risk preferences and target valuation affect the optimal defense allocation. In particular, when the attacker becomes more risk seeking/averse, the high valuable target to the attacker would receive more/less resources. The proposed model leads to a significantly lower expected damage than a model where the attacker is incorrectly considered risk neutral, especially when the attacker is risk seeking.

Percentage and Relative Error Measures in Forecast Evaluation

Properties of two large families of scale-free forecast accuracy measures that include popular measures such as mean absolute percentage error, relative error, and squared percentage error, are examined in this paper. We describe the optimal reports when forecasts are evaluated using these measures. We also provide analytic expressions for the optimal Bayes’ act associated with these measures under a general power transformation for several well-known probability distributions. We then show that using measures from these two families may inadvertently provide incentives for either pessimism or optimism among forecasters, i.e., rewarding underforecasts or overforecasts relative to some reference measure of central tendency. As an illustration of these concepts, we examine the use of these measures for model selection in a forecast aggregation example using stock price forecasts derived from the Thomson Reuters Institutional Brokers’ Estimate System. This example illustrates how aggregation methods that always yield lower estimates relative to the mean or median generally exhibit better scores using percentage error-based measures, while those that yield higher estimates compared to the mean or median will effectively rank higher when relative error-based measures are used.

Incorporating risk preferences in stochastic noncooperative games

ABSTRACT Traditional game-theoretic models of competition with uncertainty often ignore preferences and attitudes toward risk by assuming that players are risk neutral. In this article, we begin by considering how a comprehensive analysis and incorporation of expected utility theory affect players’ equilibrium behavior in a simple, single-period, sequential stochastic game. Although the literature posits that the more risk averse a first mover is, the more likely she is to compete and defend her position as the “leader”, and that the more risk seeking a “follower” is, the more likely he is willing to participate and compete, we find that this behavior may not always be true in this more general setting. Under simple assumptions on the utility function, we perform sensitivity analyses on the parameters and show which behavior changes when deviations from risk neutrality are introduced into a model. We also provide some insights on how risk preferences influence pre-emption and interdiction by looking at how these preferences affect the first mover’s advantage in a sequential setting. This article generates novel insights when a confluence of factors leads players to deviate or change their behavior in many risk analysis settings where stochastic games are used.