Yael Grushka-Cockayne

Is it Better to Average Probabilities or Quantiles

We consider two ways to aggregate expert opinions using simple averages: averaging probabilities and averaging quantiles. We examine analytical properties of these forecasts and compare their ability to harness the wisdom of the crowd. In terms of location, the two average forecasts have the same mean. The average quantile forecast is always sharper: it has lower variance than the average probability forecast. Even when the average probability forecast is overconfident, the shape of the average quantile forecast still offers the possibility of a better forecast. Using probability forecasts for gross domestic product growth and inflation from the Survey of Professional Forecasters, we present evidence that both when the average probability forecast is overconfident and when it is underconfident, it is outperformed by the average quantile forecast. Our results show that averaging quantiles is a viable alternative and indicate some conditions under which it is likely to be more useful than averaging probabilities. This paper was accepted by Peter Wakker, decision analysis.

Incentives in a Stage-Gate Process

Many large organizations use a stage-gate process to manage new product development projects. In a typical stage-gate process project managers learn about potential ideas from research and exert effort in development while senior executives make intervening go/no-go decisions. This decentralized decision making results in an agency problem because the idea quality in early stages is unknown to the executive and the project manager must exert unobservable development effort in later stages. In light of these challenges, how should the firm structure incentives to ensure that project managers reveal relevant information and invest the appropriate effort to create value? In this paper, we develop a model of adverse selection in research and moral hazard in development with an explicit go/no-go decision at the intervening gate. Our results show that the principals uncertainty regarding early-stage idea quality --- a term we refer to as idea risk --- alters the effect of late-stage development risk. The presence of idea risk can lead the firm to reject projects that otherwise seem favorable in terms of positive net present value. A simulation of early-stage ideas, found through search on a complex landscape, shows that the firm can mitigate the negative effects of idea risk by encouraging breadth of search and high tolerance for failure.

New product development flexibility in a competitive environment

Managerial flexibility can have a significant impact on the value of new product development projects. We investigate how the market environment in which a firm operates influences the value and use of development flexibility. We characterize the market environment according to two dimensions, namely (i) its intensity, and (ii) its degree of innovation. We show that these two market characteristics can have a different effect on the value of flexibility. In particular, we show that more intense or innovative environments may increase or decrease the value of flexibility. For instance, we demonstrate that the option to defer a product launch is typically most valuable when there is little competition. We find, however, that under certain conditions defer options may be highly valuable in more competitive environments. We also consider the value associated with the flexibility to switch development strategies, from a focus on incremental innovations to more risky ground-breaking products. We find that such a switching option is most valuable when the market is characterized by incremental innovations and by relatively intense competition. Our insights can help firms understand how managerial flexibility should be explored, and how it might depend on the nature of the environment in which they operate.

The Promise of Prediction Contests

This article examines the prediction contest as a vehicle for aggregating the opinions of a crowd of experts. After proposing a general definition distinguishing prediction contests from other mechanisms for harnessing the wisdom of crowds, we focus on point-forecasting contests—contests in which forecasters submit point forecasts with a prize going to the entry closest to the quantity of interest. We first illustrate the incentive for forecasters to submit reports that exaggerate in the direction of their private information. Whereas this exaggeration raises a forecasters mean squared error, it increases his or her chances of winning the contest. And in contrast to conventional wisdom, this nontruthful reporting usually improves the accuracy of the resulting crowd forecast. The source of this improvement is that exaggeration shifts weight away from public information (information known to all forecasters) and by so doing helps alleviate public knowledge bias. In the context of a simple theoretical model of overlapping information and forecaster behaviors, we present closed-form expressions for the mean squared error of the crowd forecasts which will help identify the situations in which point forecasting contests will be most useful.

Towards a Single European Sky

We describe an integrated decision-making framework and model that we developed to aid EUROCONTROL, the European air traffic management organization, in its vital role of constructing a single unified European sky. Combining multicriteria decision analysis with large-scale optimization methods, such as integer programming and column generation using branch and price, our model facilitates the process by which the numerous European aviation stakeholders evaluate and select technological enhancements to the European air traffic management system. We consider multiple objectives and potential disagreements by stakeholders regarding the impact of proposed system enhancements and allow for different priorities for each key performance area. In an earlier paper, we described the mathematical programming model in detail. In this paper, we elaborate on the broader decision framework and supporting methodologies to help EUROCONTROL in its facilitation role. Using our model and decision framework, EUROCONTROL is currently selecting a set of enhancements to the European aviation system upon which all stakeholders have agreed.

Vungle Inc. Improves Monetization Using Big Data Analytics

The advent of big data has created opportunities for firms to customize their products and services to unprecedented levels of granularity. Using big data to personalize an offering in real time, however, remains a major challenge. In the mobile advertising industry, once a customer enters the network, an ad-serving decision must be made in a matter of milliseconds. In this work, we describe the design and implementation of an ad-serving algorithm that incorporates machine-learning methods to make personalized ad-serving decisions within milliseconds. We developed this algorithm for Vungle Inc., one of the largest global mobile ad networks. Our approach also addresses other important issues that most ad networks face, such as user fatigue, budget restrictions, and campaign pacing. In an A/B test versus the company’s legacy algorithm, our algorithm generated a 23 percent increase in revenue per 1,000 impressions. Across the company’s network, this increase represents a

Forecasting Life Cycles with Exponential Smoothing

1 million increase in monthly revenue.

Perceived Repeated Risk and the Allais Paradox

We introduce an exponential smoothing model with a life-cycle trend. The trend in this model follows the density of a tilted-Gompertz distribution. When the model is used to describe a new-product diffusion, a draw from this distribution can be interpreted as a consumers adoption time. Due to its tilting parameter, this distribution is flexible and capable of describing a wide range of shapes, including right-skewed, nearly symmetric, and left-skewed life cycles. Our model leads to new behavioral interpretations about the forces of imitation and innovation that may explain why a diffusion is skewed. The smoothing equations in the model allow its parameters to be time-varying. Consequently, the model can react to local changes in the environment. In two empirical studies, one on new-product adoptions and the other on search interest in social networks, we find that our model performs well on in-sample fitting and out-of-sample forecasting, when compared to two other leading diffusion models: the Bass and gamma/shifted-Gompertz models.We introduce an exponential smoothing model with a life-cycle trend. The trend in this model follows the density of a tilted-Gompertz distribution. When the model is used to describe a new-product diffusion, a draw from this distribution can be interpreted as a consumers adoption time. Due to its tilting parameter, this distribution is flexible and capable of describing a wide range of shapes, including right-skewed, nearly symmetric, and left-skewed life cycles. Our model leads to new behavioral interpretations about the forces of imitation and innovation that may explain why a diffusion is skewed. The smoothing equations in the model allow its parameters to be time-varying. Consequently, the model can react to local changes in the environment. In two empirical studies, one on new-product adoptions and the other on search interest in social networks, we find that our model performs well on in-sample fitting and out-of-sample forecasting, when compared to two other leading diffusion models: the Bass and gamma/shifted-Gompertz models.

The impact of project portfolio management on information technology projects

Empirical evidence on the common ratio effect, a version of the Allais paradox, indicates that people are nonlinearly sensitive to probabilities. The leading explanation of this effect is prospect theory and its nonlinear weights applied to probabilities. We propose an alternative explanation: expected utility under the perception that one-shot risks will be repeated. We follow a stream of literature which suggests that evolutionary forces have shaped human tendencies to choose actions which are adapted to recurrent situations. We show that expected utility under perceived repeated risk can explain the common ratio effect and give rise to nonlinear sensitivity to probabilities. We provide several classes of utilities that can satisfy conditions that are necessary and sufficient for the existence of a common ratio effect. These utility functions have context-dependent inflection points: a zero inflection point that resets itself in each choice frame and other non-zero inflection points around which an individual is increasingly absolute risk averse. We argue that such inflection points may arise from frequent social comparison of the outcomes from one-shot risks.