Pavel Atanasov

Psychological Strategies for Winning a Geopolitical Forecasting Tournament

Five university-based research groups competed to recruit forecasters, elicit their predictions, and aggregate those predictions to assign the most accurate probabilities to events in a 2-year geopolitical forecasting tournament. Our group tested and found support for three psychological drivers of accuracy: training, teaming, and tracking. Probability training corrected cognitive biases, encouraged forecasters to use reference classes, and provided forecasters with heuristics, such as averaging when multiple estimates were available. Teaming allowed forecasters to share information and discuss the rationales behind their beliefs. Tracking placed the highest performers (top 2% from Year 1) in elite teams that worked together. Results showed that probability training, team collaboration, and tracking improved both calibration and resolution. Forecasting is often viewed as a statistical problem, but forecasts can be improved with behavioral interventions. Training, teaming, and tracking are psychological interventions that dramatically increased the accuracy of forecasts. Statistical algorithms (reported elsewhere) improved the accuracy of the aggregation. Putting both statistics and psychology to work produced the best forecasts 2 years in a row.

The psychology of intelligence analysis: Drivers of prediction accuracy in world politics

This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and open-mindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs.

Are Culturally Specific Measures of Trauma-Related Anxiety and Depression Needed? The Case of Sri Lanka

The hypothesis that psychometric instruments incorporating local idioms of distress predict functional impairment in a non-Western, war-affected population above and beyond translations of already established instruments was tested. Exploratory factor analysis was conducted on the War-Related Psychological and Behavioral Problems section of the Penn/RESIST/Peradeniya War Problems Questionnaire (PRPWPQ; N. Jayawickreme, Jayawickreme, Goonasekera, & Foa, 2009), a measure that incorporates local idioms of distress, using data from 197 individuals living in Northern and Eastern Sri Lanka. Three subscales--Anxiety, Depression, and Negative Perception--were identified. Regression analyses were conducted to test whether these 3 subscales better predicted functional impairment than the PTSD Symptom Scale-Self Report (PSS; Foa, Riggs, Dancu, & Rothbaum, 1993) and the Beck Depression Inventory (BDI; Beck & Steer, 1987), both widely used self-report measures of posttraumatic stress disorder and depression, respectively. Two of the 3 subscales from the PRPWPQ--Anxiety and Depression--were significantly associated with higher rates of functional impairment after controlling for age, the PSS and the BDI. After the inclusion of PRPWPQ, the PSS and the BDI did not significantly contribute to the final regression model predicting functional impairment. These findings suggest that the scores of measures with local idioms of distress have incremental validity in non-Western war-affected populations, predicting functional impairment above and beyond translations of established self-report measures that have been developed in the Western world.

Distilling the Wisdom of Crowds: Prediction Markets vs. Prediction Polls

We report the results of the first large-scale, long-term, experimental test between two crowd sourcing methods – prediction markets and prediction polls. More than 2,400 participants made forecasts on 261 events over two seasons of a geopolitical prediction tournament. Some forecasters traded in a continuous double auction market and were ranked based on earnings. Others submitted probability judgments, independently or in teams, and were ranked based on Brier scores. In both seasons of the tournament, last day prices from the prediction market were more accurate than the simple mean of forecasts from prediction polls. However, team prediction polls outperformed prediction markets when poll forecasts were aggregated with algorithms using temporal decay, performance weighting and recalibration. The biggest advantage of prediction polls occurred at the start of long-duration questions. Prediction polls with proper scoring, algorithmic aggregation and teaming offer an attractive alternative to prediction markets for distilling the wisdom of crowds.

Comparing physicians personal prevention practices and their recommendations to patients.

Background:Hypothetical choice studies suggest that physicians often take more risk for themselves than on their patients behalf. Objective:To examine if physicians recommend more screening tests than they personally undergo in the real-world context of breast cancer screening. Design:Within-subjects survey. Participants:A national sample of female obstetricians and gynecologists (N = 135, response rate 54%) from the United States. In total, they provided breast care to approximately 2,800 patients per week. Measures:Personal usage history and patient recommendations regarding mammography screening and breast self-examination, a measure of defensive medicine practices. Results:Across age groups, female physicians were more likely to recommend mammography screening than to have performed the procedure in the past 5 years (86% vs. 81%, p = .10). In respondents aged 40–49 this difference was significant (91% vs. 82%, p < .05), whereas no differences were detected for younger or older physicians. Among respondents in their 40s, 18% had undergone annual screenings in the past 5 years, compared to 48% of their colleagues above 50. Respondents were as likely to practice breast self-examination (98%) as to recommend it (93%), a pattern that was consistent across age groups. A logistic regression model of personal use of mammography significantly predicted recommending the procedure to patients (OR = 15.29, p = .001). Similarly, number of breast self-examinations performed over the past 2 years positively predicted patient recommendations of the procedure (OR = 1.31, p < .001). Conclusions:Obstetricians and gynecologists tended to recommend early mammography screening to their patients, though their personal practices indicated later start than their own recommendations and lower frequency of screening than peers in recent studies have recommended.

Risk Preferences in Choices for Self and Others: Meta Analysis and Research Directions

Are we more inclined to take risks for ourselves rather than on someone else’s behalf? The current study reviews and summarizes 28 effects from 18 studies (n=4,784). Across all studies, choices for others were significantly more risk-averse than choices for self (d=0.15, p=0.012). Two objective features of the choices moderated these effects: potential losses and reciprocal relationships. First, self-other differences in risk preferences were significant in the presence of potential losses (k=14, d=0.33, p<.001), and not significant (k=14, d=-0.06, p=0.473) in the gains-only domain (Q=12.56, p=<0.001). Choices for others were significantly more risk-averse when decision makers were reciprocally related to recipients (k=6, d=0.33, p=0.018) but no different in the absence of such a relationship (d=0.11, p=0.115). Reciprocal relationship was a marginally significant predictor (Q=2.02, p=0.155). Results are shown separately by publication status and by context (medical, economic game, hypothetical choice). A relational model of surrogate risk taking is proposed to explain the pattern of results, which emphasizes the importance of chooser-recipient relationships, and the tendency of choosers to minimize anticipated blame from losses, rather than maximizing credit for gains. Implications for benefits design, medical and managerial decision making are discussed.

Putting Health Back Into Health Insurance Choice

What are the barriers to voluntary take-up of high-deductible plans? We address this question using a large-scale employer survey conducted after an open-enrollment period in which a new high-deductible plan was first introduced. Only 3% of the employees chose this plan, despite the respondents’ recognition of its financial advantages. Employees who believed that the high-deductible plan provided access to top physicians in the area were three times more likely to choose it than employees who did not share this belief. A framed field experiment using a similar choice menu showed that displaying additional financial information did not increase high-deductible plan take-up. However, when plans were presented as identical except for the deductible, respondents were highly likely to choose the high-deductible plan, especially in a two-way choice. These results suggest that informing plan choosers about high-deductible plans’ health access provisions may affect choice more strongly than focusing on their financial advantages.

The marketcast method for aggregating prediction market forecasts

We describe a hybrid forecasting method called marketcast. Marketcasts are based on bid and ask orders from prediction markets, aggregated using techniques associated with survey methods, rather than market matching algorithms. We discuss the process of conversion from market orders to probability estimates, and simple aggregation methods. The performance of marketcasts is compared to a traditional prediction market and a traditional opinion poll. Overall, marketcasts perform approximately as well as prediction markets and opinion poll methods on most questions, and performance is stable across model specifications.

Taste-Based Gender Discrimination by Contestants on the Price is Right

We report evidence of gender discrimination by contestants in the One Bid game on The Price Is Right television show. One Bid contestants bid sequentially in an attempt to get closest to the price of a prize on display without exceeding it. The last bidder in the game has a dominant cutoff strategy of bidding

Leveling the playing field: Dishonesty in the face of threat

1 more than another contestant, but this strategy leaves the target contestant with almost no chance to win. Contestant groups and bidding orders within groups are plausibly exogenous with respect to gender and there are high stakes for playing impartially, yet our analysis of over 5,000 One Bid rounds shows that for last bidders of both genders, same-gender opponents are less likely to be cut offs . For rounds in which the last bid is not the lowest, cutoff s remain significantly more likely when the next lowest bidder is of the opposite-gender. Because the last bidders revealed belief is that the next lowest bid is the best to cut off , our results demonstrate gender favoritism while holding constant beliefs about the effectiveness of cutting o ff a given opponent. Our empirical strategy thus allows us to parse out tastes from other sources of discrimination such as statistical discrimination or irrational stereotyping, while also identifying which genders are discriminating. We estimate that final bidders transfer