David Rothschild

The Mythical Swing Voter

Cross-sectional surveys conducted during the 2012 U.S. presidential campaign showed large swings in support for the Democratic and Republican candidates, especially before and after the first presidential debate. Using a unique (in terms of scale, frequency, and source) panel survey, we find that daily sample composition varied more in response to campaign events than did vote intentions. Multi-level regression and post-stratification (MRP) is used to correct for selection bias. Demographic post-stratification, similar to that used in most academic and media polls, is inadequate, but the addition of attitudinal variables (party identification, ideological self-placement, and past vote) appear to make selection ignorable in our data. We conclude that vote swings in 2012 were mostly sample artifacts and that real swings were quite small. While this account is at variance with most contemporaneous analyses, it better corresponds with our understanding of partisan polarization in modern American politics.

Online and Social Media Data As an Imperfect Continuous Panel Survey

There is a large body of research on utilizing online activity as a survey of political opinion to predict real world election outcomes. There is considerably less work, however, on using this data to understand topic-specific interest and opinion amongst the general population and specific demographic subgroups, as currently measured by relatively expensive surveys. Here we investigate this possibility by studying a full census of all Twitter activity during the 2012 election cycle along with the comprehensive search history of a large panel of Internet users during the same period, highlighting the challenges in interpreting online and social media activity as the results of a survey. As noted in existing work, the online population is a non-representative sample of the offline world (e.g., the U.S. voting population). We extend this work to show how demographic skew and user participation is non-stationary and difficult to predict over time. In addition, the nature of user contributions varies substantially around important events. Furthermore, we note subtle problems in mapping what people are sharing or consuming online to specific sentiment or opinion measures around a particular topic. We provide a framework, built around considering this data as an imperfect continuous panel survey, for addressing these issues so that meaningful insight about public interest and opinion can be reliably extracted from online and social media data.

Are public opinion polls self-fulfilling prophecies?:

Psychologists have long observed that people conform to majority opinion, a phenomenon sometimes referred to as the ‘bandwagon effect’. In the political domain people learn about prevailing public opinion via ubiquitous polls, which may produce a bandwagon effect. Newer types of information – published probabilities derived from prediction market contract prices and aggregated polling summaries – may have similar effects. Consequently, polls can become self-fulfilling prophecies whereby majorities, whether in support of candidates or policies, grow in a cascading manner. Despite increased attention to whether the measurement of public opinion can itself affect public opinion, the existing empirical literature is surprisingly limited on the bandwagon effects of polls. To address this gap, we conducted an experiment on a diverse national sample in which we randomly assigned people to receive information about different levels of support for three public policies. We find that public opinion as expressed through polls affects individual-level attitudes, although the size of the effect depends on issue characteristics.

A combinatorial prediction market for the U.S. elections

We report on a large-scale case study of a combinatorial prediction market. We implemented a back-end pricing engine based on Dudik et al.s (2012) combinatorial market maker, together with a wizard-like front end to guide users to constructing any of millions of predictions about the presidential, senatorial, and gubernatorial elections in the United States in 2012. Users could create complex combinations of predictions and, as a result, we obtained detailed information about the joint distribution and conditional estimates of election results. We describe our market, how users behaved, and how well our predictions compared with benchmark forecasts. We conduct a series of counterfactual simulations to investigate how our market might be improved in the future.

The extent of price misalignment in prediction markets

We study misaligned prices for logically related contracts in prediction markets. First, we uncover persistent arbitrage opportunities for risk-neutral investors between identical contracts on different exchanges. Examining the impact of several thousand dollars of transactions on the exchanges themselves in a randomized field trial, we document that price support extends well beyond what is seen in the published order book and that arbitrage opportunities are significantly larger than purely observational measurements indicate. Second, we demonstrate misalignment among identical and logically related contracts listed on the same exchange that cluster around moments of high information flow, when related contracts systemically shut down or fail to respond efficiently. Third, we document bounded rationality in prediction markets; examples include: consistent asymmetry between buying and selling, leaving the average return for selling higher than for buying; and persistent price lags between exchanges. Despite these signs of departure from theoretical optimality, the markets studied function well on balance, considering the sometimes complex and subtle relationships among contracts. Yet, we detail how to improve prediction markets by moving the burden of finding and fixing logical contradictions into the exchange and providing flexible trading interfaces, both of which free traders to focus on providing meaningful information in the form they find most natural.

Disentangling Bias and Variance in Election Polls

ABSTRACT It is well known among researchers and practitioners that election polls suffer from a variety of sampling and nonsampling errors, often collectively referred to as total survey error. Reported margins of error typically only capture sampling variability, and in particular, generally ignore nonsampling errors in defining the target population (e.g., errors due to uncertainty in who will vote). Here, we empirically analyze 4221 polls for 608 state-level presidential, senatorial, and gubernatorial elections between 1998 and 2014, all of which were conducted during the final three weeks of the campaigns. Comparing to the actual election outcomes, we find that average survey error as measured by root mean square error is approximately 3.5 percentage points, about twice as large as that implied by most reported margins of error. We decompose survey error into election-level bias and variance terms. We find that average absolute election-level bias is about 2 percentage points, indicating that polls for a given election often share a common component of error. This shared error may stem from the fact that polling organizations often face similar difficulties in reaching various subgroups of the population, and that they rely on similar screening rules when estimating who will vote. We also find that average election-level variance is higher than implied by simple random sampling, in part because polling organizations often use complex sampling designs and adjustment procedures. We conclude by discussing how these results help explain polling failures in the 2016 U.S. presidential election, and offer recommendations to improve polling practice.

Using Big Data and Algorithms to Determine the Effect of Geographically Targeted Advertising on Vote Intention: Evidence From the 2012 U.S. Presidential Election

We develop a new conceptualization of political advertising effects by looking at the effect of the marginal advertising dollar during the heat of presidential campaigns. We argue that in contrast to other studies investigating effects of political ads, our approach is more apt to capture the natural environment in which political ads are encountered during a presidential campaign. We focus on the intense inundation of political ads voters are confronted with in swing states in the weeks leading up to the presidential election, and argue that it is unclear a priori whether we should expect advertising to affect vote intention in that critical circumstance. We empirically validate this hypothesis using a trove of data from the 2012 campaign: daily polling in media markets around the country, detailed data on all registered voters in the country, all TV advertisements by market and exact airtime, and the entire Twitter corpus. We find that neither overall increases in advertising spending nor partisan imbalances in spending expanded the candidates’ electorate. In fact, total Designated Market Area (DMA)-level spending significantly moderates a negative relationship between spending advantages and advantages in vote intention, suggesting a boomerang effect of additional spending late in the campaign. In closing, we discuss the ramifications of our findings for future research, and stress the importance of research tracking advertising effects.

Addressing Selection Bias in Event Studies with General-Purpose Social Media Panels

Data from Twitter have been employed in prior research to study the impacts of events. Conventionally, researchers use keyword-based samples of tweets to create a panel of Twitter users who mention event-related keywords during and after an event. However, the keyword-based sampling is limited in its objectivity dimension of data and information quality. First, the technique suffers from selection bias since users who discuss an event are already more likely to discuss event-related topics beforehand. Second, there are no viable control groups for comparison to a keyword-based sample of Twitter users. We propose an alternative sampling approach to construct panels of users defined by their geolocation. Geolocated panels are exogenous to the keywords in users’ tweets, resulting in less selection bias than the keyword panel method. Geolocated panels allow us to follow within-person changes over time and enable the creation of comparison groups. We compare different panels in two real-world settings: response to mass shootings and TV advertising. We first show the strength of the selection biases of keyword panels. Then, we empirically illustrate how geolocated panels reduce selection biases and allow meaningful comparison groups regarding the impact of the studied events. We are the first to provide a clear, empirical example of how a better panel selection design, based on an exogenous variable such as geography, both reduces selection bias compared to the current state of the art and increases the value of Twitter research for studying events. While we advocate for the use of a geolocated panel, we also discuss its weaknesses and application scenario seriously. This article also calls attention to the importance of selection bias in impacting the objectivity of social media data.