Ryan Kennedy

The Parable of Google Flu: Traps in Big Data Analysis

Large errors in flu prediction were largely avoidable, which offers lessons for the use of big data. In February 2013, Google Flu Trends (GFT) made headlines but not for a reason that Google executives or the creators of the flu tracking system would have hoped. Nature reported that GFT was predicting more than double the proportion of doctor visits for influenza-like illness (ILI) than the Centers for Disease Control and Prevention (CDC), which bases its estimates on surveillance reports from laboratories across the United States (1, 2). This happened despite the fact that GFT was built to predict CDC reports. Given that GFT is often held up as an exemplary use of big data (3, 4), what lessons can we draw from this error?

The Contradiction of Modernization: A Conditional Model of Endogenous Democratization

Several very influential authors in political science and economics have recently argued that the relationship between economic development and democracy is extremely weak, spurious, or that the causal arrow is reversed. This analysis argues that a fundamental conceptual issue with how democratization is studied is responsible for their findings. Namely, economic development has the opposite effect on the onset and outcome of institutional change. In an analysis of 178 countries from 1816 to 2004, this study finds that economic development decreases the probability of major institutional change, but increases the probability that such a change will be towards democracy. This finding is remarkably robust to the inclusion of control variables and fixed-effects estimation.

Growing pains for global monitoring of societal events

Automated event coding raises promise and concerns There have been serious efforts over the past 40 years to use newspaper articles to create global-scale databases of events occurring in every corner of the world, to help understand and shape responses to global problems. Although most have been limited by the technology of the time (1) [see supplementary materials (SM)], two recent groundbreaking projects to provide global, real-time “event data” that take advantage of automated coding from news media have gained widespread recognition: International Crisis Early Warning System (ICEWS), maintained by Lockheed Martin, and Global Data on Events Language and Tone (GDELT), developed and maintained by Kalev Leetaru at Georgetown University (2, 3). The scale of these programs is unprecedented, and their promise has been reflected in the attention they have received from scholars, media, and governments. However, they suffer from major issues with respect to reliability and validity. Opportunities exist to use new methods and to develop an infrastructure that will yield robust and reliable “big data” to study global events—from conflict to ecological change (3).

Improving election prediction internationally

Using global data for election predictions Assumptions underlying election result predictions have been questioned recently. Kennedy et al. assessed more than 650 executive office elections in over 85 countries and performed two live forecasting experiments. They analyzed a variety of potential predictors theorized to be of importance, ranging from economic performance to polling data. Elections were about 80 to 90% predictable, despite uncertainties with available data. Polling data were very important to successful prediction, although it was necessary to correct for systematic biases. Unexpectedly, economic indicators were only weakly predictive. As data sources improve and grow, predictive power is expected to increase. Science, this issue p. 515 Pooling data from global elections to build better models improves prediction. This study reports the results of a multiyear program to predict direct executive elections in a variety of countries from globally pooled data. We developed prediction models by means of an election data set covering 86 countries and more than 500 elections, and a separate data set with extensive polling data from 146 election rounds. We also participated in two live forecasting experiments. Our models correctly predicted 80 to 90% of elections in out-of-sample tests. The results suggest that global elections can be successfully modeled and that they are likely to become more predictable as more information becomes available in future elections. The results provide strong evidence for the impact of political institutions and incumbent advantage. They also provide evidence to support contentions about the importance of international linkage and aid. Direct evidence for economic indicators as predictors of election outcomes is relatively weak. The results suggest that, with some adjustments, global polling is a robust predictor of election outcomes, even in developing states. Implications of these findings after the latest U.S. presidential election are discussed.

Making useful conflict predictions

One of the major issues in predicting state failure is the relatively rare occurrence of event onset. This class skew problem can cause difficulties in both estimating a model and selecting a decision boundary. Since the publication of King & Zeng’s studies in 2001, scholars have utilized case-control methods to address this issue. This article builds on the landmark research of the Political Instability Task Force comparing the case-control approach to several other methods from the machine learning field and some original to this study. Case-control methods have several practical disadvantages and show no measurable advantages in prediction. The article also introduces cost-sensitive methods for determining a decision boundary. This explication raises questions about the Task Force’s formulation of a decision boundary and suggests methods for making useful predictions for policy. I find that the decision boundary chosen by the PITF implicitly assumes that the cost of intervention is about 7.7% of the cost of non-intervention when state failure will take place. These findings demonstrate that there is still much work to be done in predicting state failure, especially in limiting the number of false positives. More generally, it suggests caution in using accuracy as a measure of success when significant class imbalance exists in the data.

The role of supranational identity in promoting democratic values

This study sets forth a social identity explanation of the link between supranational identity and democratic values, and, thus, for cross-border value diffusion. This explanation is tested with data from 19 regions in Europe from the latest wave of the World Values Survey using multilevel ordered logit models. The results reveal a little-studied relationship between the supranational identification of respondents within the European framework and stronger support for the postmaterialist values that have been previously linked to successful democratization. Conversely, stronger national and local identification is associated with weaker support for these values. The use of multilevel methods also reveals several interesting patterns in more traditional explanations of the adoption of democratic values.

Expanding the Conversation: Multiplier Effects From a Deliberative Field Experiment

Do formal deliberative events influence larger patterns of political discussion and public opinion? Critics argue that only a tiny number of people can participate in any given gathering and that deliberation may not remedy—and may in fact exacerbate—inequalities. We assess these criticisms with an experimental design merging a formal deliberative session with data on participants’ social networks. We conducted a field experiment in which randomly selected constituents attended an online deliberative session with their U.S. Senator. We find that attending the deliberative session dramatically increased interpersonal political discussion on topics relating to the event. Importantly, after an extensive series of moderation checks, we find that no participant/nodal characteristics, or dyadic/network characteristics, conditioned these effects; this provides reassurance that observed, positive spillovers are not limited to certain portions of the citizenry. The results of our study suggest that even relatively small-scale deliberative encounters can have a broader effect in the mass public, and that these events are equal-opportunity multipliers.

Fading Colours? A Synthetic Comparative Case Study of the Impact of 'Colour Revolutions'

The “colour revolutions” sparked a wave of optimistic commentaries about democratization in semi-authoritarian states. Today, however, there is considerable debate over whether these “revolutions” produced real reform and what lessons can be drawn for future democratization efforts. We utilize a synthetic control method of comparative case studies to evaluate political and institutional changes following “colour revolutions.” The results show divergent patterns of reform. Serbia experienced the most thorough changes in democratization and control of corruption. Ukraines revolt increased democratic freedoms, but failed to control corruption. Georgias protests marginally improved the control of corruption, but did little to improve the political system. Kyrgyzstan appears to have become worse as a result of its revolution. The synthetic comparisons suggest that these divergent outcomes are largely due to influences present well in advance of political upheaval. These findings shed critical light on the sources of cyclical political change in semi-authoritarian countries and the effect of domestic structural factors on democracy promotion.

Supranational identity, democratic attitudes, and political participation: The EU and Moldova

Individuals self-categorize within an international context, and this supranational identity shapes expectations for their own political system. Those individuals in postCommunist countries who believe that their country’s primary international orientation should be towards the European Union have a more participatory view of politics and are more politically active. This study develops a social identity explanation of political behavior, and tests the explanation utilizing maximum likelihood and non-parametric matching methods in a 2004 survey of political attitudes in Moldova. Moldova’s patchwork of identities, mixed historical legacy, and salience of supranational association make it an excellent case for study. The analysis finds strong support for the effect of supranational identity across a wide range of attitudes and activities.

A Colorless Election: The 2005 Presidential Election in Kazakhstan, and What It Means for the Future of the Opposition

The most surprising result of the presidential election in Kazakhstan was not Nazarbayev’s victory, but the weakness of the opposition.