Gizem Korkmaz

'Beating the news' with EMBERS: forecasting civil unrest using open source indicators

We describe the design, implementation, and evaluation of EMBERS, an automated, 24x7 continuous system for forecasting civil unrest across 10 countries of Latin America using open source indicators such as tweets, news sources, blogs, economic indicators, and other data sources. Unlike retrospective studies, EMBERS has been making forecasts into the future since Nov 2012 which have been (and continue to be) evaluated by an independent T&E team (MITRE). Of note, EMBERS has successfully forecast the June 2013 protests in Brazil and Feb 2014 violent protests in Venezuela. We outline the system architecture of EMBERS, individual models that leverage specific data sources, and a fusion and suppression engine that supports trading off specific evaluation criteria. EMBERS also provides an audit trail interface that enables the investigation of why specific predictions were made along with the data utilized for forecasting. Through numerous evaluations, we demonstrate the superiority of EMBERS over baserate methods and its capability to forecast significant societal happenings.

Forecasting Social Unrest Using Activity Cascades

Social unrest is endemic in many societies, and recent news has drawn attention to happenings in Latin America, the Middle East, and Eastern Europe. Civilian populations mobilize, sometimes spontaneously and sometimes in an organized manner, to raise awareness of key issues or to demand changes in governing or other organizational structures. It is of key interest to social scientists and policy makers to forecast civil unrest using indicators observed on media such as Twitter, news, and blogs. We present an event forecasting model using a notion of activity cascades in Twitter (proposed by Gonzalez-Bailon et al., 2011) to predict the occurrence of protests in three countries of Latin America: Brazil, Mexico, and Venezuela. The basic assumption is that the emergence of a suitably detected activity cascade is a precursor or a surrogate to a real protest event that will happen “on the ground.” Our model supports the theoretical characterization of large cascades using spectral properties and uses properties of detected cascades to forecast events. Experimental results on many datasets, including the recent June 2013 protests in Brazil, demonstrate the effectiveness of our approach.

Detecting and forecasting domestic political crises: a graph-based approach

Forecasting a domestic political crisis (DPC) in a country of interest is a very useful tool for social scientists and policy makers. A wealth of event data is now available for historical as well as prospective analysis. Using the publicly available GDELT dataset, we illustrate the use of frequent subgraph mining to identify signatures preceding DPCs, and the predictive utility of these signatures through both qualitative and quantitative results.

Combining Heterogeneous Data Sources for Civil Unrest Forecasting

Detecting and forecasting civil unrest events (protests, strikes, etc.) is of key interest to social scientists and policy makers because these events can lead to significant societal and cultural changes. We analyze protest dynamics in six countries of Latin America on a daily level, from November 2012 through August 2014, using multiple data sources that capture social, political and economic contexts within which civil unrest occurs. We use logistic regression models with Lasso to select a sparse feature set from our diverse datasets, in order to predict the probability of occurrence of civil unrest events in these countries. The models contain predictors extracted from social media sites (Twitter and blogs) and news sources, in addition to volume of requests to Tor, a widely-used anonymity network. Two political event databases and country-specific exchange rates are also used. Our forecasting models are evaluated using a Gold Standard Report (GSR), which is compiled by an independent group of social scientists and experts on Latin America. The experimental results, measured by F1-scores, are in the range 0.68 to 0.95, and demonstrate the efficacy of using a multi-source approach for predicting civil unrest. Case studies illustrate the insights into unrest events that are obtained with our methods.

Human vs. Automated Text Analysis: Estimating Positive and Negative Affect

Automated text analysis (ATA) has been a widely used tool for determining the sentiment of writing samples. However, it is unclear how ATA compares to human ratings of text when estimating affect. There are costs and benefits associated with each method, and comparing the two approaches will help determine which one provides the most useful and accurate results. This study uses 279 journal entries from individuals with chronic pain in order to estimate the positive and negative affect scores reported directly by participants. We use Lasso to select the features that are most predictive of affect. Our results indicate that the model combining human coders and ATA accounts for the most variance in self-reported positive affect scores, resulting in adjusted R-squared=0.36. For negative affect scores, we obtain a lower adjusted R-squared=0.30 with the combined model, however, ATA results in significantly higher adjusted R-squared=0.27 compared to the model using only human coders, R-squared=0.14. This suggests that utilizing human coders may be the most beneficial when the focus is on positive affect, but automated text analysis may be sufficient when studying negative affect.

Multi-source models for civil unrest forecasting

Civil unrest events (protests, strikes, and “occupy” events) range from small, nonviolent protests that address specific issues to events that turn into large-scale riots. Detecting and forecasting these events is of key interest to social scientists and policy makers because they can lead to significant societal and cultural changes. We forecast civil unrest events in six countries in Latin America on a daily basis, from November 2012 through August 2014, using multiple data sources that capture social, political and economic contexts within which civil unrest occurs. The models contain predictors extracted from social media sites (Twitter and blogs) and news sources, in addition to volume of requests to Tor, a widely used anonymity network. Two political event databases and country-specific exchange rates are also used. Our forecasting models are evaluated using a Gold Standard Report, which is compiled by an independent group of social scientists and subject matter experts. We use logistic regression models with Lasso to select a sparse feature set from our diverse datasets. The experimental results, measured by F1-scores, are in the range 0.68–0.95, and demonstrate the efficacy of using a multi-source approach for predicting civil unrest. Case studies illustrate the insights into unrest events that are obtained with our method. The ablation study demonstrates the relative value of data sources for prediction. We find that social media and news are more informative than other data sources, including the political event databases, and enhance the prediction performance. However, social media increases the variation in the performance metrics.

EDISON: a web application for computational health informatics at scale

SPublic health issues, from virus and disease transmission, to the spread of unhealthy behaviors (such as smoking and obesity) are global priorities. They can lead not only to fatalities, but also to decreased quality of life, large expenditures for health care, and the onset of other ailments. Here we present EDISON: a web-based modeling environment that can be used to perform complex computational experiments involving very general (epidemiological) contagion processes over social networks. EDISON is publicly accessible by scientists and domain experts interested in carrying out in-silico social and epidemiological experiments. EDISON is unique in that: (i) the experiments can be carried out at scale (populations may range from a few hundred to millions of agents), (ii) it is web-based with an easy to use UI---it is specifically designed for use by epidemiologists, social scientists, and (government) practitioners who are not computing experts, and (iii) it has a digital library that contains a large number of open source social networks and many behavior models. EDISON has been used for a number of theoretical studies. We illustrate its utility through case studies of disease and behavioral contagion transmission.

Spreading of social contagions without key players

Contagion models have been used to study the spread of social behavior among agents of a networked population. Examples include information diffusion, social influence, and participation in collective action (e.g., protests). Key players, which are typically agents characterized by structural properties of the underlying network (e.g., high degree, high core number or high centrality) are considered important for spreading social contagions. In this paper, we ask whether contagions can propagate through a population that is devoid of key players. We justify the use of Erdős-Rényi random graphs as a representation of unstructured populations that lack key players, and investigate whether complex contagions—those requiring reinforcement—can spread on them. We demonstrate that two game-theoretic contagion models that utilize common knowledge for collective action can readily spread such contagions, thus differing significantly from classic complex contagion models. We compare contagion dynamics results on unstructured networks to those on more typically-studied, structured social networks to understand the role of network structure. We test the classic complex contagion and the two game-theoretic models with a total of 18 networks that range over five orders of magnitude in size and have different structural properties. The two common knowledge models are also contrasted to understand the effects of different modeling assumptions on dynamics. We show that under a wide range of conditions, these two models produce markedly different results. Finally, we demonstrate that the disparity between classic complex contagion and common knowledge models persists as network size increases.

Using text to predict psychological and physical health: A comparison of human raters and computerized text analysis

Abstract Given the wide-spread use of social media, text analysis has emerged as a promising way to gather information about individuals. However, it is still unclear which method of text analysis is best for determining different types of information. This study compared the utility of automated text analysis (LIWC) with human raters in predicting self-reported psychological and physical health. Expressive writing essays from chronic pain patients were used from a previous online intervention study. Results indicate that human ratings added predictive power above and beyond the LIWC on measures of depression. However, the LIWC was almost as proficient as human raters when predicting pain catastrophizing and illness intrusiveness. Neither the LIWC nor human ratings were good predictors of pain severity and life satisfaction. Overall the utility of automated text analysis over human raters depends on the individual characteristic being measured.

Can social contagion spread without key players

Contagion models have been used to study the spread of social behavior among agents of a population, such as information diffusion, social influence, and participation to collective action (e.g., protests). Key players, which are typically high-degree, -k-core or -centrality agents in a networked population, are considered important for spreading social contagions. In this paper, we ask whether contagions can propagate through a population that is void of key players. We use Erdos-Renyi random graphs as a representation of unstructured populations that lack key players, and investigate whether complex contagions - those requiring reinforcement - can spread on them. We demonstrate that two game-theoretic contagion models that utilize common knowledge for collective action can readily spread such contagions, which is a significant difference from classic complex contagion models. We compare contagion dynamics results on unstructured networks to those on more typically-studied, structured social networks to understand the role of network structure. We test a total of 14 networks. The two common knowledge models are also contrasted to understand the effects of different modeling assumptions on dynamics. We show that under a wide range of conditions, these two models produce markedly different results.