Social Bots and Social Media Manipulation in 2020: The Year in Review
Ho-Chun Herbert Chang, Emily Chen, Meiqing Zhang, Goran Muric, Emilio Ferrara
SSocial Bots and Social Media Manipulation in 2020:The Year in Review
Ho-Chun Herbert Chang ∗ a , Emily Chen ∗ b,c , Meiqing Zhang ∗ a , Goran Muric ∗ b , andEmilio Ferrara a,b,ca USC Annenberg School of Communication b USC Information Sciences Institute c USC Department of Computer Science * These authors contributed equally to this work
Abstract
The year 2020 will be remembered for two events of global significance: the COVID-19pandemic and 2020 U.S. Presidential Election. In this chapter, we summarize recentstudies using large public Twitter data sets on these issues. We have three primary ob-jectives. First, we delineate epistemological and practical considerations when combin-ing the traditions of computational research and social science research. A sensible bal-ance should be struck when the stakes are high between advancing social theory andconcrete, timely reporting of ongoing events. We additionally comment on the compu-tational challenges of gleaning insight from large amounts of social media data. Second,we characterize the role of social bots in social media manipulation around the discourseon the COVID-19 pandemic and 2020 U.S. Presidential Election. Third, we compare re-sults from 2020 to prior years to note that, although bot accounts still contribute to theemergence of echo-chambers, there is a transition from state-sponsored campaigns to do-mestically emergent sources of distortion. Furthermore, issues of public health can beconfounded by political orientation, especially from localized communities of actors whospread misinformation. We conclude that automation and social media manipulationpose issues to a healthy and democratic discourse, precisely because they distort repre-sentation of pluralism within the public sphere.
In 2013, the World Economic Forum (WEF)’s annual
Global Risk report highlighted themultidimensional problems of misinformation in a highly connected world [1]. The WEFdescribed one of the first large-scale misinformation instances that shocked America: anevent from 1938, when thousands of Americans confused a radio adaptation of the H.G.Wells novel
The War of the Worlds with an official news broadcast. Many started pan-icking, in the belief that the United States had been invaded by Martians.1 a r X i v : . [ c s . S I] F e b oday, it would be hard for a radio broadcast to cause comparably widespread confu-sion. First, broadcasters have learned to be more cautious and responsible; and second,listeners have learned to be more savvy and sceptical. However, with social media, weare witnessing comparable phenomena on a global scale and with severe geopolitical con-sequences. A relatively abrupt transition from a world in which few traditional mediaoutlets dominated popular discourse, to a multicentric highly-connected world where in-formation consumers and producers coalesced into one, can bring unparalleled challengesand unforeseen side effects. A sudden democratization in the media ecosystem enableseveryone online to broadcast their ideas to potentially massive audiences, thus allowingcontent that is not necessarily moderated or curated to be broadly accessible. Extremeopinions can become increasingly more visible and fringe groups can start gaining un-precedented attention. Eccentric ideas that would otherwise garner little support withinfringe communities, now could make their way into the mainstream. Furthermore, thefree canvas of highly connected social media systems has been reportedly exploited bymalicious actors, including foreign governments and state-sponsored groups, willing todeliberately misinform for their financial or political gain.Nowadays, the use of social media to spread false news, provoke anxiety and incite fearfor political reasons has been demonstrated around the World [2, 3, 4, 5, 6, 7, 8, 9]. How-ever, social media manipulation is not exclusively tied to political discourse. Public healthcan also be endangered by the spread of false information. For instance, in January 2019,panic erupted in Mumbai schools caused by social media rumors that the vaccines were aplot by the government to sterilize Muslim children: That led to only 50% of those whowere expected to be vaccinated to actually get the vaccine [10].Researchers from the Democracy Fund and
Omidyar Network in their investigative re-port titled “
Is Social Media a Threat to Democracy? ”, [11] warn that the fundamentalprinciples underlying democracy —trust, informed dialogue, a shared sense of reality,mutual consent, and participation— are being put to the ultimate litmus test by cer-tain features and mechanisms of social media. They point out six main issues: 1) Echochambers, polarization, and hyper-partisanship; 2) Spread of false and/or misleadinginformation; 3) Conversion of popularity into legitimacy; 4) Manipulation by populistleaders, governments, and fringe actors; 5) Personal data capture and targeted messag-ing/advertising; and 6) Disruption of the public square.As a matter of research, these six issues can be studied through multiple academic andepistemological angles.
Computational Social Science has evolved swiftly in the past fewyears: Students of the social sciences are becoming masters of machine learning, whilestudents of computer science interested in social phenomenon develop domain expertisein sociology, political science, and communication. More so than a methodological evolu-tion, it is a shared critical interest in the growing impact social media platforms play inthe very fabric of our society. A special issue documenting “Dark Participation” [12] con-trasts various issues of misinformation across different governments [13]. Scholars pointout an increasingly shared challenge: the balance of combating foreign interference with-out compromising domestic free speech [14]. The resolution of these issues requires iter-ation between computational insights and policy-makers, as any type of intervention willinadvertently attract critiques of suppression or create unforeseen side effects.2 .1 Focus of this Chapter
In this chapter, we focus on spread of false and/or misleading information across twosalient dimensions of social media manipulation, namely (i) automation (e.g., prevalenceof bots), and (ii) distortion (misinformation, disinformation, injection of conspiracies orrumors). We provide direct insight into two case studies: a) the COVID-19 pandemicand b) the 2020 U.S. Presidential Election. We detail the many aspects of large-scalecomputational projects: a) tracking and cleaning billions of tweets, b) enriching the datathrough state-of-the-art machine learning, and c) recommendation of actionable interven-tions in regards to platform governance and online speech policy.While misleading information can materialize in many different forms, it is often scru-tinized in the context of current events. Social media allows users to actively engage indiscourse in real-time, reacting to breaking news and contributing to the conversationsurrounding a particular topic or event with limited filters for what can or cannot beposted prior to publication. Although many social media companies have terms of ser-vices and automated filters that remove posts that violate their community guidelines,many of these posts are either able to evade detection long enough such that a wideaudience has already seen or engaged with a post, or elude these automated or human-assisted filters completely.Politics and current events as a whole have created an environment that is rife and con-ducive to the spread of misleading information. Regardless of the alacrity of a post’sremoval and the original poster’s broader visibility, as long as misinformation has beenposted online, there is the potential for this information to have been seen and conse-quently consumed by others who can further disseminate it. Social media companiessuch as Twitter, Facebook and YouTube have recently begun active campaigns to reducethe spread of misinformation and conspiracy theories [15, 16]. Fact checkers activelymonitor rumors and events. However, the virality and speed at which this informationpropagates makes it difficult to catch and contain, particularly as alternative social me-dia platforms, such as Parler and Gab, with fewer mitigation measures emerge to allowfurther misinformation circulation in the ecosystem [17, 18].With the recent 2020 U.S. Presidential Election and ongoing COVID-19 pandemic, theneed to understand the distortion of information becomes ever more urgent. When wediscuss distortion of information, we note a subtle but important distinction between(a) misinformation, the organic spread of false or inaccurate information, and (b) disin-formation, the deliberate spread of misinformation. Although the two terms are closelyrelated, the nuance of purpose differentiates the intent of the distortion. Disinformation,in particular, is often promulgated on social media platforms not only by human users,but also by bots [19, 20, 21]. A “bot”, which is shorthand for the word “software robot”,is a software based unit whose actions are controlled by software instead of human inter-vention. While there are many disciplines that leverage this term, we use the term “bot”in the context of “social bots”, which are social media accounts that are either fully con-trolled by software or have some level of human intervention (semi-automated) [22].
The term computational social science evokes not just two disciplines, but their ownpractices and traditions. In the following, we highlight some important epistemological3oncepts that inform the study of social media manipulation through the lens of compu-tational and social science theory.
Although both inductive and deductive reasoning is common in social science researchmethods, quantitative social science research traditionally holds deductive methods inhigher regard. A deductive approach starts from theories and uses data to test the hy-potheses stemmed from the theories. Computational social science work conducted bycomputer scientists often exhibits a data-driven, inductive approach. However, as datascience and domain expertise in the social sciences are brought together, computationalsocial science bears great promise to reconcile inductive and deductive reasoning [23].Exploring large volumes of data, even sans prior theoretical assumptions, may yield newinsights or surprising evidence. The findings from this initial, data-driven step will guideus to discern emerging hypotheses and collect new data to test them. This is called the abductive analysis [24]. It starts with observations, which serve to generate new hypothe-ses or filter existing hypotheses. The promising hypotheses emerged from data analysiscan then be tested deductively with new data.This deductive approach can be used to study the relationship between social mediaand democratic discourse, which is hardly a direct or linear one. Social media do notinherently undermine or improve democracy. Instead, they affects the quality of democ-racy through multiple mechanisms such as political polarization and disinformation [25].These intermediate variables operate in varying contexts shaped by political institutions,political culture and media ecosystems. Therefore, the effects of social media on democ-racy differ despite the same technological affordances [26]. The political system, ideolog-ical distribution, how political elites use social media and the behavioral patterns of dif-ferent political actors in a given context interact with one another to determine whetherpolitical polarization and disinformation are amplified on social media platforms. The in-teractions amongst all potential political, social and technological variables form a com-plex system. Data exploration and analysis can help uncover crucial variables operatingin a specific context. Our case studies of misinformation in the context of the COVID-19pandemic and the 2020 U.S. Presidential Election described next will reveal significantfactors underlying the relationship between social media use and democracy in the U.S.context and help identify social scientific hypotheses that are worth further investigation.
We recently found ourselves at the intersection of two important events that have changedthe way the world has functioned. 2020 was already going to be a big year for U.S. poli-tics due to the contentious nature of the current political climate. The United States hasbecome more polarized, leading to high anticipation over whether or not the then incum-bent President Trump would win re-election. While Trump cinched the Republican nom-ination, there was a high anticipated battle for the Democratic Presidential nominee [27].In the midst of the political furor, in late December 2019, the first cases of novel SARS-COV-2 Coronavirus (whose caused disease was later named COVID-19) were reported4rom Wuhan, China [28]. As the world began to understand the severity of the illness,whose status was later classified as a pandemic, many countries began to impose lock-downs in attempts to contain the outbreaks [29, 28].For years, our conversations had already been shifting toward online, with the advent ofsocial media platforms that foster environments for sharing information. Social mediahas also become more integrated into the fabric of political communication [30]. Withthe lockdowns that closed offices and forbade gatherings, the discourse surrounding cur-rent events was pushed even further onto online platforms [31, 32, 33, 34]. This created abreeding ground for potential misinformation and disinformation campaigns to flourish,particularly surrounding health initiatives during a time of heightened political tensionsduring the 2020 U.S. Presidential Election [35]. In our paper published in the
HarvardMisinformation Review special issue on
U.S. Elections and Disinformation , we studythe politicization of and misinformation surrounding health narratives during this time.We found several major narratives present in our data, and further explored two health-related narratives that were highly politicized: mask wearing and mail-in ballots . We have been actively collecting and maintaining two publicly released Twitter datasets:one focusing on COVID-19 related discourse and the other on the 2020 U.S. PresidentialElection [36, 37]. We began the former collection in late January 2020 and the latter inlate May 2019. These tweets are collected using the Twitter streaming API, which en-ables us to gather tweets that match specific keywords or accounts [38]. We note herethat, at the time of this writing, the free Twitter streaming API only returns 1% of thefull Twitter data stream. Because of this limitation, we are unable to collect all tweetsrelevant to COVID-19 and the elections. However, the 1% returned is still a representa-tive sample of the discourse occurring during that day [39].In this particular case study, we capitalized on both our COVID-19 (v1.12) and elections(v1.3) Twitter datasets, with a focus on the time period from March 1, 2020 through Au-gust 30, 2020. At the time that this study was conducted, we had only processed ourelection data from March 1, 2020 onward. This timeframe covers from Super Tuesday,when a significant number of states hold their primaries, through the end of the Demo-cratic presidential primaries.
We first filtered our COVID-19 dataset for keywords related to the elections, includ-ing the last names of the candidates as well as general elections-related keywords (vote,mailin, mail-in, mail in, ballot). We then conducted Latent Dirichlet allocation (LDA) toidentify 8 topics present within the data, using the highest coherence score to determinethe optimal number of topics [40]. After sorting tweets into their most probable topic,we leveraged the most frequent hashtags, keywords, bigrams and trigrams to understandthe narratives within each identified topic. Four broader narratives emerged: generalCoronavirus discourse, lockdowns, mask wearing and mail-in balloting. We then filteredour general COVID-19 and elections dataset for tweets that contained at least one of theaforementioned elections-related keywords and a representative keyword or hashtag fromthe four major identified topics. This netted us a final dataset of 67,846,555 tweets, with50,536,524 general Coronavirus tweets, 619,914 regarding lockdowns, 1,283,450 tweets onmask-wearing and 5,900,737 on mail-in balloting.
We first wanted to understand how discourse surrounding our four narratives (Coron-avirus, lockdowns, mask wearing and mail in balloting) fluctuated over time (see figures1 and 2). We tracked the percentage of all collected tweets on a particular day that con-tained selected keywords and hashtags that are representative of each narrative.Figure 1: Coronavirus and mail-in ballot related tweets within primaries related tweets,plotted as a 3-day rolling average of the percentage of primary related tweets. State ab-breviations aligned with the day on which the respective state conducted their Demo-cratic primary.Figure 2: Lockdown and mask related tweets within primaries related tweets, plotted asa 3-day rolling average of the percentage of primary related tweets. State abbreviationsaligned with the day on which the respective state conducted their Democratic primary.6 oronavirus.
The pervasiveness of Coronavirus-related tweets in our Twitter datasetis by construction hence unsurprising. Not only was our COVID-19 dataset trackingCoronavirus-related keywords, but this topic has dominated political discourse in theUnited States since the first case was reported in Washington state on January 21, 2020.In this narrative, we find several prevalent misinformation subnarratives — including thebelief that COVID-19 is a hoax created by the Democratic party and that COVID-19will disappear by itself [41]. This has also been driven in tandem with the anti-vaccinemovement, which has staged protests at COVID-19 vaccine distribution locations [42].Hydroxychloroquine (HCQ) also became a highly divisive topic within the Twitter com-munity debating its effectiveness as treatment for COVID-19. During a press conference,then-President Trump stated that he was taking HCQ as a preventative measure [43].The
United States Food and Drug Administration (FDA) initially issued an emergencyuse authorization (EUA) for HCQ and the
World Health Organization included it in itstreatment trials. However, the EUA was rescinded and the trials halted as results beganto show that HCQ was not an effective treatment or preventative for COVID-19 [44, 45].The controversy surrounding HCQ shows a shift in factuality surrounding the viabilityof HCQ, as it was initially unknown if HCQ was indeed viable. Information can developinto misinformation as its factuality changes, which further emphasizes the dangers ofspreading medical information without substantive, corroborated scientific evidence. De-spite evidence showing that HCQ should not be used as a treatment for COVID-19, thisnarrative promoting HCQ continued to spread and for many to seek this treatment.
Mail-in Ballots.
As fears surrounding COVID-19 began to grow throughout the UnitedStates, one of the major concerns with the U.S. Democratic primaries and the upcom-ing Presidential Election was how voters would be able to vote safely [46]. This causedmany states to begin promoting mail-in ballots as a way to safely vote from home dur-ing the Democratic primaries. In August 2020, then-President Trump appointed Post-master Louis DeJoy began reappropriating the United States Postal Service resources,making budget cuts and changing standard mail delivery protocols. This led to a signif-icant slowdown of mail being processed and delivered, including the delivery of ballots,particularly as the U.S. began to prepare for the Presidential Election [47, 48].While many were advocating for mail in ballots to be more widely used as COVID-19precaution, others pushed the narrative that mail in ballots would increase ballot fraud.This misinformation has been proven false by fact checkers, as no evidence in previ-ous election cycles have indicated that mail in ballots or absentee ballots increase voterfraud [49]. This misinformation narrative that was incubating during the primaries sea-son became an even larger misinformation campaign during the U.S. Presidential Elec-tion.
Lock downs and Masking.
Finally, lock downs and masks were also major themesin our dataset. This is expected, as the United States began to implement social distanc-ing ordinances, such as stay-at-home orders, in March 2020. As more states held theirprimaries, we see that mentions of lock downs and masks increase, suggesting that on-line conversation surrounding social distancing and mask wearing is driven by currentevents. This included misinformation narratives that claimed masks are ineffective andharmful towards one’s health, when studies have shown that masks can effectively reduceCOVID-19 transmission rates [50, 49, 51]. 7 .1.4 Echo chambers and populist leaders
Out of the four narratives, we further investigate mask-wearing and mail-in balloting,as these two topics contain health-related discourse that became highly politicized andsubsequently prone to misinformation. One of the more startling findings was the sourceof misinformation, specifically the communities in which distortions were concentrated.Figure 3 shows the network topology of Twitter users who have engaged in COVID-19related elections discourse (see [52] for details on the methodology to generate this plot).Figure 3: Community structure of COVID-19 related elections discourse [52]. a) Showsthe political diet of users. b) shows where general misinformation is found. c) shows thedistribution of mail-in voting and mask wearing, and the position of the Twitter users.Figure 3a shows the users in our dataset, each data point being colored by “political in-formation diet”. In order to categorize a user’s information diet, we labeled users whohave shared at least 10 posts containing URLs that have been pre-tagged by the Media-Bias/Fact-Check database. This database contains a political leanings-tagged list ofcommonly-shared domains (left, center-left, center, center-right and right). We foundthat the majority of the users are center or left-leaning. However, there is also a fairlyclear distinction between more homogeneous conservative and liberal clusters near thetop of the topology. This suggests that while the majority of users ingest a variety of in-formation from both sides of the aisle, there are still clear signs of polarization based onpolitical views that can be detected in the network topology. This polarization of highlyconnected clusters also indicates the presence of “echo chambers” [53, 54].Media-Bias/Fact-Check also contains a list of domains which they deem “questionablesources”, or sources that are known to prompt conspiracy theories and misinformation.We use this to tag each user with both their political affiliation (Left or Right) and their https://mediabiasfactcheck.com/ There is a well known saying that “the first casualty of war is truth”. In times of unusualsocial tensions caused by the political struggle with relatively high stakes, the prolifer-ation of false news, misinformation and other sorts of media manipulation is to be ex-pected. The importance of voter competence is one of the postulates of modern democ-racy [61, 62] and information vacuums can undermine electoral accountability [63]. Anideal democracy assumes an informed and rational voter, but the former aspect is some-thing that can be undermined or compromised. During the 2020 U.S. Presidential Elec-tion, social media manipulation has been observed in the form of (i) automation, thatis the evidence for adoption of automated accounts governed predominantly by softwarerather than human users, and (ii) distortion, in particular of salient narratives of discus-sion of political events, e.g., with the injection of inaccurate information, conspiracies orrumors. In the following, we describe ours and others’ findings in this context.9 .2.1 Dataset
For this study, we again leverage one of our ongoing and publicly released Twitter datasetscentered around the 2020 U.S. Presidential Election. Please refer to Section 3.1.1 formore details on the collection methods; this particular dataset is further described in[36]. While this dataset now has over 1.2 billion tweets, we focused on tweets posted be-tween June 20, 2020 and September 9, 2020 in advance of the November 3, 2020 election.This subset yielded 240 million tweets and 2 TB of raw data. The period of observationincludes several salient real-world political events, such as the
Democratic National Com-mittee (DNC) and
Republican National Committee (RNC) conventions.
The term bot (shorthand for robot) in Computational Social Science commonly refers tofully automated or semi-automated accounts on social media platforms [22]. Researchinto automation on social media platforms has spurned its own sub-field not only incomputational social sciences but in social media research at large [22, 19, 64, 65, 66].One of the major challenges with automation is the ability to detect accounts that arebots as opposed to accounts fully operated by humans. Although there are benign ac-counts that publicly advertise the fact that they are automated, bots used for maliciouspurposes try to evade detection. As platforms and researchers study the behavior of botsand devise algorithms and systems that are able to automatically flag accounts as bots,bot developers are also actively developing new systems to subvert these detection at-tempts by mimicking behavioral signals of human accounts [67, 68]Botometer is a tool developed and released by researchers at
Indiana University , as partof the
Observatory on Social Media (OSoMe [69]), that allows users to input a Twitteruser’s screen name, and returns a score of how likely an account is to be automated. These scores range from 0 to 5, with 0 indicating that the account has been labeled asmost likely human and 5 indicating that the account is most likely a bot account. Wewill be referring to accounts that are most likely human accounts as “human” and bot-like accounts as “bots” for brevity. Botometer itself has gone through several iterations,with the most recent version Botometer v4 released in September 2020 [67]. Botometerv4 extracts thousands of features from an input account and leverages machine learningmodels trained on a large repository of labeled tweets to predict the likelihood of an ac-count being a bot. Botometer v4 [68] can identify different types of bots, including botsthat are fake followers, spammers and astroturfers [66, 70].
In the following analysis, we leveraged Botometer v3 [66], as that was the latest versionat the time we performed our study [71]. We tagged 32 percent of the users within ourcomplete dataset, and removed all tweets not posted by the users for whom we have botscores for. We labeled the top decile of users according to Botometer scores as “bots”and the bottom decile as “humans” [72]. Our final dataset contains more that four mil-lion tweets posted by bots and more than one million tweets posted by humans. We https://botometer.osome.iu.edu/ Next, we broaden an analysis to distortion , an umbrella concept that also includes com-pletely fabricated narratives that do not have a hold in reality. Fake news are an exam-ple of distorted narratives and are conceptualized as distorted signals uncorrelated withthe truth [75]. To avoid the conundrum of establishing what is true and what is false toqualify a piece of information as fake news (or not), in this study we focus on conspir-acy theories, another typical example of distorted narratives. Conspiracy theories canbe (and most often are) based upon falsity, rumors, or unverifiable information that re-sist falsification; other times they are instead postulated upon rhetoric, divisive ideology,and circular reasoning based on prejudice or uncorroborated (but not necessarily false)evidence. Conspiracies can be shared by users or groups with the aim to deliberately de-ceive or indoctrinate unsuspecting individuals who genuinely believe in such claims [76].Conspiracy theories are attempts to explain the ultimate causes of significant social andpolitical events and circumstances with claims of secret plots by powerful actors. Whileoften thought of as addressing governments, conspiracy theories could accuse any group12erceived as powerful and malevolent [77]. They evolve and change over time, depend-ing on the current important events. Upon manual inspection, we found that some of themost prominent conspiracy theories and groups in our dataset revolve around topics suchas: objections to vaccinations, false claims related to 5G technology, a plethora of Coro-navirus related false claims and the flat earth movement [72]. Opinion polls carried outaround the world reveal that substantial proportions of population readily admit to be-lieving in some kind of conspiracy theories [78]. In the context of democratic processesincluding the 2020 U.S. Presidential Election, the proliferation of political conspiratorialnarratives could have an adverse effect on political discourse and democracy.In our analysis, we focused on three main conspiracy groups:1.
QAnon conspiracies:
A far-right conspiracy movement whose theory suggeststhat President Trump has been battling against a Satan worshipping global childsex-trafficking ring and an anonymous source called ’Q’ is cryptically providing se-cret information about the ring [79]. The users who support such ideas frequentlyuse hashtags such as ”@potus @realDonaldTrump was indeed correct,the beruit fire was hit bya missile, oh and to the rest of you calling this fake,you are not a qanonyou need to go ahead and change to your real handles u liberal scumbagsjust purpously put out misinfo and exposed yourselves,thnxnan””I’ve seen enough. It’s time to “gate” conspiracies: Another indicator of conspiratorial content is signalled bythe suffix ’-gate’ with theories such as pizzagate, a debunked claim that connectsseveral high-ranking Democratic Party officials and U.S. restaurants with an al-leged human trafficking and child sex ring. The examples of the typical conspirato-rial tweets related to these two conspiracies are: ” Covid conspiracies:
A plethora of false claims related to the Coronavirus emergedright after the pandemic was announced. They are mostly related to scale of thepandemic and the origin, prevention, diagnosis, and treatment of the disease. Thefalse claims typically go alongside the hashtags such as @fyjackson @rickyb_sports @rhus00 @KamalaHarris @realDonaldTrumpThe plandemic is a leftist design. And it’s backfiring on them. We’vehad an effective treatment for COVID-19, the entire time. Leftists hateTrump so much, they are willing to murder 10’s of thousands of Ameri-cans to try to make him look bad. The jig is up.””The AUS Govt is complicit in the global scare
During the period preceding the 2020 U.S. Presidential Election,
QAnon related materialhas more highly active and engaged users than other narratives. This is measured by theaverage number of tweets an active user has made on a topic. For example, the mostfrequently used hashtag,
QAnon community has a more active user base strongly dedicated to the narrative.When we analyze how the conspiratorial narratives are endorsed by the users, condi-tioned upon where they fall on the political spectrum, we discover that conspiratorialideas are strongly skewed to the right. Almost a quarter of users who endorse predom-inantly right-leaning media platforms are likely to engage in sharing conspiracy narra-tives. Conversely, out of all users who endorse left-leaning media, approximately two per-cent are likely to share conspiracy narratives.Additionally, we explore the usage of conspiracy language among automated accounts.Bots can appear across the political spectrum and are likely to endorse polarizing views.Therefore, they are likely to be engaged in sharing heavily discussed topics includingconspiratorial narratives. Around 13% of Twitter accounts that endorse some conspir-acy theory are likely bots. This is significantly more than users who never share con-spiracy narratives, which have only 5% of automated accounts. It is possible that suchobservations are in part the byproduct of the fact that bots are programmed to interactwith more engaging content, and inflammatory topics such as conspiracy theories providefertile ground for engagement [80]. On the other hand, bot activity can inflate certainnarratives and make them popular.The narratives of these conspiracy theories during the 2020 U.S. Presidential Electioncall attention to the so-called “new conspiracism” and the partisan differences in prac-ticing it [81]. Rosenblum and Muirhead argue that the new conspiracism in the contem-porary age is “conspiracy without theory”. Whereas the “classic conspiracy theory” stillstrives to collect evidence, find patterns and logical explanations to construct a “theory”of how malignant forces are plotting to do harm, the new conspiracism skips the bur-dens of “theory construction” and advances itself by bare assertion and repetition [81].Repetition produces familiarity, which in turn increases acceptance [82, 83]. A conspir-acy becomes credible to its audience, simply because many people are repeating it [81].The partisan asymmetry in the circulation of conspiracy theories is also consistent withothers’ claims that the new conspiracism is asymmetrically aligned with the radical rightin the U.S. context [26, 81], although this species of conspiracism is not ideologically at-tached to liberals or conservatives [81]. Our analysis shows the promising direction oftesting the theories of asymmetrical polarization and exploring the nature and conse-quences of asymmetrical media ecosystem, ideally using multi-platform data.The findings about the bot behaviors relative to humans on Twitter reveal some patterns14f conspiracy transmission in the 2020 U.S. Presidential Election. Their high-volume andecho-chamber retweeting activities attest to the role that automation plays in stoking thenew conspiracism. Bots are capable of retweeting and repeating the same information ef-ficiently. However, bots are not solely to blame for the prevalence of conspiracy-theorystories. False information are found to spread faster than true information due to the hu-man tendency to retweet it. A comprehensive study conducted by Vosoughi et al. com-pared the diffusion of verified true and false news stories on Twitter from 2006 to 2017.They discovered that falsity travels wider and deeper than truth, even after bots wereremoved, suggesting that humans are more likely to retweet false rumors than true infor-mation. Among all topics, political rumors are particularly viral. False rumors peakedbefore and around the 2012 and 2016 U.S. Presidential Election [84]. Additionally, auto-mated accounts that are part of an organized campaign can purposely propel some of theconspiracy narratives, further polarizing the political discourse.Although bots present a threat to the ideal, well-informed democratic citizenship, thesusceptibility of humans to believing and spreading false information is worth equal at-tention. Further examinations of how distorted narratives go viral will help us betterdiagnose the problem. Some new research points to the hypothesis that the nature andstructure of false rumors and conspiracy-theory stories evoke human interest. For ex-ample, Vosoughi et al. suggested that false rumors tend to be more novel, hence moresalient. False rumors also elicit stronger emotions of surprise and disgust [84]. Tangher-lini et al. studied the conspiracy theory narrative framework using the cases of
Bridge-gate and
Pizzagate . They deconstructed those stories into multi-scale narrative networksand found that conspiracy theories are composed of a small number of entities, multipleinterconnected domains and separable disjoint subgraphs. By construction, conspiracytheories can form and stabilize faster. In contrast, the unfolding of true conspiracy sto-ries will admit new evidence and result in a denser network over time [85]. Therefore,true stories could be at a disadvantage when competing with false rumors as they areless stable and grow in complexity as events develop.
In this chapter, we presented the findings that emerged from two significant events of2020. In the first study, we showed how political identity aligns with narratives of pub-lic health. Four narratives were identified: (i) mail-in ballots, (ii) reference to the pan-demic, (iii) lock-downs, and (iv) mask-wearing. Spikes in these narratives were foundto be driven by predetermined events, predominantly the primaries. When observingthe policy stance of mail-in ballots and mask-wearing, we observe users against mask-wearing and mail-in ballots arise from a dense group of conservative users separate fromthe majority. Topological distinctions between these two groups are further observed.Further details are found in our recent paper [14].When investigating the 2020 U.S. Presidential Election more broadly, we find bots notonly generate much higher volumes of election-related tweets per capita, but also tweetprimarily within their own political lines (more than 80% for both left- and right-leaningcommunities). An analysis of content from QAnon-driven conspiracies, politicized “gate”-related, and COVID-related conspiracies suggested that users self-organize to promulgatefalse information and also leverage automation to amplify hyperpartizan and conspirato-rial news sites: more details are discussed in our associated study [72].15hat do these results tell us? First, although bots still generate significant distortionsin volume and self-reinforcement across party lines as observed in the 2016 U.S. Presi-dential Election [2], this is overshadowed by the self-organization of extremism and “newconspiracism” in the public sphere. A further contrast is the shift from foreign interfer-ence in 2016 to domestic, ingrown social media manipulation in 2020. This phenomenoncan be observed across a variety of case studies, including the populism in EU [86], xeno-phobia in Russia, hate speech in Germany [87], and foreign interference in Taiwan [14].Finally, the case study of COVID-19 demonstrates the interplay between public healthand politics on a national level. In the past, computational studies on anti-vaccinationfocused on smaller, community level scales [42]. Given the high levels of alignment be-tween political information diet and health misinformation, the polarization and sub-sequent distortions not only can have ramifications on the democratic process, but alsotangible effects on public health.
References [1] World Economic Forum. Global Risks 2013. Technical report, World EconomicForum, 2013.[2] Alessandro Bessi and Emilio Ferrara. Social bots distort the 2016 U.S. Presidentialelection online discussion.
First Monday , 21(11), nov 2016.[3] Chengcheng Shao, Pik-Mai Hui, Lei Wang, Xinwen Jiang, Alessandro Flammini,Filippo Menczer, and Giovanni Luca Ciampaglia. Anatomy of an online misinforma-tion network.
PLOS ONE , 13(4):e0196087, apr 2018.[4] Adam Badawy, Kristina Lerman, and Emilio Ferrara. Who falls for online politicalmanipulation? In
Companion Proceedings of The 2019 World Wide Web Conference ,pages 162–168, 2019.[5] Leon Derczynski, Torben Oskar Albert-Lindqvist, Marius Venø Bendsen, NannaInie, Viktor Due Pedersen, and Jens Egholm Pedersen. Misinformation on Twit-ter During the Danish National Election: A Case Study. TTO Conference Ltd, oct2019.[6] Sumitra Badrinathan. Educative Interventions to Combat Misinformation: EvidenceFrom a Field Experiment in India. aug 2020.[7] Who Believed Misinformation during the 2019 Indonesian Election?
Asian Survey ,60(6):1029–1043, dec 2020.[8] Ralph Schroeder. Even in Sweden?
Nordic Journal of Media Studies , 2(1):97–108,jun 2020.[9] Luca Luceri, Ashok Deb, Adam Badawy, and Emilio Ferrara. Red bots do it better:Comparative analysis of social bot partisan behavior. In
Companion Proceedings ofthe 2019 World Wide Web Conference , pages 1007–1012, 2019.[10] Heidi J. Larson.
Stuck - How Vaccine Rumors Start and Why They Don’t go Away .Oxford University Press, 1 edition, 2020.[11] Anamitra Deb, Stacy Donohue, and Tom Glaisyer. Is Social Media a Threat toDemocracy? Technical report, Omydiar Group, 2017.1612] Thorsten Quandt. Dark participation.
Media and Communication , 6(4):36–48, 2018.[13] Thorsten Quandt. Can we hide in shadows when the times are dark?
Media andCommunication , 9(1):84–87, 2021.[14] Ho-Chun Herbert Chang, Samar Haider, and Emilio Ferrara. Digital civic participa-tion and misinformation during the 2020 taiwanese presidential election.
Media andCommunication , 9(1):144–157, 2021.[15] Roy Cellan-Jones. Youtube, facebook and twitter align to fight covid vaccine con-spiracies, Nov 2020.[16] Kaya Yurieff. How twitter, facebook and youtube are handling election misinforma-tion, Nov 2020.[17] L. Lima, J. C. S. Reis, P. Melo, F. Murai, L. Araujo, P. Vikatos, and F. Ben-evenuto. Inside the right-leaning echo chambers: Characterizing gab, an unmod-erated social system. In , pages 515–522, 2018.[18] Max Aliapoulios, Emmi Bevensee, Jeremy Blackburn, Emiliano De Cristofaro, Gi-anluca Stringhini, and Savvas Zannettou. An early look at the parler online socialnetwork, 2021.[19] Chengcheng Shao, Giovanni Luca Ciampaglia, Onur Varol, Alessandro Flammini,and Filippo Menczer. The spread of fake news by social bots. arXiv preprintarXiv:1707.07592 , 96:104, 2017.[20] Emilio Ferrara. Measuring social spam and the effect of bots on information dif-fusion in social media. In
Complex spreading phenomena in social systems , pages229–255. Springer, 2018.[21] Kate Starbird. Disinformation’s spread: bots, trolls and all of us.
Nature ,571(7766):449–450, 2019.[22] Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flam-mini. The rise of social bots.
Commun. ACM , 59(7):96–104, June 2016.[23] James A Evans and Pedro Aceves. Machine translation: Mining text for social the-ory.
Annual Review of Sociology , 42:21–50, 2016.[24] Stefan Timmermans and Iddo Tavory. Theory construction in qualitative research:From grounded theory to abductive analysis.
Sociological theory , 30(3):167–186,2012.[25] Joshua A Tucker, Andrew Guess, Pablo Barberá, Cristian Vaccari, Alexandra Siegel,Sergey Sanovich, Denis Stukal, and Brendan Nyhan. Social media, political polar-ization, and political disinformation: A review of the scientific literature.
Politicalpolarization, and political disinformation: a review of the scientific literature (March19, 2018) , 2018.[26] Yochai Benkler, Robert Faris, and Hal Roberts.
Network propaganda: Manipulation,disinformation, and radicalization in American politics . Oxford University Press,2018.[27] Louis Jacobson. Politifact - the record-setting 2020 democratic primary field: Whatyou need to know, Jan 2020. 1728] Derrick Bryson Taylor. A timeline of the coronavirus pandemic, Jan 2021.[29] Jiachuan Wu, Savannah Smith, Mansee Khurana, Corky Siemaszko, and BriannaDeJesus-Banos. Coronavirus lockdowns and stay-at-home orders across the u.s., Apr2020.[30] Daniel M. Romero, Brendan Meeder, and Jon Kleinberg. Differences in the mechan-ics of information diffusion across topics: idioms, political hashtags, and complexcontagion on twitter.
Proceedings of the 20th international conference on World wideweb - WWW ’11 , 2011.[31] Joey Hadden and Laura Casado. Here are the latest major events that have beencanceled or postponed because of the coronavirus outbreak, including the 2020tokyo olympics, burning man, and the 74th annual tony awards, Apr 2020.[32] Joey Hadden, Laura Casado, Tyler Sonnemaker, and Taylor Borden. 21 major com-panies that have announced employees can work remotely long-term, Dec 2020.[33] Sara Fischer. Social media use spikes during pandemic, Apr 2020.[34] Andreas Jungherr. Twitter use in election campaigns: A systematic literature re-view.
Journal of Information Technology & Politics , 13(1):72–91, 2016.[35] Yochai Benkler, Casey Tilton, Bruce Etling, Hal Roberts, Justin Clark, RobertFaris, Jonas Kaiser, and Carolyn Schmitt. Mail-in voter fraud: Anatomy of a dis-information campaign.
SSRN Electronic Journal , 2020.[36] Emily Chen, Ashok Deb, and Emilio Ferrara. arXiv , oct 2020.[37] Emily Chen, Kristina Lerman, and Emilio Ferrara. Tracking social media discourseabout the covid-19 pandemic: Development of a public coronavirus twitter data set.
JMIR Public Health and Surveillance , 6(2):e19273, 2020.[38] Consuming streaming data | twitter developer.[39] Is the sample good enough? comparing data from twitter’s streaming api with twit-ter’s firehose. 7.[40] David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation.
J.Mach. Learn. Res. , 3(null):993–1022, March 2003.[41] Lauren Egan. Trump calls coronavirus democrats’ ’new hoax’, Feb 2020.[42] Alicia Victoria Lozano. Anti-vaccine protest briefly shuts down dodger stadiumvaccination site, Jan 2021.[43] Caitlin Oprysko. Trump says he’s taking hydroxychloroquine, despite scientists’concerns, May 2020.[44] Erika Edwards. World health organization halts hydroxychloroquine study, Jun2020.[45] Who discontinues hydroxychloroquine and lopinavir/ritonavir treatment arms forcovid-19, Jul 2020.[46] Jacob Bogage, Lisa Rein, and Josh Dawsey. Postmaster general eyes aggressivechanges at postal service after election, Aug 2020.1847] Noah Pransky. U.s. mail slowed down just before the election. these states are mostat risk, Oct 2020.[48] Emily Cochrane, Hailey Fuchs, Kenneth P Vogel, and Jessica Silver-Greenberg.Postal service suspends changes after outcry over delivery slowdown, Aug 2020.[49] Robert Farley. Trump’s latest voter fraud misinformation, Apr 2020.[50] Amy V. Mueller, Matthew J. Eden, Jessica M. Oakes, Chiara Bellini, andLoretta A. Fernandez. Quantitative method for comparative assessment of parti-cle removal efficiency of fabric masks as alternatives to standard surgical masks forppe.
Matter , 3(3):950–962, 2020.[51] Olga Khazan. How a bizarre claim about masks has lived on for months, Oct 2020.[52] Emily Chen, Herbert Chang, Ashwin Rao, Emilio Ferrara, Kristina Lerman, andGeoffrey Cowan. COVID-19 misinformation and the 2020 U.S. presidential election.
Harvard Kennedy School Misinformation Review , Feb 2021.[53] Kathleen Hall Jamieson and Joseph N. Cappella.
Echo chamber: Rush Limbaughand the conservative media establishment . Oxford University Press, 2008.[54] Siying Du and Steve Gregory. The echo chamber effect in twitter: does communitypolarization increase? In Hocine Cherifi, Sabrina Gaito, Walter Quattrociocchi, andAlessandra Sala, editors,
Complex Networks & Their Applications V , pages 373–378,Cham, 2017. Springer International Publishing.[55] Amy Sherman. Politifact - trump said the obama admin left him a bare stockpile.wrong, Apr 2020.[56] Edward G Carmines. Class politics, american-style: A discussion of winner-take-allpolitics: How washington made the rich richer—and turned its back on the middleclass.
Perspectives on Politics , 9(3):645–647, 2011.[57] Jacob S Hacker and Paul Pierson.
Off center: The Republican revolution and theerosion of American democracy . Yale University Press, 2006.[58] Sean M Theriault.
The Gingrich senators: The roots of partisan warfare inCongress . Oxford University Press, 2013.[59] Robert Faris, Hal Roberts, Bruce Etling, Nikki Bourassa, Ethan Zuckerman, andYochai Benkler. Partisanship, propaganda, and disinformation: Online media andthe 2016 us presidential election.
Berkman Klein Center Research Publication , 6,2017.[60] William J Brady, Julian A Wills, Dominic Burkart, John T Jost, and Jay JVan Bavel. An ideological asymmetry in the diffusion of moralized content on so-cial media among political leaders.
Journal of Experimental Psychology: General ,148(10):1802, 2019.[61] Michael M Gant and Dwight F Davis. Mental Economy and Voter Rationality: TheInformed Citizen Problem in Voting Research.
The Journal of Politics , 46(1):132–153, 1984.[62] Iris Stucki, Lyn E. Pleger, and Fritz Sager. The Making of the Informed Voter: ASplit-Ballot Survey on the Use of Scientific Evidence in Direct-Democratic Cam-paigns.
Swiss Political Science Review , 24(2):115–139, jun 2018.1963] Scott Ashworth and Ethan Bueno D E Mesquita. Is Voter Competence Good forVoters?: Information, Rationality, and Democratic Performance.
The AmericanPolitical Science Review , 108(3):565–587, 2014.[64] Luca Maria Aiello, Martina Deplano, Rossano Schifanella, and Giancarlo Ruffo.People are strange when you’re a stranger: Impact and influence of bots on socialnetworks.
Proceedings of the International AAAI Conference on Web and SocialMedia , 6(1), May 2012.[65] Asta Zelenkauskaite and Marcello Balduccini. “information warfare” and online newscommenting: Analyzing forces of social influence through location-based comment-ing user typology.
Social Media + Society , 3(3):2056305117718468, 2017.[66] Kai-Cheng Yang, Onur Varol, Clayton A. Davis, Emilio Ferrara, Alessandro Flam-mini, and Filippo Menczer. Arming the public with artificial intelligence to countersocial bots.
Human Behavior and Emerging Technologies , 1(1):48–61, 2019.[67] Mohsen Sayyadiharikandeh, Onur Varol, Kai-Cheng Yang, Alessandro Flammini,and Filippo Menczer. Detection of novel social bots by ensembles of specializedclassifiers.
Proceedings of the 29th ACM International Conference on Information& Knowledge Management , Oct 2020.[68] Kai-Cheng Yang, Onur Varol, Pik-Mai Hui, and Filippo Menczer. Scalable and gen-eralizable social bot detection through data selection.
Proceedings of the AAAI Con-ference on Artificial Intelligence , 34(01):1096–1103, Apr. 2020.[69] Clayton A Davis, Giovanni Luca Ciampaglia, Luca Maria Aiello, Keychul Chung,Michael D Conover, Emilio Ferrara, Alessandro Flammini, Geoffrey C Fox, Xiaom-ing Gao, Bruno Gonçalves, et al. Osome: the iuni observatory on social media.
PeerJ Computer Science , 2:e87, 2016.[70] Emilio Ferrara. The history of digital spam.
Communications of the ACM , 62(8):82–91, 2019.[71] Emilio Ferrara, Herbert Chang, Emily Chen, Goran Muric, and Jaimin Patel. Char-acterizing social media manipulation in the 2020 U.S. presidential election.
FirstMonday , oct 2020.[72] Emilio Ferrara. What types of covid-19 conspiracies are populated by twitter bots?
First Monday , 2020.[73] Julie Jiang, Emily Chen, Shen Yan, Kristina Lerman, and Emilio Ferrara. Politicalpolarization drives online conversations about covid-19 in the united states.
HumanBehavior and Emerging Technologies , 2(3):200–211, 2020.[74] Christopher A. Bail, Lisa P. Argyle, Taylor W. Brown, John P. Bumpus, HaohanChen, M. B. Fallin Hunzaker, Jaemin Lee, Marcus Mann, Friedolin Merhout, andAlexander Volfovsky. Exposure to opposing views on social media can increase po-litical polarization.
Proceedings of the National Academy of Sciences , 115(37):9216–9221, 2018.[75] Hunt Allcott and Matthew Gentzkow. Social media and fake news in the 2016 elec-tion, mar 2017.[76] Jan-Willem van Prooijen. Belief in Conspiracy Theories. In
The Social Psychologyof Gullibility , pages 319–332. Routledge, apr 2019.2077] Karen M. Douglas, Joseph E. Uscinski, Robbie M. Sutton, Aleksandra Cichocka,Turkay Nefes, Chee Siang Ang, and Farzin Deravi. Understanding Conspiracy Theo-ries.
Political Psychology , 40(S1):3–35, feb 2019.[78] Jovan Byford.
Conspiracy Theories - A critical Intoduction . 2011.[79] Ethan Zuckerman. QAnon and the Emergence of the Unreal.
Journal of Design andScience , (6), jul 2019.[80] Massimo Stella, Emilio Ferrara, and Manlio De Domenico. Bots increase expo-sure to negative and inflammatory content in online social systems.
Proceedings ofthe National Academy of Sciences of the United States of America , 115(49):12435–12440, dec 2018.[81] Nancy L Rosenblum and Russell Muirhead.
A lot of people are saying: The newconspiracism and the assault on democracy . Princeton University Press, 2020.[82] Christopher Paul and Miriam Matthews. The russian “firehose of falsehood” propa-ganda model.
Rand Corporation , pages 2–7, 2016.[83] Stephan Lewandowsky, Ullrich KH Ecker, Colleen M Seifert, Norbert Schwarz, andJohn Cook. Misinformation and its correction: Continued influence and successfuldebiasing.
Psychological science in the public interest , 13(3):106–131, 2012.[84] Soroush Vosoughi, Deb Roy, and Sinan Aral. The spread of true and false newsonline.
Science , 359(6380):1146–1151, 2018.[85] Timothy R Tangherlini, Shadi Shahsavari, Behnam Shahbazi, Ehsan Ebrahimzadeh,and Vwani Roychowdhury. An automated pipeline for the discovery of conspiracyand conspiracy theory narrative frameworks: Bridgegate, pizzagate and storytellingon the web.
PloS one , 15(6):e0233879, 2020.[86] Jasper Muis and Tim Immerzeel. Causes and consequences of the rise of populistradical right parties and movements in europe.
Current Sociology , 65(6):909–930,2017.[87] Ekaterina Zhuravskaya, Maria Petrova, and Ruben Enikolopov. Political effects ofthe internet and social media.