Volunteer contributions to Wikipedia increased during COVID-19 mobility restrictions
Thorsten Ruprechter, Manoel Horta Ribeiro, Tiago Santos, Florian Lemmerich, Markus Strohmaier, Robert West, Denis Helic
VVolunteer contributions to Wikipedia increasedduring COVID-19 mobility restrictions
Thorsten Ruprechter , Manoel Horta Ribeiro , Tiago Santos , Florian Lemmerich ,Markus Strohmaier , Robert West , and Denis Helic Graz University of Technology, 8010 Graz, Austria EPFL, 1015 Lausanne, Switzerland RWTH Aachen University, 52062 Aachen, Germany GESIS – Leibniz Institute for the Social Sciences, 50667 Cologne, Germany * Corresponding author ([email protected])
Wikipedia, the largest encyclopedia ever created, is a global initiative driven by volunteer contribu-tions. When the COVID-19 pandemic broke out and mobility restrictions ensued across the globe,it was unclear whether Wikipedia volunteers would become less active in the face of the pandemic,or whether they would rise to meet the increased demand for high-quality information despite theadded stress inflicted by this crisis. Analyzing million edits contributed from 2018 to 2020 acrosstwelve Wikipedia language editions, we find that Wikipedia’s global volunteer community respondedremarkably to the pandemic, substantially increasing both productivity and the number of newcom-ers who joined the community. For example, contributions to the English Wikipedia increased byover % compared to the expectation derived from pre-pandemic data. Our work sheds light onthe response of a global volunteer population to the COVID-19 crisis, providing valuable insightsinto the behavior of critical online communities under stress. Wikipedia is the world’s largest encyclopedia, one of the most prominent volunteer-based information systems inexistence [18, 29], and one of the most popular destinations on the Web [2]. On an average day in 2019, users fromaround the world visited Wikipedia about 530 million times and editors voluntarily contributed over thousandedits to one of Wikipedia’s language editions (Supplementary Table 1).Amidst the COVID-19 pandemic and the “infodemic” [15] that ensued, Wikipedia played and continues to playan important role in supplying information about the COVID-19 crisis [9, 19, 45, 46]. Notably, the increase in accessrelated to all kinds of articles—not only those related to the pandemic—suggests that Wikipedia’s role in this timeof crisis transcends mere COVID-19-related information seeking [24]. However, page views are but a single aspect ofthe pandemic’s impact on Wikipedia, an aspect that ignores the fundamental contribution of editors, who performunpaid volunteer work to maintain and develop content on the website. If the pandemic negatively impacted theproductivity and number of editors on Wikipedia, the world’s largest online encyclopedia could be at peril [21, 33].We can devise two competing hypotheses on how the COVID-19 crisis may have impacted editors on Wikipedia.First, the editor community may have shrunk in response to COVID-19 and corresponding mobility restrictions. Asalmost everyone, Wikipedia volunteers may have been affected by the negative economic and social ramifications ofthe pandemic [4, 7, 39], especially after most governments enforced mobility restrictions [11, 13, 54]. The challengesassociated with this new reality may have led editors to withdraw from volunteer work for Wikipedia while focusingtheir efforts on personal issues and on dealing with the crisis. Alternatively, editors may have increased theirvolunteer work. This could be due to a personal response to the increased demand for high-quality information,as previously observed during locally confined disease outbreaks [1] and extraordinary events [52], or simply due tomobility restrictions resulting in individuals spending more time at home in front of computer screens [40] or onthe Internet [12]. Whether Wikipedia editors withdraw from volunteering or increase their activity during distressdetermines the overall quality of information that Wikipedia serves to a global audience of readers. Therefore,understanding how editors responded to the COVID-19 pandemic and the accompanying mobility restrictions iscrucial to assess Wikipedia’s capacity to act as a global information medium during worldwide disasters.After careful quantitative analyses of large-scale edit logs on Wikipedia, we present robust evidence that vol-unteer contributions significantly increased during the COVID-19 crisis across many language editions. During thepandemic, the Wikipedia editor community not only generated many more edits than what we would expect given1 a r X i v : . [ c s . C Y ] F e b an2020 Feb Mar Apr May Jun Jul Aug Sep Oct Date E d i t s i n t h e E n g li s h W i k i p e d i a J a n s t D e a t h i n C h i n a F e b D i s e a s e N a m e d " C O V I D - " M a r L o c k d o w n i n I t a l y A p r M illi o n C a s e s A p r , D e a t h s M a y M illi o n C a s e s J u l M illi o n C a s e s S e p M illi o n D e a t h s M a y : + , E d i t s Figure 1:
Edit volume in the English Wikipedia increased during COVID-19 mobility restrictions.
Wevisualize the rolling 7-day average edit volume in the English Wikipedia from January to October 2020 alongsidethe daily mean of 2019 and 2018, only considering non-bot edits to Wikipedia articles. Vertical lines mark majordevelopments during the COVID-19 pandemic in 2020. After the first Western countries (e.g., Italy) enforcedmobility restrictions in early March, edit volume stagnated briefly before rising sharply—a trend that prevailed untillate May, where the maximum difference in rolling 7-day average edit volume reached
20 970 . Although this initialsharp increase in edits declined, a surplus persisted until late September. Until September 31 st , editors produced . ( . ) more edits in 2020 than in 2019 (2018), an increase of . million ( million) edits (SupplementaryTable 2). Much of this edit surplus appears to stem from periods of mobility restrictions in the spring of 2020. Extracted from https://wikimediafoundation.org/covid19/data historical baselines, but also acquired many more newcomers than in recent history, demonstrating the remarkableresilience of this online community in the face of adverse conditions.Figure 1 depicts the increase in volunteer edits in the English Wikipedia during the COVID-19 timeline in2020 compared to previous years. Whereas no increase in edit volume was apparent in early 2020, the mobilityrestrictions in Western countries seemed to first slightly dampen edit activity, before triggering a strong upwardtrend towards the end of March. In the weeks thereafter, a considerable edit surplus developed in comparison toprevious years, which lasted until its peak in late May. As the pandemic subsided over the summer, the growthin edit volume also continuously decreased until fall. By October, the relative increase in edit volume, and thusvolunteer contribution, from 2019 to 2020 (about . , or . million edits) was about double that from 2015 to2019 (about . , or . million edits; see Supplementary Table 2). In summary, visual representation of editvolume in the English Wikipedia suggests a considerable contribution surplus in 2020.Beyond the mere descriptive analysis of a single Wikipedia language edition, we systematically analyzed a variedsample of 12 Wikipedia language editions (“Wikipedias”), consisting of four large, medium, and small languageeditions each (Methods), with over million edits spread through . million articles. In accordance with thedescriptive analysis shown in Figure 1, our quasi-experimental difference-in-differences analysis finds a significantincrease in edit volume after COVID-19 mobility restrictions came into effect for many of the Wikipedia editions,and an influx of new editors that is particularly salient for larger Wikipedias. Our study sheds light on the impactof the COVID-19 mobility restrictions on Wikipedia volunteer contributions and provides a reusable framework tomeasure user activity under stress. More broadly, the evident increase in edit volume and newcomers across most2 .891.041.19 e n f r d e i t e v English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o e v Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.4-0.20.00.20.40.60.81.01.2 e v Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t V o l u m e M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 2:
Edit volume during COVID-19 mobility restrictions.
We show edit volume findings in large(top), medium (middle), and small (bottom) Wikipedias during COVID-19 mobility restrictions, which in thefigure we delineate using mobility (when restrictions become effective) and normality (when restrictions are lifted)changepoints. a , We show rolling 7-day average edit volume generated by human editors for 2018, 2019, and 2020until October. After a slight retraction of editing around mobility changepoints in most Wikipedias, the number ofcontributions recovers to previous levels within a few days. Editors contribute substantially more in all large andsome medium Wikipedias in the weeks after the mobility restrictions in 2020, compared to historical baselines. b , Wedepict the relative change in edit volume ( ev ) as retrieved from DiD via δ ev (95% confidence interval as two standarddeviations) and plot δ ev for 120 left-aligned seven-day-windows (see Methods), with the x-axis describing days afterthe respective mobility changepoint. We observe that editing in large and medium Wikipedias significantly increasesafter their mobility changepoint, while most small Wikipedias show neither significant increase nor decrease.observed Wikipedias is a finding of interest not only to Wikipedia itself but also to researchers and managers ofother online collaboration systems, as it provides valuable insight into user behavior during a global crisis. Results
Edit volume during COVID-19 mobility restrictions
We observe an increase in edit volume (the number of edits made by non-bot users) on Wikipedia during the period ofCOVID-19 mobility restrictions in the spring of 2020, which is particularly evident in large and medium Wikipedias.Figure 2a depicts the rolling 7-day average edit volume for large (top), medium (middle), and small (bottom)Wikipedias in the context of COVID-19 mobility restrictions, which we delineate via automatically detected mobility(i.e., restrictions take effect) and normality (i.e., restrictions are lifted) changepoints (see Methods). We also reportedit volumes for 2018 and 2019 as a reference for 2020. We observe substantial drops in edit volume around themobility changepoint for almost all Wikipedias, indicating a shock to the Wikipedia ecosystem. In particular, largerWikipedias experience a considerable short-lived decrease in edit volume but are able to recover quickly. English,Italian, German, French, Korean, and Japanese even clearly surpass their pre-shock volume levels, leading to anoverall edit surplus. On the contrary, some smaller Wikipedias (e.g., Finnish) exhibit a steady decline in edit volumeafter the mobility changepoint. To better relate edit volume during the COVID-19 pandemic to reference values3rom previous years and pre-pandemic periods, we employ a difference-in-differences regression (DiD) that controlsfor the year, period, and language, as well as their interactions. For all Wikipedias, we compute the effective changein edit volume ( ev ) after the mobility changepoint from the three-way interaction of year, period, and language, anddenote this effective change as δ ev . We apply the DiD analysis to a sequence of seven-day-windows post-changepoint,always retaining the 30-day pre-changepoint period, and plot the time series of logarithmic effects for edit volumeaccording to δ ev in Figure 2b. We describe this DiD setup in more detail in Methods. The DiD analysis validatesthat all large and most medium Wikipedias significantly increase their edits following the mobility restrictionsaccording to δ ev (95% confidence interval), while no general statement can be made for small Wikipedias.For the rolling 7-day average edit volume in large Wikipedias, we identify an upward trend in 2020 immediatelyafter mobility restrictions took place (Figure 2a, top). In the English, French, and Italian language editions, editvolume steadily increases for nearly two months after a dip around the date of the mobility changepoint, beforeslowly reverting to prior levels. The steady initial increase in edit volume leads to outstanding peaks—approximately
120 000 edits for English,
28 000 for French,
15 000 for Italian, and
24 000 for German, which exhibits a decline backto pre-crisis levels earlier than other large Wikipedias. DiD results confirm the edit volume surplus visible in thetime series for large Wikipedias in 2020 (Figure 2b, top). δ ev for French, Italian, and English depicts an immediaterelative increase in edits after the mobility restrictions take place, leading to over 100 days of significant increases forall three of these Wikipedias, whereas German declines earlier. Approximately 35 days after the mobility restrictionstake effect, French ( e . = 144% , a surplus of ), Italian ( +42% ), and German ( +25% ) reach their highestsignificant relative increase for edit volume. The higher short-term increases in French, German, and Italian maybe related to more detailed reporting of local issues in these language editions. On the contrary, English shows alonger, sustained upward trend for δ ev , with a maximum significant increase of after 69 days. In conclusion,edit volume significantly increases in large Wikipedias after mobility restrictions come into effect.Edit volume in most medium and small Wikipedias slightly drops around the respective mobility changepointsin 2020. However, virtually all Wikipedias quickly recover from the initial shock, with most maintaining a stableedit volume in the ensuing weeks and some even generating an edit surplus. While Figure 2a (middle) shows thatmedium Wikipedias do not homogeneously increase their edit volume, Korean and Japanese surpass their pre-mobility-restriction levels about a month post changepoint, peaking at about and
14 500 edits, respectively.For small Wikipedias, edit volume only decreases slightly right after the mobility changepoint (Figure 2a, bottom).Afterward, edit volume recovers to previous baselines within thirty days, before following similar trends and levels asin previous years. DiD analysis and corresponding values for δ ev reveal that, in fact, medium Wikipedias experiencevarying periods of significant relative increases in edit volume (Figure 2b, middle). For example, when compared topre-pandemic years around the same time period, the Korean and Dutch Wikipedias produce a consistent relativeincrease (peaking at +40% ), whereas Swedish and Japanese exhibit shorter significant periods ( +30% and +38% inmaximum, resp.). Furthermore, the relative change for small Wikipedias (Figure 2b, bottom) signals brief periods ofsubstantial relative increases for Danish and Norwegian (peaks of +69% and +43% , resp.). Most notably, Serbianexhibits a considerate increase during the first month after mobility restrictions take place, with volume nearlytripling (logarithmic effect of . ). Lastly, we note that out of our twelve investigated Wikipedias only Finnishshows a significant decrease in δ ev over longer stretches of the observed period. In any case, small and mediumWikipedias are mostly resilient to the initial shock to edit volume triggered by COVID-19, with some even surpassingtheir pre-pandemic baselines after a few weeks. Newcomers during COVID-19 mobility restrictions
We find that all large and medium Wikipedias acquire considerably more newcomers (the number of registeredusers who made their first edit) for most of the study period, while the remaining Wikipedias exhibited resilienceand do not decrease their levels significantly. We visualize the 7-day rolling averages for newcomer counts duringthe COVID-19 pandemic for large (top), medium (middle), and small (bottom) Wikipedias in Figure 3a, while alsoshowing values for previous years as well as mobility and normality changepoints. Newcomer counts plummet aroundthe mobility changepoint, in particular for large Wikipedias, but this attenuation in newcomer recruitment onlypersists for a brief period. Shortly thereafter, newcomer counts increase considerably in all but a few medium andsmall Wikipedias (e.g., Swedish or Finnish). Again, we build a DiD model for newcomers ( nc ) to quantify effectivechanges during the period of COVID-19 mobility restrictions in spring 2020, again controlling for year, period, andlanguage. We again perform our DiD analysis for a sequence of seven-day windows after the mobility changepoint(see Methods) and show the logarithmic effects for newcomers ( δ nc ) in Figure 3b. This newcomer DiD analysisconfirms that while all large Wikipedias acquire significantly more new editors after mobility restrictions take effect,some medium and small Wikipedias seem to be resilient and exhibit no significant long-term changes ( CI).Large Wikipedias appear to recover rapidly from the initial negative effect of mobility restrictions in terms of4 .321.722.13 e n f r d e i t n c English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o n c Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.50.00.51.0 n c Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) N e w c o m e r s M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 3:
Newcomers during COVID-19 mobility restrictions.
We visualize newcomer results in large (top),medium (middle), and small (bottom) Wikipedias during COVID-19 mobility restrictions, which we delineate viamobility (when restrictions become effective) and normality (when restrictions are lifted) changepoints. a , We depictrolling 7-day average newcomer counts until October of 2018, 2019, and 2020. For many Wikipedias, newcomeracquirement strongly declines right around their mobility changepoint, but then quickly rises to or even exceedspre-pandemic baselines. b , We investigate the relative change in newcomers ( nc ) via δ nc as computed from DiDanalysis (95% confidence intervals as two standard deviations) and plot δ nc for 120 left-aligned seven-day windows(see Methods), starting with the respective mobility changepoint. In large Wikipedias, considerably more newcomersjoin in the weeks after mobility restrictions come into effect, relative to before the changepoint and previous years.Results for medium and small Wikipedias are non-conclusive, with some showing increases in the number of newlyacquired editors and others not significantly changing their values.newcomer counts (Figure 3a, top). Most notably, Italian registers a downright newcomer surge until late April,recruiting over newcomers on a rolling 7-day average. English and French show similar patterns of perpetualincreases, reaching respective peaks of approximately and new editors. Although German exhibits nearly newcomers shortly after the mobility changepoint, the surplus in 2020 seems not as considerable as for otherlarge Wikipedias. We further note that newcomer counts for large Wikipedias start to steadily decline in May.However, this seasonal trend also appears to be prevalent in previous years. Our DiD analysis, which captures thechange in newcomers via δ nc , for the most part confirms these findings (Figure 3b, top). During the first two tothree weeks past the mobility changepoint, large Wikipedias steadily recover from the COVID-19 shock withoutsignificant overall gains according to δ nc . However, right after this recovery phase significant peaks arise for English( e . = 130% of previous levels), French ( ), and German ( ). For the Italian Wikipedia, which belongsto a region with particularly strict mobility restrictions, we confirm an even stronger newcomer surge, leading to a relative increase. Furthermore, English generates a notably stable, significant long-term growth in newcomersthat is possibly owed to editors from all over the world joining this language edition during mobility restrictionsin their regions, as the English Wikipedia serves as a global repository of knowledge. Ultimately, positive effectsprevail for large Wikipedias and solidify a newcomer surplus after the mobility restrictions come into force.Similar to large Wikipedias, most medium and small Wikipedias experience a decline in newcomers right aroundtheir mobility changepoints before then increasing their counts to previous baselines (Figure 3a, middle and bottom).Some of these Wikipedias (e.g., Norwegian, Finnish, Danish, Swedish) recover to previous levels within the first5onth and exhibit no long-term effects afterwards. However, others recruit a surplus of newcomers during thiscrisis. Japanese, Dutch, Korean, and Serbian show short-term newcomer influxes about one to two months after theinitial mobility restrictions take effect, with maximum respective values of approximately , , , and dailynewcomers. We also observe these effects in δ nc as captured by DiD (Figure 3b, middle and bottom), which confirmsbrief relative increases for Japanese ( +30% ), Dutch ( +47% ), Korean ( +28% ), and Serbian ( +179% ). Finally, thenewcomer DiD analysis corroborates that some medium and most small Wikipedias do not significantly deviatefrom baselines prior to the mobility restrictions over much of the observed time span. Discussion
As the COVID-19 pandemic erupted on a global scale, it was unclear how this incisive event would affect Wikipedia’svolunteer community. Over the course of the last few years, both human editing [38] and newcomer recruitment [22]on Wikipedia have stagnated or even decreased (Supplementary Table 2). Accordingly, the pandemic could haveaccelerated the decline of the online encyclopedia as the hardships of this global crisis may even further decreasevolunteer activity. However, our study, in which we analyze million edits from 12 Wikipedia language editions,reveals that the COVID-19 pandemic and its accompanying mobility restrictions have substantially boosted vol-unteer activity on Wikipedia. By performing a difference-in-differences analysis, we show that edit volume as wellas the influx of newcomers has generally increased after COVID-19 mobility restrictions went into effect. In whatfollows, we discuss the implications and limitations of this finding.
Mechanisms behind contribution growth.
We observe significant increases in edit volume and newcomersduring the COVID-19 pandemic across multiple Wikipedias, making it their most active period in at least the lastthree years. While our quantitative study sheds light on the extent of contribution growth, there are several possiblemechanisms behind this effect, which may or may not impact the collaborative structure of editor communities.Firstly, Wikipedia received significantly more page views during the COVID-19 crisis [24]. The increase inedits and newcomers may partially be due to the prior increase in Wikipedia readership, as a certain proportionof readers turns into contributors because of various motivational factors [37, 48]. In addition, we theorize thatincreased screen time and Internet exposure [12, 40] during the mobility restrictions lead to Wikipedia readersspending more time editing, possibly increasing the reader-to-editor turnover rate. Tracing the transformation ofreaders into editors during this pandemic in more detail is a promising avenue for future work.Secondly, the increase in contributions may be due to the rapidly changing information and new knowledgethat the COVID-19 pandemic generates about the world. Past literature has suggested that Wikipedia growth isconstrained by the amount of knowledge available, as editors have already contributed most of the easily obtainableand verifiable information [38]. The fact that volunteers have been “running out of easy topics” to contribute tohas made it difficult for non-specialists to provide new content with little effort [21]. As the COVID-19 pandemicdramatically changes the status quo of our world today, it is generating new knowledge about many fields and thusmay provide fresh opportunities for both novel and veteran editors to contribute to Wikipedia.Moreover, the observed edit surplus may have been caused by the high-intensity activity of a core group ofeditors rather than the broader editor population. We therefore investigate the number of editors active on anygiven day according to their activity level: to , to , to , or more than daily edits (Methods). TheDiD analysis for editor counts depicts increases across all activity levels after mobility changepoints for all large andmost medium Wikipedias, while small Wikipedias show non-conclusive effects (Supplementary Figs. 1, 2, 3, and 4).These findings indicate that the editor population as a whole intensified their contribution during the COVID-19pandemic, causing the overall increase in volunteer activity.Finally, we detected a contribution disparity with respect to Wikipedia size, meaning that the smaller Wikipediaswe studied did not benefit to the same degree as larger or medium Wikipedias. The observed discrepancy in editand newcomer increases for large, medium, and small Wikipedias may stem from a difference in community sizeand structure, or these Wikipedias’ specific rules [5, 21, 26, 34, 49]. Moreover, the amount of content for certaintopical categories diverges due to cultural contextualization in different language editions [32]. Specifically, a strong(hypothetical) affinity for topics not directly related to the pandemic (e.g., Sports ) in medium or smaller Wikipediasmight change the effect of this crisis on their edit volume, in comparison to larger Wikipedias. As an example, incase such Wikipedia language editions focused more on updating sports articles, edit volume would decrease moreduring the pandemic. The magnitude of such an effect may further depend on a region’s more (e.g., Italy) or lessstrict (e.g., Sweden) mobility restrictions. Future research may explore language-specific collaboration mechanismsin more detail, for example by attempting to topically analyze Wikipedia contributions during the pandemic.
Resilience of Wikipedia communities.
Although we did not find the same surplus in contributions across large,medium, and small Wikipedia language editions, volunteer communities in all studied Wikipedias demonstrated6esilience by quickly recovering from the initial negative impact of the pandemic on their contributions. While slowresponse to negative events or other shocks causes severe problems in social-ecological systems [35, 36], resilient sys-tems are adaptable and manage to withstand such shocks, even bearing the capacity to cross previous performancethresholds [14]—a behavior observed in this study. The strongest resilience and subsequent crossing of earlier thresh-olds in large Wikipedias during the pandemic may be partially explained by the difference in community size [51].For example, in larger communities it may not be as problematic that leaders are limited due to the pandemic, asa greater number of other veteran members can take over their work. This conjecture borrows from critical masstheory, in the sense that a critical mass of core members is the fundamental source of content [48]. Future researchmight investigate the aspect of Wikipedia resilience during the pandemic in more detail, for example by consideringthreat rigidity [51] or building a model [47] that considers COVID-19 as an attack on the community structure.
Revert rate during COVID-19 mobility restrictions.
The observed simultaneous increase in newcomers andedits may suggest that the edit surplus was partially caused by low-quality edits by first-time editors. Frequently,veteran editors or bots would then completely undo (i.e., identity revert) these newcomer revisions, which representsa common behavioral pattern on Wikipedia [21, 22, 53], in turn generating further revisions. To investigate whetheran increase in such reverts occurred, we performed a cursory analysis of the revert rate, which is defined as the ratioof reverted edits to edit volume (see Methods and Supplementary Information). Supplementary Figure 5a visualizesthe rolling 7-day average revert rate ( rr ), while Supplementary Figure 5b plots the relative change in revert rate( δ rr ) as captured by a DiD analysis (Methods). Interestingly, we detect a significant increase of the revert rate inonly one language (Korean). By contrast, several Wikipedias exhibit significantly decreased revert rates shortlyafter the mobility restrictions come into force. For example, the large Italian, French, and German Wikipediasall show reduced revert rates by about one quarter. This suggests that less valuable revisions, possibly made bynewcomers, and their immediate reversal do not cause the reported increase in edit volume. Furthermore, potentialmisbehavior or conflict on Wikipedia, such as vandalism or edit wars, is prominently characterized by large numbersof identity reverts, as they undo these unwanted contributions [28, 41, 50]. Therefore, reduced revert rates mayindicate that editors refrain more from confrontational behavior and thus demonstrate higher levels of solidarityduring the pandemic, which is a common phenomena within collectives during crises [16]. However, a decline inrevert rate could also imply that bots and administrators may be unable to keep up with the influx of edits, leavinglow quality or malicious edits undetected and thus diminishing quality in the long term. We see the detection andanalysis of behavioral patterns and collaborative structure of online communities as a promising path for futureresearch. In addition, it may be valuable to further study the treatment and retention of newcomers [8, 22, 34]during and after the pandemic once more longitudinal data is available. Contribution to COVID-19 articles.
One might speculate that the increase in edit volume is mostly due toedits in articles that are strongly related to COVID-19. However, many of those articles were protected from publicediting early in the pandemic to prevent spread of misinformation [27], and we find that only a negligibly smallfraction of edits (at most for most Wikipedias) goes towards articles with a primary focus on COVID-19 (seeMethods) between January 1 st and September 31 st . of edits performed in 2020 by the end of September concernthemselves with such articles. This may indicate higher coverage of local COVID-19 outbreaks in German thanin other languages. We consequently repeat our DiD analysis for edit volume, this time excluding edits to articlesstrongly related to COVID-19 (Supplementary Fig. 7). The results support the previous findings and confirm thatthe reported edit volume increase is not due to COVID-19 articles. In this way, our work extends previous studies,which focused on a smaller subset of pandemic-related articles [19, 27]. Other limitations.
Even though our work covers a large portion of Wikipedia’s content and editor population, itcomes with several limitations. First, we do not consider a variety of different Wikipedias associated with languageswidely spoken in the global south, including Spanish, Portuguese, Arabic, Hindi, or any African Wikipedias (seeMethods for how we chose language editions). Future work analyzing these Wikipedias could improve our under-standing of the impact of the pandemic on volunteer contribution in other parts of the world. Second, contenton Wikipedia is predominantly edited by white males between the ages of 17 and 40 [10, 23]. It may be that theCOVID-19 crisis has disparately impacted contributors of less represented demographics, as certain racial or so-cioeconomic groups are particularly disadvantaged by the pandemic [3, 6, 25]. In addition, bots have an importantrole in the creation and management of Wikipedia content [43, 53]. We excluded bots from our analysis as wespecifically focused on edits performed by human volunteers. Nevertheless, other studies may choose to considerbot activities as valid contributions to Wikipedia.In conclusion, our study provides evidence for a substantial surplus of volunteer contributions to multipleWikipedia language editions during COVID-19 mobility restrictions, which shines light on the resilience of theWikipedia community under times of stress. The methodological framework used in this work can easily be adapted7or similar domains. We believe that our work provides valuable insights into contributor behavior on onlineplatforms during the COVID-19 pandemic and illustrates a plethora of possibilities for future work.
Methods
Data procurement and preprocessing.
We utilize the openly available MediaWiki history dataset dumps toanalyze a varied sample of 12 Wikipedia language editions (“Wikipedias”).
Wikipedia language editions.
We investigate 12 Wikipedias (Supplementary Table 3), consisting of languages pri-marily spoken in European countries that were exposed to the outbreak of COVID-19 in the spring of 2020, as wellas two Asian Wikipedias. Our choice of language editions takes into consideration: (i) the size of the Wikipediaedition, (ii) whether the language is spoken in relatively few countries, and (iii) the mobility restrictions imposed inthese countries—three criteria that are often very difficult to simultaneously satisfy. Overall, we aim to capture rel-evant Wikipedias that represent different attitudes towards the crisis, preferably from languages easily attributableto a single country or region. Accordingly, our sample contains regions with strict (e.g., Italian, Serbian, or French)and less stringent mobility restrictions (e.g., Japanese, Korean, or Swedish). Although it can not be attributed toa single country, we include English as it is the largest language edition. We employ the number of edits in 2019 asa metric to categorize the 12 Wikipedias we studied as either large (English, French, German, Italian, with morethan million edits), medium (Swedish, Korean, Japanese, Dutch, with . million to million edits), or small (Serbian, Norwegian, Danish, Finnish, with less than . million edits). MediaWiki history dataset dumps.
We retrieve the monthly updated MediaWiki history dataset dumps providedby the Wikimedia Foundation (WMF) and perform additional preprocessing before computing as well as plottingour results. The denormalized MediaWiki history dumps are generated from the full history logs stored in theWMF’s MediaWiki databases. During their generation, WMF’s automatic scripts reconstruct and enrich user andpage history with additional data, and also automatically validate the dumps to prevent errors. After WMF’spreprocessing, the dataset contains fields with precomputed standard metrics, such as revert information, bot users,number of user contributions, or time since a user’s last revision. The technical documentation on Wikitech closerdescribes the dataset dumps’ schema and contained fields. Overall, each entry in the dump consists of 70 fields withevent information. Fields are grouped into entities, which bear information about either revision , page , or user . Preprocessing.
In the MediaWiki history dataset, we only consider edits to articles by excluding all pages not inthe Wikipedia article namespace (“ns0”), thus removing revisions to talk pages or other content. Furthermore, weutilize corresponding dataset fields to distinguish human editors (anonymous or registered) from bots and markcertain revisions as reverts. Moreover, we convert MediaWiki history timestamps from Coordinated Universal Time(UTC) to the timezone of the local Wikipedia language edition. For Wikipedias in which languages can not beattributed to a single timezone (e.g., French), we choose the timezone with the highest volunteer population forthe given Wikipedia. We do not apply timestamp conversion for the English Wikipedia. Lastly, we detect articleswhich are strongly related to COVID-19 via an algorithm by Diego Sáez-Trumper , which recognizes COVID-19articles based on their Wikidata [44] links to the main COVID-19 pages. Metrics.
To make sense of which exact data fields in the MediaWiki history dumps we utilize to compute ourmetrics, please refer to the code repository (see Code availability).
Edit Volume.
We define edit volume as the number of daily revisions to pages in the article namespace (“ns0”) bynon-bot users (anonymous or registered).
Newcomers.
For each Wikipedia language edition and day, we specify the amount of newcomers as the numberof registered editors which perform their first article edit in that Wikipedia language on the given day. Throughrecognizing new editors by their first edit, we measure the exact day they become a contributor in a languageedition. Note that the number of daily registered users is generally much higher than the number of newcomersas computed in this work. However, as our study aims to quantify volunteer contribution, we choose to identifynewcomers by their first actual contribution in a given Wikipedia.
Revert rate.
Editors and bots revert article revisions to undo changes which they deem unwarranted. Frequently,these reverts correct revisions which arise from conflicts, edit wars, or vandalism [50]. Additionally, literature showsthat revisions by newcomers are more likely to be reverted than those of veteran editors [22]. For this research,we only consider reverts to articles that undo all changes and subsequently create a new revision which exactlymatches a previous article version (i.e., identity reverts). We calculate the daily revert rate by dividing the number https://dumps.wikimedia.org/other/mediawiki_history/readme.html https://w.wiki/uzW https://covid-data.wmflabs.org/
8f identity reverts (by humans or bots) by the number of non-bot edits on this given day. Correspondingly, revertrate relates the amount of reverts to the amount of human contribution.
Daily editors by activity level.
We measure daily active editors in a Wikipedia by counting the number of registered,non-bot users which perform revisions in the article namespace. To detect effects across the editor population, wecollect data for multiple activity levels, keeping count of how many editors perform 1 to 4, 5 to 24, 25 to 99, ormore than 99 daily edits. In contrast to other metrics, we do not compute the number of daily editors from theWikimedia history dumps, but retrieve it via the Wikimedia REST API instead. Changepoint detection.
We adopt the approach by Horta Ribeiro et al. [24] to automatically detect mobility and normality changepoints via Google and Apple mobility reports. These reports capture population-wide movementpatterns based on cellphone location signals and specify, on a daily basis, the percentage of time spent in variety oflocations (e.g., residential areas, workplaces, or retail). Government-mandated lockdowns and self-motivated socialdistancing measures manifest themselves as sharp changes in these mobility time series. To detect changepointsin mobility, the approach consists of a simple binary segmentation algorithm [42]. For Wikipedias of languageswidely spoken across many countries (e.g. English, German, etc), we determine a changepoint by aggregatingmobility reports for the countries in which the language is official with weights proportional to the population ofeach of these countries. Notice that the link between Wikipedia and language editions is merely approximate—inparticular for English, which is accessed from all over the world. We use the changepoints at which mobility dropsas heuristics for dates when people started spending substantially more time in their homes and term them mobilitychangepoints . To detect normality changepoints , we compute the point in time for which the future average mobilityremains within a 10% band around baseline levels before the initial mobility changepoint (defined as pre-pandemicmobility levels by Google and Apple). For languages spoken across multiple countries, we maintain the sameaggregation scheme as before. Compared to choosing specific dates, this changepoint detection approach leads tomore comparable treatments across different regions. Supplementary Table 3 summarizes the detected changepointsfor the investigated Wikipedias, which we also make available in our code repository.
Difference-in-differences setup.
To compare values of metrics during the COVID-19 pandemic with referencevalues from previous years and pre-pandemic periods, we employ a difference-in-differences regression (DiD). DiDallows us to quantify changes in these metrics in multiple Wikipedia language editions around times of region-specificmobility changepoints in early spring, while controlling for (long-term) temporal trends.Our basic DiD equation models a dependent variable’s value ( V ) as a function of the independent variables year( Y ), period ( P ), and Wikipedia language ( L ), as well as their interactions. Year is a binary variable which differ-entiates between pre-pandemic (2018 and 2019) and pandemic years (2020), whereas period encodes the treatmentperiod via a binary variable, in our case represented by the pre- and post-phases of the region-specific mobilitychangepoints. Lastly, we model our 12 Wikipedia language versions with a categorical variable to control forlanguage-specific effects. To account for outliers and normalize regression results across various-sized Wikipedias,we use logarithmic scales for V . Literature often refers to our setup, which uses three independent variables, as“triple-difference” or “difference-in-difference-in-difference” estimators [20, 31]. Mathematically, our DiD setup is: V = β + β (cid:62) L + β Y + β P + β (cid:62) ( Y L ) + β (cid:62) ( P L ) + β ( Y P ) + β (cid:62) ( Y P L ) + ε (1)We depict the 12 Wikipedia language versions as a vector of 11 binary indicators ( L ). Scalar coefficients ( β , β , β , β ) describe effects for the reference language (i.e., baseline). Coefficient vectors ( β , β , β , β , printedin bold) collect language-specific effects of non-baseline Wikipedias. Lastly, ε is the normally distributed residual.Given this mathematical formulation, the coefficient β captures the change in V post mobility changepoint relativeto the baseline Wikipedia, after accounting for differences stemming from year or period alone ( β and β , resp.).We therefore compute the effect of interest for all Wikipedias via summation of β and β . For each Wikipedia, weterm this effective change in V as δ m , where m stands for the metric representing the dependent variable. Interpretation of DiD coefficients.
We now elaborate in more detail on how to interpret the coefficients of our DiDmodel. We model the categorical language variable via vector L containing 11 binary indicator variables for the 12Wikipedias. As is customary, the regression utilizes a “reference Wikipedia” baseline, which is represented by theintercept of the model given Y = 0 and P = 0 . In our setup, we arbitrarily choose Danish as the baseline. Conse-quently, β describes the respective difference between the baseline Wikipedia and the 11 non-baseline Wikipediasusing indicator variables. Thus, adding β and β yields the intercept of each language’s sub-model.The binary year variable ( Y ) indicates whether a data point lies in 2020 ( = 1 ) or in the previous two pre-pandemicyears ( = 0 ), regardless of period. As Danish represents the arbitrary baseline, the corresponding coefficient β is ascalar which describes the overall change between the pre-pandemic years (2018 and 2019) and 2020 for Danish. For https://wikimedia.org/api/rest_v1/ Y L models the language-specific effects for the change in years relative tothe baseline Wikipedia and is quantified by the corresponding coefficient vector β . Therefore, the summation of β and β is equal to the effective overall difference of 2020 to the previous two years for all Wikipedias.We model seasonal differences between pre- and post-changepoint windows via the binary period indicator ( P ).The corresponding scalar coefficient ( β ) measures the difference between before and after the mobility changepointover all years for the baseline. Consequently, P L and coefficient vector β describe the period effect for non-baseline Wikipedias in relation to the baseline. Calculating the sum of β and β then gives the total pre- andpost-changepoint effects.Lastly, the interaction between year and period ( Y P ) enables our model to capture the change in V for thebaseline Wikipedia via β , after accounting for change in Y (via β ) and P (via β ) alone. To measure this effectivechange for all Wikipedias, we employ the coefficient vector β of the three-way interaction Y P L . While β describesthe baseline’s effect, β contains the aforementioned change relative to the baseline Wikipedia. Therefore, the sumof β and β captures the effective change in V for all Wikipedias. For a single Wikipedia l and metric m , we namethis effect of interest δ m . Correspondingly, δ m describes language-specific post-changepoint effects in 2020, as itexcludes differences that are due to year or period alone. Quantifying changes in volunteer contribution.
Wikipedia is a dynamic ecosystem, in which edit behaviorand the amount of volunteer contribution can change rapidly—especially in times of turmoil. To track thesechanges and detect short-, medium-, and long-term effects of mobility restrictions on volunteer contributions, wefit our statistical model on different data-points obtained from the same longitudinal dataset. This methodology,pioneered by Gelman and Huang [17], allows us to observe trends rather than mere point estimates.We compute our DiD analysis for a sequence of post-changepoint windows, always retaining the Wikipedias’pre-changepoint periods. For each language version, we choose a fixed 30-day period before the respective mobilitychangepoint as the pre-changepoint baseline. As post-changepoint analysis intervals, we then extract a sequenceof 120 overlapping left-aligned seven-day-windows starting with the changepoints. Mathematically, we set thetreatment period to days { n, n + 1 , . . . , n + 6 } , ∀ n ∈ { , , . . . , } . For each post-changepoint window n , weperform a separate DiD analysis across all languages using the retained baseline periods. By doing so, each DiDanalysis compares the week starting at day n after the language-specific changepoint to the baseline periods. In thisdefault setup, each of the 12 Wikipedias is represented by 37 data points for every year in the DiD regression (2018,2019, and 2020), yielding a total of data points ( = (30 pre-changepoint days + 7 post-changepoint days ) × years × Wikipedias) for each of the experiments. For each Wikipedia, we conservatively detect outliersvia the Median Absolute Deviation (MAD) approach [30] with a threshold of ∗ MAD from the monthly medianand replace such outliers by the monthly median. We then build a time series of the
DiD results using δ m andapproximate the 95% two-sided confidence intervals (CI) as two standard errors. As robustness checks, we computevariations of our DiD experiments with wider window size ( days) and slightly varied mobility changepoint dates( ± days) as described in Supplementary Information (Supplementary Figs. 8, 9, 10, 11, 12, 13, 14, 15, and 16).These results corroborate the findings reported under Results. Data availability
The openly accessible MediaWiki history dataset dumps are available at https://dumps.wikimedia.org/other/mediawiki_history/readme.html . We further provide preprocessed data and results relevant to the manuscriptin the code repository at https://github.com/ruptho/wiki-volunteers-covid . Any other supplementary datais available upon request from the corresponding author.
Code availability
The code repository for this paper can be found at https://github.com/ruptho/wiki-volunteers-covid . Author contributions
T.R. retrieved the dataset, processed the data, and performed the experiments. T.R. and M.H.R. wrote the code.T.R., M.H.R., and T.S. analyzed the data. T.R., M.H.R., T.S., F.L., M.S., and D.H. conceived and designed theexperiments, developed the arguments, and wrote the paper.10 ompeting interests
The authors declare no competing interests.
References [1]
Al Tamime, R., Giordano, R., and Hall, W.
Observing burstiness in Wikipedia articles during newdisease outbreaks. In
Proceedings of the 10th ACM Conference on Web Science (2018), pp. 117–126.[2]
Alexa . The top 500 sites on the Web. , 2021. Accessed 2021-01-07.[3]
Atchison, C. J., Bowman, L., Vrinten, C., Redd, R., Pristera, P., Eaton, J. W., and Ward, H.
Perceptions and behavioural responses of the general public during the COVID-19 pandemic: A cross-sectionalsurvey of UK adults. medRxiv (2020).[4]
Bloom, N., Bunn, P., Mizen, P., Smietanka, P., and Thwaites, G.
The impact of COVID-19 onproductivity. Working Paper 28233, National Bureau of Economic Research, December 2020.[5]
Butler, B., Joyce, E., and Pike, J.
Don’t look now, but we’ve created a bureaucracy: the nature and rolesof policies and rules in Wikipedia. In
Proceedings of the SIGCHI conference on human factors in computingsystems (2008), pp. 1101–1110.[6]
Chang, S., Pierson, E., Koh, P. W., Gerardin, J., Redbird, B., Grusky, D., and Leskovec, J.
Mobility network models of COVID-19 explain inequities and inform reopening.
Nature (2020), 1–6.[7]
Chetty, R., Friedman, J. N., Hendren, N., Stepner, M., and Team, T. O. I.
The economic impactsof COVID-19: Evidence from a new public database built using private sector data. Working Paper 27431,National Bureau of Economic Research, June 2020.[8]
Ciampaglia, G. L., and Taraborelli, D.
MoodBar: Increasing new user retention in Wikipedia throughlightweight socialization. In
Proceedings of the 18th ACM Conference on Computer Supported CooperativeWork & Social Computing (2015), pp. 734–742.[9]
Colavizza, G.
COVID-19 research in Wikipedia. bioRxiv (2020).[10]
Collier, B., and Bear, J.
Conflict, criticism, or confidence: An empirical examination of the gender gapin Wikipedia contributions. In
Proceedings of the ACM 2012 Conference on Computer Supported CooperativeWork (New York, NY, USA, 2012), CSCW ’12, Association for Computing Machinery, p. 383–392.[11]
Desvars-Larrive, A., Dervic, E., Haug, N., Niederkrotenthaler, T., Chen, J., Di Natale, A.,Lasser, J., Gliga, D. S., Roux, A., Chakraborty, A., et al.
A structured open dataset of governmentinterventions in response to COVID-19. medRxiv (2020).[12]
Feldmann, A., Gasser, O., Lichtblau, F., Pujol, E., Poese, I., Dietzel, C., Wagner, D., Wichtl-huber, M., Tapiador, J., Vallina-Rodriguez, N., et al.
The lockdown effect: Implications of theCOVID-19 pandemic on internet traffic. In
Proceedings of the ACM Internet Measurement Conference (2020),pp. 1–18.[13]
Flaxman, S., Mishra, S., Gandy, A., Unwin, H. J. T., Mellan, T. A., Coupland, H., W-taker,C., Zhu, H., Berah, T., Eaton, J. W., et al.
Estimating the effects of non-pharmaceutical interventionson COVID-19 in Europe.
Nature 584 , 7820 (2020), 257–261.[14]
Folke, C., Carpenter, S. R., Walker, B., Scheffer, M., Chapin, T., and Rockström, J.
Resiliencethinking: integrating resilience, adaptability and transformability.
Ecology and society 15 , 4 (2010).[15]
Gallotti, R., Valle, F., Castaldo, N., Sacco, P., and De Domenico, M.
Assessing the risks ofinfodemics in response to COVID-19 epidemics. arXiv preprint arXiv:2004.03997 (2020).[16]
Garcia, D., and Rimé, B.
Collective emotions and social resilience in the digital traces after a terroristattack.
Psychological science 30 , 4 (2019), 617–628.1117]
Gelman, A., and Huang, Z.
Estimating incumbency advantage and its variation, as an example of abefore–after study.
Journal of the American Statistical Association 103 , 482 (2008), 437–446.[18]
Giles, J.
Internet encyclopaedias go head to head, 2005. Nature Publishing Group.[19]
Gozzi, N., Tizzani, M., Starnini, M., Ciulla, F., Paolotti, D., Panisson, A., and Perra, N.
Collective response to the media coverage of COVID-19 pandemic on Reddit and Wikipedia. arXiv preprintarXiv:2006.06446 (2020).[20]
Gruber, J.
The incidence of mandated maternity benefits.
The American economic review (1994), 622–641.[21]
Halfaker, A., Geiger, R. S., Morgan, J. T., and Riedl, J.
The rise and decline of an open collaborationsystem: How Wikipedia’s reaction to popularity is causing its decline.
American Behavioral Scientist 57 , 5(2013), 664–688.[22]
Halfaker, A., Kittur, A., and Riedl, J.
Don’t bite the newbies: how reverts affect the quantity andquality of Wikipedia work. In
Proceedings of the 7th international symposium on wikis and open collaboration (2011), pp. 163–172.[23]
Hill, B. M., and Shaw, A.
The Wikipedia gender gap revisited: Characterizing survey response bias withpropensity score estimation.
PloS one 8 , 6 (2013), e65782.[24]
Horta Ribeiro, M., Gligorić, K., Peyrard, M., Lemmerich, F., Strohmaier, M., and West, R.
Sudden attention shifts on Wikipedia during the COVID-19 crisis. arXiv preprint arXiv:2005.08505 (2020).[25]
Jay, J., Bor, J., Nsoesie, E. O., Lipson, S. K., Jones, D. K., Galea, S., and Raifman, J.
Neigh-bourhood income and physical distancing during the COVID-19 pandemic in the United States.
Nature humanbehaviour (2020), 1–9.[26]
Keegan, B., and Fiesler, C.
The evolution and consequences of peer producing Wikipedia’s rules.
Pro-ceedings of the International AAAI Conference on Web and Social Media 11 , 1 (May 2017).[27]
Keegan, B. C., and Tan, C.
A quantitative portrait of Wikipedia’s high-tempo collaborations during the2020 coronavirus pandemic. arXiv preprint arXiv:2006.08899 (2020).[28]
Kittur, A., Suh, B., Pendleton, B. A., and Chi, E. H.
He says, she says: conflict and coordination inWikipedia. In
Proceedings of the SIGCHI conference on Human factors in computing systems (2007), pp. 453–462.[29]
Lemmerich, F., Sáez-Trumper, D., West, R., and Zia, L.
Why the world reads Wikipedia: Beyondenglish speakers. In
Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining (2019), pp. 618–626.[30]
Leys, C., Ley, C., Klein, O., Bernard, P., and Licata, L.
Detecting outliers: Do not use standard devi-ation around the mean, use absolute deviation around the median.
Journal of Experimental Social Psychology49 , 4 (2013), 764 – 766.[31]
Mian, A., and Sufi, A.
House prices, home equity-based borrowing, and the US household leverage crisis.
American Economic Review 101 , 5 (2011), 2132–56.[32]
Miquel-Ribé, M., and Laniado, D.
Cultural identities in Wikipedias. In
Proceedings of the 7th 2016International Conference on Social Media & Society (2016), pp. 1–10.[33]
Ransbotham, S., and Kane, G.
Membership turnover and collaboration success in online communities:Explaining rises and falls from grace in Wikipedia.
MIS Quarterly 35 (09 2011), 613–628.[34]
Robert Jr, L. P., and Romero, D. M.
The influence of diversity and experience on the effects of crowdsize.
Journal of the Association for Information Science and Technology 68 , 2 (2017), 321–332.[35]
Scheffer, M., Westley, F., and Brock, W.
Slow response of societies to new problems: causes andcosts.
Ecosystems 6 , 5 (2003), 493–502.[36]
Scheffer, M., and Westley, F. R.
The evolutionary basis of rigidity: locks in cells, minds, and society.
Ecology and Society 12 , 2 (2007). 1237]
Shaw, A., and Hargittai, E.
The pipeline of online participation inequalities: The case of Wikipediaediting.
Journal of communication 68 , 1 (2018), 143–168.[38]
Suh, B., Convertino, G., Chi, E. H., and Pirolli, P.
The singularity is not near: Slowing growth ofWikipedia. In
Proceedings of the 5th International Symposium on Wikis and Open Collaboration (New York,NY, USA, 2009), WikiSym ’09, Association for Computing Machinery.[39]
Suh, J., Horvitz, E., White, R. W., and Althoff, T.
Population-scale study of human needs during theCOVID-19 pandemic: Analysis and implications. arXiv preprint arXiv:2008.07045 (2020).[40]
Sultana, A., Tasnim, S., Bhattacharya, S., Hossain, M. M., and Purohit, N.
Digital screen timeduring COVID-19 pandemic: A public health concern. osf.io/preprints/socarxiv/e8sg7 , Sep 2020.[41]
Sumi, R., Yasseri, T., et al.
Edit wars in wikipedia. In (2011),IEEE, pp. 724–727.[42]
Truong, C., Oudre, L., and Vayatis, N.
Selective review of offline change point detection methods.
SignalProcessing 167 (2020), 107299.[43]
Tsvetkova, M., García-Gavilanes, R., Floridi, L., and Yasseri, T.
Even good bots fight: The caseof wikipedia.
PloS one 12 , 2 (2017), e0171774.[44]
Vrandečić, D., and Krötzsch, M.
Wikidata: a free collaborative knowledgebase.
Communications of theACM 57 , 10 (2014), 78–85.[45]
Wikimedia Foundation . Responding to COVID-19: How we can help in this time of uncertainty, Oct 2020.[46]
World Health Organization . The world health organization and wikimedia foundation expand access totrusted information about COVID-19 on Wikipedia, Oct 2020.[47]
Xiong, W., and Lagerström, R.
Threat modeling–a systematic literature review.
Computers & security84 (2019), 53–69.[48]
Xu, B., and Li, D.
An empirical study of the motivations for content contribution and community partici-pation in Wikipedia.
Information & management 52 , 3 (2015), 275–286.[49]
Yasseri, T., Sumi, R., and Kertész, J.
Circadian patterns of wikipedia editorial activity: A demographicanalysis.
PloS one 7 , 1 (2012), e30091.[50]
Yasseri, T., Sumi, R., Rung, A., Kornai, A., and Kertész, J.
Dynamics of conflicts in Wikipedia.
PloSone 7 , 6 (2012), e38869.[51]
Zhang, A. F., Livneh, D., Budak, C., Robert, L., and Romero, D.
Shocking the crowd: The effect ofcensorship shocks on chinese Wikipedia.
Proceedings of the International AAAI Conference on Web and SocialMedia 11 , 1 (May 2017).[52]
Zhang, A. F., Wang, R., Blohm, E., Budak, C., Robert Jr, L. P., and Romero, D. M.
Participationof new editors after times of shock on Wikipedia. In
Proceedings of the International AAAI Conference onWeb and Social Media (2019), vol. 13, pp. 560–571.[53]
Zheng, L., Albano, C. M., Vora, N. M., Mai, F., and Nickerson, J. V.
The roles bots play inWikipedia.
Proceedings of the ACM on Human-Computer Interaction 3 , CSCW (2019), 1–20.[54]
Zheng, Q., Jones, F. K., Leavitt, S. V., Ung, L., Labrique, A. B., Peters, D. H., Lee, E. C., andAzman, A. S.
HIT-COVID, a global database tracking public health interventions to COVID-19.
Scientificdata 7 , 1 (2020), 1–8. 13 upplementary Information
Wikipedia Statistics
This supplementary section summarizes Wikipedia statistics relevant to this study. First, we visualize the total,monthly, weekly, as well as daily edits to Wikipedia in the year before the COVID-19 pandemic began ( ) inSupplementary Table 1. Secondly, Supplementary Table 2 shows statistics describing the yearly growth of Wikipediabetween 2015 and 2020 until October of each year. Finally, Supplementary Table 3 lists all Wikipedias we studied,alongside their automatically detected mobility changepoints (Methods).Table 1:
Edits to Wikipedia in 2019.
Overall, Wikipedia language editions were subject to
193 412 million visitsand . million content edits by non-bot users in . This translates to about million visits and thousand edits per day. Total page views, via Wikimedia Statistics ( https://w.wiki/vkf ) User edits, via Wikimedia Statistics ( https://w.wiki/vkm ) In 2019 Views (in Millions) Edits (in Millions)Total
193 412 .
78 319 . Monthly
16 117 .
73 26 . Weekly .
48 6 . Daily . . Table 2:
Growth between 2019 and 2020 in the English Wikipedia nearly doubles the relative increasebetween 2015 and 2019.
Edit growth in Wikipedia stagnated in recent years. From 2019 to 2020, edits grewby . ( . million edits), that is nearly double the growth between 2015 and 2019 ( . , or about . millionedits). In 2018 and 2019, human edits even declined in comparison to previous years. Therefore, the developmentsin 2020 mark a clear difference to the downward trend of edits in Wikipedia in the last 5 years. Edits by anonymous or registered users, via Wikimedia Statistics ( https://w.wiki/u8R ) Year Non-Bot Edits(until September 31 st ) Difference to Previous Year Difference to 2019Edits Percent Edits Percent
25 943 729 – – − − .
27 095 495 1 151 766 4 . −
11 485 − .
27 406 797 311 302 1 .
15 299 817 1 .
27 374 765 −
32 032 − .
12 267 785 0 .
27 106 980 −
267 785 − . – –2020
29 386 750 2 279 770 8 .
41 2 279 770 7 . Number of editors by daily activity level during COVID-19 mobility restrictions
On Wikipedia, human users contribute edits with varying daily intensity. Wikipedia categorizes editors into fivegroups, according to their daily activity: 1 to 4, 5 to 24, 25 to 99, and more than 99 daily edits. We retrieve thenumber of registered editors (and their activity level) via the Wikimedia REST API and apply DiD analysis todetect significant changes across the editor population (Methods).We again remove outliers before performing DiD analysis (see Methods) and visualize the results for all dailyactivity levels in Supplementary Figures 1 (1 to 4 edits), 2 (5 to 24 edits), 3 (25 to 99 edits), and 4 (more than 99edits). The results corroborate our previous newcomer and edit volume findings, as the number of editors increasessignificantly after mobility changepoints. Our findings signal an increase in contribution across all activity levelsfor the editor population, particularly in large and medium Wikipedias, while results for small Wikipedias remainconsistent with pre-pandemic baselines.
Revert rate during COVID-19 mobility restrictions
The supplementary explanations in this section extend the revert rate analysis carried out in Discussion. We plot therolling 7-day average revert rate in Supplementary Figure 5a as well as logarithmic effects for δ rr captured by DiDanalysis with revert rate as the dependent variable in Supplementary Figure 5b for large (top), medium (middle),and small (bottom) Wikipedias. We find recession of revert rates for most Wikipedias during the initial weeks14able 3: Wikipedia language versions.
The 12 Wikipedia language editions relevant to this study, ordered bythe total number of edits (bot or non-bot) in 2019, including mobility and normality changepoint dates (Methods).
Language Changepoints (2020) Wikipedia VersionMobility Normality Code Edits in 2019 (Millions)
English 03/16 05/21 en 40.56French 03/16 07/02 fr 7.45German 03/16 07/10 de 7.33Italian 03/11 06/26 it 5.80Japanese 03/31 06/14 ja 3.84Swedish 03/11 06/05 sv 2.73Dutch 03/16 05/29 nl 1.78Korean 02/25 04/15 ko 1.61Serbian 03/16 05/02 sr 1.29Norwegian 03/11 06/04 no 0.71Finnish 03/16 05/21 fi 0.65Danish 03/11 06/05 da 0.31of mobility restrictions, possibly indicating a reduction in negative contributions that need to be reverted (e.g.,vandalism). Coefficient values for δ rr support this sentiment for large and particular medium or small Wikipedias.Nevertheless, it must be mentioned that revert rate can not be interpreted so simply, as specific bots periodicallyrefactor revisions (e.g., monthly, quarterly) or some editor groups conduct article maintenance in coordinated events.Such difficult-to-predict patterns might be especially notable in Wikipedias with a generally lower amount of reverts,which is often the case in smaller Wikipedias. However, even these spontaneous patterns that would normally driveup revert rates appear to be mostly muted during the COVID-crisis.Revert rates in the large English, German, and French Wikipedias drop after mobility restrictions come intoeffect in March 2020 (Supplementary Fig. 5a, top). For English, we observe an average revert rate of . inthe month before mobility restrictions take effect and . in the month after. Revert rate for both German andFrench averages approximately . before the changepoints, but reaches respective minima of . and . inthe subsequent weeks. DiD analysis and corresponding δ rr (Supplementary Fig. 5b, top) confirm significant relativedecreases by measuring respective logarithmic effects of − . and − . for the French and German Wikipedia,signaling a -decline for both Wikipedias ( e − . ≈ e − . ≈ of previous levels). Italian shows a similardrop in revert rate ( − ). Relative decrease for English is considerably lower ( − ), but is deemed significantby our DiD analysis. Altogether, we find significant decreases in revert rate for all large Wikipedias.Most medium and small Wikipedias seem to not exhibit considerable negative effects for revert rates (Supple-mentary Fig. 5a, middle and bottom). However, instead of explicitly showing visible dips in the revert rate graphs,the general level seems to be subdued during the investigate periods in 2020, especially close to the mobility change-points. As an exception, Korean is the only language version that increases its revert rate during times of mobilityrestrictions, from . before the changepoint to a maximum of . in the month thereafter. DiD analysis revealssignificant relative decreases in δ rr (Supplementary Fig. 5b, middle and bottom) for the medium Japanese ( − ),Dutch ( − ), and Swedish ( − ) Wikipedias within two months post changepoint, as well as for the smallerNorwegian ( − ), Serbian ( − ), and Danish ( − ) Wikipedias. We explain some of these significant effectsby the generally higher revert rate in the same period in previous years. Although our DiD analysis uncovers thesesignificant short-term declines in revert rates for medium and small Wikipedias, results must be taken with a grainof salt due to the aforementioned nature of reverts in smaller Wikipedias. Edit volume in articles not related to COVID-19
We investigate the impact of articles strongly related to COVID-19 (Methods) on the edit volume on Wikipedia.Supplementary Table 4 lists information about the total percentage of edits to COVID-19 articles, as well as thepercentage of edited articles that are related to COVID-19. Supplementary Figure 6 shows the percentage of editsgoing towards articles strongly related to COVID-19 articles, as well as the overall percentage of edited articles thatwere strongly related to COVID-19. It appears that edits to COVID-19 articles make up an insignificant accountof daily activity in most Wikipedias (mostly < ), whereas some Wikipedias, for example German (at one point daily COVID-19 edits), have a somewhat higher but short-lived affinity for COVID-19 topics.To quantify the effect of COVID-19 articles on overall edit volume, we perform the same DiD analysis for15 .057.278.50 e n f r d e i t English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.4-0.20.00.20.40.6
Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t o r s ( t o e d i t s ) M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 1:
Editors with 1 to 4 Edits during COVID-19 mobility restrictions. a , We depict the 7-day rollingaverage of active registered editors with an activity level of 1 to 4 daily edits in the context of COVID-19 mobilityrestrictions, delineated via mobility and normality changepoints. b , We show the relative change in active registerededitors ( ae ) with 1 to 4 edits per day as retrieved from the DiD via δ ae (95% confidence intervals as two standarddeviations. The number of editors with 1 to 4 daily edits increases significantly after mobility restrictions come intoeffect, for all but a few medium and small Wikipedias.edit volume as in Results, this time specifically excluding edits to articles that are strongly related to COVID-19.Supplementary Figure 7 visualizes the performed DiD analysis for edit volume, excluding edits to COVID-19 articles.We find that excluding edits to COVID-19 articles does not significantly alter the results reported beforehand. Robustness checks for edit volume, newcomers, and revert rate
As robustness checks, we perform variations of our DiD experiments for edit volume, newcomers, and revert rate.Supplementary Figures 8, 9, and 10 visualize DiD with a 14-day post-changepoint period. Supplementary Figures 11,12, and 13 depict DiD with 7-day post-changepoint periods, but move mobility changepoints to seven days before theactual dates. Similarly, Supplementary Figures 14, 15, and 16 move mobility changepoints to seven days after theactual changepoints. Our DiD robustness checks show that longer post-changepoint periods or modified changepointdates do not significantly influence results and prove the robustness of our methodology.16 .932.322.70 e n f r d e i t English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.8-0.6-0.4-0.20.00.20.40.6
Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t o r s ( t o e d i t s ) M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 2:
Editors with 5 to 24 Edits during COVID-19 mobility restrictions. a , We depict the 7-dayrolling average of active registered editors with an activity level of 5 to 24 daily edits in the context of COVID-19mobility restrictions, delineated via mobility and normality changepoints. b , We show the relative change in activeregistered editors ( ae ) with 5 to 24 edits per day as retrieved from the DiD via δ ae (95% confidence intervals astwo standard deviations. In all large and most medium Wikipedias the number of editors with 5 to 24 daily editssignificantly increases over longer periods of time, while smaller Wikipedias do not consistently increase their editornumbers for this activity level.Table 4: COVID-19 edits and edited COVID-19 articles.
We list the total percentage of non-bot edits toCOVID-19 articles as well as the percentage of edited articles that were related to COVID-19 for the total timespan between January 1 st and September 31 st Code % of Edits toCOVID-19Articles % of Edited Articlesthat areCOVID-19Articles Max. Daily %of Edits toCOVID-19Articles Max. Daily % ofEdited Articlesthat areCOVID-19Articles de 2.40 0.44 17.0 1.28fr 0.73 0.24 3.44 1.00it 0.29 0.12 1.78 0.72sr 0.02 0.02 1.27 0.65no 0.12 0.07 2.19 1.10ko 0.04 0.03 1.15 0.26da 0.33 0.16 6.35 1.55sv 0.25 0.07 3.99 0.49ja 0.22 0.11 1.49 0.46nl 0.64 0.21 4.23 1.49fi 0.75 0.37 4.12 1.43en 1.03 0.33 4.44 0.8217 .345.286.21 e n f r d e i t English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -1.0-0.50.00.51.0
Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t o r s ( t o E d i t s ) M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 3:
Editors with 25 to 99 Edits during COVID-19 mobility restrictions. a , We depict the 7-dayrolling average of active registered editors with an activity level of 25 to 99 daily edits in the context of COVID-19mobility restrictions, delineated via mobility and normality changepoints. b , We show the relative change in activeregistered editors ( ae ) with 25 to 99 edits per day as retrieved from the DiD via δ ae (95% confidence intervals astwo standard deviations. We observe significant increases for the number of registered editors who perform 25 to 99daily edits during COVID-19 mobility restrictions in large Wikipedias. Results for medium and small Wikipediasare mostly inconsistent. 18 .590.811.02 e n f r d e i t English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.573.506.43 k o Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Jan Feb Mar Apr May Jun Jul Aug Sep Oct
Date(a) d a Day after Mobility Changepoint(b) -1.0-0.50.00.51.0
Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t o r s ( > E d i t s ) M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 4:
Editors with more than 99 Edits during COVID-19 mobility restrictions. a , We depict the 7-dayrolling average of active registered editors with an activity level of 25 to 99 daily edits in the context of COVID-19mobility restrictions, delineated via mobility and normality changepoints. b , We show the relative change in activeregistered editors ( ae ) with 25 to 99 edits per day as retrieved from the DiD via δ ae (95% confidence intervals as twostandard deviations. For all large Wikipedias, besides German, we find increased counts for editors with more than99 edits per day. Additionally, some medium Wikipedias exhibit significantly more of these high-intensity editors,while smaller Wikipedias show no strong significant trends.19 .750.961.18 e n
1e 1 f r
1e 24.35.87.4 d e
1e 2Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.620.871.12 i t
1e 1 0 10 20 30 40 50 60 70 80 90 100 110-0.4-0.20.00.2 rr English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a
1e 20.370.871.36 s v
1e 10.330.821.31 n l
1e 1Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.430.751.07 k o
1e 1 0 10 20 30 40 50 60 70 80 90 100 110-0.6-0.4-0.20.00.20.40.6 rr Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r
1e 10.240.671.09 n o
1e 13.05.98.8 f i
1e 2Jan Feb Mar Apr May Jun Jul Aug Sep Oct
Date(a) d a
1e 1 0 10 20 30 40 50 60 70 80 90 100 110
Day after Mobility Changepoint(b) -1.0-0.50.00.51.0 rr Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) R e v e r t R a t e M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 5:
Revert rate during COVID-19 mobility restrictions.
We depict results for revert rate in large(top), medium (middle), and small (bottom) Wikipedias during COVID-19 mobility restrictions, which in thefigure we delineate using mobility (when restrictions become effective) and normality (when restrictions are lifted)changepoints. a , We plot rolling 7-day average revert rate in 2018, 2019, and 2020 up until October. Even thoughour analysis showed that newcomers and edit volume increased in 2020, especially after the mobility changepoint,we do not observe increases for revert rate in any Wikipedias. b , We calculate the relative change in revert rate( rr ) to pre-changepoint periods from the DiD via δ rr (95% confidence intervals as two standard deviations), andplot δ rr for 120 left-aligned seven-day-windows (see Methods), beginning with the respective mobility changepoint.We detect no significant increase after mobility restrictions come into effect in virtually all Wikipedias, with theexception of Korean, and even find decreased revert rates for most large and medium Wikipedias.20 .0%5.0%10.0%15.0% L a r g e W i k i p e d i a s Percentage of edits aimed toward COVID-19 Articles
Percentage of edited articles that are COVID-19 articles
EnglishFrenchGermanItalian0.0%2.0%4.0% M e d i u m W i k i p e d i a s Date(a) S m a ll W i k i p e d i a s Jan2020 Feb Mar Apr May Jun Jul Aug Sep
Date(b)
Figure 6:
COVID-19 edits and edited COVID-19 articles.
We visualize the percentage of COVID-19 articleedits per day and daily edited COVID-19 articles for large (top), medium (middle), and small (bottom) Wikipedias. a , We show the daily percentage of non-bot edits to COVID-19 articles between January 1 st and September 31 st b , We visualize the daily percentage of edited articles that were related to COVID-19 in 2020 until October.21 .891.041.18 e n f r d e i t e v English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o e v Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.20.00.20.50.81.01.2 e v Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t V o l u m e ( N o n - C o v i d ) M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 7:
Edit volume in articles not related to COVID-19 during mobility restrictions.
We showedit volume in non-COVID-19 articles in large (top), medium (middle), and small (bottom) Wikipedias during themobility restrictions, which we delineate using mobility (when restrictions become effective) and normality (whenrestrictions are lifted) changepoints. a , We show rolling 7-day average edit volume generated by human editors for2018, 2019, and 2020 until October. b , We depict relative change in edit volume ( ev ) as retrieved from DiD via δ ev (95% confidence interval as two standard deviations) and plot δ ev for 120 left-aligned seven-day-windows. Edits toarticles closely related to COVID-19 mostly only make up a small fraction of daily edits (Supplementary Table 4and Supplementary Fig. 6). Accordingly, we observe that findings for edit volume excluding COVID-19 edits barelydiffer from those for overall edit volume depicted in Figure 2. Although a minimal visual effect is observable for afew select Wikipedias (e.g., German), it does not affect significance of our results.22 .941.061.19 e n f r d e i t e v English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o e v Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.5-0.20.00.20.50.81.0 e v Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t V o l u m e M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 8:
Edit volume during COVID-19 mobility restrictions (14-day windows).
We show results ofour 14-day window robustness experiment for edit volume in large (top), medium (middle), and small (bottom)Wikipedias during COVID-19 mobility restrictions, delineated using mobility (when restrictions become effective)and normality (when restrictions are lifted) changepoints. a , We show rolling 14-day average daily edit volumegenerated by human editors for 2018, 2019, and 2020 until October. b , We depict relative change in edit volume( ev ) as retrieved from DiD via δ ev (95% confidence interval as two standard deviations) and plot δ ev for 120left-aligned fourteen-day-windows. Edit volume results for 14-day windows represent the same trends and similarsignificant effects as previous experiments (Figure 2), only smoothening the 7-day-window results more.23 .401.762.11 e n f r d e i t n c English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o n c Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.50.00.51.0 n c Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) N e w c o m e r s M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 9:
Newcomers during COVID-19 mobility restrictions (14-day windows).
We show results ofour 14-day window robustness experiment for newcomers in large (top), medium (middle), and small (bottom)Wikipedias during COVID-19 mobility restrictions, delineated using mobility (when restrictions become effective)and normality (when restrictions are lifted) changepoints. a , We show rolling 14-day average newcomer counts for2018, 2019, and 2020 until October. b , We depict relative change in newcomers ( nc ) as retrieved from DiD via δ nc (95% confidence interval as two standard deviations) and plot δ nc for 120 left-aligned fourteen-day-windows. New-comer results for 14-day windows represent the same trends and similar significant effects as previous experiments(Figure 3), only smoothening the 7-day-window results more.24 .770.941.11 e n
1e 1 f r
1e 24.45.87.3 d e
1e 2Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.660.861.07 i t
1e 1 0 10 20 30 40 50 60 70 80 90 100 110-0.4-0.20.00.2 rr English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a
1e 20.390.811.22 s v
1e 10.410.781.14 n l
1e 1Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.460.731.01 k o
1e 1 0 10 20 30 40 50 60 70 80 90 100 110-0.4-0.20.00.20.40.6 rr Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r
1e 12.96.49.8 n o
1e 23.45.78.0 f i
1e 2Jan Feb Mar Apr May Jun Jul Aug Sep Oct
Date(a) d a
1e 1 0 10 20 30 40 50 60 70 80 90 100 110
Day after Mobility Changepoint(b) -1.0-0.50.00.51.0 rr Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) R e v e r t R a t e M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 10:
Revert rate during COVID-19 mobility restrictions (14-day windows).
We show results ofour 14-day window robustness experiment for revert rate in large (top), medium (middle), and small (bottom)Wikipedias during COVID-19 mobility restrictions, delineated using mobility (when restrictions become effective)and normality (when restrictions are lifted) changepoints. a , We show rolling 14-day average revert rate for 2018,2019, and 2020 until October. b , We depict relative change in revert rate ( rr ) as retrieved from DiD via δ rr (95%confidence interval as two standard deviations) and plot δ rr for 120 left-aligned fourteen-day-windows. Revertrate results for 14-day windows represent the same trends and similar significant effects as previous experiments(Supplementary Fig. 5), only smoothening the 7-day-window results more.25 .891.041.19 e n f r d e i t e v English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o e v Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.5-0.20.00.20.50.81.0 e v Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t V o l u m e M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 11:
Edit volume during COVID-19 mobility restrictions (beginning 7 days earlier).
We varyour DiD for edit volume by changing the mobility (when restrictions take effect) and normality changepoint (whenrestrictions are lifted) to 7 days earlier. a , We show rolling 7-day average edit volume for 2018, 2019, and 2020 untilOctober. b , We depict relative change in edit volume ( ev ) as retrieved from DiD via δ ev (95% confidence interval astwo standard deviations) and plot δ ev for 120 left-aligned seven-day-windows. As mobility changepoints generallymark dates of decreased activity, moving the changepoint before these declines lead to the first few days representinga more negative trend and values for later days are slightly lower than in the original experiment (Figure 2), whichis an expected effect. 26 .321.722.13 e n f r d e i t n c English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o n c Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.50.00.51.0 n c Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) N e w c o m e r s M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 12:
Newcomers during COVID-19 mobility restrictions (beginning 7 days earlier).
We varyour DiD for newcomers by changing the mobility (when restrictions take effect) and normality changepoint (whenrestrictions are lifted) to 7 days earlier. a , We show rolling 7-day average newcomers for 2018, 2019, and 2020until October. b , We depict relative change in newcomers ( nc ) as retrieved from DiD via δ nc (95% confidenceinterval as two standard deviations) and plot δ nc for 120 left-aligned seven-day-windows. As mobility changepointsgenerally mark dates of decreased activity, moving the changepoint before these declines lead to the first few daysrepresenting a more negative trend, before than recovering and increasing to slightly lower values than in the originalexperiment (Figure 3), which is an expected effect. 27 .750.961.18 e n
1e 1 f r
1e 24.35.87.4 d e
1e 2Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.620.871.12 i t
1e 1 0 10 20 30 40 50 60 70 80 90 100 110-0.4-0.20.00.2 rr English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a
1e 20.370.871.36 s v
1e 10.330.821.31 n l
1e 1Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.430.751.07 k o
1e 1 0 10 20 30 40 50 60 70 80 90 100 110-0.4-0.20.00.20.40.6 rr Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r
1e 10.240.671.09 n o
1e 13.05.98.8 f i
1e 2Jan Feb Mar Apr May Jun Jul Aug Sep Oct
Date(a) d a
1e 1 0 10 20 30 40 50 60 70 80 90 100 110
Day after Mobility Changepoint(b) -1.0-0.50.00.51.0 rr Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) R e v e r t R a t e M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 13:
Revert rate during COVID-19 mobility restrictions (beginning 7 days earlier).
We varyour DiD for revert rate by changing the mobility (when restrictions take effect) and normality changepoint (whenrestrictions are lifted) to 7 days earlier. a , We show rolling 7-day average revert rate for 2018, 2019, and 2020 untilOctober. b , We depict relative change in revert rate ( rr ) as retrieved from DiD via δ rr (95% confidence interval astwo standard deviations) and plot δ rrrr
We varyour DiD for revert rate by changing the mobility (when restrictions take effect) and normality changepoint (whenrestrictions are lifted) to 7 days earlier. a , We show rolling 7-day average revert rate for 2018, 2019, and 2020 untilOctober. b , We depict relative change in revert rate ( rr ) as retrieved from DiD via δ rr (95% confidence interval astwo standard deviations) and plot δ rrrr for 120 left-aligned seven-day-windows. We find no strong differences to theoriginal experiment (Supplementary Fig. 5), as revert rates are relatively stable close to the mobility changepoint.28 .891.041.19 e n f r d e i t e v English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o e v Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.50.00.51.0 e v Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) E d i t V o l u m e M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 14:
Edit volume during COVID-19 mobility restrictions (beginning 7 days later).
We vary ourDiD for edit volume by changing the mobility (when restrictions take effect) and normality changepoint (whenrestrictions are lifted) to 7 days later. a , We show rolling 7-day average edit volume for 2018, 2019, and 2020 untilOctober. b , We depict relative change in edit volume ( ev ) as retrieved from DiD via δ ev (95% confidence interval astwo standard deviations) and plot δ ev for 120 left-aligned seven-day-windows. As mobility changepoints generallymark dates of decreased activity, moving the changepoint past these declines leads to these lower values now beingcounted towards the 30-day baseline period, generating overall higher post-changepoint values than in the originalexperiment (Figure 2). 29 .321.722.13 e n f r d e i t n c English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a s v n l k o n c Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r n o f i Date(a) d a Day after Mobility Changepoint(b) -0.50.00.51.0 n c Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) N e w c o m e r s M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 15:
Newcomers during COVID-19 mobility restrictions (beginning 7 days later).
We vary ourDiD for newcomers by changing the mobility (when restrictions take effect) and normality changepoint (whenrestrictions are lifted) to 7 days later. a , We show rolling 7-day average newcomers for 2018, 2019, and 2020 untilOctober. b , We depict relative change in newcomers ( nc ) as retrieved from DiD via δ nc (95% confidence interval astwo standard deviations) and plot δ nc for 120 left-aligned seven-day-windows. As mobility changepoints generallymark dates of decreased activity, moving the changepoint past these declines leads to these lower values now beingcounted towards the 30-day baseline period, generating overall higher post-changepoint values than in the originalexperiment (Figure 3). 30 .750.961.18 e n
1e 1 f r
1e 24.35.87.4 d e
1e 2Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.620.871.12 i t
1e 1 0 10 20 30 40 50 60 70 80 90 100 110-0.4-0.20.00.20.4 rr English (en)French (fr)German (de)Italian (it)Significance (solid) or non-significance (dashed) Normality j a
1e 20.370.871.36 s v
1e 10.330.821.31 n l
1e 1Jan Feb Mar Apr May Jun Jul Aug Sep Oct0.430.751.07 k o
1e 1 0 10 20 30 40 50 60 70 80 90 100 110-0.4-0.20.00.20.40.6 rr Japanese (ja)Swedish (sv)Dutch (nl)Korean (ko) s r
1e 10.240.671.09 n o
1e 13.05.98.8 f i
1e 2Jan Feb Mar Apr May Jun Jul Aug Sep Oct
Date(a) d a
1e 1 0 10 20 30 40 50 60 70 80 90 100 110
Day after Mobility Changepoint(b) -1.0-0.50.00.51.0 rr Serbian (sr)Norwegian (no)Finnish (fi)Danish (da) R e v e r t R a t e M e d i u m W i k i p e d i a s S m a ll W i k i p e d i a s L a r g e W i k i p e d i a s Figure 16:
Revert rate during COVID-19 mobility restrictions (beginning 7 days later).
We varyour DiD for revert rate by moving the mobility (when restrictions take effect) and normality changepoint (whenrestrictions are lifted) to 7 days later. a , We show rolling 7-day average revert rate for 2018, 2019, and 2020 untilOctober. b , We depict relative change in revert rate ( rr ) as retrieved from DiD via δ rr (95% confidence interval astwo standard deviations) and plot δ rrrr