A Deep Dive on the Impact of COVID-19 in Software Development
Paulo Anselmo da Mota Silveira Neto, Umme Ayda Mannan, Eduardo Santana de Almeida, Nachiappan Nagappan, David Lo, Pavneet Singh Kochhar, Cuiyun Gao, Iftekhar Ahmed
NNoname manuscript No. (will be inserted by the editor)
A Deep Dive on the Impact of COVID-19 inSoftware Development
Paulo Anselmo da Mota Silveira Neto · Umme Ayda Mannan · Eduardo Santanade Almeida · Nachiappan Nagappan · David Lo · Pavneet Singh Kochhar · Cuiyun Gao · Iftekhar Ahmed
Received: date / Accepted: date
Abstract Context.
COVID-19 pandemic has impacted different businesssectors around the world.
Objective.
This study investigates the impactof COVID-19 on software projects and software development professionals.
Method.
We conducted a mining software repository study based on 100GitHub projects developed in Java using ten different metrics. Next, we sur-veyed 279 software development professionals for better understanding theimpact of COVID-19 on daily activities and wellbeing.
Results.
We identified12 observations related to productivity, code quality, and wellbeing.
Con-clusions.
Our findings highlight that the impact of COVID-19 is not binary(reduce productivity vs. increase productivity) but rather a spectrum. Formany of our observations, substantial proportions of respondents have dif-fering opinions from each other. We believe that more research is needed touncover specific conditions that cause certain outcomes to be more prevalent.
Keywords
COVID-19 · Empirical Study · Survey · Mining SoftwareRepository (MSR)
The latest threat to global health is the ongoing outbreak of the respiratorydisease that was recently given the name Coronavirus Disease 2019 (COVID-19) [13]. COVID-19 was recognized in December 2019, in Wuhan, a large cityin central China [24]. Three months later, on March 11, 2020, the World HealthOrganization (WHO) characterized COVID-19 as a pandemic . According toWHO: “We have never before seen a pandemic sparked by a coronavirus. Thisis the first pandemic caused by a coronavirus. And we have never before seen a Address(es) of author(s) should be given WHO characterizes COVID-19 as a pandemic. a r X i v : . [ c s . S E ] A ug Paulo Anselmo da Mota Silveira Neto et al. pandemic that can be controlled, at the same time.”
In August 2020, COVID-19 was spread out more than 200 countries with more than 500k confirmeddeaths.The COVID-19 pandemic is considered the most crucial global healthcalamity of the century and the greatest challenge humankind faced since the2nd World War [8]. It has enormously impacted how we live and interact witheach other (social distancing, wearing masks, washing hands frequently, quar-antine, etc.) Besides the health problems, the action taken by the countriesaround the world to manage the COVID-19 pandemic, e.g., restricting travel,shuttering nonessential businesses, and implementing universal social distanc-ing policies, are having drastic economic consequences. In the United States,for example, more than 30 million Americans have filed initial unemploymentclaims since March 2020. Business sectors, such as travel and transportation, manufacturing, hotels,restaurants, live entertainment and movie, and sports are strongly impactedand they are having to adapt to this new situation. The software developmentsector is not an exception. Besides the economic aspects, suddenly, companieshad to support and (in some cases) equip office workers who quickly transi-tioned to a work-from-home set up because of the pandemic.Working from home is not a new reality for software development. Manycompanies adopt different approaches for it, such as remote teams that haveemployees in a single country or even in one city, and some/all of them workfrom home without having to go to the office every day. Professional websitessuch as Upwork, Linkedin, and Stack Overflow present several job offers towork remotely. However, working from home during a pandemic is not likeregular remote work. Additional difficulties are involved due to the lack ofproper physical infrastructure, the need to care for children with school anddaycare being closed, fear and anxiety of contracting COVID-19, etc.Previous research studies [18], [16], [7], [17] have investigated remote workand responses for a better understanding of pandemic [9], [12], [37], [38], [40].However, just a few studies [2], [31], [32] have started to investigate the im-pact of a pandemic in software development. This aspect can be justified sinceCOVID-19 is the first pandemic after World Wide Web development. Thus,to gain insights into the impact of COVID-19 in software development, ourfirst research question (
RQ1: What is the Impact of COVID-19 on Projects? )explores how the pandemic impacts open source projects considering differentperspectives. In addition to quantitative information about the projects, it isalso important to understand the impact on developers’ wellbeing; this moti-vates us to investigate another research question (
RQ2: What is the Impact ofCOVID-19 on Developers WellBeing? ).To answer these questions, we conducted a mining software repositorystudy based on 100 GitHub projects developed in Java. The Java projectswere selected according to different criteria ranging from the last update to https://edition.cnn.com/2020/04/30/economy/unemployment-benefits-coronavirus/index.html Deep Dive on the Impact of COVID-19 in Software Development 3 the number of commits. We collected ten metrics for analyzing the projects.Next, we surveyed 279 software development professionals from 32 countries.The survey asked respondents to provide feedback on the impact of COVID-19on software projects and their wellbeing.Overall, the paper makes the following contributions: – We perform a large scale quantitative study to investigate the impact ofCOVID-19 on software development based on ten different metrics. – We complement this study with a survey of how software developmentprofessionals perceive the impact of COVID-19 on daily activities. – Based on a set of observations from the mining software repository studyand survey, we provide some recommendations for practitioners, organiza-tions, and researchers. – For replication and reproducible research, we make our materials availableon our project website. These include our repository mining data (projectdata, including metrics and time series data) and survey instrument. Ourartifacts can be found at the accompanying website. In this section, we discuss the main work related to our study.2.1 COVID-19 StudiesThe COVID-19 pandemic originated the development of several multidisci-plinary initiatives around the world. The Center for Systems Science and Engi-neering (CSSE) at Johns Hopkins University, created an interactive web-baseddashboard to visualize and track COVID-19 reported cases in real-time [12].The dashboard illustrates the location and number of confirmed COVID-19cases, deaths, and recoveries for all affected countries. Zhang et al. [40] created,Neural Covidex, a search engine for clinicians, researchers, and other expertstrying to better understand COVID-19. The system offers access to the AllenInstitute for AIs COVID-19 Open Research Dataset (CORD-19). CORD-19is a curated public resource of more than 40,000 scholarly articles, medicalreports, journal articles, and preprints about COVID-19 and the coronavirusfamily of viruses. Researchers from MRC Centre for Global Infectious DiseaseAnalysis, from Imperial College London, have developed a set of tools andprediction models based on different scenarios (social distancing, shielding theelderly, and healthcare availability) [37], [38]. Chen et al. [9] created a publicCoronavirus Twitter dataset with more than 100 million tweets. According tothe authors, the dataset can help track scientific coronavirus misinformationand unverified rumors, and contribute towards enabling informed solutionsand prescribing targeted policy interventions. https://github.com/pamsn/covid-study Paulo Anselmo da Mota Silveira Neto et al. In the software engineering area, Ralph et al. [32] conducted a survey with2225 software developers to understand the effects of the COVID-19 pandemicon developers’ wellbeing and productivity. They identified that developers hadlower wellbeing and productivity while working from home due to COVID-19.In addition, disaster preparedness, fear related to the pandemic and homeoffice ergonomics all affect wellbeing and productivity; and women, parents,and people with disabilities may be disproportionately affected.Rahman and Farhana [31] conducted an empirical study with 129 opensource COVID-19 projects hosted on GitHub to identify what categories ofbugs exist in this kind of project. Initially, they identified seven categoriesof COVID-19 projects (aggregation, education, medical equipment, mining,user tracking, statistical modeling, and volunteer management). Next, apply-ing open coding on 550 bug reports, they identified eight bug categories forthese projects (algorithm, data, dependency, documentation, performance, se-curity, syntax, and user interface). Based on this taxonomy, user interface wasthe most frequent category using the proportion of bugs across all projects,and documentation was the least frequent category.Bao et al. [2] collected data from a Chinese company, which contains thedevelopment activities from 139 developers working in 8 projects over a pe-riod of 138 working days, and compared developer productivity when workingfrom home and working onsite. Part of records in the dataset included a pe-riod where developers worked from home due to COVID-19 pandemic. Theyfound that working from home has both positive and negative impacts ondeveloper productivity in terms of different metrics, such as the number ofbuilds/commits/code reviews. In addition, they identified that working fromhome has different impacts on different kinds of projects. For example, workingfrom home has a negative impact on developer productivity for large projects.They also found that the majority of developers’ productivity when work-ing from home is similar to that when working onsite. For a small portion ofdevelopers, working from home had different impacts on their productivity.2.2 ProductivityThere are a number of studies [11], [19], [28], [35], [36],that investigate devel-oper productivity. For example, Meyer et al. [26] investigated how developerworkday looks like and the relationships between their activities and perceivedproductivity. The study uncovered that productivity is a personal matter, andfactors such as emails and meetings are often considered detrimental to pro-ductivity. In another work, Murphy-Hill et al. [29] surveyed 622 developersacross 3 companies regarding self-perceived productivity and its contribut-ing factors. They uncovered that developers perceived productivity is stronglycorrelated to job enthusiasm, peer support, and reception of useful feedback.Moreover, they reported that compared to other knowledge workers, ability towork remotely is more strongly related to perceived productivity. Unlike the
Deep Dive on the Impact of COVID-19 in Software Development 5 aforementioned studies, in this work, we investigate the impact of COVID-19on software projects.2.3 WellbeingMany prior works investigate wellbeing in the workplace, e.g., [4]. However,not many investigate it in the context of software development. One such workis by Kuutila et al. [23] that investigated the relationship between developerwellbeing and software repository metrics. They reported that developers whoreported “high hurry” were less productive. Moreover, factors that hamperwellbeing (such as stress, sleeping problems, etc.) are negatively related tothe number of chat messages. Another work by Graziotin et al. [15] investi-gated what happens when developers are happy and unhappy. They foundthat unhappiness impacts the developer’s own being (in terms of low cogni-tive performance, mental unease, and disorder, etc.) and their work products(in terms of low productivity, low code quality, etc.). Happiness leads to theopposite effect (e.g., high cognitive performance, high productivity, high codequality, etc.).Johnson et al. [19] conducted a mixed-method study (two surveys andinterviews) with 1159 participants from Microsoft to understand the effectof work environments on productivity and satisfaction of software engineers.They found that personalization, social norms and signals, room compositionand atmosphere, work-related environment affordances, work area, and furni-ture, and productivity strategies were considered important factors for workenvironments. In addition, the ability to work privately with no interruptionsand the ability to communicate with the team and leads were important factorsrelated to satisfaction.Meyer et al. [25] investigated what is a good and typical workday for soft-ware developers at Microsoft. They conducted a survey with 5971 developersand identified that on good workdays, developers make progress and valueprojects they consider meaningful and spend their time efficiently, with littlerandomization, administrative work, and infrastructure issues.Fucci et al. [14] performed a quasi-experiment with 45 undergraduate stu-dents to investigate whether, and to what extent, sleep deprivation impactsthe performance of novice software developers using the agile practice of test-first development (TFD). The students were divided into two groups, where23 stayed awake the night before carrying out the tasks, while 22 slept nor-mally. They identified that a single night of sleep deprivation reduced 50% inthe quality of the implementations. In addition, the study found that sleep-deprived developers make more fixes to syntactic mistakes in the source code.
This section describes the research design used in our study. Firstly, a (i)Repository Mining analysis was performed to understand the impact of COVID-
Paulo Anselmo da Mota Silveira Neto et al.
19 pandemic on software projects. It considered different perspectives such asnumber of commits, issues, pull requests, branches, comments, time to fix apull request and an issue, and the number of active and new contributors.Next, a (ii) Survey with software professionals was performed to get insightson how the pandemic impact on their wellbeing. Figure 1 shows the overallresearch design used in our study.
Fig. 1
Research Design.
Metrics.
The metrics used in our analysis are presented in Table 1. For each ofthe metrics, we collected information for one year and four months (2019-Janto 2020-May). It was important to show the behavior before and during thepandemic period. We also want to investigate how the metrics evolve monthby month.
Deep Dive on the Impact of COVID-19 in Software Development 7
Metric Description Ref.
N of Active Contributors The developers that had at least one commitin the study date range [11]N of New Developers The developers that committed at first timeduring the study date range [11]N of Branches List of all public branches of each repository [3]N of Created Pull Requests Number of pull requests created per monthconsidering the study date range. [29,36]N of Closed Pull Requests Number of pull requests updated per monthconsidering the study date range. [29,36]Pull Request created andclosed per month Number of pull requests created and closed inthe same month considering the study daterange. [29,36]N of Commits Number of commits considering the study daterange [11]N of Created Pull RequestsComments Number of comments created related to a pre-vious created pull request. [20]N of Updated Pull RequestsComments Number of comments updated related to a pre-vious created pull request. [20]N of Bug-fix commits Number of project commits related to bug fix. [6]
Table 1
Metrics used in the analysis.
Selecting and Filtering Repositories.
First, we choose GitHub repos-itories written in the Java language, since it is one of the most popular pro-gramming languages. Next, in order to detect pandemic impact on GitHubrepositories, we retrieved Java repositories created from 2019 and sorted bytheir popularity [5]. In addition, as recommended by Munaiah et al. [27], wefiltered out the noise in such large repositories by applying different inclusionand exclusion criteria, as follows: – Inclusion Criteria 1:
The repository has been updated at least once inthe last year (2019-Jan to 2020-May); – Inclusion Criteria 2:
The repository must have at least 34 commits inthe study period (2019-Jan to 2020-May); this corresponds to two commitsper month in the 17-month study period. This criteria was used to filterout inactive repositories; – Inclusion Criteria 3:
The repository must have at least 10 contributorsin the study period (2019-Jan to 2020-May). This criteria was used in orderto eliminate irrelevant repositories, c.f., [1], [22], [30]; – Exclusion Criteria 1:
Repositories that did not have their artifacts anddescription in English were not considered in the study; – Exclusion Criteria 2:
Repositories corresponding to tutorials, books, andclassroom materials were also removed from our analysis.After the filtering step, we selected the Top 100 Java repositories. As de-scribed in Figure 1, the repositories were cloned and the git log and gitbranch commands were used to get all projects commits and branches. Usingthe .csv file generated, a Python script was used to extract the number ofcommits, time to fix a issue and pull request, the number of new contributors (i.e. those who made their first contribution in the study period (2019-Jan to2020-May)), the number of contributors (i.e. those who collaborated in thestudy period) and number of remote branches for each repository. When theinformation could not be collected from local cloned repositories, we used theGitHub API. Specifically, pull requests and their comments were retrieved byGitHub REST API requests .Next, we present how each metric was identified and measured for therepository mining process. The number of activecontributors was identified by first cloning each repository and collecting allcommits in the study period. Once the commits were collected in a .csv file, a Python script was used to identify each developer responsible for eachcommit. This way, we considered an active contributor to be a developer whichhas at least one commit in the study period. The number of new develop-ers was also collected from the commits extracted from each repository. ThePython script searched for developers that had their first commit in the repos-itory during the study period. It is important to know if most of the newdevelopers initiated their contribution before or during the pandemic.
The number of brancheswas also collected from the cloned repositories throughout the GitHub RESTAPI. It is important to mention that we used an API Endpoint that retrievesall protected and unprotected repository branches. Finally, a Python scriptwas used to collect the branch creation date based on its commits.
Thecreated and closed pull requests were collected using the GitHub REST API since this information is not available at local cloned repositories. Accordingto GitHub REST API documentation, both issues and pull requests should beretrieved by the same GET /issues endpoint. In order to identify pull requests, https://developer.github.com/v3/ All commits were extracted using the git command: gitlog − − all − − format = H, aE, ci > commit f ile.csv https://developer.github.com/v3/ Deep Dive on the Impact of COVID-19 in Software Development 9 we need to filter the results searching for the “pull request” string in eachissue occurrence. With this information, we were able to identify how manypull requests were created and closed, considering the study date interval.In addition, we also identified the pull requests created and closed in thesame months. With this metric, we analyze how the pandemic impacts teamproductivity.
The project commitswere collected from the cloned repository using git log command. Next, thePython script was used to group all commits by month-year in a .csv file.This information is important to understand how the pandemic impacts onteam productivity over the months.
The GitHub REST API was also used to collect informationregarding to the pull requests comments. This metric collected both new andupdated comments related to repository pull requests. The comments showimportant insights regarding to the project activity over the study interval. To gather thenumber of commits related to bug-fixing, we used the GitcProc Tool whichuses a keyword search to determine if that commits is related to a bug fixingor not. It searches for related words such as error, bug, defect, and fix withineach commit message [6].3.2 Survey Protocol.
We created a 20-minute survey designed to understand the impactof COVID-19 on software development from the perspective of projects anddevelopers’ wellbeing. It was composed of seventeen closed questions on a Lik-ert scale (Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree, andN/A) and three open questions. We included the option N/A to prevent re-spondents providing arbitrary ratings to questions that they find to be unclear.The survey also collected demographic information from respondents. For thedesign of the survey, we followed the Kitchenham and Pfleeger’s guidelines forpersonal opinion surveys [21]. https://developer.github.com/v3/ https://github.com/caseycas/gitcproc0 Paulo Anselmo da Mota Silveira Neto et al. We piloted our survey with three researchers (2 with Ph.D. degrees and 1Ph.D. student) with experience in the area to get feedback on the questions andtheir corresponding answers; difficulties faced to answer the survey and time tofinish it. As these pilot respondents were experts in the area, we also would liketo know if we were asking the right questions. We conducted several iterationsof the survey and rephrased some questions and removed others to make thesurvey easier to understand and answer. Then, we ended up with 20 questions.Another concern in this stage was also to ensure that the participants couldfinish it in 20-minutes. The pilot survey responses were used solely to improvethe questions, and these responses were not included in the final results. Wekept the survey anonymous, but in the end, the respondents could inform theiremail to receive a summary of the study. The survey instrument can be seenin the accompanying website . Respondents . We followed a three steps approach to recruit survey re-spondents: initially, we posted survey information on personal accounts ofsocial media (Twitter, LinkedIn, Instagram). Next, most authors contactedpotential respondents by email (convenience sample) and asked them to shareit with other potential respondents (snowballing). Because of this process, wewere not able to track the total number of invitations. In total, we received287 responses. We disqualified six responses without actual data (without re-sponses to any of the survey questions of interest to the study despite respond-ing to basic demographics questions, such as the role of the respondent) andtwo repeated ones, leading to 279 valid responses that were considered.The respondents spread out in 32 countries across four continents. The topthree countries where the respondents come from are Brazil, United States,and Germany. The professional experience of these 279 respondents varies fromno experience to 50 years, with an average of 12.5 years and a median of 10years.Regarding the size of the organizations, 34% of the respondents work forcompanies with 1000 to 9999 employees, 23% of the participants work forcompanies with 100 to 999 employees, 16% of the respondents work for com-panies with 10 to 99 employees, 11% of the respondents work for companieswith 10000 to 99999 employees, 8% work for companies with 100000 or moreemployees, and 8% of the respondents work for companies with 0 to 9 employ-ees. The majority of the respondents have a Bachelor’s degree (41%) and anadvanced degree (41%), i.e., Master’s or Ph.D.95% of the respondents were paid professional, and 5% were volunteers.90% of the respondents work full time, and 94% had no disability. On average,the participants live with three people (including himself/herself), and only32% live with children under the age of 12. The home office experience ofthe 279 respondents varies from no experience to 35 years, with an averageof 3.5 years and a median of 1 year. Finally, 82% of the participants wereworking from office before the pandemic and switched to a home office, 17% https://github.com/pamsn/covid-study Deep Dive on the Impact of COVID-19 in Software Development 11 were working the whole time remotely, and 1% were working at the office thewhole time.
Data Analysis . We collected the ratings that our respondents providedfor each question. Next, we converted these ratings to Likert scores from 1(Strongly Disagree) to 5 (Strongly Agree), We computed the average Likertscore of each statement related to productivity and wellbeing perspectivesand the plot Likert scale graph. A Likert scale graph is a bar chart thatshows the number of responses corresponding to strongly disagree, disagree,neutral, agree, strongly agree, and N/A. In addition, we used open coding [33]to analyze the answers that the survey respondents gave to explain the openquestions related to productivity and wellbeing, and their perceptions abouthow the COVID-19 pandemic has affected him/her and his/her team.3.3 Statistical AnalysisIn the mining part of the study, we calculated the number of each metric men-tioned in Table 1 for each project for 17 months (for details, see section 3.1).To investigate the impact of COVID-19 on these metrics, we partition the timeinto six time-windows. As all the collected data related to project metrics aretime-series [34], we did a time series analysis for each metric in each timeframe across the projects to see the changes. We also performed a pairwisetwo-sample t-test to check if there is any statistically significant difference inthe mean of each metric among the time-frames. The time series plots for eachmetric for each category and for each time-frame is available at the accompa-nying Website . This section presents our findings from analyzing both mining data and surveyresponses. The content is organized around our two research questions.4.1 RQ1: What is the impact of COVID-19 on projects?Our goal is to identify the impact of COVID-19 on software projects withthe first research question. For this, we conducted a mining study of 100 Javaprojects where we investigate the change of 10 different project-related met-rics (defined in Table 1) during the COVID-19 period. In order to do thiscomparison, we divided our dataset of 17 months into six time-windows:1.
Jan’19-May’ 20 (TW1) : This time-window includes time starting fromJanuary 2019 to May 2020.2.
Jan’19-Jun’19 (TW2) : This time-window includes the time startingfrom January 2019 to June 2019. https://github.com/pamsn/covid-study Jul’ 19-Dec’19 (TW3) : This time-window includes the time startingfrom July 2019 to December 2019. This time-window includes the initialCOVID-19 spread time.4.
Jan’20-Mar’ 20 (TW4) : This time-window includes time starting fromJanuary 2020 to March 2020. During this time-window, WHO (WorldHealth Organization) declared COVID-19 as a global pandemic on March11,2020 [10]. Moreover, people started to work from home during this time-window.5.
Apr’ 20-May’ 20 (TW5) : This time-window includes the months ofApril and May 2020.6.
Jan’20-May’ 20 (TW6) : This time-window includes time staring fromJanuary 2020 to May 2020.After dividing the metrics data into six time-windows (including the win-dow with full-time length TW1), we investigated the changes of each metric(Table 1). We tracked the evolution of each metric for identifying their trendsin the projects during the time-windows. For this purpose, we calculated theeffect size of the month on each metric using a linear regression model, givingus how much that metric changed for each project per month. Then, basedon the effect size, we categorized each project into one of two categories: in-creasing or decreasing. If the effect was positive, we marked those projectsas increasing for that metric. For a negative effect, we marked the project asdecreasing. For example, Figure 2 shows that for the metric “number of activecontributors”, project “Brave” (Figure 2.a) shows an increasing trend for thetime-window January 2020 to May 2020 and “ExoPlayer” project (Figure 2.b)shows a decreasing trend according to our time series analysis. All the figuresgenerated from the analysis are provided in the accompanying website . InTable 2, we report the number of projects belonging to each trend for the sixtime-windows mentioned earlier.In the following sections, we list the observations pertaining to the impactof COVID-19 on different project activities. From our mining analysis, we found that in general, more projects are showingdecreasing trends in the number of bug-fix commits per month during the pan-demic period. For example, time-window between Jan’20-May’20 (TW6) showsmore projects with a decreasing trend in comparison to the time-windows be-tween Jan’19-Dec’19 (See Table 2). Next, we performed a pairwise two-samplet-test to check if there is any statistically significant difference in the meanbetween these time-windows and found that between the time-windows Jul’19-Dec’19 and Jan’20-May’20, there is a statistically significant difference in their https://github.com/pamsn/covid-study Deep Dive on the Impact of COVID-19 in Software Development 13
Fig. 2
Month wise trends for Active Contributors across all projects.
Table 2
Summary of time series analysis for each categoryMetric TW 1 TW 2 TW 3 TW 4 TW 5 TW 6 ↑ ↓ ↑ ↓ ↑ ↓ ↑ ↓ ↑ ↓ ↑ ↓
ActiveContributors 42 58 42 58 45 55 31 69 47 50 18 82NewDevelopers 49 49 41 59 29 69 30 66 49 41 18 70Branches 93 7 26 28 35 39 91 9 33 29 77 23CreatedPullRequests 81 11 29 25 35 35 73 15 49 21 54 30ClosedPullRequests 45 54 45 53 49 50 34 64 49 48 25 71Pull requestopened &closed insame month 46 53 42 58 44 55 39 58 52 43 20 75Commits 45 55 40 60 44 56 32 68 55 44 23 77CreatedPull requestComments 56 41 36 64 38 46 34 55 50 34 23 59UpdatedPull requestcomments 58 39 38 62 37 45 38 51 49 35 32 51Bug-fixCommits 40 60 38 62 37 62 32 65 49 47 25 70 mean (p-value < Table 3
Summary of t-test among projects between different time range.Metric TW 1 TW 2 Mean1 Mean2 p-valueActive Contributors Jul19-Dec19 Jan20-May20 19.13 29.02 0.019Closed Pull Requests Jan20-Mar20 Apr20-May20 78.79 46.94 0.047Created Pull Requests Jul19-Dec19 Jan20-May20 6.79 12.48 0.04New Developers Jan20-Mar20 Apr20-May20 8.88 4.46 0.00012Pull Request opened &closed in the same month Jan20-Mar20 Apr20-May20 62.25 35.7 0.048Bug-fix Commits Jul19-Dec19 Jan20-May20 69.92 32.54 0.028 (cid:23)(cid:22) (cid:20)(cid:21)
Observation 1:
Though the majority of projects show decreasing trendsin the number of bug-fix commits most of the survey respondents disagreeregarding decreased number of bug-fix commits during the pandemicperiod.We also asked the respondents about their view on the statement: “The num-ber of bugs in the project has increased since they began working from home”.39, 91, and 107 respondents strongly disagree, disagree, and neutral with thisstatement. The average Likert score for this statement is 2.47 (i.e., between”disagree” and ”neutral”). The following are some comments that refute orconfirm the statement: (cid:55) ”Overall we’ve been finishing more features at a higher quality. ” (cid:51) ”The overall level of engagement has increased, but I feel that the qualityhas suffered somewhat. ” (cid:11)(cid:10) (cid:8)(cid:9) Observation 2:
From our mining part of the study, we found that in case of creating andupdating pull request comments (which is a way of discussion among devel-opers), more projects are showing decreasing trends during the pandemic pe-riod. For example, time-windows between Jan’20-May’20 (TW6) show moreprojects with a decreasing trend in comparison to the time-windows betweenJan’19-Dec’19 (See Table 2) in both creating and updating comments. Next,we performed a pairwise two-sample t-test to check if there is any statistically
Deep Dive on the Impact of COVID-19 in Software Development 15
Fig. 3
Summary of the survey results on the impact of COVID-19 on the bug-fixing activ-ities. significant difference in the mean between these time-windows and did not findany statistically significant difference in mean between any two time-windows(p-value > (cid:55) ”I having more structure timed schedule. Review with the team, coding,review with the team. ”We also asked about the rate of discussion among team members in oursurvey. 37.63% respondents did not think there is an increment in discussionamong team members since they began working from home in the pandemicperiod. 19, 86, and 73 respondents strongly disagree, disagree, and are neutralin this, respectively. The average Likert score for this statement is 3.0 (”neu-tral”). The following are some comments that refute or confirm the statement: (cid:55) ”Normally during work days, at least on the team/office I worked with/onwe would have some time to discuss about problems that occur to us during thedevelopment of some task. Now with everyone so distant, even though we have tools to workaround that problem it still feels like you are significantly distantfrom that person, both physically and mentally. ” (cid:51) ”On our company we have a permanent virtual room opened to attenddoubts and have discussions. ” (cid:23)(cid:22) (cid:20)(cid:21) Observation 3:
Though the number of pull request comments createdand updated decreased over the pandemic period, discussion among de-velopers related to their work and review activity did not decrease duringthe pandemic.53.04% of the respondents disagree on the quality degradation of discussionamong team members since they began working from home, while 26.16% agreethat there is a degradation in discussion quality. The average Likert score forthis statement is 2.63 (i.e., between ”disagree” and ”neutral”). The followingare some comments that refute or confirm the statement: (cid:55) ”I make calls with my colleagues the whole time, to discuss and cheaptalk, this helps focus and fell like we are at the same room. ” (cid:51) ”In general, people are more nervous and stressed, which makes discus-sions less constructive and more stressful. ”Figure 4 shows the summary of the survey results on the impact of COVID-19 on discussion among developers. Fig. 4
Summary of the survey results on the impact of COVID-19 on discussions amongdevelopers. Deep Dive on the Impact of COVID-19 in Software Development 17
To investigate the impact of COVID-19 in code contribution, we analyzed thenumber of pull requests created, closed each month during and before thepandemic period.From our mining analysis, we found that in general, more projects showincreasing trends in the number of created and closed pull requests per monthduring the pandemic period. For example, time-windows between Jan’20-May’20 (TW6) show more projects with an increasing trend in comparison tothe time-windows between Jan’19-Dec’19 (See Table 2). Next, we performeda pairwise two-sample t-test to check if there is any statistically significantdifference in the mean of the number of created pull requests between thesetime-windows and found that between the time-windows Jul’19-Dec’19 andJan’20-May’20, there is a statistically significant difference in their means (p-value < < (cid:55) ”I am doing home office. It’s effective to keep the features delivery. ” (cid:55) ”Overall we’ve been finishing more features at a higher quality. However,we have limited QA resources (we only have one full-time QA staff member),so the fact that the developers are churning out code is probably making itmuch more stressful for our QA lead. ”48.75% of our survey respondents disagree that the number of refactoringis lower than usual since they began working from home. 36, 100, and 92respondents strongly disagree, disagree, and are neutral with this statement,respectively. The average Likert score for this statement is 2.50 (i.e., between”disagree” and ”neutral”). The following are some comments that confirm thestatement: (cid:51) ”Besides of interpersonal communication be affected, I perceived that longterm strategies about the quality and refactoring, for example, are depreciated instead of short term tasks, such as new implementations and tests automationrelated to the implementations. ” (cid:11)(cid:10) (cid:8)(cid:9) Observation 4:
In general, working from home during pandemic perioddoes not impact code contribution.Figure 5 shows the summary of the survey results on the impact of COVID-19 on code contribution.
Fig. 5
Summary of the survey results on the impact of COVID-19 on code contribution.
In our survey, we also asked the survey respondents about task completiontime, productivity, and quality of their work during the pandemic period.47% survey respondents disagree that it takes longer to complete the tasksthan usual since they began working from home. 58, 73, and 52 respondentsstrongly disagree, disagree, and are neutral with this statement, respectively.The average Likert score for this statement is 2.72 (i.e., between ”disagree”and ”neutral”). The following are some comments that refute or confirm thestatement: (cid:55) ”I’ve more time to study my tasks, with more attention and patience, I’vecreated step by steps before to start working and this is more effective. ” (cid:55) ”We scheduled more small conversations during the day and splitted morethe tasks. We are keeping the same productivity as before or even higher. ” Deep Dive on the Impact of COVID-19 in Software Development 19 (cid:51) ”Tried using the Pomodoro method of time boxing tasks. Otherwise, noth-ing. I dont think it helped much. ” (cid:51) ”Work more hours to complete necessary work, try to be more organizedwith tasks and time available (prioritization). Not so effective, I still have towork more hours. ”On the overall productivity, 56% of the survey respondents disagree theyare less productive since they began working at home due to the COVID-19 pandemic. 89, 67, and 39 respondents strongly disagree, disagree, and areneutral, respectively. The average Likert score for this statement is 2.43 (i.e.,between “disagree” and “neutral”). The following are some comments thatrefute or confirm the statement: (cid:55) ” My productivity at work increased a lot in this pandemic and I was evenrecognized as an QA MVP (Most Valuable Player) for the last 2 months. How-ever, I had to keep track of my working hours because I started working a lotmore than I was supposed to. ” (cid:55) ” My team became more productive due to the absence of office distrac-tions. ” (cid:51) ” We had been working remotely for years already. However, our produc-tivity still took a big hit - not due to working from home specifically, but dueto the cognitive overload we’re all experiencing these days. It’s hard to concen-trate and be productive when we’re all worried about our families, our societyand our own health. ” (cid:51) ” Decreased one-on-one conversations, low internet bandwidth, irregularpower supplies and many other factors have impacted our effectiveness. ”In general, respondents disagree or are neutral considering that the qualityof their work is lower than it should have been since they began working fromhome. 83, 89, and 40 respondents strongly disagree, disagree, and are neutralwith this statement, respectively. The average Likert score for this statementis 2.33 (”i.e., between ”disagree” and ”neutral”). (cid:11)(cid:10) (cid:8)(cid:9)
Observation 5:
Overall productivity and task completion time do notdecrease during the pandemic period compare to the usual time.Figure 6 shows the summary of the survey results on the impact of COVID-19 on task completion time, productivity, and quality of the work.In general, respondents disagree or are neutral, considering that the amountof testing is lower than usual since they began working from home during thepandemic. 54, 110, and 69 respondents strongly disagree, disagree, and areneutral with this statement, respectively. The average Likert score for thisstatement is 2.28 (i.e., between “disagree” and “neutral”). The following is acomment that confirms the statement: (cid:51) ”Besides of interpersonal communication be affected, I perceived that longterm strategies about the quality and refactoring, for example, are depreciated
Fig. 6
Summary of the survey results on the impact of COVID-19 on task completion time,productivity, and quality of the work. instead of short term tasks, such as new implementations and tests automationrelated to the implementations. ”Many respondents disagree or are neutral, considering that the code qual-ity has decreased since they began working from home. 62, 112, and 66 re-spondents strongly disagree, disagree, and are neutral with this statement,respectively. The average Likert score for this statement is 2.21 (”i.e., between”disagree” and ”neutral”), which is the lowest among all survey questions. (cid:11)(cid:10) (cid:8)(cid:9)
Observation 6:
According to our survey respondents working fromhome during pandemic period does not impact code quality.Figure 7 shows the summary of the survey results on the impact of COVID-19 on code quality.4.2 RQ2: What is the impact of COVID-19 on Developers Wellbeing?With the second research question our goal is to identify the impact of COVID-19 on developers wellbeing. To achieve this goal, we asked the survey respon-dents regarding their stress, emotional condition, getting help from others,etc. In the following sections, we present our findings regarding the impact ofCOVID-19 on developers wellbeing.
Deep Dive on the Impact of COVID-19 in Software Development 21
Fig. 7
Summary of the survey results on the impact of COVID-19 on code quality.
Sleep disorder could hamper developer’s productivity. In our survey, we askedthe respondents if they think their sleeping disorder increased during the pan-demic period. Among 279 respondents, 39.40% of the respondents agreed thattheir sleep disorder increased since they began working from home. The av-erage Likert score for this statement is 3.0 (i.e., ”neutral”). The following aresome comments that refute or confirm the statement: (cid:55) ”This situation is quite complex, especially when we perceive a completeneglect on the part of the federal authorities in Brazil with the pandemic and atthe same time see the increasingly devastating advance of the virus in Brazil-ian territory. The most important action has been to control the amount ofinformation to maintain emotional control. Apparently just for sleep it hasn’tbeen enough. ” (cid:51) ”I’m going to sleep at the same time. It was effectively strong. ” (cid:51) ”I gave sleep a priority. Discontinues some non-essential activities. ” (cid:11)(cid:10) (cid:8)(cid:9) Observation 7:
We found that 141 out of 279 survey respondents agreed that the level ofstress increased in the pandemic period since they began working from home.
A substantial number of respondents choose to be neutral or disagree (44neutral respondents, and 85 respondents who disagree or strongly disagree).The average Likert score for this statement is 3.33 (”i.e., ”neutral”) which isthe highest among the survey questions. The following are some commentsthat refute or confirm the statement: (cid:55) ”Constant breaks (10minutes every 50 minutes). No overtime. When myworkday is done, I close my laptop and put in a drawer (”Out of sight, out ofmind”) and get it out only on the next working day. I feel less stressed becauseI have more time overall (no commuting). ” (cid:55) ”The pandemic is serious, but it is far away to be my source of stress.I have low need for social interaction. Sunbathing with the baby is enough torelieve the ”quarantine”. ” (cid:51) ”Working more flexible hours to watch family more during day and shiftmore at night. This has maintained work output and quality but increasesstress. ” (cid:51) ”In general, people are more nervous and stressed, which makes discus-sions less constructive and more stressful. ” (cid:11)(cid:10) (cid:8)(cid:9) Observation 8:
Many respondents disagree or are neutral considering that the level of happi-ness increased since they began working from home. 27, 70, and 95 respondentsstrongly disagree, disagree, and are neutral with this statement, respectively.The average Likert score for this statement is 2.91 (”i.e., between ”disagree”and ”neutral”). The following are some comments that refute or confirm thestatement: (cid:55) ”I am depressed and taking anti-depression pills along with therapy. Icheck a doctor online. ” (cid:55) ”Slightly increased the frequency of my panic attacks. ” (cid:55) ”I try to get just enough news to stay informed but other than that I tryto avoid news because the more I hear, the more depressed I get. I have aregular schedule of zoom dates with my boyfriend on top of our impromptucalls/facetimes to make sure that we are staying as connected as we can. Thathelps to some extent because we know we’ll get through it, but it still completelysucks that we don’t get to see each other in person. To make up for it, weprobably text more frequently during work hours than we would have beforethe pandemic. At work, my team has weekly 30min meetings to just hangouttogether on zoom. They were good at first to help us stay connected and sane,but now I think we’re kind of getting bored with them. We probably don’t needthem so frequently. ” (cid:51) ”My overall wellbeing was not affected. Just the mood and motivation isdown due to the situation. There is not much to do about it. We just have toget through it. ” Deep Dive on the Impact of COVID-19 in Software Development 23 (cid:11)(cid:10) (cid:8)(cid:9)
Observation 9:
Fig. 8
Summary of the survey results on the impact of COVID-19 on the developer’swellbeing.
101 out of 279 survey respondents agreed that the mentoring activities fornewcomers has decreased in the project since they began working from homein pandemic. A substantial number of respondents choose to be neutral ordisagree (66 neutral respondents, and 81 respondents with disagree or stronglydisagree with this statement). The average Likert score for this statementis 3.15 (i.e., ”neutral”). The following are some comments that confirm thestatement: (cid:51) ” I think it affected the welcoming of the youngest to the team. Face-to-faceactivities generate more intimacy between people. ” (cid:51) ” The COVID-19 has influenced the acceptance of new employees. Theefficiency of the discussion among team members was much lower than face-to-face discussion. ” (cid:19)(cid:18) (cid:16)(cid:17) Observation 10:
Fig. 9
Summary of the survey results on the impact of COVID-19 on the developer’smentoring activity.
In general, respondents disagree or are neutral considering that newly intro-duced interruptions along with the old ones are negatively impacting theirproductivity since they began working from home. 35, 81, 64 respondentsstrongly disagree, disagree, and are neutral with this statement, respectively.The average Likert score for this statement is 2.87 (i.e., between ”disagree”and ”neutral”). The following are some comments that refute or confirm thestatement: (cid:55) ”Working remotely, others can’t interrupt my work so quickly, so thisincreased my productivity in general. ” Deep Dive on the Impact of COVID-19 in Software Development 25 (cid:55) ”Home office leads to much less interruptions. This helps productivity alot! ” (cid:51) ”I had to improvise a home office in my daughter’s room so I can isolatemyself there to avoid too many interruptions or background noises. It helped alittle, but my wife and daughter still eventually interrupt me during my workinghours. ” (cid:51) ”We tried to schedule a series of video conferences to keep in touch withthe team. It helped a lot, but is not as good as talking directly to the persons.The rate of interruptions did increase. ” (cid:19)(cid:18) (cid:16)(cid:17) Observation 11: . .
4, indicating the perceptionthat the number of people living in the household is not having a major impact.We also investigated whether interruptions are associated with number ofchildren below 12 years of age living in the household. We analyzed the resultsfrom the survey and our results show that the average Likert score rangesfrom 2 . .
2, indicating the perception that the number of children alsodoes not have a major impact. On the contrary, people mostly disagree withthis perception. Figure 10 shows the summary of the survey. (cid:11)(cid:10) (cid:8)(cid:9)
Observation 12:
Majority of the respondents do not perceive the num-ber of kids and people living in the house as a source of interruption.
From our study, we found several suggestions for the practi-tioners to maintain a healthy work-life balance during a pandemic. Creatingvarious “forms of entertainment at home” can help divert the mind away fromall the negative news. Staying at home provides opportunities for practition-ers to expand their knowledge and utilize this time to study and take onlinecourses as several universities and platforms have their offerings. While pan-demic has been tough for everyone, it has been positive for many with “bettersleep, more time with family, less travel time”. Practitioners recommend tak-ing “more small breaks during the day to spend time with family and pets”.Others mentioned the benefit of being disciplined in separating work time fromrest/family time; for example, one respondent mentions: “No overtime. Whenmy workday is done, I close my laptop and put in a drawer (“Out of sight, outof mind”) and get it out only on the next working day.”
Several respondents
Fig. 10
Survey Results Summary. mention using the Pomodoro technique for time management, which uses atimer to break down work into intervals, has helped “improve productivity”.
Organizations:
Based on our analysis, we found that the COVID-19 situationdoes not necessarily result in reduced productivity and inferior code quality.Organizations can take active steps to help developers cope with COVID-19 and remain productive and produce high-quality code. We received manyinputs from our respondents including the following: “On our company we havea permanent virtual room opened to attend doubts and have discussions.”, “Atwork, my team has weekly 30min meetings to just hangout together on zoom.They were good at first to help us stay connected and sane”, “Daily meetingsduring the beginning and end of the day, it allows visibility for the whole teamand visibility about what is being executed. It was too efficient that the team isproducing more”, “My team opened up a remote call which everybody shouldbe present during all day. This way we ’simulate’ the real-world environmentwhich we can talk with the other team members anytime. We noticed thatthe team collaboration has not decreased”.
From the above, we can note thatorganizations can positively impact their developers ability to cope well withCOVID-19 in various ways to simulate their previous working environment(before the pandemic) and facilitate more interactions between the developers.In such a way, developers can effectively collaborate and feel less isolated indoing their tasks.
Researchers:
Our findings highlight that the impact of COVID-19 is notbinary (reduce productivity vs. increase productivity) but rather a spectrum.For many of our observations, substantial proportions of respondents havediffering opinions from each other. For example, for Observation 3 (
Though
Deep Dive on the Impact of COVID-19 in Software Development 27 the number of pull request comments created and updated decreased over thepandemic period, discussion among developers related to their work and reviewactivity did not decrease during the pandemic. ), 37.63% of respondents did notthink there is an increment in discussion among team members since theybegan working from home in the pandemic period, with an average Likertscore of 3.0 (”neutral”).More research is needed to uncover specific conditions that cause certainoutcomes to be more prevalent. These conditions can correspond to personali-ties of different developers, their roles, their organization structure, and char-acteristics, their home conditions (e.g., “low internet bandwidth”, “irregularpower supplies”), etc. To illustrate the impact of these factors, one of our re-spondents mentions “... [I] have a low need for social interaction. Sunbathingwith the baby is enough to relieve the “quarantine” while another mentions“ decreased one-on-one conversations ... impacted our effectiveness” . Yet an-other mention: “
Slightly increased the frequency of my panic attacks” . Thissuggests that personality plays a role in the impact of COVID-19 to develop-ers (c.f., [39] that also shows the impact of personality on the developer andteam effectiveness prior to COVID-19). Developers with some physical/mentalhealth conditions (e.g., panic attacks) may also be more adversely impactedby COVID-19.The understanding gained from further research can result in: (1) creationof specific guidelines that can help developers or organizations adversely af-fected by COVID-19 to learn from other developers or organizations that havecoped well with COVID-19, (2) Organizations to adopt different strategies tohelp developers of different personalities and conditions to cope with COVID-19. We do not view our study as a final definitive study, but rather one ofthe many that can shed “full” light into COVID-19 (or other pandemics), itsimpacts, and ways to mitigate those impacts. Some of the future studies canconsider performing a smaller scale but more in-depth and focused study on aparticular aspect (e.g., the impact of personality on how developers cope withCOVID-19).
Our research findings may be subject to the concerns that we list below. Wehave taken all possible steps to reduce the impacts of these possible threats,but some could not be mitigated and it’s possible that our mitigation strategiesmay not have been effective.Our samples have been from a single source (Github) and single program-ming language (Java). This may be a source of bias, and our findings maybe limited to open source programs from Github. However, we believe thatthe large number of projects sampled more than adequately addresses thisconcern.
The set of analyzed metrics spans across 10 different categories and widelyused in literature. However, we cannot guarantee that our set of metric areexhaustive. We plan to expand our metric set in future work.It is possible that there are defects in the implementation of our miningscripts. To that end, we have extensively tested our implementation, and man-ually verified sampled counts of different metrics.Finally, it is possible that the survey participants misunderstood some ofthe survey questions. To mitigate this threat, we conducted a pilot study withdevelopers with different experience levels from both open-source communityand industry. We also conducted a pilot study with survey design experts. Weupdated the survey based on the findings of these pilot studies.
The COVID-19 pandemic has impacted the whole world in different ways. Assoftware is still ”eating the world”, it is essential to understand COVID-19’simpact on software projects. We conducted a mining software repository studybased on 100 GitHub projects developed in Java using ten different metrics.Next, we surveyed 279 software development professionals from 32 countriesfor gathering more insights about the impact of COVID-19 on daily activitiesand wellbeing.Based on our findings, we derived 12 observations that can be used bypractitioners, organizations, and researchers. Practitioners can use our rec-ommendations to maintain a healthy work-life balance during a pandemic.Organizations can learn from our survey respondents and take steps to remainproductive while creating high-quality code. The research community can ex-plore the social and human aspects to understand the impact of developerpersonality during a pandemic.As future work, we intend to expand the analysis of the projects (consid-ering a wider time window) to better understand the impact of a year-longpandemic on software development.
References
1. Agrawal, A., Rahman, A., Krishna, R., Sobran, A., Menzies, T.: We dont need an-other hero? Proceedings of the 40th International Conference on Software EngineeringSoftware Engineering in Practice - ICSE-SEIP 18 (2018)2. Bao, L., Li, T., Xia, X., Zhu, K., Li, H., Yang, X.: How does working from home affectdeveloper productivity? – a case study of baidu during covid-19 pandemic (2020)3. Barr, E.T., Bird, C., Rigby, P.C., Hindle, A., Germ´an, D.M., Devanbu, P.T.: Cohesiveand isolated development with branches. In: FASE (2012)4. Bhutkar, G., Roto, V., Clemmensen, T., Barricelli, B.R., Abdelnour-Nocera, J.L.,Meschtscherjakov, A., Lopes, A.G., Campos, P.F., Gon¸calves, F.: User experiences andwellbeing at work. In: Human-Computer Interaction - INTERACT 2019 - 17th IFIPTC 13 International Conference, Paphos, Cyprus, September 2-6, 2019, Proceedings,Part IV, pp. 754–758 (2019) Deep Dive on the Impact of COVID-19 in Software Development 295. Borges, H., Tulio Valente, M.: Whats in a github star? understanding repository starringpractices in a social coding platform. Journal of Systems and Software , 112129(2018)6. Casalnuovo, C., Suchak, Y., Ray, B., Rubio-Gonz´alez, C.: Gitcproc: A tool for pro-cessing and classifying github commits. In: Proceedings of the 26th ACM SIGSOFTInternational Symposium on Software Testing and Analysis, ISSTA 2017, p. 396399.Association for Computing Machinery, New York, NY, USA (2017)7. Cataldo, M., Herbsleb, J.D.: Communication patterns in geographically distributed soft-ware development and engineers’ contributions to the development effort. In: Proceed-ings of the 2008 International Workshop on Cooperative and Human Aspects of SoftwareEngineering, CHASE 2008, Leipzig, Germany, Tuesday, May 13, 2008, pp. 25–28 (2008)8. Chakraborty, I., Maity, P.: Covid-19 outbreak: Migration, effects on society, global en-vironment and prevention. Science of The Total Environment , 138882 (2020)9. Chen, E., Lerman, K., Ferrara, E.: Tracking social media discourse about the covid-19pandemic: Development of a public coronavirus twitter data set. JMIR Public HealthSurveill (2), e19273 (2020)10. Cucinotta, D., Vanelli, M.: Who declares covid-19 a pandemic. Acta bio-medica: AteneiParmensis (1), 157–160 (2020)11. Delorey, D.P., Knutson, C.D., Chun, S.: Do programming languages affect productivity?a case study using data from open source projects. In: First International Workshop onEmerging Trends in FLOSS Research and Development (FLOSS’07: ICSE Workshops2007), pp. 8–8 (2007)12. Dong, E., Du, H., Gardner, L.: An interactive web-based dashboard to track covid-19in real time. The Lancet Infectious Diseases (2020)13. Fauci, A., Lane, H., Redfield, R.: Covid-19 navigating the uncharted. New EnglandJournal of Medicine (2020)14. Fucci, D., Scanniello, G., Romano, S., Juristo, N.: Need for sleep: The impact of a nightof sleep deprivation on novice developers performance. IEEE Transactions on SoftwareEngineering (1), 1–19 (2020)15. Graziotin, D., Fagerholm, F., Wang, X., Abrahamsson, P.: What happens when softwaredevelopers are (un)happy. J. Syst. Softw. , 32–47 (2018)16. Herbsleb, J.D.: Global software engineering: The future of socio-technical coordination.In: 2007 Future of Software Engineering, FOSE 07, p. 188198. IEEE Computer Society,USA (2007)17. Herbsleb, J.D.: Technical perspective - maintaining quality in the face of distributeddevelopment. Commun. ACM (8), 84 (2009)18. Herbsleb, J.D., Mockus, A.: An empirical study of speed and communication in globallydistributed software development. IEEE Trans. Software Eng. (6), 481–494 (2003)19. Johnson, B., Zimmermann, T., Bird, C.: The effect of work environments on productivityand satisfaction of software engineers. IEEE Transactions on Software Engineering (01),1–1 (5555). DOI 10.1109/TSE.2019.290305320. Kalliamvakou, E., Gousios, G., Blincoe, K., Singer, L., German, D.M., Damian, D.: Anin-depth study of the promises and perils of mining github. Empirical Softw. Engg. (5), 20352071 (2016)21. Kitchenham, B.A., Pfleeger, S.L.: Personal opinion surveys. In: Guide to AdvancedEmpirical Software Engineering, pp. 63–92 (2008)22. Krishna, R., Agrawal, A., Rahman, A., Sobran, A., Menzies, T.: What is the connec-tion between issues, bugs, and enhancements? Proceedings of the 40th InternationalConference on Software Engineering Software Engineering in Practice - ICSE-SEIP 18(2018)23. Kuutila, M., M¨antyl¨a, M.V., Claes, M., Elovainio, M., Adams, B.: Using experience sam-pling to link software repositories with emotions and work well-being. In: Proceedings ofthe 12th ACM/IEEE International Symposium on Empirical Software Engineering andMeasurement, ESEM 2018, Oulu, Finland, October 11-12, 2018, pp. 29:1–29:10 (2018)24. Li, Q., Guan, X., Wu, P., Wang, X., Zhou, L., Tong, Y., Ren, R., Leung, K., Lau, E.,Wong, J.Y., Xing, X., Xiang, N., Wu, Y., Li, C., Chen, Q., Li, D., Liu, T., Zhao, J., Li,M., Feng, Z.: Early transmission dynamics in wuhan, china, of novel coronavirusinfectedpneumonia. New England Journal of Medicine (2020)0 Paulo Anselmo da Mota Silveira Neto et al.25. Meyer, A., Barr, E.T., Bird, C., Zimmermann, T.: Today was a good day: The daily lifeof software developers. IEEE Transactions on Software Engineering (01), 1–1 (5555)26. Meyer, A.N., Murphy, G.C., Zimmermann, T., Fritz, T.: Retrospecting on work and pro-ductivity: A study on self-monitoring software developers’ work. PACMHCI (CSCW),79:1–79:24 (2017)27. Munaiah, N., Kroh, S., Cabrey, C., Nagappan, M.: Curating github for engineered soft-ware projects (2016). DOI 10.7287/PEERJ.PREPRINTS.261728. Murphy-Hill, E., Jaspan, C., Sadowski, C., Shepherd, D.C., Phillips, M., Winter, C.,Dolan, A.K., Smith, E.K., Jorde, M.A.: What predicts software developers productivity?Transactions on Software Engineering (2019)29. Murphy-Hill, E., Jaspan, C., Sadowski, C., Shepherd, D.C., Phillips, M., Winter, C.,Dolan, A.K., Smith, E.K., Jorde, M.A.: What predicts software developers productivity?Transactions on Software Engineering (2019)30. Rahman, A., Agrawal, A., Krishna, R., Sobran, A.: Characterizing the influence of con-tinuous integration: empirical results from 250+ open source and proprietary projects.Proceedings of the 4th ACM SIGSOFT International Workshop on Software Analytics- SWAN 2018 (2018)31. Rahman, A., Farhana, E.: An exploratory characterization of bugs in covid-19 softwareprojects (2020)32. Ralph, P., Baltes, S., Adisaputri, G., Torkar, R., Kovalenko, V., Kalinowski, M., Novielli,N., Yoo, S., Devroey, X., Tan, X., Zhou, M., Turhan, B., Hoda, R., Hata, H., Robles,G., Fard, A.M., Alkadhi, R.: Pandemic programming: How covid-19 affects softwaredevelopers and how their organizations can help (2020)33. Saldana, J.: The Coding Manual for Qualitative Researchers. SAGE Publications (2015)34. Shumway, R., Stoffer, D.: Time Series Analysis and Its Applications With R Examples,vol. 9 (2011). DOI 10.1007/978-1-4419-7865-335. Storey, M., Zimmermann, T., Bird, C., Czerwonka, J., Murphy, B., Kalliamvakou, E.:Towards a theory of software developer job satisfaction and perceived productivity.IEEE Transactions on Software Engineering (01), 1–1 (5555)36. Vasilescu, B., Yu, Y., Wang, H., Devanbu, P., Filkov, V.: Quality and productivityoutcomes relating to continuous integration in github. pp. 805–816 (2015). DOI 10.1145/2786805.278685037. Verity, R., Okell, L., Dorigatti, I., Winskill, P., Whittaker, C., Imai, N., Cuomo-Dannenburg, G., Thompson, H., Walker, P., Fu, H., Dighe, A., Griffin, J., Baguelin,M., Bhatia, S., Boonyasiri, A., Cori, A., Cucunub, Z.M., FitzJohn, R., Gaythorpe, K.,Ferguson, N.: Estimates of the severity of coronavirus disease 2019: a model-based anal-ysis. The Lancet Infectious Diseases20