Onboarding in Open Source Software Projects: A Preliminary Analysis
Fabian Fagerholm, Patrik Johnson, Alejandro Sánchez Guinea, Jay Borenstein, Jürgen Münch
!!!!!!!!!!!!
This is an author-generated version. ! ! The final publication is available at http://ieeexplore.ieee.org. ! ! DOI: 10.1109/ICGSEW.2013.8 ! URL: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6613445 ! ! Bibliographic information: ! ! Fabian Fagerholm, Patrik Johnson, Alejandro Sánchez Guinea, Jay Borenstein, Jürgen Münch. Onboarding in Open Source Software Projects: A Preliminary Analysis. In Proceedings of the 8th International Conference on Global Software Engineering (ICGSE 2013), Compendium Proceedings (VirtuES Workshop), Bari, Italy, August 2013. IEEE. nboarding in Open Source Software Projects:A Preliminary Analysis
Fabian Fagerholm ⇤ , Patrik Johnson ⇤ , Alejandro S´anchez Guinea ⇤ , Jay Borenstein † , and J¨urgen M¨unch ⇤⇤ Department of Computer Science † Department of Computer Science † FacebookUniversity of Helsinki Stanford University 1601 Willow RoadP.O. Box 68 (Gustaf H¨allstr¨omin katu 2b) 353 Serra Mall Menlo Park,FI-00014, Finland Stanford, CA 94305 USA CA 94025 USAEmail: { firstname.lastname } @cs.helsinki.fi Email: [email protected] Abstract —Nowadays, many software projects are partially orcompletely open-source based. There is an increasing need forcompanies to participate in open-source software (OSS) projects,e.g., in order to benefit from open source ecosystems. OSS projectsintroduce particular challenges that have to be understood inorder to gain the benefits. One such challenge is getting newcom-ers onboard into the projects effectively. Similar challenges maybe present in other self-organised, virtual team environments.In this paper we present preliminary observations and resultsof in-progress research that studies the process of onboardinginto virtual OSS teams. The study is based on a programcreated and conceived at Stanford University in conjunction withFacebook’s Education Modernization program. It involves thecollaboration of more than a dozen international universities andnine open source projects. More than 120 students participatedin 2013. The students have been introduced to and supportedby mentors experienced in the participating OSS projects. Ourfindings indicate that mentoring is an important factor foreffective onboarding in OSS projects, promoting cohesion withindistributed teams and maintaining an appropriate pace.
Keywords — onboarding; open source software projects; virtualteams; mentoring; global software development; distributed softwaredevelopment; case study I. I
NTRODUCTION
For years, software companies around the world haveengaged with Open Source Software (OSS) projects andcommunities. The motivation for participating in, supporting,or actually establishing open source projects can stem from adesire to reduce development costs and increase the levels ofinnovation [1]. OSS development is similar to Global SoftwareDevelopment (GSD) in many ways and projects are oftenhighly decentralised with participants from a wide range ofgeographical locations and cultural backgrounds.In many application domains, engagement with OSS is aninevitable part of the computing business. In order to participatein existing software ecosystems (such as Android or the LAMPstack), companies may need to adopt open source developmentand licensing approaches in order to combine their offeringwith the infrastructure provided by the ecosystem. Anotherreason to participate in open source projects is to be ableto conduct projects of a size that exceeds the capabilitiesof an individual organisation. In addition, engagement inopen source projects can serve as a recruitment strategy thatallows companies to analyse the performance and talents ofdifferent developers around the world without the need to set up special infrastructure for that purpose [2]. OSS alsoplays an important role in government IT and the open sourceapproach is often considered an enabler for technology andknowledge transfer to developing countries (e.g. [3], [4]). TheOpen Source development approach has also greatly influencedsoftware businesses using other kinds of licensing models [5].Despite the benefits that OSS projects can offer for compa-nies, actually obtaining those benefits requires understandingand managing a number of challenges. However, guidance toaddress these challenges is widely missing in many areas [6].The most important among such challenges probably relates tothe nature of the development process followed in many OSSprojects.Open source developers have a large degree of freedomto decide how to manage themselves within their projects.OSS development teams can be characterised as self-organisedvirtual teams . We refer to “virtual teams” as “teams whosemembers use technology to varying degrees in working acrosslocational, temporal, and relational boundaries to accomplishan interdependent task” [7]. Self-organisation in open sourceprojects refers to the lack of formally appointed leaders orindications of rank or role, and to a large degree of sharedpower [8]. Such teams accomplish organisation of their workthrough self-assignment and “soft delegation” where partici-pants ask each other to perform tasks rather than command ordirect task assignments.Because of the need to quickly get involved in OSS projects,an important question is how to support the entry of newmembers into OSS projects.
Onboarding , or organisationalsocialisation , is “the process that helps new employees learnthe knowledge, skills, and behaviours they need to succeedin their new organizations” [9]. Studies on onboarding haveexamined different essential aspects such as mentoring [10],virtual teams [11], and other factors that impact the process [12],[13]. Despite the importance attributed to onboarding, only fewstudies have directly examined onboarding in open sourceprojects, but some work does exist (e.g. [14], [15]). However,factors that have been identified as highly important in previousresearch on onboarding have been largely neglected. Onboard-ing in open source projects is still poorly understood: littleresearch on social processes related to team maintenance hasbeen conducted [6], and as far as we can see, the gap still exists.Due to their similarities, results from studying onboarding inOSS projects may also be applicable to GSD projects.
This is the author’s version of an IEEE-copyrighted article. The definitive version was published in Proceedings of the 8th International Conference on Global Software Engineering (ICGSE 2013), Compendium Proceedings (VirtuES Workshop), Bari, Italy (available at http://ieeexplore.ieee.org). n this paper, we present a preliminary analysis of a studyon onboarding in open source projects. The ultimate aimis to derive guidelines for onboarding in different kinds ofenvironments, while this paper describes initial results. Therest of this paper is organised as follows. In Section II, wedescribe the international collaborative program which forms thecontext for this study. We then present the preliminary researchdesign, hypotheses, and method in Section III. In Section IV, wepresent preliminary results from the study. Finally, we discussthe results and future research in Section VI.II. C
OLLABORATION P ROGRAM
The context for our study is a large collaborative programwith several participating open source communities, companies,and universities. The project was established and coordinatedthrough Facebook’s Education Modernization program. Whilethe program itself was educational in nature, its other propertiesmake it suitable for examining onboarding in an open sourceenvironment. In this study, we focus on a subset of the programand examine onboarding in four open source projects. In thissection, we describe the program with particular emphasis onaspects that are relevant for this study.The program was initially piloted by Jay Borenstein,Education Modernizer at Facebook and lecturer in ComputerScience at Stanford University. The pilot was conductedthrough a collaboration between a Stanford University courseand the Phabricator Open Source Project and its principaldeveloper, Evan Priestley. Based on the success of the initialpilot, the program was expanded substantially in 2013. Theinternational reach of Facebook’s Education Modernizationprogram was leveraged to introduce the element of crossuniversity collaboration. More than a dozen universities werepart of the collaboration in 2013.Each university integrated the program into their computerscience curriculum in the way that was suited to their academiccalendar and infrastructure. For example, at the Department ofComputer Science, University of Helsinki, students joined theprogram through the Software Factory laboratory for researchand education [16], [17].Nine open source projects (see Table I) were carefullyselected by Mr. Borenstein to participate in the critical men-toring role for this collaborative program. The main criteriafor being a mentor in this program is the capacity to beresponsive to students, overall quality as a software engineer,and a high activity level in the open source project itself. Atotal of approximately 130 students participated, referred tobelow as developers. In this study, a subset of these developersare compared against developers in the same OSS projectswho did not receive the onboarding support provided by thecollaborative program.Prior to the January start of the program, developerssubmitted their OSS project preferences through their respectivefaculty members to Mr. Borenstein. They were then assignedto virtual teams with 4–8 (median: 5) participants from at leasttwo different universities. Each OSS project had two separatevirtual teams working for it, and provided an experienced seniordeveloper to mentor the developer teams.Two activities supported onboarding in the projects: akickoff activity, and continuous mentoring during the project. At the start in January 2013, developers gathered for a three-day kickoff Hackathon event at Facebook’s Menlo Park head-quarters in California, USA. This component of the programis noteworthy in that it takes significant resources to bringinvolved faculty members, OSS mentors, and students togetherto a single location. Facebook’s Education Modernization teamhandled all financial, logistic, and coordination-related aspectsof the event. We emphasise that only the student developersparticipated in this event; other developers already participatingin the same OSS projects were not present.During the event, developers got acquainted with othermembers of their virtual teams and met their open sourcementors. The mentors provided hands-on, practical training tothe developers, so that they could gain the basic technical skillsrequired to participate in and contribute to their projects.After the kickoff session, developers returned to their homeuniversities and continued work as virtual teams. A set ofpractices were suggested for the mentors, but each mentor wasfree to apply them in their own way. In particular, the practicesof conducting daily meetings, being available via email or chaton a daily basis, and setting a clear, high-level goal after aperiod of familiarisation were suggested. The mentors continuedto support the teams during the remainder of the project by,e.g., participating in online forum and mailing list discussions,conducting or participating in online meetings through videoconferencing and chat, helping developers find and understandtasks, reviewing code contributions and giving feedback onthem, and helping to coordinate work through issue trackingsystems.Developers were free to work on any tasks relevant to theirprojects. Initially, mentors would typically direct developersto small tasks suitable for novices, assuming that as thedevelopers became more proficient, they would begin to take theinitiative and tackle tasks of greater complexity. Most tasks wereprogramming tasks of different kinds, ranging from small bug-fixes to complicated new features. Other tasks included writingtest cases, creating new issues in tracking systems when newbugs were found, and improving some non-functional aspect ofthe software, taking into account maintainability, performance,and user experience.The developers were integrated into each open sourceproject and community by the regular procedures of thatcommunity. They have thus been exposed to the regular normsand implicit policies of each community. In addition, they havereceived support from their mentors, from their local and remoteteam members, as well as any support provided by their homeuniversities. We consider the context suitable for examining theonboarding process and the effect of these additional supportstructures on the process. In company settings, it appearsrealistic that similar support structures can be enacted both toenable entry into external open source projects as well as toenable third parties to enter projects driven by the companyitself. III. P
RELIMINARY R ESEARCH S TUDY
The goal of this study is to characterise and understandthe onboarding process in the OSS projects under study, withparticular consideration of the role of mentoring as part of thatprocess. Our study focuses on the following four open source
ABLE I. P
ARTICIPATING O PEN S OURCE P ROJECTS . projects: Kotlin, Phabricator, Ruby on Rails, and Socket.IO(see Table I). We chose to examine these projects because thestudent developers enrolled in the Software Factory Projectcourse at the Department of Computer Science, University ofHelsinki, were members of the virtual teams working on theseprojects. By examining these projects first, we were able toobserve them more closely and make appropriate choices in theresearch design. The long-term goal is to develop guidelines foronboarding in open source projects, but at this stage, preliminaryresults of characterisation and understanding are the initial focus.In this section, we describe the study as far as necessary inorder to understand the preliminary results.As noted, open source teams can be characterised as self-organised virtual teams. Such teams are also possible in otherkinds of GSD settings. These teams are not formed once andfor all, but are in a continuous state of change. The formationstrategy requires members to engage with each other, as theycannot rely on management to assign members to their teams.New members often need to learn technical and social skillsas they enter the project. Personal motivation may play asignificant role in the ability to enter projects and remain anactive contributor.We examine how certain supporting activities affect on-boarding and thus the productivity and communication activityamong team members. We hypothesise that factors such as face-to-face, workshop-like meetings, interaction with co-locatedparticipants, and mentoring can serve to support onboarding byincreasing the chance that participants are exposed to, select,and perform tasks in the project on their own.One way of analysing the issue described above is tolook at the initiation of developers in open source projects,comparing developers who receive specific onboarding supportwith developers who enter the projects in the “usual” way –without onboarding support. A. Research Design
The study follows a mostly quantitative multiple-case studyapproach [18]. We replicate the same study across severalprojects in order to increase the generalisability of the results.Following the Goal-Question-Metric approach [19], [20], wehave operationalised the goals and questions of the studyinto quantitative metrics. Analysing the metrics will provideinformation to answer the questions and thus address the overallgoal of the study. In addition, qualitative observations madeduring the study will be used to help interpret the metrics andthe contribution of each question to the overall goal. The GQMmeasurement goal is shown in tabular form in Table II. In thisstudy, we focus mainly on the following questions:
TABLE II. D
EFINITION OF
GQM M
EASUREMENT G OAL . Object Onboarding processPurpose Characterisation and UnderstandingFocus Contribution level over timeViewpoint / Stakeholder Project manager / Open source mentorContext International collaboration project (see main text) Q1 How much time does a developer communicate in theproject over time? Q2 How much time and effort does a developer put into theproject over time? Q3 How much does a developer contribute to the project overtime? Q4 How much mentoring does each mentor give to theirrespective team(s)? Q5 What aspects of onboarding are considered important byproject mentors and developers? Q6 What is the influence of mentoring on the performance ofthe developers?
B. Metrics
We derived several metrics from measurement goals follow-ing the GQM approach [19], [20]. Among these, most of ourattention in this preliminary stage has been put on activity , dueto its intuitive role as an objective indicator of onboarding. Wedefine activity as a compound of metrics that together measurethe effective participation of a developer in a particular project.Fritz et al. present and use a metric for onboarding, compris-ing the interest, knowledge and interaction of developers [21].Inspired by this work, we define activity as a linear combinationthat considers number of commits, number of pull requestsand number of interactions. All these direct metrics correspondto data that can be gathered from the public GitHub revisioncontrol system that is used by all OSS projects considered.A commit is understood in the context of revision controlsystems as representing submission of the latest changes inthe source code to the repository. A pull request in turn ischaracterized by GitHub as a notification that is made toothers about changes pushed to the repository, so that interestedparties can review changes, discuss potential modifications andpush follow-up commits if necessary [22]. An interaction isdefined as a single message posted in a GitHub discussionforum. These discussions may be attached to commits or pullrequests, and may concern potential changes to the source code,messages justifying changes, and general messages related tothe development of a pull request or commit. Each such messageis counted as one interaction.A refinement of the coefficients of the linear combinationfor the activity measure is expected as the study continues.Also, more parameters will probably be added to include theknowledge of developers with respect to their projects, whichhas currently been left out of consideration in this preliminarywork.
C. Sample
For the purpose of comparing between mentored and non-mentored developers, we sampled two groups among developers.To form the group of mentored developers, we took a random
ABLE III. T
OTAL ACCUMULATED ACTIVITY . Developers Commits Pull requests Interactions
Activity
Mentored 64 111 251
Non-mentored 19 37 92 sample of 20 developers involved in the collaboration project.For the non-mentored group, we took a random sample of thesame size of developers who have contributed to the same OSSprojects as the developers of the mentored group, but who havenot been involved in the collaboration project.For the purpose of comparison over time, we defined atime series sampling strategy for obtaining the accumulatedactivity over time. We sampled the activity for each developerevery week for the first 12 weeks of the project involvement.The definition of the first week required some assumptions tobe made regarding the non-mentored group. For the mentoredgroup, we simply took the initial week of the collaborationproject as week 1. For the non-mentored group, we definedthe initial week as the week previous to the date when thefirst activity count was found for each developer. The weeksare thus relative to when the developer began their onboardingprocess. IV. P
RELIMINARY R ESULTS
We obtained both the total accumulated activity and theaccumulated activity over time for both groups. Table III showsthe absolute numbers for each direct metric and the compoundactivity metric. Developers in the mentored group show roughlythree times more activity than the non-mentored group.While the activity metric cannot completely answer thequestions described in Section III-A, it contributes to answeringquestions 1, 2, 3, and 6. In Table III, we can see that mentoreddevelopers interact more than non-mentored developers (Q1),presumably spending more time on the project (Q2). Mentoreddevelopers also produce more commits and pull requests, whichindicates that they may contribute more to the project (Q3).The accumulated activity over time is shown in Figure 1.The numbers in the plot have been scaled to the range 0–10, inorder to make the projects comparable. The figure depicts thedata collected for the mentored and non-mentored groups withthe corresponding linear regression line that helps to see thegrowth pattern more clearly. As can be seen in the figure, theactivity of developers that have received onboarding supporthas grown considerably more with time compared to those thatdid not receive such support. This indicates that onboardingsupport has a big impact in the projects under study (Q6).Closer examination of Figure 1 shows that initially, activityin both groups is roughly the same. At week three, however,the supported group dramatically increases its activity, whichthen levels off for another three weeks. After that, the activityagain increases, and after a plateau, starts increasing steadily.There are some fluctuations in the non-supported group, butthe changes in activity level are smaller. At this stage of thestudy, we do not have a well-grounded explanation for whythe increases in activity are timed at approximately three-weekintervals. While it may be an anomaly, other possible reasonscould be that task size or effort tends to converge so thattasks generally require three weeks to be completed, or that y"="0.2676x"+"0.033"R²"="0.97315"y"="0.7681x"+"0.2451"R²"="0.88592" A c v i t y ( Weeks(
Non mentored Mentored Linear (Non mentored) Linear (Mentored)
Fig. 1. Cumulative activity of supported and non-supported developers overtime. there is something inherent in human information processingor communication abilities that sets the pace of understandingamong software developers in distributed projects to progressin approximately three-week intervals. However, we stress thatthese are merely guesses.In summary, the results indicate that onboarding supporthas a large influence on activity in the projects under study.The data analysed here, while incomplete, indicates that theonboarding process may benefit from the involvement of amentor. It would be interesting to observe if the growth inactivity will continue being larger even when mentoring hasstopped, for those that have received it – in other words, whethermentoring has a permanent effect on developer performance.This however, cannot be supported by solid evidence at thisstage of the project. For such purpose the collaboration projectwill have to come to its end and our study continued, in order toobserve the developers that were involved. Thus, an interestingpath of research can be drawn by following up this study.V. L
IMITATIONS
Onboarding is a complex construct with multiple impactfactors and context factors that may limit the applicability ofthe results in other cases. Being a preliminary analysis, we havenot yet examined the exact factors in detail. Based on the dataanalysed in this study, we cannot distinguish between mentoringand other onboarding support factors. Other contextual factorsmay play a role. For example, developers in the group receivingonboarding support may be more involved, i.e., spend moretime, more regularly, with the project. Also, the kickoff eventmay be another factor which impacts the results.In this preliminary analysis, we included only part of thedata available. We focused on four of the nine projects, and thusthe results may be biased towards the particular characteristicsof these projects. Expansion of the sample is necessary toconfirm that the results apply to all the projects in the opensource collaboration program.The observed effect may stem from factors that are eithernot present in other cases, or that are moderated by otherfactors not present in this study. For example, the use of studentsubjects could cause the study to be less applicable amongprofessional developers, either because of different levels ofxpertise, because of differing organisational constraints, orbecause of different reward mechanisms (salary versus creditpoints).A further limitation is that we have not examined whetherdevelopers in the non-supported group has received anytreatment during the observed period of time that would impactthe results. We can thus not be entirely sure what kinds offactors influence their activity. However, we expect our sampleof developers to represent the variety of conditions in whichopen source developers in the projects under study normallywork. Furthermore, even though the exact conditions of thisgroup is unknown, our results do indicate that the onboardingsupport given to the project teams has an effect on activity.We have not statistically evaluated the construct validityof our activity metric. Since this paper presents preliminaryresults, it can be considered a pilot study which contributesto the evaluation of construct validity. There are theoreticalgrounds to expect that supporting onboarding by differentmeans and in different stages of involvement will shorten thetime required for developers to become effectively onboard inan organisation. Furthermore, there are theoretical groundsto expect that individuals who have progressed further intheir onboarding process will display higher productivity thanother individuals. Our activity metric appears to correlate withthese theoretical assumptions. Still, no single study can proveconstruct validity, and further work is needed to evaluate thevalidity of our activity metric.One possible limitation of this study is that subjects mayhave altered their behaviour in response to the study or to thefact that they are being assessed as students by their universities.While this possible limitation needs to be taken into account,we note that i) the subjects did not receive information abouttheir performance from the researchers, ii) the educationalassessment of the subjects was not performed by the researchers,and iii) developers in companies are also frequently evaluated.We thus argue that any biasing effect of the study itself on theonboarding process is likely to be extremely low.VI. C
ONCLUSION
In this study, we examined the onboarding process of asubset of developers engaged in a collaborative project withseveral participating open source communities, companies, anduniversities. The aim of the study was to characterise andunderstand the onboarding process in these projects. The studyis a preliminary analysis which we expect to lead to guidelinesfor onboarding in open source projects.In the study, we used the GQM approach to derive anactivity metric, which measures the amount of commits, pullrequests, and interactions in online discussion fora that adeveloper has produced during a specific period of time. Wecompared the cumulative activity of developers who receivedonboarding support as part of the collaborative project with thatof developers working in the same open source projects, butwho did not receive the onboarding support. In our preliminaryresults, we found that the developers in the first group showedmore activity during the first 12 weeks of their participationthan developers in the second group.Our interpretation of the results is that the onboardingsupport given to the first group of developers has influenced their onboarding process, allowing them to become more activeat an earlier stage. We thus find support for the hypothesis thatonboarding support increases the chance of developers to beexposed to, select, and perform tasks in a proactive and self-directed manner in open source projects. We assume that thisapplies in other kinds of GSD settings where the organisationaland team structure is similar.Among the factors involved in onboarding support, whichinclude an initial co-located kickoff session, interaction withco-located participants, and mentoring, we assume that thelast factor is highly important for maintaining developers’motivation and for attaining good cohesion within virtual teams,since it involves expert guidance that is sustained during along period. It thus has the potential to be one of the largestfactors that influence onboarding. We also assume that it helpsmaintain clear objectives in open source projects, which directsthe efforts of developers to work collaboratively on mutual,interdependent goals.This study has opened a number of questions to be addressedin future work. First, this paper examined only one metric, i.e.,activity. Even our preliminary characterisation of onboarding(see Section III-A) includes several GQM questions with othermetrics. A more comprehensive set of metrics promises tomeasure onboarding better.Another question is whether the effect of onboarding supportis permanent even after some or all of the support activities areremoved. Future work could assess to what degree the effect ispresent, taking into account that the organisational context willchange since the universities involved will no longer rewardcredit points to the students involved. However, if onboardinghas been very successful, students may continue working fortheir open source projects as volunteers.The construct validity of both the activity metric and theentire set of metrics in our GQM graph should be evaluated. Infuture work, validity may be evaluated by eliciting expertassessment as to how far developers have come in theironboarding process. Also, an estimation of the difference intime spent by each developer on project-related tasks wouldallow us to reduce some of the identified limitations. Separatingthe effect of different components of the onboarding supportactivities can allow us to assess the relative importance ofmentoring for onboarding. Expansion of the the sample willincrease the validity of the results. Finally, taking other impactand context factors into account, we can begin to assess theapplicability of the construct in different kinds of environments.Our ultimate goal is to provide guidelines for companieswishing to accelerate onboarding in open source projects ofimportance to them. We expect that some companies are lookingfor guidelines on how to involve their own employees intoexternal open source projects, while other companies are moreinterested in guidelines on how to accelerate onboarding ofexternal contributors to open source projects conducted bythem companies themselves. Some of the guidelines are likelyapplicable in both cases, while other guidelines are specific toone of the scenarios. For example, mentoring is likely to beapplicable in both cases, while arranging Hackathon events orother kinds of co-located workshops may be more realistic inthe second case. With appropriate modification, the guidelinesmay also be useful in other kinds of GSD settings.
EFERENCES[1] E. Von Hippel and G. Von Krogh, “Open source software and the“private-collective” innovation model: Issues for organization science,”
Organization Science , vol. 14, no. 2, pp. 209–223+225, 2003.[2] R. T. Watson, M.-C. Boudreau, P. T. York, M. E. Greiner, and J. D.Wynn, “The business of open source,”
Communications of the ACM ,vol. 51, no. 4, pp. 41–46, Apr. 2008.[3] G. Cˆamara and F. Fonseca, “Information policies and open sourcesoftware in developing countries,”
Journal of the American Society forInformation Science and Technology , vol. 58, no. 1, pp. 121–132, 2007.[4] M. Li, Z. Lin, and M. Xia, “Leveraging the Open Source SoftwareMovement for Development of China’s Software Industry,”
InformationTechnologies and International Development , vol. 2, no. 2, pp. 45–64,Dec. 2004.[5] J. Matusow, S. McGibbon, and D. Rowe, “Shared source and opensolutions: An e-government perspective,” in
Proceedings of the FirstInternational Conference on Open Source Systems , Genova, 2005, pp.263–266.[6] K. Crowston, K. Wei, J. Howison, and A. Wiggins, “Free/Libre open-source software development: What we know and what we do not know,”
ACM Computing Surveys , vol. 44, no. 2, pp. 7:1–7:35, Mar. 2008.[7] L. L. Martins, L. L. Gilson, and M. T. Maynard, “Virtual Teams: What DoWe Know and Where Do We Go From Here?”
Journal of Management ,vol. 30, no. 6, pp. 805–835, 2004.[8] K. Crowston, Q. Li, K. Wei, U. Y. Eseryel, and J. Howison, “Self-organization of teams for free/libre open source software development,”
Information and Software Technology , vol. 49, no. 6, pp. 564–575, 2007.[9] T. N. Bauer and B. Erdogan, “Organizational socialization: The effectiveonboarding of new employees.” 2011.[10] S. E. Sim and R. C. Holt, “The ramp-up problem in software projects:A case study of how software immigrants naturalize,” in
SoftwareEngineering, 1998. Proceedings of the 1998 International Conferenceon . IEEE, 1998, pp. 361–370.[11] L. Hemphill and A. Begel, “Not Seen and Not Heard: OnboardingChallenges in Newly Virtual Teams.” [12] M. Zhou and A. Mockus, “Developer fluency: Achieving true masteryin software projects,” in
Proceedings of the eighteenth ACM SIGSOFTinternational symposium on Foundations of software engineering . ACM,2010, pp. 137–146.[13] A. Begel and B. Simon, “Struggles of new college graduates in theirfirst software development job,” in
ACM SIGCSE Bulletin , vol. 40, no. 1.ACM, 2008, pp. 226–230.[14] J. Wang and A. Sarma, “Which bug should I fix: helping new developersonboard a new project,” in
Proceedings of the 4th International Workshopon Cooperative and Human Aspects of Software Engineering . ACM,2011, pp. 76–79.[15] G. Von Krogh, S. Spaeth, and K. R. Lakhani, “Community, joining,and specialization in open source software innovation: a case study,”
Research Policy , vol. 32, no. 7, pp. 1217–1241, 2003.[16] F. Fagerholm, N. Oza, and J. M¨unch, “A Platform for Teaching AppliedDistributed Software Development: The Ongoing Journey of the HelsinkiSoftware Factory,”
Collaborative Teaching of Globally DistributedSoftware Development , 2013.[17] P. Abrahamsson, P. Kettunen, and F. Fagerholm, “The set-up of a softwareengineering research infrastructure of the 2010s,” in
Proceedings of the11th International Conference on Product Focused Software . New York,NY, USA: ACM, 2010, pp. 112–114.[18] R. Yin,
Case study research: design and methods , 4th ed. SAGEPublications, Inc., 2009.[19] V. Basili and D. Weiss, “A Methodology for Collecting Valid SoftwareEngineering Data,”
IEEE Transactions on Software Engineering , vol.SE-10, no. 6, pp. 728–738, 1984.[20] L. C. Briand, C. M. Differding, and H. D. Rombach, “Practicalguidelines for measurement-based process improvement,”
SoftwareProcess: Improvement and Practice , vol. 2, no. 4, pp. 253–280, 1996.[21] T. Fritz, J. Ou, G. C. Murphy, and E. Murphy-Hill, “A degree-of-knowledge model to capture source code familiarity,” in