Mining for Process Improvements: Analyzing Software Repositories in Agile Retrospectives
aa r X i v : . [ c s . S E ] J u l Mining for Process Improvements:Analyzing Software Repositories in Agile Retrospectives
Christoph Matthies [email protected] Plattner InstituteUniversity of Potsdam, Germany
Franziska Dobrigkeit [email protected] Plattner InstituteUniversity of Potsdam, Germany
Guenter Hesse [email protected] Plattner InstituteUniversity of Potsdam, Germany
ABSTRACT
Software Repositories contain knowledge on how software engi-neering teams work, communicate, and collaborate. It can be usedto develop a data-informed view of a team’s development process,which in turn can be employed for process improvement initia-tives. In modern, Agile development methods, process improve-ment takes place in Retrospective meetings, in which the last devel-opment iteration is discussed. However, previously proposed activ-ities that take place in these meetings often do not rely on projectdata, instead depending solely on the perceptions of team mem-bers. We propose new Retrospective activities, based on mining thesoftware repositories of individual teams, to complement existingapproaches with more objective, data-informed process views.
CCS CONCEPTS • Software and its engineering → Agile software develop-ment . KEYWORDS
Agile Software Development, Software Process Improvement, Min-ing Software Repositories, Retrospective
ACM Reference Format:
Christoph Matthies, Franziska Dobrigkeit, and Guenter Hesse. 2020. Min-ing for Process Improvements: Analyzing Software Repositories in AgileRetrospectives. In
IEEE/ACM 42nd International Conference on Software En-gineering Workshops (ICSEW’20), May 23–29, 2020, Seoul, Republic of Korea.
ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3387940.3392168
Retrospective meetings are commonly held at the end of a projectto review the past work and to identify improvement opportunities.The practice of Retrospectives was embraced by the Agile commu-nity, which focuses on light-weight software development meth-ods, iterations, and feedback [6]. Instead of waiting until the endof a project, Agile practitioners began running Retrospective meet-ings more frequently, e.g. at the end of Scrum Sprints [9]. Today,regular Retrospective meetings are a popular practice in profes-sional software engineering [16].
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).
ICSEW’20, May 23–29, 2020, Seoul, Republic of Korea © 2020 Copyright held by the owner/author(s).ACM ISBN 978-1-4503-7963-2/20/05.https://doi.org/10.1145/3387940.3392168
Team activities for Retrospectives have been proposed to struc-ture meetings and to encourage the sharing of ideas [5]. Derbyand Larsen defined five consecutive phases for Retrospectives insoftware engineering: set the stage , gather data , generate insights , decide what to do , and close [5]. More recently, Baldauf introducedthe Retromat , a book [2] and online tool, which includes most ofthe previously proposed exercises in a structured format. Most pro-posed Retrospective exercises focus on gathering the perceptionsand experiences of team members and extracting improvement op-portunities from them. Another view of the project reality is avail-able through the artifacts that are produced by software develop-ers in the course of their daily work [13]. Table 1 lists an extract ofpopular tools and the data that can be extracted from them. Thisdata is useful for process improvement as it provides evidence forproject problems, e.g. when tests fail [20]. Large-scale analysis ofthis valuable project data is the focus of the
Mining Software Repos-itories (MSR) research field [8]. However, their approaches to ex-tract insights from vast collections of software repositories havenot yet been applied to software process improvement in small,Agile teams. We propose employing the software project data ofdevelopment teams, to enable an additional, data-informed viewof the executed process in Retrospective meetings. Our vision in-cludes new activities for the gather data phase, based on softwarerepository analyses.In the following, we present two use cases: (i)
Action Item Discov-ery , i.e. discovering opportunities for improvement and (ii)
ProgressCheck , i.e. assessing the team’s progress on improvement actions.
The outcome of a Retrospective is a list of “action items” [5], thatthe team will work on in the next development iteration. Of themany proposed activities to gather data, only extremely few have aconnection to project data [2]. We propose using data-driven activ-ities to discover new action items. Assessments of project data canbe drawn from measurements designed for Agile software engi-neering best practices. Examples include code coverage over time, [4],the regularity of commits to the VCS [12] or the percentage of sto-ries implemented using Pair Programming [4].
Proposed Activity: Health Check.
The Retrospective exercise is basedon the established software development best practices of a team’sorganization, with the goal of revealing violations of these prac-tices in the project data. To gather data , project data measurementsconcerning a practice should be collected. For example, for the“commit early, commit often” principle [1], this can include theaverage amount of commits per developer or the average time be-tween commits during core working hours. In the generate insights
CSEW’20, May 23–29, 2020, Seoul, Republic of Korea Matthies, Dobrigkeit, Hesse
Table 1: Extract of types of tools that produce project data which can be employed in data-informed Retrospective activities.Tool Type Function Examples of Extractable Data Points Tool Example
Version Control Track code changes, communicate rationales [15] Code diffs, committer details, timestamps git
Issue Tracker Manage detailed information on work items [14] Developer assignments, status updates
Jira
Software Tests Present the status of current software builds [3] Integration logs, test run logs, build status
Jenkins
Status Monitor Inform/alert regarding availability of systems [7] Accumulated uptime, downtime events
Nagios
Code Review Share knowledge, gather critique of peers [19] Time to completion, reviewer details, verdicts
Gerrit
Code Analysis Provide automated feedback on code quality [18] Code coverage results, coding style checks
Lint phase, the team members can inspect the results and note whetherthey are outside the expected range, i.e. when adhering to the rule.The team members can compare their interpretations of analysisresults, debate rationales for their observations and can find a con-sensus on action items for the next iteration, e.g. to commit theirwork to the VCS after each finished work item. In the case thatresults are considered to be flawed or false positives, the measure-ment parameters can be fine-tuned for the next iteration.
Without a method to gain insight into the effectiveness of Ret-rospectives and few tangible results, an organization might findit hard to justify the time and expense of performing Retrospec-tives [10]. Project artifact measurements, based on Retrospectiveaction items, are one avenue to provide these quantifiable improve-ment results. Once a measurement is defined for a given actionitem, the results for the current (without the change) and the nextiteration (with the enacted change) can be compared.
Proposed Activity: Remedy Appraisal.
Suppose that in a previousRetrospective the team identified the issue of a single person com-mitting most of the team’s code changes, which slowed down theteam. As an action item, all team members were trained in VCS us-age. To track progress, the team can decide to employ the numberof unique contributors to their code repository as a measurement.In the following Retrospective, the team appraises the effect of theremedy. The VCS can provide evidence of whether the trainingshowed effects and whether more team members contributed code,by rerunning the previously defined measurements and compar-ing results. Depending on whether the results improve, i.e. showa higher contributor count, the action item can be considered re-solved or can be discussed further.
Modern software engineers depend on digital collaboration, com-munication and development tools. Integrations between these toolsare becoming more prevalent. An increasing amount of informa-tion on developers’ interactions and behaviors is available in projectartifacts, which allows improving cooperative and developmentprocesses [17]. However, these concepts have not yet fully estab-lished themselves in the domain of Agile process improvement. Wepropose new Retrospective activities based on project data mea-surements both for discovering process improvement opportuni-ties and progress inspection. Our proposal represents initial stepsin integrating the promises of the field of
Mining Software Repos-itories into Agile process improvement approaches. Future work includes research on automating data-informed insights, such asthrough chatbots supporting Agile Retrospectives [11].
REFERENCES [1] Jeff Atwood. 2008. Check In Early, Check In Often.http://blog.codinghorror.com/check-in-early-check-in-often/[2] Corinna Baldauf. 2018.
Retromat - Run great agile retrospectives!
Leanpub.com.239 pages. https://leanpub.com/retromat-activities-for-agile-retrospectives[3] Moritz Beller, Georgios Gousios, and Andy Zaidman. 2017. Oops, My Tests Brokethe Build: An Explorative Analysis of Travis CI with GitHub. In
IEEE/ACM 14thInternational Conference on Mining Software Repositories . IEEE, 356–367.[4] Bernardo Estacio, Rafael Prikladnicki, Michael Mora, Gabriel Notari, Paulo Car-oli, and Alejandro Olchik. 2014. Software Kaizen: Using Agile to Form High-Perfomance Software Development Teams. In . IEEE, 1–10.[5] Derby Esther and Diana Larsen. 2006.
Agile Retrospectives: Making Good TeamsGreat . Pragmatic Bookshelf. 200 pages.[6] Martin Fowler and Jim Highsmith. 2001. The Agile Manifesto.
Software Devel-opment
9, 8 (2001), 28–35.[7] Marc Haberkorn and Kishor Trivedi. 2007. Availability monitor for a softwarebased system. In .IEEE, 321–328. https://doi.org/10.1109/HASE.2007.49[8] Ahmed E. Hassan. 2008. The road ahead for Mining Software Repositories. In
Frontiers of Software Maintenance . IEEE, 48–57.[9] Henrik Kniberg. 2015.
Scrum and XP From the Trenches (2nd ed.). C4Media.[10] David G Marshburn. 2018. Scrum retrospectives: Measuring and improving ef-fectiveness. In
SAIS 2018 Proceedings .[11] Christoph Matthies, Franziska Dobrigkeit, and Guenter Hesse. 2019. An Addi-tional Set of (Automated) Eyes: Chatbots for Agile Retrospectives. In
Proceedingsof the 1st International Workshop on Bots in Software Engineering . IEEE Press, 34–37. https://doi.org/10.1109/BotSE.2019.00017[12] Christoph Matthies, Thomas Kowark, Matthias Uflacker, and HassoPlattner. 2016. Agile Metrics for a University Software Engineer-ing Course. In
IEEE Frontiers in Education Conference . IEEE, 1–5.https://doi.org/10.1109/FIE.2016.7757684[13] Christoph Matthies, Ralf Teusner, and Guenter Hesse. 2018. Be-yond Surveys: Analyzing Software Development Artifacts to AssessTeaching Efforts. In
IEEE Frontiers in Education Conference . IEEE, 1–9.https://doi.org/10.1109/FIE.2018.8659205[14] Marco Ortu, Giuseppe Destefanis, Bram Adams, Alessandro Murgia, MicheleMarchesi,and Roberto Tonelli. 2015. The JIRA Repository Dataset. In
Proceedingsof the 11th International Conference on Predictive Models and Data Analytics inSoftware Engineering . ACM Press, 1–4. https://doi.org/10.1145/2810146.2810147[15] Eddie Antonio Santos and Abram Hindle. 2016. Judging a commit by its cover.In
Proceedings of the 13th International Workshop on Mining Software Repositories .ACM Press, 504–507. https://doi.org/10.1145/2901739.2903493[16] Scrum Alliance. 2018. State of Scrum 2017-2018: Scaling and Agile Transforma-tion. , 36 pages. http://info.scrumalliance.org/State-of-Scrum-2017-18[17] Leif Singer, Margaret-Anne Storey, Fernando Figueira Filho, Alexey Zagalsky,and Daniel M German. 2017. People Analytics in Software Development. In
Grand Timely Topics in Software Engineering . Springer, 124–153.[18] C. C. Williams and J. K. Hollingsworth. 2005. Automatic mining of source coderepositories to improve bug finding techniques.
IEEE Transactions on SoftwareEngineering
31, 6 (June 2005), 466–480. https://doi.org/10.1109/TSE.2005.63[19] Xin Yang, Raula Gaikovina Kula, Norihiro Yoshida, and Hajimu Iida. 2016. Min-ing the modern code review repositories. In
Proceedings of the 13th InternationalWorkshop on Mining Software Repositories . ACM Press, 460–463.[20] Celal Ziftci and Jim Reardon. 2017. Who broke the build? Automatically identi-fying changes that induce test failures in continuous integration at Google Scale.In