How Can Human Values Be Addressed in Agile Methods? A Case Study on SAFe
Waqar Hussain, Mojtaba Shahin, Rashina Hoda, Jon Whittle, Harsha Perera, Arif Nurwidyantoro, Rifat Ara Shams, Gillian Oliver
11 How Can Human Values Be Addressed in AgileMethods? A Case Study on SAFe
Waqar Hussain, Mojtaba Shahin, Rashina Hoda,Jon Whittle, Harsha Perera, Arif Nurwidyantoro, Rifat Ara Shams, and Gillian Oliver
Abstract —Agile methods are predominantly focused on delivering business values. But can Agile methods be adapted to effectivelyaddress and deliver human values such as social justice, privacy, and sustainability in the software they produce? Human values arewhat an individual or a society considers important in life. Ignoring these human values in software can pose difficulties or risks for allstakeholders (e.g., user dissatisfaction, reputation damage, financial loss). To answer this question, we selected the Scaled AgileFramework (SAFe), one of the most commonly used Agile methods in the industry, and conducted a qualitative case study to identifypossible intervention points within SAFe that are the most natural to address and integrate human values in software. We present fivehigh-level empirically-justified sets of interventions in SAFe: artefacts , roles , ceremonies , practices , and culture . We elaborate howsome current Agile artefacts (e.g., user story), roles (e.g., product owner), ceremonies (e.g., stand-up meeting), and practices (e.g.,business-facing testing) in SAFe can be modified to support the inclusion of human values in software. Further, our study suggests newand exclusive values-based artefacts (e.g., legislative requirement), ceremonies (e.g., values conversation), roles (e.g., valueschampion), and cultural practices (e.g., induction and hiring) to be introduced in SAFe for this purpose. Guided by our findings, weargue that existing Agile methods can account for human values in software delivery with some evolutionary adaptations. Index Terms —Human Values, Scaled Agile Framework (SAFe), Agile Methods, Case Study. (cid:70)
NTRODUCTION
Human values—such as transparency, social responsibility,diversity, and integrity—are increasingly recognised as crit-ical to software development [1]. This realisation has in partcome about as a result of high profile media cases wherea failure to address human values has led to significantreputation damage, financial loss, and/or user dissatisfac-tion. The Cambridge Analytica and Facebook scandal [2],Google’s Project Maven [3], and the use of social mediaplatforms to spread propaganda during elections [4], are justa few cases where the software industry in general and largesoftware firms in particular have failed to properly addressa range of values such as trust, inclusion, fairness, empathyduring software development. As a result, there are nowcountless examples of large-scale software systems thatcan be considered ethically dubious, morally suspicious, orsimply not aligned to the value set of the user base [5].In addressing values in Software Engineering (SE), re-search and practice have thus far focused on a limited setof values such as security, privacy, and accessibility, largelyignoring the vast majority of human values such as so-cial justice, cultural traditions, curiosity, and environmentalsustainability [6]. Perera et al. [6] found that only 16% ofthe publications between 2015 and 2018 in the leading SE • Waqar Hussain, Mojtaba Shahin, Rashina Hoda, Harsha Perera, ArifNurwidyantoro, Rifat Ara Shams and Gillian Oliver are with the Facultyof Information Technology, Monash University, Australia.E-mail: { Waqar.Hussain, Mojtaba.Shahin, Rashina.Hoda, Har-sha.Perera, Arif.Nurwidyantoro, Rifat.Shams, Gillian.Oliver } @monash.edu • Jon Whittle is with CSIRO’s Data61, Australia.E-mail: [email protected] submitted to IEEE Transactions on Software Engineering (2021) venues were directly relevant to human values. Some of thereasons for the lack of attention to values range from a lackof diversity amongst the leaders in the technology industry,relentless drive to profit in an increasingly competitivemarket, to lack of legislation to regulate the use of newtechnologies [7].Another key barrier to the development of more values-conscious software is the lack of guidelines in softwaredevelopment methodologies for addressing values. As aresult, even when regulations are introduced—such as theEU’s General Data Protection Regulation (GDPR) [8] andvarious governmental frameworks on the ethical use ofAI introduced internationally [9]—software practitionersstruggle to implement those guidelines in practice. GDPRis overwhelmingly lengthy but, at the same time, not veryprecise. On the other hand, AI ethics frameworks typicallyare relatively less drawn out but still very high level. Whileimportant in their own right, these two arguably ‘edge-of-spectrum’ approaches illustrate the lack of actionableknowledge on how to introduce human values into existingsoftware development processes and frameworks [10].What is needed is practical and effective adaptations toeveryday software development practices so that values canbe explicitly and effectively addressed during software de-velopment, and their absence is not retrospectively regrettedor justified.Agile software development methods are undeniably themost popular software development methodologies, driv-ing small and large-scale development of software [11]. Dueto their focus on people and interactions [12], agile methodsare particularly amenable to incorporating human values.Of the various agile methods at scale such as DisciplinedAgile Delivery (DAD), Large Scale Scrum (LeSS), Scaled a r X i v : . [ c s . S E ] F e b Agile Framework (SAFe), and Spotify, SAFe [13] has gainedprominence in recent years because of its focus on effectivelyapplying core agile principles and a range of agile practicesat scale within large organisations.We undertook a qualitative, exploratory case study [14]to identify the intervention points within SAFe that appearto be more practical and natural to allow consideration ofhuman values into the SAFe framework. In particular, ouraim was to address the overarching research question: whereand how can human values be most effectively addressed inthe Scaled Agile Framework?
Over a twelve-month period, our research team workedclosely with the IT business unit of a major public organ-isation. This business unit (henceforth, the case company)develops a range of software products and services for usewithin the broader organisation. For the last three years, ithas been undergoing an agile transformation, gradually in-troducing the SAFe framework. The case company currentlyuses the
Essential configuration of SAFe (see Section 2.2) andcombines it with practices from human centred design anddesign thinking.The contribution of this paper is twofold: Firstly, ourstudy empirically identifies five high-level interventionpoints namely, artefacts , roles , ceremonies , practices , and culture to introduce human values in the SAFe framework. Wespecifically articulate the modifications required for eachof these intervention points to ensure human values areconsidered as primary artefacts in software developmentpractice. Moreover, we introduce a set of new values-basedartefacts, roles, ceremonies and cultural practices that canhelp agile software development organisations transitionfrom being mainly driven by productivity or business valuetowards one having a human values-oriented developmentmindset. Secondly, we present the template of how thesepractices can be linked together within an overall agileprocess. The Paper Organisation:
Section 2 provides the back-ground for this study and summarises the related research.We describe our research method in Section 3. Section 4reports our findings which we discuss and reflect upon inSection 5. The possible threats to the validity of our studyare discussed in Section 6. Finally, Section 7 concludes ourstudy with some research directions for the future.
ACKGROUND AND R ELATED W ORKS
Human values are the goals that serve as guiding prin-ciples in people’s lives. These values transcend situationsor specific actions and vary in importance across differentsituations [15]. Synthesising the literature on human needs,social motives, and institutional demands of social groups,Schwartz and Bilsky [16] elaborate on the source of humanvalues and propose that at the root of human values arethree basic human needs — ‘human survival as biologicalorganisms’ , ‘requisites of coordinated social interaction’ , and ‘survival and welfare of groups’ [17].In his seminal paper [17], Schwartz presented the the-ory of human values, which postulated ten (potentiallyuniversal) types of values that are distinguished by theirmotivational goals. Schwartz also explicated a structure to explain the dy-namic relationship among these value types (such as Power,Achievement, Self-determination, Universalism, Benevo-lence, Safety, etc.). Schwartz’s theory of human values , which is the mostcomprehensive and empirically tested universal values the-ory to date [18], also introduced 56 ‘value items’ that areused to measure the ten (broad) value types [17]. Some ofthese values are well known in Software Engineering andArtificial Intelligence such as privacy, security, responsibil-ity, equality and social justice.Schwartz argues that values can be arranged in a circularstructure that reflects the motivations of each value; one thatcaptures the conflicts and compatibility among them. Forexample, value items adjacent to each other share a commonmotivation such as broad-mindedness and freedom, whilethose in opposing direction are in tension such as obedienceand freedom [15]. This circular structure can be referred towhen it is required to do values trade-offs [19].
Agile software development appeared in the mid-1990s as aresponse to plan-driven approaches in software engineering[20]. Since then, it has been increasingly adopted by organi-sations to iteratively and incrementally develop and deliversoftware-intensive systems [11], [21], [22].Broadly speaking, Agile development is characterisedby four core values [12]: “ individuals and interactions overprocess and tools, customer collaboration over contract negoti-ation, working software over comprehensive documentation , and responding to change over following a plan. ”. Respect, openness,transparency and trust are core Agile values.The concept of ‘Values’ has a different connotation in theAgile community compared to a social sciences or psychol-ogy standpoint. As discussed in Section 2.1, social scienceshave a broader view on human values, one that encom-passes anything that humans consider as important, withoutany moral imperative. While at its core, Agile values doshare some common grounds with human values of trust,helpfulness, respect, honesty, creativity, and so on, however,in the Agile community, it is tended to be used as a synonymfor SE “principles”. Agile values encourage continuous dis-covery of ‘better ways of developing software’ comparedto how it is done in heavyweight methodologies. A widerange of Agile methods [23], such as eXtreme Programming(XP) [24], Scrum [25], and Crystal methodologies [26] man-ifest the core values and principles of the Agile Manifesto[20], [21]. While the software industry has witnessed bothsuccessful and failed Agile adoptions [27], [28], Scrum, orits combination with other Agile methods, remains the mostwidely adopted Agile development method in the industry[22].The benefits of Agile methods experienced by smallteams led to an increasing interest of large organisations inagility [28], [29]. Nonetheless, some characteristics of largeorganisations make it difficult to adopt the Agile methodsdesigned for small, co-located teams in large organisa-tions with several teams [27], [28]. Increased coordinationand communication needs at the inter-team level, betweensoftware development teams and other units within the organisation (e.g. human resources), and geographicallydistributed settings [28], [30] imply Agile practices need toscale beyond single team application. Several frameworkshave been proposed for this purpose, such as Scaled AgileFramework (SAFe) [13], Large-Scale Scrum (LeSS) [31], andDisciplined Agile Delivery (DAD) [32]. These frameworksscale basic Agile values and introduce new practices, cer-emonies, and roles particularly suited to the large-scalecontext [28].In particular, SAFe has become a very popular approachfor industrial large-scale agile transformations [22]. SAFeis an elaborate, comprehensive framework based on Agileand Lean principles [13], [33]. It promotes four core values: alignment, built-in quality, transparency, and program execution with the aim to help organisations assess their large-scaleAgile transformation and to satisfy their business goals [13].SAFe version 4.6 comes with four ‘out-of-the-box’ con-figurations that organisations can adapt based on the scaleand complexity of the systems they develop. From the basicto the most comprehensive, these configurations are (a)
Essential SAFe , (b)
Large Solution SAFe , (c)
Portfolio SAFe , and(d)
Full SAFe , respectively. The case company has adopted
Essential SAFe ; the simplest starting point to implementSAFe. As shown in Figure 1, the Essential configurationoperates under two coordination levels namely,
Team level and
Program level . • The team level includes Agile teams (i.e., each has 7to 9 members with all necessary roles) that use Scrumor Kanban methods to collaboratively develop anditeratively release parts of a solution. Here, solutioncan be a service or a product released to the cus-tomer. The integration among Agile teams is facili-tated through synchronised, fixed-length iterations.This level provides a wide range of build-in-quality practices for Agile teams; for example, Agile teamscan leverage user stories from XP but deliver valuein Sprints adopted from Scrum [13], [33]. • SAFe introduces Agile Release Train (ART) at the program level to organise and synchronise Agileteams. At the team level, ART corresponds to sprints,but has a longer time box [34]. Each release at thislevel is planned and executed by all relevant Agileteams (typically 5 to 12 teams) and stakeholders.A recently published review study [28] found that thereis little evidence-based knowledge about the usage of Agilescaling frameworks. The authors call for more empiricalstudies to understand how Agile scaling frameworks areadopted in the industry and how they may be customised.Nevertheless, agile methods are popular with practitionersand therefore are a critical subject of study in softwareengineering.
Human values are discussed as a part of technology de-sign since the 1970s [35].
Value Sensitive Design (VSD) isrecognised as one of the most prominent design method-ologies to embed social and moral values in technologydevelopment [35]. To date, VSD has mainly been appliedin human-computer interaction (HCI) [36]. Despite a few isolated attempts to apply VSD in the software developmentlife cycle (SDLC) [37], [38], it is yet to gain a foothold in dayto day software development practice [39]. Moreover, to thebest of our knowledge, there is no systematic approach inthe mainstream SE practice that embeds human values intoSDLC from start to finish, i.e., from requirements elicitationto software delivery. However, a limited number of humanvalues such as security, privacy, and accessibility, has beenstudied in SE as software quality attributes [40].A few emerging studies have attempted to incorporatehuman values into different phases of SDLC. For exam-ple, Thew et al. [41] proposed the
Value-Based RequirementsEngineering (VBRE) method to guide the analyst on howto elicit values, emotions, motivations from requirementsengineering artefacts. Other techniques such as
Values-First SE [42] and
Values Q-sort [43] emphasise that valuesshould be one of the main drivers in SE decision making.These techniques address one phase of SDLC. In contrast,Aldewereld and colleagues attempted to cover all phases ofSDLC with their
Value-Sensitive Software Development (VSSD) framework [39], which has not yet gained wide popularityor adoption among practitioners. Overall, these techniquesare either at a conceptual level or require industrial trialsbefore they can be part of everyday software engineeringpractice.
Miller and Larson [44] argued that human values andethical principles should be the backbone of evaluatingAgile methods. This can help and inform developers tohave more productive discussions about the strengths andweaknesses of Agile methods. They suggested two ethicalanalysis techniques for this purpose: utilitarian analysis and deontological analysis . The former helps software engineersunderstand and analyse the potential consequences of theiractions and decisions on the stakeholders who are directlyand indirectly affected by the software. The latter focuses onthe intentions of software engineers and encourages themto consider human values and human needs as the leadingdriver in their decisions.Several studies have attempted to extend or augmentAgile methods to put the needs of their customers and end-users at the centre of the software development process,particularly at the design phase [45], [46]. In an extensiveinvestigation of literature, Sch ¨on et al. [46] highlighted
Human-Centered Design, Design Thinking, Contextual Inquiry ,and
Participatory Design as prominent approaches used inAgile software development for this purpose. However,these methods focus more on commercial or business valuefor the customers and are less concerned, if at all, withhuman values.More recent studies have attempted to address humanvalues in agile software development. Detweiler and Har-bers [47] proposed the idea of
Value Stories Workshop toguide Agile teams in identifying and analysing what thestakeholders of a system consider as important in their dailylife (i.e., human values). Fagerholm and Pagels [48] exploredthe values of developers working in Lean and Agile contextsand compared them with the basic human values (i.e.,the Schwartz values). Their study, which is based on the
Fig. 1. Essential SAFe Configuration (V4.6) (adopted from [13]) perspective of 61 developers, shows that Lean and Agilevalues can be organised into 11 groups (e.g., the freedomto organise , collaboration and so on). Fagerholm and Pagelsfound that while there exists a relation between Lean andAgile values (e.g., reliance on people ) and the basic humanvalues (e.g., benevolence ), these are two different concepts.In another study [49], Lawrence and Rodriguez anal-ysed two sources of Agile documentation (i.e., the vendor-sponsored whitepapers and IT and business magazine ar-ticles) to understand how basic values in the Lasswellvalue framework are interpreted and expressed by the Agilecommunity. The Lasswell value framework includes eightvalues: power , respect , rectitude , affection , well-being , wealth , skill , and enlightenment . The study found that power, enlight-enment, wealth , and skill are the most frequently expressedvalues among the contributors to the Agile method. How-ever, the level of correlation of these four strong values tothe values expressed in the Agile Manifesto is different. Enlightenment and skill are highly correlated to the AgileManifesto, while wealth and power are not explicitly mappedto the values in the Agile Manifesto. It was also observedthat rectitude, respect, affection , and well-being were less ex-pressed.To the best of our knowledge, we are the first to empir-ically investigate the need for human values in agile meth-ods, in particular, in SAFe, and to map a comprehensiveset of opportunities in SAFe where human values can beeffectively embedded.
ESEARCH M ETHOD
This research uses a qualitative, exploratory case study [14]to investigate where and how to introduce human values inthe Scaled Agile Framework (SAFe). Exploratory case studyaims to investigate a phenomenon, in the form of a causalrelationship, that has not been widely researched [14], [50].We decided to employ exploratory study since they serve asa prelude to social research [51], this truly fits the objectivesof our study since the social construct of human values isgenerally an unexplored phenomenon in large scale agile development. The exploratory nature of the study is aimedat informing software engineering practice with respect tohuman values; a concept which currently does not enjoyan established theory with SE [6], [52]. Furthermore, ourresearch questions (e.g., asking how questions) can be moreappropriately answered using an exploratory approach.Overall our qualitative approach facilitated a holisticand in-depth understanding of the software practitioners’perspective of human values derived from the direct con-versations we had with the participants [53], [52].We utilised Schwartz theory of values that explains thenature of and origins of human values and the underlyingmotivational goals to further strengthen our understand-ing of practitioners’ values systems and hierarchies [54].Because SAFe does not explicitly address human values,a study of how human values are currently being addressedin SAFe in practice was not possible. However, the combinedexperience, expertise, and wisdom of SAFe practitioners canbe harnessed to explore how human values can be addressed in SAFe. This is what we did.
Selection of a relevant case was crucial for our study. Ourprimary criterion for selecting a case was to identify a soft-ware company that had articulated their corporate valuesand was interested in including these values into their soft-ware development methodologies. This could enable us toinvestigate how human values translated into SE practices.We purposefully selected an IT business unit (we referto it as the case company) of a large public organisationwho met our primary selection criteria that is, they hadcodified their organisational values in their ‘Cultural Hand-book’ to govern their development operations. As valuesare an emergent concept in the software industry, randomlychoosing a case might not offer adequate insights into thephenomenon of interest. On the other hand, a purposiveselection approach which we applied in this research, max-imises the likelihood of collecting relevant information fromthe participants [55] and aids the inferential process [56].
The selected case company has a significant SE operationand mainly does in-house software development but alsoutilises external contractors. It provides a wide range ofIT services for the public organisation with a clear valuesstatement. While the large public organisation was foundedin the 1950s and currently has 18000 staff, their IT businessunit is relatively new and consists of approximately 500staff. We chose this case company for our study as it waswilling, able and enthusiastic to support the research.The case company was previously applying a mix ofAgile and Waterfall methods for their development but hasbeen using the Essential configuration of SAFe [57] for itsprojects and services development for the last three years.Essential configuration is the simplest starting point of SAFeimplementation, which “is the basic building block for all theother configurations” [57]. The company has applied a Lean-Agile Leadership Model as well as Team and TechnicalAgility principles as prescribed in Essential SAFe. WithinSAFe, product teams at the case company use a combinationof practices from Scrum, Lean, and Kanban and utilisehuman-centred design and design thinking.
The main source of data was from the semi-structuredinterviews and observations. Other sources of data for thisstudy included digital artefacts.
Protocol:
Interviews consisted of open ended questions tounderstand perceptions of software practitioners regardingvalues. The main theme of interview enquiry was relatedto participants’ views on areas within SAFe (e.g. roles,practices, tools or events) to introduce human values. Thegeneral pattern of the interview was to provide a brief back-ground of our research objectives and human values, andask about the roles and responsibilities of the interviewee inthe case company. This was followed by questions relatingto how human values can be introduced into the existingSAFe framework (i.e., anywhere from project inception tosoftware delivery that may include software artefacts, de-velopment tools, practices or people). Examples of questionswe asked from the participants during interviews include: • What are your personal and organisational values? • Can you identify some of the potential intervention pointswithin the SAFe framework where human values can beconsidered/introduced? • In your opinion, how can the interventions you identifiedbe introduced in practice?
We followed the best practice interviewing techniquesrecommended in [58] to allow adaptability to the roles andindividual experiences of the interviewees but at the sametime making sure that relevant areas of our research wereexplored. All interviews were carried out face-to-face wherepossible, else Skype or Zoom video calls. Each interviewlasted between 30 and 45 minutes. Interviews were audio-recorded and used for analysis.In line with our project’s ethics approval, research par-ticipants were provided a project information sheet, outlin-ing brief but necessary information on the nature of the investigation, mutual expectations, information on humanvalues, and sample interview questions. However, in somecases, where a participant did not read the material in detailand/or wanted additional information, they were shownthe Schwartz model of human values. This was used as aframe of reference for the participants to facilitate discussionon the topic. Interviews were concluded by asking theinterviewees for their impressions of the interview and theirexpectations from the research.
Participants:
Once University ethics approval was ob-tained, we distributed research study descriptions to poten-tial participants by working through the company point-of-contact. Interested participants were asked to directly con-tact the researchers via email. To take part in the research,the participants had to be members of an active softwareproduct team. All participants who replied and fulfilledthis criterion were subsequently interviewed. In total, weinterviewed 16 software practitioners with an average of16 years of experience in the IT industry. The majority ofthe participants were female (9 participants, 59%), while therest were male. A range of software engineering practitionerroles was interviewed, including Developer, Delivery Lead,Release Train Engineers, Scrum Master, Digital Transforma-tion Manager, Business Analysts, SAFe Agile Trainer, andthe CIO (see Table 1).
Fig. 2. A snapshot observation of a PI planning meeting
Protocol:
The case company allowed researchers to jointhree types of team events: daily stand-up, pre-programincrement planning (PI), and PI planning. In our case com-pany, stand-ups were used mainly to discuss developmentupdates and issues as well as to plan for the next 24hours. Pre-PI planning meetings were held once every threemonths where key members of cross-functional agile teamsparticipated in order to align work tasks and coordinatemilestones to create realistic objectives for the PI planningsession.This event was followed by a major PI planningevent, which is at the heart of SAFe [13].Our observation of the four PI planning meetings wasof the longest duration (approximately 45 hours in total).Each PI planning meeting in the case company span overtwo working days. Figure 2 shows a snapshot of one ofour PI planning observations. It hosted 25 teams (i.e., eachteam had approximately 10 members) from the same ART(Agile Release Train) to synchronise the teams to delivera shared vision. The activities therein included: announce-ments from the organisational executives, status reportingfrom the program leadership, planning activities (featureand user stories development for the PI), and discussions
TABLE 1Interviewee demographics (M: Male, F: Female, y: Year, m: Month)
ID Role Gender Experience in ID Role Gender Experience inIT Case Co. IT Case Co.
I01 Release Train Engineer M 22y 9m 1y 6m I09 Strategic Business Analyst F 24y 2y 1mI02 Scrum Master F 22y 5m 1y 3m I10 SAFe Agile Trainer M 17y 9m 4y 3mI03 Business Transformation Manager F 20y 20y I11 Product Manager F 20y 11m 5y 6mI04 Digital Transformation Manager M 8y 10m 2y 9m I12 Strategic Business Analyst F 14y 1m 7y 11mI05 Business Analyst M 12y 2y I13 Developer M 2y 1m N/AI06 Change Specialist F 9m N/A I14 Release Train Engineer M 28y 2m 10mI07 Strategic Business Analyst M 19y 2m 6y 1m I15 Delivery Lead F 7y 10m 7y 10mI08 Business Change Specialist F 11y 9m 3y 4m I16 Chief Information Officer F 18y 2m 10y 8m
16 interview audios Lead Coder 3 Coders 3 CodersLead Coder 3 CodersOne interview transcript 16 interview transcripts
Findings
Lead Coder Lead CoderMain Analysis (4 transcripts for each coders)Workshop for calibrating codesPilot Analysis (1 transcript for all coders)Preliminary Analysis Supplementary Materials
Observation Digital Artefacts
Fig. 3. Data analysis process using Thematic Analysis about software artefacts such as storyboards, feature plans,workflow, PI objectives, Burn Down Charts, resource capac-ity, and allocation.Our observations of team events allowed us to gainfirst-hand insight into the extent to which SAFe-prescribedactivities were followed during these events. These obser-vations also allowed us to collect first-hand informationon whether human values were (expressly or implicitly)discussed during these events. Moreover, the observationshelped us understand the perceptions of the participants re-garding human values, which they might have been unableto communicate through the interviews [59].The researchers took pictures and field notes during theobservation of these events and activities. We also utilisedobservation opportunities to interact with team membersparticipating in the events and ask questions about theactivities and artefacts for understanding and clarificationthat helped in data analysis, e.g., about PI objectives, userstories, technical debt cards, NFRs, etc. The participantsapproached during these events included business owners,delivery leads, and the current and former CEOs.
The case company also made available additional datasources such as project plans, vision statements, user stories,change documents, and issue logs. Data from the digi-tal artefacts were used to triangulate findings from othersources, such as the interviews and observations.
Thematic Analysis (TA) was used to analyse the qualitativedata collected from the interviews [60]. The NVivo softwarewas used to facilitate the coding process across the researchteam. As shown in Figure 3, the data analysis process was carried out in four steps. In the first step, the first author(i.e., lead coder), who conducted the interviews, listenedto the audio recordings of the 16 interviews to becomefamiliar with the interview data and extract the domi-nant themes. This step resulted in the construction of fivepreliminary high-level themes to describe the interventionpoints within SAFe where human values can be introduced:(a) software/IT artefacts , (b) (practitioner) roles , (c)
SAFegroup activities/ceremonies , (d) software practices includ-ing Lean, Scrum and Kanban practices, and (e) tools . In Step2, the lead coder and three authors of the paper (coders)conducted a pilot coding exercise on a common transcribedinterview. While the coders were given the five high-levelthemes to guide their data analysis process, they were freeto add new codes and themes.Step 2 was followed by a half-day workshop (Step 3),where all coders compared the similarities and differencesof their coded transcript with each other and explainedtheir rationales. A codebook was produced as an outcomeof the workshop that included five themes, (a) to (e) men-tioned above, and the underlying codes along with theirdescriptions. In the next step, each coder transcribed fourinterviews assigned to them and used the codebook toseparately code the assigned interviews. The lead coder wasadditionally responsible for overseeing and ensuring con-sistency across all coders. The coders held weekly meetingsto discuss their work status and resolve issues encounteredduring coding. As the analysis progressed, the
Tools themewas dropped from the codebook because of the lack ofenough evidence.
Organisational culture was added as anew high-level theme to the codebook due to repeatedpatterns found in the data. Table 2 shows examples of opencoding. Furthermore, two types of triangulation were usedin this study [61]: (a) investigator triangulation , by involv- ing multiple researchers to bring confirmation of findingsand to benefit from multiple perspectives; and (b) datasource triangulation , by collecting data from multiple sources(i.e., interviews, observations, and digital artefacts). Mostsupplementary material confirmed our findings from theinterviews. Where we found any discrepancy, we consultedwith participants and made necessary corrections. Findingsfrom the study were anonymised and fed back to membersof the case company for confirmation and feedback.
INDINGS
Our findings reveal that SAFe is seen as a frameworkprimarily focused on business values and lacking humanvalues (Section 4.1). We present a comprehensive set of in-terventions in SAFe to introduce human values in artefacts (Section 4.2), roles (Section 4.3), ceremonies (Section 4.4), practices (Section 4.5), and culture (Section 4.6). In doing so,we address our overarching research question, where andhow can human values be most effectively addressed in theScaled Agile Framework ?Table 3 presents an overview of these intervention areas,which are detailed in the following subsections. We includeseveral quotations from the participant interviews as aglimpse into their perceptions and ideas and as examplesof the underlying empirical data collected and analysed inour study.
As summarised in Section 2.2, SAFe promotes four ‘values’ alignment, built-in quality, transparency, and program execu-tion for organisation-wide adoption that primarily representtechnical and business concerns of software delivery.The Delivery Lead (I16), for example, confirmed that “SAFe is very much [focused] on technical delivery...the whole[SAFe] framework has been designed around getting an efficientand effective enterprise wide delivery”.
It is this technical focus,she added, that does not allow human values to be suffi-ciently addressed in practice.Other participants perceived SAFe to be less accommo-dating towards Human-Centred activities. They pointed outthe absence of human-centred design and design thinkingfrom the framework itself and found it hard to simply ‘fit in’.The Release Train Engineer contended that ‘out of the box’SAFe framework ignores the human component and doesnot allow human values integration into software delivery,making it rather ‘ soulless ’: “SAFe is very much focused around the practices and princi-ples that deliver business value which makes it a little soulless [emphasis added]” Release Train Engineer ( I15).He added that software teams’ alignment to the organ-isational values remains unaddressed in SAFe: “
There is nomention as to how do teams organise to the organisations’ valuesand behaviours, it is not part of any of the documentations either”
Release Train Engineer (I15).While human values can potentially be introduced any-where during software development, participants suggestedthat interventions would only work if they were made in theareas and in ways practitioners were already familiar with. “The major thing is that we need to frame the conversation insomething they already know and something that their frameworkalready handles and manages”
S. Business Analyst (I09).When categorising the suggested interventions intothemes, it became apparent that most fit under the interven-tions made to software artefacts, roles, ceremonies, practices ,and culture . In the following sections, we describe the inter-ventions suggested for each of these themes.
Software artefacts help communicate the design of softwareor describe the process of development itself. Examplesinclude project plans, personas, class diagrams, user stories,software code, and test cases, and the like. Participantssuggested values-based interventions made through mod-ifications to the existing artefacts in SAFe or Agile as wellas by introducing some new artefacts . L User stories are the most basic of Agile artefacts. In ourstudy, user stories were identified as one of the main pointsof intervention by a majority of our participants (n=11, 73%).They believed it made sense to include values into userstories as they aim to address the real needs or goals ofthe users (see Figure 4). They perceived goals to be theclosest concept related to values, which somewhat verifiesSchwartz view, from a social sciences perspective, on thegoal-values relationship or proximity [16].From a more practical and pragmatic stance, the prac-titioners believed that for values to have a consistent rep-resentation across the board, the Delivery Lead (I16), forexample, believed values need to be included at the userstories level because that is “where the rubber hits the road” .She elaborated that from a practical perspective, includingvalues increases the chances of their implementations be-cause “that is the piece of work that gets done”
Delivery Lead(I16). A simplified example was provided by one of theanalysts depicting how such values ’modification’ to userstories can be done: “Just add the value to the story; ‘I wantto be able to share the system with trusted advisers so that I feel empowered ‘” Strategic Analyst (I12).To modify a user story, she explained, first think aboutthe ultimate user goal achieved by successfully implement-ing the user story and then add that as a value to thestory. She added that while the goal of the example userstory above is to deliver some functionality, the higherpurpose is to enable user empowerment. Stories that deliverfunctionality to support or ensure users privacy and securitycan be similarly modified. L Epic (called feature in the case organisation) wasanother important artefact that could represent values ashigh level requirements. An epic is a work item, too large tobe completed by an Agile team in one iteration and thereforehas to be divided into several user stories. From an organi-sational perspective, the purpose or mission statements, asper Release Train Engineer (I01) can “probably translate downto the epics and capabilities in the features”.
From the users perspective, requirements that closelyrelate to values (e.g., security, privacy, usability, etc.) and areelicited during RE or assessed from user experience could bemade part of epics/feature and translated into user stories.
TABLE 2Examples of open coding
Interview Excerpt Code Refined Code IntermediateTheme Theme ”Just add the value to the [user] story.”
Adding values to user stories Modifying existing artefacts: userstories Artefact Modifica-tion/Valuefication Artefact ”You might equip 3 or 4 people to go back totheir teams and try and implement the valuesand meeting with this mentor person so thenthey’ll discuss.”
Adding a new role- ValuesMentor Specialised Values Roles: Mentor Specialisedvalues-basedRole Role ”I would even bring it [values] back to testing....actually testing against human values require-ments”
Adding values to testing andvalidation criteria Modifying testing practices: ValuesBased Testing, Values Based Re-quirements Validation Practice Modifica-tion/Valuefication Practice ”If the foundation and culture of the team re-main consistent, people are more likely to assim-ilate into a founded culture”
Keeping cultural foundationsconsistent Building consistency in the team’sculture Fostering valuesculture Culture ”That’s not going to happen in one hit rightat the end. It’s got to be continuously revisitedevery sprint every PI”
Maintaining ongoing valuescommunication to influencesoftware development (Multi-layered multi-faceted) on-going values communication Communicationculture Culture
TABLE 3Findings Summary: intervention points in SAFe to introduce human values in software › : new ideas emerged; L : modifying existing items; (cid:143) : Culture; T: SAFe Team
Level; P: SAFe
Program
Level
Intervention point LevelArtefacts L User story: state human values in the ‘ so that ’ part of a user story to capture its purpose or the bigger ‘why’. T L Epic/Feature: state human values as high level requirements in the epic/feature definition. T L Persona: state users values as higher-order goals in persona definition as potential expectations to be satisfied by the system . T L User Journey Map: describe the values of users affected by system usage when they try to achieve their goals. T › Checklist: list values to be explicitly considered or addressed during development and validated through users feedback of the system. T › Legislative Requirement: incorporate regulatory guidelines into development processes to avoid violations of privacy/security etc. P › Corporate Directive: introduce corporate directives encouraging values consideration in software beyond security/privacy. P
Roles L Product Owners: optimise business value by ensuring human values are also addressed in the system delivered. T L Business Owners: provide inputs on what satisfies users values expectations to maximise value (creation) for stakeholders. T L Release Train Engineer: ensure alignment between the Agile process and human values is never derailed during development. T › Values Champions: actively advocate for values inclusion and keep it alive in people’s minds to ensure success. T › Values Mentors: guide development teams and encourage continuous adherence to values while developing software. T › Values Translators: communicate values concepts in a terminology understandable for the development teams. T › Values Promoters: advocate for inclusion of human values in software and help overcome barriers encountered. T › Values Officers: answer values-specific questions and ensure values related directives and guidelines are adhered to. T
Ceremonies L Stand-ups: align development work with its ‘higher propose’; reflect on ’how this work upholds that human value’. P L Inspect and Adapt: reflect on current practices and propose new ways to embed human values in software. P L PI Planning: align organisational vision with stakeholder values to plan software delivery accordingly. P L Kick Offs: allow key stakeholders to define major human values objectives for the entire project. P › Values Conversations: prioritise human values and contemplate the implications of values breaches by software use. T
Practices L Business Facing Testing: seek user feedback beyond usability, functionality to capture users experiences of their values expectations. T
Culture (cid:143)
Hiring/Induction:
Prioritise hiring for values alignment; foster a culture of recognition for behaviour exemplifying desired values. P (cid:143)
Leadership: role-model values-based thinking and action as an example behaviour for the (development) teams to follow. P (cid:143)
Collective Responsibility: share responsibility to address human values throughout the development life cycle. P
Commuter
Michael Baumann
Personal InformationTransport profile Expectations/goalsDaily Routine
Commuter
Michael Baumann
Personal Information Expectations/goalsDaily RoutineTransport profile Human Values expectations P e r s ona Persona template for a commuter
Human Values Stream
Value Stream at SAFe framework (V4.6 - Portfolio level)[13] Extended persona template with human values expectationsValue Stream with an additional stream layer of human values V a l ue S t r ea m o f SA F e An actual user story template Changed user story examples with human values interventions
Example 4:
As an applicant, I want to view the status of my application in real-time so that
I can promptly respond to any requests for documentation and not feel frustrated by a lack of progress updates U s e r s t o r y Example 1: I want to be able to share the system with trusted Advisers so that
I feel empowered.
Example 2:
As an assessor,
I want to ask for further information from the applicant so that
I can give every applicant a fair chance to complete the application to feel that I have done my job responsibly
Example 3:
As an assessor of applications,
I want to view all relevant details of an applicant so that
I can make informed decisions and feel I have treated everyone fairly
After values interventionBefore values intervention
Fig. 4. Examples of human values interventions in three artefacts. “You can have some feature [or epic] statements that aretalking to the human value that you want to deliver as a partof this feature”
Delivery Lead (I16).From the development perspective, values need to beassociated with features since they clarify the main purposeor the ’bigger why’ for developing a feature. “Values need to be at the feature [epic] level, like we arebuilding feature X so how does that align to the value... you shouldhave some ability to tie it back to the value”
Developer (I13). L Persona is an exemplar fictional-user that representsthe characteristics and goals of a specific group of users[62]. Personas are requirements related artefacts that reflecthuman personality and, therefore, can portray values peoplemay hold. Personas were believed to be a good candidatefor introducing values into SAFe. While human values werenot being represented in personas in the case organisation,the participants acknowledged that their existing practiceneeded a change. For example, Product Manager (I11),while reflecting over potential intervention points for valuesinclusion, emphasised, “I would definitely capture this as partof the persona definition”.
Modifying personas seemed natural to include humanvalues in software artefacts to one of the analysts. She notedthat eliciting user values and reflecting them in personasshould not be difficult if the analysts truly empathised withtheir target users to understand their real needs and con- cerns. “The whole idea of personas is, you’re trying to empathisewith the user and you bring what user would want out of thesystem”
Strategic Business Analyst (I07). L User Journey Maps are visual descriptions of theemotions a user goes through to accomplish their goal.Making values part of journey maps was also consideredimportant for the purpose of visualisation and teams com-munication. “...capture [values] as kind of user journey, not only the painpoints or the opportunities but the emotions or the salient values[to deliver]”
Product Manager (I11).
In addition to SAFe and Agile artefacts, participants rec-ommended introducing other artefacts that are commonlyassociated with traditional disciplines and developmentmethodologies. › Checklists.
Use of checklists is common in disciplinessuch as civil engineering, health, and aviation [63]. Morerecently, in machine learning based medical systems, auto-mated implementation of checklists is considered to provideadditional safety to surgeons while operating and avoidsurgical errors [64]. Traditionally checklists have been usedfor software inspection [65], [66], especially for militaryprojects [67] but are not common practice in Agile. Someinterviewees considered checklists should be introduced as new artefacts in SAFe to ensure values inclusion in softwaredevelopment. “You could create a checklist to make sure that it [a value] isactually covered” Delivery Lead (I16). › Legislative Requirements.
Regulations like GeneralData Protection Regulation (GDPR) require organisationsto safeguard public interests of privacy, data security, andautonomy. Some respondents believed such regulations arenecessary to ensure human values are considered duringsoftware development. One participant pointed out thatcompanies fear monetary implications like “Are we goingto get fined?” , and therefore invest heavily in GDPR com-pliance. He added that companies are well aware that “the potential impact of not complying is greater than [profits]”
Digital Transformation Manager (I09). › Corporate Directives refer to written communicationsthat govern policies and direct organisational actions. In thecase company, corporate directives guided user privacy andsecurity considerations in software development practice.Although seemingly in conflict with agile values such asteam autonomy, collaboration, and freedom, enforcing com-pliance through executive orders was noted as a possiblesolution: “Compliance is hugely important. Compliance baseddirective would help change behaviour or would force people toincorporate values”
Business Change Specialist (I08).Privacy directives were seen as effective in aligningsoftware with privacy regulations and therefore were rec-ommended as beneficial for potentially introducing themfor other human values in the development process.
Much like artefacts, participants felt the need to introducevalues related roles in SAFe, arguing: “[human values] needvoice. The role for that voice doesn’t actually exist in the frame-work”
Delivery Lead (I16). L Product Owner (PO) is the core of every product de-velopment cycle. POs have the high level perspective ofthe product vision and clients’ strategic needs. They liaisewith the client and work closely with the development teamas 1) the communicators of the product’s vision or goals,2) prioritisers of requirements or features, 3) maximisersof product’s profit, 4) optimisers of value creation in theproduct and development work and 5) deciders of workresults.POs were considered suitable for keeping values as partof the vision at the front of the team’s collective mind. Theywere seen influential enough to make values part of existingartefacts to ensure reflection and accountability: “Productowners are really, really critical to making sure that the focuson those values get retained, get written up into things like PIobjectives”
Business Change Specialist (I08).As guardians of value inclusion, they could also po-tentially influence and persuade the development team bypulling back everything to link them to values. A developershared his experience from a previous company where thePO ensured the overarching messages, roughly translatedto values for our purpose, were made part of their product: “Our product owner... was just like ‘these are the things we can never deviate from, ‘messaging must be clear’, ‘style must conformto this’, ‘this is how the feeling should be’, like there were four keyoverarching messages and every decision was made in contrast tothat and you were constantly pulled back to what the overarchingmessage is rather than iterating forward and then slowly deviatingovertime”
Developer (I13). L Business Owners (BOs) represent the stakeholdersand sponsorship of the product. They guide POs on businessneeds. While BOs rely on the POs to provide the capacityof the team to handle work, their timely input as to whatmaximises the value for the stakeholders generally setsthe expectation of the teams work. For example, BusinessChange Specialist (I08), while reflecting on the most suitablerole in SAFe or Agile, saw BOs as critical in ensuring valuesthe product should embody: “I would expect the businessowners to come in championing the vision across the team”. L Release Train Engineer (RTE) being the shepherdsfor the delivery teams, RTEs were seen as crucial to ensuringthat the alignment of delivery with values is never derailedthroughout the course of software development. “I would expect RTE to be really aware of [values] as the RTEis looking at how the things are working and is checking whetherthey’re actually on track with what they were asked to do, thatthey [development teams] haven’t lost sight of it as they’ve takenon the ownership of that project or initiative”
Business ChangeSpecialist (I08).
While some existing roles could be modified to focus ondelivering software with considerations of human values,some new roles were still highlighted to ensure its imple-mentation. › Values Champions.
Champions are individuals whoinformally emerge to promote an idea with conviction, per-sistence, and energy. They willingly risk their position andreputation to ensure the success of their cause [68]. Whileplenty of empirical evidence exists in the organisationalinnovation literature of the effectiveness of this role [69], theparticipants believed such roles were needed in SAFe forsuccessful implementation of values in software. StrategicBusiness Analyst (I07) noted that inclusion of values in theartefacts alone can not guarantee their inclusion in software.He explained: “Once recorded or visually captured in some SAFeartefact, values need to be ’championed’ by someone who keepsthem at the front of people’s mind throughout the development”.
He argued further that ”even when legislation and standardsare available, people need to champion individual values to ensurethe delivery of that value is made possible in the software”. › Values Promoters.
Promoters utilise every possible op-portunity to reiterate the importance of the idea or objectivethey promote and overcome barriers they encounter. Unlikechampions with a generalist role, contributions of promotersin projects are more specific, e.g., power promoter, expertpromoter, technological gatekeeper, relationship promoter,and so on [70]. Values need constant ’promotion’ to keepthem ’alive’ in the team conversations, as noted by thedelivery lead. She believed this increases their chances ofbeing included in the software. “Having one person promoting for values is better than havingnone...one who is charged with spruiking that, making sure that[in] every ceremony have we thought about this in key areas, PI planning and showcases, bringing it back to the users and thevalues.” Delivery Lead (I15). › Values Mentor.
Agile mentors provide initial guidanceand support for development teams in adopting Agile. Theyremove the team’s misconceptions, enable teams to becomeconfident in Agile, and encourage continued adherence toAgile practices [71]. Agility, however, does not come naturalto many practitioners and software engineers, especiallythose trained in traditional ways of developing software.Dealing with the abstract and ’mushy’ concept of valuescould be very daunting for software developers, accordingto Release Train Engineer (I14) as “their view of the world tendsto be more logical more methodical.”
To work with something asabstract as values, for those who are so engineering driven, “takes a bit of time and coaching to help them understand...whyare we doing what we are doing”
Release Train Engineer (I14).A values mentor role could fulfil this need, one who ’tellsthe story’ by drawing links between the technical work andits higher purpose (i.e., human values). The mentor doesthis by explaining the benefits of software use, connectingit with stakeholder values, and helping the team to becomemore confident in dealing with them. Change Specialist (I06)recommended to groom mentors from the existing systemby creating ‘ values communities of practice ’ that nominatesValues Mentors “who meet regularly to discuss [values-related]ideas and send them back to the teams”. › Values Translators.
Translators understand the busi-ness language used by customer representatives and thetechnical language used and understood by the develop-ment team. Acting as a bridge between the two groups, theyhelp overcome the language barrier between these groups[71]. Not having someone with the ability to understand hu-man values from the users’ perspective and explaining thatin the language of developers poses significant challengesfor them. Developer (I13) highlighted the obvious need forsuch values translator noting, “One key role that is missing inSAFe is [someone with] the ability to translate what the businesslayer is saying what the people are trying to achieve and link that[with] overarching program goal”.
Chief Information Officer (I16) added that “they [devel-opers] ‘get it’ when it [value] is articulated from a businessperspective”.
Commenting on the abilities of a values trans-lator, Developer (I13) highlighted that it “needs to be both,strategist and delivery person” , but acknowledged “those peopleare currently ‘unicorns’ they are hard to find”.
One participant reported that he translated trust fordevelopers: “[I] took the organisational statements and goals thatcontain corporate values and transformed them into a languagethat [the implementers] would understand”
Strategic BusinessAnalyst (I07). › Values Officers.
Officer role is not part of the SAFeframework, but one such role existed in the case organisa-tion, called the ‘Privacy Officer’. This individual answeredquestions related to privacy regulations and compliancestandards to ensure regulatory directives were adhered to.The Strategic Business Analyst (I07) explained the involve-ment of the privacy officer role in development projects: “Part of the project governance process involves a big conversationwith the universal Privacy Officer about where data is being storedwhat data was collected so that ends up being formally builtinto project management”.
Similar roles were envisaged by participants for values other than privacy and security.While there can be a variety of roles that can be in-troduced to bring about the values-based culture in theorganisation, for practical reasons, however, for organisa-tions using agile methodologies, these responsibilities couldbe performed by either the product owner or the businessowner (or a similar role) depending on their ability, willing-ness, and awareness of human values.
Participants generally agreed that SAFe ceremonies allowample opportunities for technical discussions and reflec-tions, but they could also include and benefit from valuesconversations to improve the alignment of software deliverywith stakeholder values. We now describe some existingand new ceremonies that could be ’valuefied’, i.e., to con-sciously and explicitly address the values concerns of abroad range of stakeholders. L Program Increment Planning.
In PI planning, teams a)estimate delivery targets for the upcoming increment, b)highlight their dependencies with other Agile teams, andc) come up with a set of PI Objectives (or summary ofthe business and technical goals) to be achieved at the endof the PI. On average, around 200 members belonging tovarious teams in the program participated in these meetingsheld every four months for two consecutive days (usuallyoutside the organisational premises). The relatively longerduration of these ceremonies allowed the reflective as wellas envisioning type of team interactions. The participantssuggested that a subset of human values could be added toPI objectives. “[values] really needs to be in PI planning... it gives a goodleveller for everyone to go OK, we understand what the key metricsor values [are], what we are working towards”
Developer (I13).Participants believed the inter-team and intra-team dis-cussions opportunities afforded by PI planning meetingscould facilitate human values discussion to understand theirsignificance for the project and how they can be weaved intothe future PIs. L Inspect and Adapt Meetings.
Agile principles em-phasise reflection on the work done to improve team per-formance. In SAFe, the Retrospectives and Inspect & Adaptmeetings offer opportunities for ’reflection-on-action’ whereagile teams reflect on the work finished (e.g., sprint or pro-gram increment) [72]. Several recommendations to embedreflection in Agile are found in the literature [72], [73]. Thereflection aspect of these meetings was appealing for prac-titioners as they could collectively contemplate how andwhere values should be introduced to have the maximumimpact on software delivery. Some considered these SAFeceremonies to be ideal for pondering over the real meaningof their work and whether that aligned with the team ororganisational values. “The places where the most reflection on how we actually workis the Retro[spective] and then the Inspect and Adapt...they areprobably the two best suited ceremonies [for values consideration]”
Business Change Specialist (I08). L Daily Stand-Ups are daily Agile team meetings todisseminate information, share work progress, flag barriers to achieving tasks, synchronise activities, and plan for thenext 24 hours [74]. Some participants believed that valuescould be discussed in stand-ups as they allow, to someextent, an opportunity to reflect and plan ahead. Anotherreason could be to link the development work with a ‘higherpurpose’ and keep it fresh in the teams’ collective mind.From the implementation aspect, it was believed thatsome additional questions could simply be added to thestandard discussion points during stand-ups to cover hu-man values. “We already have a list of the questions that we ask at standup...so it could very much be, how is the work you are doing todayupholding that value” Strategic Business Analyst (I12).However, the opinion was divided on whether or not todiscuss values on a daily basis. A developer pointed out thatthere was “too much overhead.. people will just ignore and it willbe too much constantly reviewing it”
Developer (I13). L Kick Off is one of the five main activities of theScrum process where the high-level backlog for the project,and the major project goals are defined. Kickoffs provide anopportunity to build a consensus among key stakeholders.They were also considered to be useful for defining valuesobjectives for the entire project, as the CTO (I16) providedan example: “when we do the kickoffs we can reiterate why it[values] is important... [for example] sometimes it’s not about the[customer], sometimes it is about even the mental health of theprofessional staff member”.
Some participants believed that a new ceremony was nec-essary to better address human values while developingsoftware using SAFe, as discussed in the following section. › Values Conversations.
Agile methods, including SAFe,provide considerable opportunities for teams for commu-nication and conversations, e.g., daily stand-ups, planningmeetings, inspect and adapt, and so on. These, however,are conventionally more focused on technical discussionsregarding performance and productivity. Some participantsbelieved a new value oriented meeting, say values con-versation, should also be introduced. The purpose of suchmeeting would be to solely focus on discussing humanvalues and identifying ways to embed them into softwaredelivery.Participants generally acknowledged that the concept ofvalue, which is at the core of Agile methods (e.g., SAFe), isdefined rather narrowly, i.e., from a business or economicperspective. Human values are inadequately discussed of-ten remain unrealised in the software. “[values] .... is just notsomething that software delivery teams are used to having conver-sations around... it is not a voice that we commonly represent inthe way that the teams operate”
Delivery Lead (I15).These thoughts were echoed by Digital TransformationLead (I04), who elaborated further by saying “there’s anopportunity for us to start talking more about not just the softwarebut also... How did it make you feel?... and bringing in users totalk about the level of helpfulness and empowerment that they’regetting”.
He suggested that values conversations need to beheld with actual people: “these are values that you need to hearfrom real people”.
The main practice to address stakeholder values and con-cerns was identified as values based testing and validation. L Business Facing Agile Testing.
Agile methodologies,including SAFe, utilise several types of business facing oruser testing to ensure software quality and improve user ac-ceptance. Examples include Alpha/Beta testing, prototypes,usability testing, user acceptance testing, and simulations.The current practice is to seek feedback on usability, func-tionality, and user experience. The participants believe thiscould very well be extended to “get some feedback to make surethat we are now considering those [values]”
RTE (I01).While testing for functionality is necessary, testing toascertain whether users’ values expectations are met orviolated by the software is even more important, as the De-livery Lead noted (I15): “I need to test whether it is functionallysound and robust but actually [also] testing against human valuesrequirements” . No ‘new’ practices were recommended. Participants con-tended that existing practices could easily be modified todiscuss human values in the SAFe framework.
Implementation of values in software was seen as the man-ifestation of a values-based culture introduced by organisa-tional leadership. “In order for the values to come to life, it has to be entrenchedin the culture”
Digital Transformation Manager (I04).The participants identified the following components asintervention points to strengthen organisational culture tomake it values-based and fit for human values delivery insoftware. (cid:143)
Hiring and Induction.
Values are at the core of organ-isational culture and often drive actions. The participantsbelieved a bottom-up approach to introducing a culturaltransformation is needed to make it values-driven. For ex-ample, Digital Transformation Manager (I04) suggested: “Interms of delivering software and delivering solutions that bringhuman values to life you have to start at the most basic layer whichis the people you hire how they are on-boarded into the company”.
Similarly, having consistent values within the team wasnecessary, according to Developer (I13), who noted that it “speaks to the health of the team and the values of what they holdcollectively” but felt “that’s really hard to foster unless you bringpeople from the very very start”. (cid:143)
Leadership.
Leaders of Agile teams set project direc-tion, align people, obtain resources, and motivate the teams[71]. The participants considered that it was the responsi-bility of leaders to ensure values are considered in everyproject. “Leaders, at any capacity need to take on that role of pro-moting values to allow trickling down of values to team members– not a single special role designated for values”
Delivery Lead(I15). Since the management style in Agile is ‘leadership-collaboration’ instead of command and control [73], values-conscious behaviour ’rubs off’ onto the development teams,only when it is ‘role modelled’ by the leadership. Such behaviours do not take root if they are hurled down to thedevelopment teams as orders that have to be followed. (cid:143) Collective Responsibility in Agile methods facili-tates vigilance and enhances perceived integrity throughshared work ethic [75]. Some participants believed that theresponsibility to include values in the team conversationsand to embed them in software could not be placed on asingle person or role. In fact, it had to be distributed andcollectively shared by everyone, i.e., from the leadershipdown to individual team members. “It should be everybody, it should be all of these people in bothof these layers the program and the team level, should be trying toreach this”
Business Change Specialist (I08).
ISCUSSION
Our study is the first to empirically investigate how humanvalues can be introduced into software systems developedusing scaled Agile methods. The previous section provideddetailed findings on the recommended intervention pointsto introduce values in the SAFe framework by participantsfrom our case company. The interventions relate to theinclusion of values in general. While it can be argued that inevery software some ‘non-negotiable’ values like privacy,security, social justice or fairness, should be introducedhowever, the participants made no such recommendations.We also believe such suggestions are beyond the scope ofthis study. The decision for the type of values to be includedtherefore rests upon the stakeholders in a given context andproject.Our findings provide empirical evidence of the needto introduce values in Agile frameworks such as SAFe.Practitioners may be able to relate to the findings betterwhen looking at SAFe Principles. What jumps out especiallyfrom the SAFe (4.6) principles is its narrow view of value,defined as “the economic worth of the capability to the businessand the customer” . Business or economic value is no doubtimportant in commercial software development, the addedemphasis that an economic rather than a human-centricview, is the right way to go about it is hard to justify. Notethe pronounced emphasis on the economic aspect in thistext from the SAFe (4.6) documentation: “It [economic worth]is Principle 1 for a reason. Economics should inform and drivedecisions at all levels, from Portfolio to Development Teams” .No wonder some of the more conscientious practitionersthought that such a view/stance makes SAFe rather ‘soul-less’ .The recent version of SAFe (5.0) introduces customer-centricity as a ‘mindset’ that enterprises should focus on. Itpromotes aligning solution delivery to end-users so that thecustomers enjoy a positive experience. While this is far bet-ter than the transactional or utilitarian approach discussedpreviously/above, the delivery of customer value in SAFe(5.0) even with newly introduced customer-centricity con-tinues to guide the economics of the problem solving/thesolution by documented statements like: “There are twoprimary means by which a customer derives value from productsand solutions, 1) reducing their costs and 2) increasing theirrevenue”.
This suggests that with the current version, SAFe isstill focused on business/economic value and needs certaininterventions to steer its focus on solution delivery which isbased on human values.
Values, admittedly are subjective. Software engineers inparticular consider them as “mushy stuff” [44], [76]. Since1994, when Jim Collins and Jerry Porras published theirbook “Built to Last” making the case that the best com-panies adhered to certain core values, many executives feltpressured to conjure up some values of their own. However,too often these value statements turn out to be rather blandor toothless; something which stands for nothing but adesire to be au courant or politically correct [76]. This leavescompanies with an ambiguous identity, one without anyrallying point for the employees. The ambiguity of valueshinders the understanding necessary for their translationinto business contexts and implementation in software.This resonated with the participants who emphasisedthe introduction of values oriented roles who not onlygenuinely believe in the values cause but have the necessaryexpertise or skills, for example, to help reduce the ambiguityand translate these concepts in a language that softwareengineers can understand and implement. This is impor-tant because defining value statements more clearly andconsciously enhances their chances of implementation. Fur-thermore, when developed conscientiously, communicatedclearly, and adhered to devoutly, values can foster a culturethat sets companies apart from the competition. Therefore,instead of using vague nouns for values which often do notprompt any action, values (statements) should make sense andhave to be made actionable [77].
Addressing human values might be seen as requiring fun-damentally new software techniques and methods: a rev-olution in the way that software practitioners think andwork [1]. However, our study provides evidence that recom-mended interventions to incorporate values can be imple-mented fairly easily using a combination and/or adaptationof existing building blocks of Agile methods: artefacts, prac-tices and roles with an overarching values-driven mindset.Where existing methods fail, however, is knowing whenand how to combine and/or adapt existing artefacts ortechniques. We saw evidence of lost opportunities: inter-viewees, for example, suggest modifying user stories to“values stories” to capture user values, but this was notdone. This could arguably be attributed to SAFe’s focuson the ’creation’ of business value rather than aligningsolutions delivery with human values. Evidently, the ideaof value ’creation’ relates more closely to the functionalor economic view of value rather than a more social orethical view of values. With small adaptations, like spellingout human values as overarching software requirementsin user stories, we can trigger conversions that can leadto some implementable functional requirements which canbe ‘satisfied’ through software. This is one example of arelatively simple adaptation of a requirements artefact canlead to a significant impact on practice, as noted by ourparticipants in Section 4.2.1 (User Stories) and other evolu-tionary interventions noted in Figure 4. Similar adaptationsare possible in existing roles and practices.
Guidelines foradapting practices, roles, artefacts, and culture, need to be in- cluded in SAFe documentation and coaching material to facilitatevalues inclusion. To become a values-based organisation, something else thatneeds to be fixed that has to do with neither the domainof development nor sound engineering practice, and thatis culture . When something is amiss with certain elementsof the organisational culture it does not remain conduciveto progress. This is because change in any aspect of anorganisational system is inextricably linked with the cul-tural context, and values have a significant impact on thesuccess and failure of change efforts [78]. Deficiencies inorganisational culture therefore, must be acknowledged andaddressed, if software organisations are to move forward.The case organisation was on its way towards a values-based culture. This was in part evident by some of theinitial steps taken by the leadership e.g. introduction ofthe cultural handbook of organisational values to guidetheir development practices and instigate a cultural change(noted in Section 3.1). It was also evident from the partici-pants‘ intent manifested in their practical recommendationsto ‘valuefy’ the existing development processes and prac-tices (i.e. to integrate values in them) as summarised inTable 3 and covered in Section 4.2 through to Section 4.6.Among others, these steps included a) introducing multi-threaded multi-layered communication processes to under-stand, promote and implement human values, b) advocatingshared responsibility c) the promoting the induction and on-boarding of individuals that are more aligned to the organi-sational values, and d) targeting development outcomes thatare informed by the value-systems of their users and arecognisant of potential social implications of the software.These encouraging and less ’intrusive’ initiatives to cultivatechange are more likely to take roots even in somewhatmore‘technically-oriented’ software teams. This in essencemay be the precursor to a values-driven organisationalculture.While the desire was present, the awareness of the im-portance of integrating values into the development processand understanding the implication when done wrong, e.g.,for the organisation, users, and the society, varied amongthe participants in the case organisation. As with the otherkind of organisational culture change, this is possible withthe appropriate motivation, incentives, mentoring, and lead-ership role modelling. Since the recommended interventionpoints stem from our study of the SAFe framework, which isessentially based on Scrum and Kanban, recommendationshighlighted in this study could easily be adapted or appliedin other Agile methods.
Cultural change in organisations is notoriously challengingand often invokes reactions of intense nature [79], [80]. Therecommended interventions in this study thus may bear sig-nificant monetary and emotional costs. For example, mov-ing from essentially a profit driven towards values drivenphilosophy can bear a positive outcome when participants’values are congruent with those of the organisation. This,however, may require significant investment in making the hiring, recruitment, and training processes values-oriented,e.g., hiring an ethically conscious workforce, training exist-ing ones, and recognising and rewarding employees thatexemplify behaviour aligned to the desired values (Table 3).Both monetary cost and effort overheard of such concreteinitiatives may or may not be feasible for some organisa-tions.
Even when values are considered explicitly, organisationslike the one in this study tend to consider them mainlyin the early phases of a project, with values discussionstaking place mainly during project selection and early re-quirements of design stages [76]. Occasional considerationis also given to some values like privacy and security duringAlpha/Beta testing as well, but it tends to receive lessattention from the team members. One possible reason forthis is that there are more well-known techniques that applyin the early development phases, such as personas that helpsoftware engineers to understand the behaviours of usersand to design efficient feedback acquisition to meet users’needs with lower functional capability during the designprocess. Another reason might be the development team’sperceived low return on effort on downstream developmentactivities [76]. Given the nature of the interventions recom-mended that span the whole life cycle activities and roles,our findings suggest that instead of focusing on individualdevelopment phases and development practices softwaremethodologies need to be adapted to address values throughoutthe development life cycle.
HREATS TO V ALIDITY
We discuss the potential threats to our research method andfindings from the qualitative research perspective [81], [82].
Transferability . We collected data from a single or-ganisation with relatively small sample size. Hence, mostobservations are exclusive to the case company and maynot be representative of other organisations or softwarepractitioners. We limited the impact of this threat by inter-viewing practitioners with varying levels of seniority andexperiences and different roles (e.g., CIO, Senior Analysts,Developers, etc.). Moreover, the research community, inparticular social sciences research, values single-case studiesas a potential source of significant contributions to scientificdiscovery [83], [84], [85].
Credibility . The approach used for data collection, de-veloping the interview questions, and the selection of thecase company and participants could introduce possiblethreats to the credibility of this study. We are confident thatour approach of multi-source data collection and triangu-lation increased the plausibility of our results significantly.To mitigate the potential threats stemming from the inter-view questions, (1) we kept the interview questions open-ended; (2) follow-up questions were customised accordingto the interviewees’ responses; and (3) we allowed theinterviewees to share any significant experiences they hadrelated to integrating and/or ignoring human values in anysoftware development methods, not limited to SAFe [86]. Recruiting participants that could reflect unbiased opinionsabout integrating human values in SAFe was a challenge.The participants came from a purposively selected companythat had a clear values statement. This may have triggeredthe participants to exaggerate the role of human values insoftware development methods and how they were fol-lowed in the case organisation as well as provide moresocially acceptable responses (i.e., social desirability bias)[87]. We mitigated these threats to some extent by assuringthe participants that their responses and personal detailswould not be identifiable [88]. It was also crucial to recruitparticipants with the right competencies. To this end, wediscussed our study’s objective with the senior members ofthe case company before the study started, which helpedthem introduce the right candidates. To further alleviatethis threat, we shared the relevant references on humanvalues (e.g., Schwartz model) with the participants in ad-vance, leading to increased preparation and engagement indiscussions [86].
Confirmability . We minimised researcher bias by involv-ing four coders in the analysis process (i.e., investigatortriangulation) [61]. We also set up several internal discus-sions to review and verify the codes identified by the fourcoders involved in the study. We further limited this threatby sharing an early version of this paper with one of thecase company representatives and seeking his feedback. Stolet al. [82] argue that such data triangulation reduces thesubjectivity of the researcher’s interpretation and judgementand, in turn, helps establish confirmability.
ONCLUSION AND F UTURE W ORK
Agile methods, as the most popular software developmentmethodologies, lack specific and specific guidelines in theircurrent form to address human values such as accessibil-ity, social responsibility, equality. However, their focus onpeople and interactions makes them amenable to adaptingand incorporating human values. In this paper, we reportfindings from our qualitative case study to reveal the po-tential intervention points within one of the most widelyused large scale Agile methods, the Scaled Agile Framework(SAFe), that can be leveraged to address human values. Theidentified intervention points can be broadly classified intofive categories: artefacts, roles, practices, ceremonies, andculture. Our study has shown that some existing artefacts(user story, epic, persona, user journey map), roles (productowner, business owner, release train engineer), ceremonies(stand-up meeting, inspect and adapt meeting, PI plan-ning meeting, kick off meeting), and practice (business-facing testing) in SAFe can be changed to consciously andexplicitly identify and address the values concerns of abroad range of stakeholders. Furthermore, we found newroles (values champion, values promoter, values translator,values offices, values mentor), artefacts (checklist, legislativerequirement, corporate directive), ceremonies (values con-versation) and cultural practices (hiring/induction, leader-ship, collective responsibility) that can be added to SAFe tosupport human values inclusion in software.Although the focus of this case study was specificallyon the implementation of SAFe and its capacity for takinghuman values into account, findings are also relevant more broadly and further research is needed to investigate Agilemethodologies in general. Key questions arising from thisstudy are whether Agile is conducive to the incorporationof human values, or whether its core principles and the tech-niques developed for practice settings might inadvertentlyhinder the recognition of human values. R EFERENCES [1] J. Whittle, M. A. Ferrario, W. Simm, and W. Hussain, “A case forhuman values in software engineering,”
IEEE Software
Proceedings of the 2017 11th JointMeeting on Foundations of Software Engineering , 2017, pp. 498–510.[6] H. Perera, W. Hussain, J. Whittle, A. Nurwidyantoro,D. Mougouei, R. A. Shams, and G. Oliver, “A study on theprevalence of human values in software engineering publications,2015 – 2018,” in
Proceedings of the 42nd International Conference onSoftware Engineering , 2020.[7] T. Philbeck, N. Davis, and A. M. E. Larsen, “Values, ethics andinnovation: Rethinking technological development in the fourthindustrial revolution,” Tech. Rep., 2018.[8] European Parliament and of the Council, “General Data ProtectionRegulation,” 2016, regulation (EU) 2016/679, GDPR.[9] F. A. Raso, H. Hilligoss, V. Krishnamurthy, C. Bavitz, and L. Kim,“Artificial intelligence & human rights: Opportunities & risks,”Tech. Rep., 2018.[10] S. Sirur, J. R. Nurse, and H. Webb, “Are we there yet? understand-ing the challenges faced in complying with the general data pro-tection regulation (GDPR),” in
Proceedings of the 2nd InternationalWorkshop on Multimedia Privacy and Security , 2018, pp. 88–95.[11] R. Hoda, N. Salleh, and J. Grundy, “The rise and evolution of agilesoftware development,”
IEEE software , vol. 35, no. 5, pp. 58–63,2018.[12] M. Fowler and J. Highsmith, “The agile manifesto,”
SoftwareDevelopment
Case study research: Design and methods (applied socialresearch methods) . Sage Publications, 2014.[15] S. H. Schwartz, “Are there universal aspects in the structure andcontents of human values?”
Journal of Social Issues , vol. 50, no. 4,pp. 19–45, 1994.[16] S. H. Schwartz and W. Bilsky, “Toward a universal psychologicalstructure of human values,”
Journal of Personality and Social Psy-chology , vol. 53, no. 3, pp. 550–562, 1987.[17] S. H. Schwartz, “Universals in the content and structure of values:Theoretical advances and empirical tests in 20 countries,”
Advancesin Experimental Social Psychology , vol. 25, pp. 1–65, 1992.[18] D. Seddig and E. Davidov, “Values, attitudes toward interpersonalviolence, and interpersonal violent behavior,”
Frontiers in Psychol-ogy , vol. 9, p. 604, 2018.[19] H. Perera, G. Mussbacher, W. Hussain, R. Ara Shams, A. Nur-widyantoro, and J. Whittle, “Continual human value analysis insoftware development: A goal model based approach,” in ,2020, pp. 192–203.[20] L. Williams and A. Cockburn, “Agile software development: it’sabout feedback and change,”
IEEE computer , vol. 36, no. 6, pp.39–43, 2003.[21] T. Dingsøyr, S. Nerur, V. Balijepally, and N. B. Moe, “A decade ofagile methodologies: Towards explaining agile software develop-ment,”
Journal of Systems and Software , vol. 85, no. 6, pp. 1213–1221,2012, special Issue: Agile Development. [22] CollabNet VersionOne, “13th annual state of agile report,” Tech.Rep., 2019.[23] P. Abrahamsson, O. Salo, J. Ronkainen, and J. Warsta, Agile Soft-ware Development Methods: Review and Analysis . VTT Publications,2002.[24] K. Beck,
Extreme Programming Explained: Embrace Change .Addison-Wesley Professional, 2000.[25] K. S. Rubin,
Essential Scrum: A Practical Guide to the Most PopularAgile Process . Addison-Wesley, 2012.[26] A. Cockburn,
Crystal Clear: A Human-powered Methodology for SmallTeams . Pearson Education, 2004.[27] B. Boehm, “Get ready for agile methods, with care,”
Computer ,vol. 35, no. 1, pp. 64–69, 2002.[28] K. Dikert, M. Paasivaara, and C. Lassenius, “Challenges andsuccess factors for large-scale agile transformations: A systematicliterature review,”
Journal of Systems and Software , vol. 119, pp. 87–108, 2016.[29] K. Conboy and N. Carroll, “Implementing large-scale agile frame-works: challenges and recommendations,”
IEEE Software , vol. 36,no. 2, pp. 44–50, 2019.[30] M. Paasivaara, B. Behm, C. Lassenius, and M. Hallikainen, “Large-scale agile transformation at Ericsson: a case study,”
EmpiricalSoftware Engineering , vol. 23, no. 5, pp. 2550–2596, 2018.[31] C. Larman and B. Vodde. LeSS framework. [Online]. Available:https://less.works/less/framework/index.html[32] S. W. Ambler and M. Lines,
Disciplined agile delivery: A practitioner’sguide to agile software delivery in the enterprise . IBM press, 2012.[33] O. Turetken, I. Stojanov, and J. J. Trienekens, “Assessing theadoption level of scaled agile development: a maturity model forScaled Agile Framework,”
Journal of Software: Evolution and Process ,vol. 29, no. 6, p. e1796, 2017.[34] C. Ebert and M. Paasivaara, “Scaling agile,”
IEEE Software , vol. 34,no. 6, pp. 98–103, 2017.[35] J. van den Hoven, P. E. Vermaas, and I. van de Poel, “Design forvalues: An introduction,” in
Handbook of Ethics, Values, and Tech-nological Design: Sources, Theory, Values and Application Domains .Springer, 2015, pp. 1–7.[36] J. Davis and L. P. Nathan, “Value sensitive design: Applications,adaptations, and critiques,” in
Handbook of Ethics, Values, and Tech-nological Design: Sources, Theory, Values and Application Domains ,J. van den Hoven, P. E. Vermaas, and I. van de Poel, Eds. SpringerNetherlands, 2015, pp. 11–40.[37] B. Friedman, P. Kahn, and A. Borning, “Value sensitive design:Theory and methods,” University of Washington, Tech. Rep., 2002.[38] M. Van der Velden and C. M¨ortberg, “Participatory design anddesign for values,” in
Handbook of Ethics, Values, and TechnologicalDesign: Sources, Theory, Values and Application Domains . Springer,2015, pp. 1–22.[39] H. Aldewereld, V. Dignum, and Y.-h. Tan, “Design for values insoftware development,” in
Handbook of Ethics, Values, and Tech-nological Design: Sources, Theory, Values and Application Domains .Springer, 2015, pp. 831–845.[40] J. Whittle, “Is your software valueless?”
IEEE Software , vol. 36,no. 3, pp. 112–115, 2019.[41] S. Thew and A. Sutcliffe, “Value-based requirements engineering:method and experience,”
Requirements Engineering , vol. 23, no. 4,pp. 443–464, 2018.[42] M. A. Ferrario, W. Simm, S. Forshaw, A. Gradinar, M. T. Smith, andI. Smith, “Values-first SE: research principles in practice,” in , 2016, pp. 553–562.[43] E. Winter, S. Forshaw, and M. A. Ferrario, “Measuring human val-ues in software engineering,” in
Proceedings of the 12th ACM/IEEEInternational Symposium on Empirical Software Engineering and Mea-surement , 2018, pp. 1–4.[44] K. W. Miller and D. K. Larson, “Agile software development:human values and culture,”
IEEE Technology and Society Magazine ,vol. 24, no. 4, pp. 36–42, 2005.[45] M. Brhel, H. Meth, A. Maedche, and K. Werder, “Exploring prin-ciples of user-centered agile software development: A literaturereview,”
Information and Software Technology , vol. 61, no. C, pp.163–181, 2015.[46] E.-M. Sch¨on, J. Thomaschewski, and M. J. Escalona, “Agile re-quirements engineering: A systematic literature review,”
ComputerStandards & Interfaces , vol. 49, pp. 79–91, 2017.[47] C. Detweiler and M. Harbers, “Value stories: Putting humanvalues into requirements engineering,” in
Proceedings of the 4th International Workshop on Creativity in Requirements Engineering(CreaRE) , 2014, pp. 2–11.[48] F. Fagerholm and M. Pagels, “Examining the structure of leanand agile values among software developers,” in
InternationalConference on Agile Software Development , 2014, pp. 218–233.[49] C. Lawrence and P. Rodriguez, “The interpretation and legitimiza-tion of values in agile’s organizing vision,” in
European Conferenceon Information Systems (ECIS) , 2012.[50] A. J. Mills, G. Durepos, and E. Wiebe,
Encyclopedia of Case StudyResearch . Sage Publications, 2009.[51] W. Tellis, “Introduction to case study,”
The qualitative report , vol.269, 1997.[52] S. R. Ponelis, “Using interpretive qualitative case studies forexploratory research in doctoral studies: A case of informationsystems research in small and medium enterprises,”
InternationalJournal of Doctoral Studies , vol. 10, no. 1, pp. 535–550, 2015.[53] S. B. Merriam and E. J. Tisdell,
Qualitative research: A guide to designand implementation . John Wiley & Sons, 2015.[54] S. H. Schwartz and A. Bardi, “Value hierarchies across cultures:Taking a similarities perspective,”
Journal of cross-cultural Psychol-ogy , vol. 32, no. 3, pp. 268–290, 2001.[55] S. Easterbrook, J. Singer, M. A. Storey, and D. Damian, “Selectingempirical methods for software engineering research,” in
Guide toAdvanced Empirical Software Engineering . Springer, 2008, pp. 285–311.[56] J. Seawright and J. Gerring, “Case selection techniques in casestudy research: A menu of qualitative and quantitative options,”
Political Research Quarterly , vol. 61, no. 2, pp. 294–308, 2008.[57] D. Leffingwell,
SAFe 4.5 Reference Guide: Scaled Agile Framework forLean Enterprises . Addison-Wesley Professional, 2018.[58] M. Q. Patton, “Qualitative interviewing,” in
Qualitative research andevaluation methods , 2002, vol. 3, no. 1, pp. 344–347.[59] N. Malhotra and D. Birks,
Marketing Research: An Applied Approach:3rd European Edition . Pearson Education, 2007.[60] V. Braun and V. Clarke, “Using thematic analysis in psychology,”
Qualitative Research in Psychology , vol. 3, no. 2, pp. 77–101, 2006.[61] N. Carter, D. Bryant-Lukosius, A. DiCenso, J. Blythe, and A. J.Neville, “The use of triangulation in qualitative research,”
Oncol-ogy Nursing Forum , vol. 41, no. 5, 2014.[62] R. Miller and L. A. Williams, “Personas: Moving beyond role-based requirements engineering,” North Carolina State University.Dept. of Computer Science, Tech. Rep., 2006.[63] R. Clay-Williams and L. Colligan, “Back to basics: checklists inaviation and healthcare,”
BMJ Quality & Safety , vol. 24, no. 7, pp.428–431, 2015.[64] A. Rajkomar, J. Dean, and I. Kohane, “Machine learning inmedicine,”
New England Journal of Medicine , vol. 380, no. 14, pp.1347–1358, 2019.[65] B. Anda and D. I. Sjøberg, “Towards an inspection technique foruse case models,” in
Proceedings of the 14th International Conferenceon Software Engineering and Knowledge Engineering , 2002, pp. 127–134.[66] B. Brykczynski, “A survey of software inspection checklists,”
ACMSIGSOFT Software Engineering Notes , vol. 24, no. 1, p. 82, 1999.[67] E. Palmer and A. Degani, “Electronic checklists: Evaluation of twolevels of automation,” in
Proceedings of the International Symposiumon Aviation Psychology . The Ohio State University Columbus,1991, pp. 178–183.[68] J. Kratzer and I. Michelfelder, “The social footprint of championsand promoters as creative leaders in innovating and executing,”in
Handbook of Research on Leadership and Creativity . Edward ElgarPublishing, 2017, pp. 182–202.[69] U. Lichtenthaler and H. Ernst, “The role of champions in the exter-nal commercialization of knowledge,”
Journal of Product InnovationManagement , vol. 26, no. 4, pp. 371–387, 2009.[70] H. G. Gem ¨unden, S. Salomo, and K. H¨olzle, “Role models forradical innovations in times of open innovation,”
Creativity andInnovation Management , vol. 16, no. 4, pp. 408–421, 2007.[71] R. Hoda, J. Noble, and S. Marshall, “Self-organizing roles onagile software development teams,”
IEEE Transactions on SoftwareEngineering , vol. 39, no. 3, pp. 422–444, 2012.[72] J. Babb, R. Hoda, and J. Nørbjerg, “Embedding reflection andlearning into agile software development,”
IEEE software , vol. 31,no. 4, pp. 51–57, 2014.[73] A. Cockburn and J. Highsmith, “Agile software development, thepeople factor,”
Computer , vol. 34, no. 11, pp. 131–133, 2001. [74] S. Augustine, B. Payne, F. Sencindiver, and S. Woodcock, “Agileproject management: steering from the edges,” Communications ofthe ACM , vol. 48, no. 12, pp. 85–89, 2005.[75] O. McHugh, K. Conboy, and M. Lang, “Agile practices: The impacton trust in software project teams,”
IEEE Software , vol. 29, no. 3,pp. 71–76, 2012.[76] W. Hussain, H. Perera, J. Whittle, A. Nurwidyantoro, R. Hoda,R. A. Shams, and G. Oliver, “Human values in software engi-neering: Contrasting case studies of practice,”
IEEE Transactions onSoftware Engineering , pp. 1–1, 2020.[77] P. M. Lencioni, “Make your values mean something,”
HarvardBusiness Review , vol. 80, no. 7, pp. 113–117, 2002.[78] A. Danıs¸man, “Good intentions and failed implementations: Un-derstanding culture-based resistance to organizational change,”
European Journal of Work and Organizational Psychology , vol. 19,no. 2, pp. 200–220, 2010.[79] R. F. Baumeister and M. Muraven, “Identity as adaptation to so-cial, cultural, and historical context,”
Journal of Adolescence , vol. 19,no. 5, pp. 405–416, 1996.[80] R. K. Smollan and J. G. Sayers, “Organizational culture, changeand emotions: A qualitative study,”
Journal of Change Management ,vol. 9, no. 4, pp. 435–457, 2009.[81] E. G. Guba, “Criteria for assessing the trustworthiness of natural-istic inquiries,”
ECTJ , vol. 29, no. 2, p. 75, 1981.[82] K.-J. Stol, P. Avgeriou, M. A. Babar, Y. Lucas, and B. Fitzgerald,“Key factors for adopting inner source,”
ACM Transactions onSoftware Engineering and Methodology , vol. 23, no. 2, pp. 1–35, 2014.[83] B. Flyvbjerg, “Five misunderstandings about case-study research,”
Qualitative Inquiry , vol. 12, no. 2, pp. 219–245, 2006.[84] E. Kalliamvakou, C. Bird, T. Zimmermann, A. Begel, R. DeLine,and D. M. German, “What makes a great manager of softwareengineers?”
IEEE Transactions on Software Engineering , vol. 45, no. 1,pp. 87–106, 2017.[85] A. Kuper,
The Social Science Encyclopedia . Routledge, 2013.[86] S. E. Hove and B. Anda, “Experiences from conducting semi-structured interviews in empirical software engineering research,”in , 2005, pp. 10–23.[87] A. Furnham, “Response bias, social desirability and dissimula-tion,”
Personality and Individual Differences , vol. 7, no. 3, pp. 385–400, 1986.[88] G. Gousios, M.-A. Storey, and A. Bacchelli, “Work practices andchallenges in pull-based development: the contributor’s perspec-tive,” in2016 IEEE/ACM 38th International Conference on SoftwareEngineering (ICSE)