Timo O.A. Lehtinen

Perceived causes of software project failures - An analysis of their relationships

Context: Software project failures are common. Even though the reasons for failures have been widely studied, the analysis of their causal relationships is lacking. This creates an illusion that the causes of project failures are unrelated. Objective: The aim of this study is to conduct in-depth analysis of software project failures in four software product companies in order to understand the causes of failures and their relationships. For each failure, we want to understand which causes, so called bridge causes, interconnect different process areas, and which causes were perceived as the most promising targets for process improvement. Method: The causes of failures were detected by conducting root cause analysis. For each cause, we classified its type, process area, and interconnectedness to other causes. We quantitatively analyzed which type, process area, and interconnectedness categories (bridge, local) were common among the causes selected as the most feasible targets for process improvement activities. Finally, we qualitatively analyzed the bridge causes in order to find common denominators for the causal relationships interconnecting the process areas. Results: For each failure, our method identified causal relationships diagrams including 130-185 causes each. All four cases were unique, albeit some similarities occurred. On average, 50% of the causes were bridge causes. Lack of cooperation, weak task backlog, and lack of software testing resources were common bridge causes. Bridge causes, and causes related to tasks, people, and methods were common among the causes perceived as the most feasible targets for process improvement. The causes related to the project environment were frequent, but seldom perceived as feasible targets for process improvement. Conclusion: Prevention of a software project failure requires a case-specific analysis and controlling causes outside the process area where the failure surfaces. This calls for collaboration between the individuals and managers responsible for different process areas.

Development and evaluation of a lightweight root cause analysis method (ARCA method) – Field studies at four software companies

Abstract in Undetermined Context The key for effective problem prevention is detecting the causes of a problem that has occurred. Root cause analysis (RCA) is a structured investigation of the problem to identify which underlying causes need to be fixed. The RCA method consists of three steps: target problem detection, root cause detection, and corrective action innovation. Its results can help with process improvement. Objective This paper presents a lightweight RCA method, named the ARCA method, and its empirical evaluation. In the ARCA method, the target problem detection is based on a focus group meeting. This is in contrast to prior RCA methods, where the target problem detection is based on problem sampling, requiring heavy startup investments. Method The ARCA method was created with the framework of design science. We evaluated it through field studies at four medium-sized software companies using interviews and query forms to collect feedback from the case attendees. A total of five key representatives of the companies were interviewed, and 30 case participants answered the query forms. The output of the ARCA method was also evaluated by the case attendees, i.e., a total 757 target problem causes and 124 related corrective actions. Results The case attendees considered the ARCA method useful and easy to use, which indicates that it is beneficial for process improvement and problem prevention. In each case, 24–77 target problem root causes were processed and 13–40 corrective actions were developed. The effort of applying the method was 89 man-hours, on average. Conclusion The ARCA method required an acceptable level of effort and resulted in numerous high-quality corrective actions. In contrast to the current company practices, the method is an efficient method to detect new process improvement opportunities and develop new process improvement ideas. Additionally, it is easy to use.

Time pressure: a controlled experiment of test case development and requirements review

Time pressure is prevalent in the software industry in which shorter and shorter deadlines and high customer demands lead to increasingly tight deadlines. However, the effects of time pressure have received little attention in software engineering research. We performed a controlled experiment on time pressure with 97 observations from 54 subjects. Using a two-by-two crossover design, our subjects performed requirements review and test case development tasks. We found statistically significant evidence that time pressure increases efficiency in test case development (high effect size Cohen’s d=1.279) and in requirements review (medium effect size Cohen’s d=0.650). However, we found no statistically significant evidence that time pressure would decrease effectiveness or cause adverse effects on motivation, frustration or perceived performance. We also investigated the role of knowledge but found no evidence of the mediating role of knowledge in time pressure as suggested by prior work, possibly due to our subjects. We conclude that applying moderate time pressure for limited periods could be used to increase efficiency in software engineering tasks that are well structured and straight forward.

A tool supporting root cause analysis for synchronous retrospectives in distributed software teams

Context: Root cause analysis (RCA) is a useful practice for software project retrospectives, and is typically carried out in synchronous collocated face-to-face meetings. Conducting RCA with distributed teams is challenging, as face-to-face meetings are infeasible. Lack of adequate real-time tool support exacerbates this problem. Furthermore, there are no empirical studies on using RCA in synchronous retrospectives of geographically distributed teams. Objective: This paper presents a real-time cloud-based software tool (ARCA-tool) we developed to support RCA in distributed teams and its initial empirical evaluation. The feasibility of using RCA with distributed teams is also evaluated. Method: We compared our tool with 35 existing RCA software tools. We conducted field studies of four distributed agile software teams at two international software product companies. The teams conducted RCA collaboratively in synchronous retrospective meetings by using the tool we developed. We collected the data using observations, interviews and questionnaires. Results: Comparison revealed that none of the existing 35 tools matched all the features of our ARCA-tool. The team members found ARCA-tool to be an essential part of their distributed retrospectives. They considered the software as efficient and very easy to learn and use. Additionally, the team members perceived RCA to be a vital part of the retrospectives. In contrast to the prior retrospective practices of the teams, the introduced RCA method was evaluated as efficient and easy to use. Conclusion: RCA is a useful practice in synchronous distributed retrospectives. However, it requires software tool support for enabling real-time view and co-creation of a cause-effect diagram. ARCA-tool supports synchronous RCA, and includes support for logging problems and causes, problem prioritization, cause-effect diagramming, and logging of process improvement proposals. It enables conducting RCA in distributed retrospectives.

What are Problem Causes of Software Projects? Data of Root Cause Analysis at Four Software Companies

Root cause analysis (RCA) is a structured investigation of a problem to detect the causes that need to be prevented. We applied ARCA, an RCA method, to target problems of four medium-sized software companies and collected 648 causes of software engineering problems. Thereafter, we applied grounded theory to the causes to study their types and related process areas. We detected 14 types of causes in 6 process areas. Our results indicate that development work and software testing are the most common process areas, whereas lack of instructions and experiences, insufficient work practices, low quality task output, task difficulty, and challenging existing product are the most common types of the causes. As the types of causes are evenly distributed between the cases, we hypothesize that the distributions could be generalizable. Finally, we found that only 2.5% of the causes are related to software development tools that are widely investigated in software engineering research.

Bottom-up Adoption of Continuous Delivery in a Stage-Gate Managed Software Organization

Context: Continuous delivery (CD) is a development practice for decreasing the time-to-market by keeping software releasable all the time. Adopting CD within a stage-gate managed development process might be useful, although scientific evidence of such adoption is not available. In a stage-gate process, new releases pass through stages and gates protect low-quality output from progressing. Large organizations with stage-gate processes are often hierarchical and the adoption can be either top-down, driven by the management, or bottom-up, driven by the development unit. Goal: We investigate the perceived problems of bottom-up CD adoption in a large global software development unit at Nokia Networks. Our goal is to understand how the stage-gate development process used by the unit affects the adoption. Method: The overall research approach is a qualitative single case study on one of the several geographical sites of the development unit. We organized two 2-hour workshops with altogether 15 participants to discover how the stage-gate process affected the adoption. Results: The stage-gate development process caused tight schedules for development and process overhead because of the gate requirements. Moreover, the process required using multiple version control branches for different stages in the process, which increased development complexity and caused additional branch overhead. Together, tight schedule, process overhead and branch overhead caused the lack of time to adopt CD. In addition, the use of multiple branches limited the available hardware resources and caused delayed integration. Conclusions: Adopting CD in a development organization that needs to conform to a stage-gate development process is challenging. Practitioners should either gain support from the management to relax the required process or reduce their expectations on what can be achieved while conforming to the process. To simplify the development process, the use of multiple version control branches could be replaced with feature toggles.

Diagrams or structural lists in software project retrospectives - An experimental comparison

Root cause analysis (RCA) is a recommended practice in retrospectives and cause-effect diagram (CED) is a commonly recommended technique for RCA. Our objective is to evaluate whether CED improves the outcome and perceived utility of RCA. We conducted a controlled experiment with 11 student software project teams by using a single factor paired design resulting in a total of 22 experimental units. Two visualization techniques of underlying causes were compared: CED and a structural list of causes. We used the output of RCA, questionnaires, and group interviews to compare the two techniques. In our results, CED increased the total number of detected causes. CED also increased the links between causes, thus, suggesting more structured analysis of problems. Furthermore, the participants perceived that CED improved organizing and outlining the detected causes. The implication of our results is that using CED in the RCA of retrospectives is recommended, yet, not mandatory as the groups also performed well with the structural list. In addition to increased number of detected causes, CED is visually more attractive and preferred by retrospective participants, even though it is somewhat harder to read and requires specific software tools. Root cause analysis is a recommended practice in retrospectives.We compare the use of cause-effect diagram to the use of structural lists in root cause analysis.Cause-effect diagram improves the root cause analysis.Cause-effect diagram is preferred by participants (75%).

Why the Development Outcome Does Not Meet the Product Owners’ Expectations?

Many software development projects fail due to problems in requirements, scope, and collaboration. This paper presents a case study of the mismatch between the expectations of Product Owners and the outcome of the development in a large distributed Scrum organization. The data was collected in retrospective meetings involving a team of Product Owners and two software development teams. A focused root cause analysis of the problem “Why the expectations of Product Owners do not meet the outcome of development teams?” was conducted. The analysis aimed at explaining why the problem occurred and how the causes were related to one another. The outcomes were analyzed both quantitatively and qualitatively. Our results illustrate the challenges of implementing the Product Owner role in the context of complex, high-variability requirements and distributed development. We highlight the importance of true collaboration, effective requirements specification activities, and sufficient resources for the Product Owner role.