Hennie Huijgens

How to build a good practice software project portfolio

What can we learn from historic data that is collected in three software companies that on a daily basis had to cope with highly complex project portfolios? In this paper we analyze a large dataset, containing 352 finalized software engineering projects, with the goal to discover what factors affect software project performance, and what actions can be taken to increase project performance when building a software project portfolio. The software projects were classified in four quadrants of a Cost/Duration matrix: analysis was performed on factors that were strongly related to two of those quadrants, Good Practices and Bad Practices. A ranking was performed on the factors based on statistical significance. The paper results in an inventory of ‘what factors should be embraced when building a project portfolio?’ (Success Factors), and ‘what factors should be avoided when doing so?’ (Failure Factors). The major contribution of this paper is that it analyzes characteristics of best performers and worst performers in the dataset of software projects, resulting in 7 Success Factors (a.o. steady heartbeat, a fixed, experienced team, agile (Scrum), and release-based), and 9 Failure Factors (a.o. once-only project, dependencies with other systems, technology driven, and rules- and regulations driven).

An exploratory study on functional size measurement based on code

In this paper we explore opportunities, challenges, and obstacles that Functional Size Measurement (FSM) experts assume to be in automatically derived functional size, directly from the software project code itself. We designed a structured survey, that was answered by 336 FSM specialists. A majority of the respondents consider FSM to be an important tool for decision making. No indications are found for any perceived impact of agile methodology on the difficulty of applying FSM. Respondents overall think of automated FSM as important, but also difficult to realize. 54% of the respondents think that automated FSM will help measurement specialists, while 44% thinks that it will help decision makers too. The most preferred FSM method for automation is COSMIC (25%), followed by IFPUG (21%) and Nesma (16%). Respondents perceive automated FSM to be most suitable for baselining, benchmarking, and maintenance and legacy purposes.

An exploratory study on the effects of perceived value and stakeholder satisfaction on software projects

Context: In this paper we present an exploratory study on the insights of organizations into the perceived value of their software projects. Our study is based on the notion that quantifying and qualifying project size, cost, duration and defects needs to be done in relation with stakeholder satisfaction and perceived value. Objectives: We expect that bringing perceived value into the equation will help in increasing the impact such organizations deliver. Method: In order to find out whether our approach is practically feasible in an industrial setting, we performed an exploratory study in a Belgian telecom company. Results: In this study we evaluate 22 software projects that were delivered during one release. Fifty-three (53) key stakeholders provide stakeholder satisfaction and perceived value measurements in 103 completed surveys. Conclusions: We conclude that a focus on shortening overall project duration, and improving communication on intermediate progress improved stakeholder satisfaction and perceived value. Our study does not provide any evidence that steering on costs helped to improve these.

Pricing via Functional Size - A Case Study of a Company's Portfolio of 77 Outsourced Projects

A medium-sized west-European telecom company experienced a worsening trend in performance, indicating that the organization did not learn from history, in combination with much time and energy spent on preparation and review of project proposals. In order to create more transparency in the supplier proposal pro-cess a pilot was started on Functional Size Measurement pricing (FSM-pricing). In this paper we evaluate the implementation of FSM-pricing in the software engineering domain of the company, as an instrument useful in the context of software management and supplier proposal pricing. We analyzed 77 finalized software engineering projects, covering 14 million Euro project cost and a project portfolio size of more than 5,000 function points. We found that a statistical, evidence-based pricing approach for software engineering, as a single instrument (without a connec-tion with expert judgment), can be used in the subject companies to create cost transparency and performance management of software project portfolios.

Measuring Best-in-Class Software Releases

In this research we aimed to identify distinguishing factors in software releases. For this purpose we analyzed the metrics of 26 software projects. These projects were release-based deliveries from two stable, experienced development teams. During the measurement period both teams transformed from a plan-driven delivery model (waterfall) to an agile approach (Scrum). Overall, we observed that these small release-based projects differ largely from non-release-based projects. Our research indicates that a combination of release-based working, a fixed and experienced development team, and a steady heartbeat contribute to performances that can be characterized as best-practice. The main contribution of this paper is that we found five success factors (all reducing development complexity) that result in best-of-class performance for small software releases.

Success factors in managing legacy system evolution: a case study

In this paper, we attempt to understand what contributes to a successful process for managing legacy system evolution. We provide an analysis of a number of key performance indicators such as cost, duration, and defects. By normalizing through function points, we furthermore compare to a larger benchmark. To do so we performed a mixed, retrospective case study on a series of nine software releases and eight single once-only releases, all performing on a single, legacy software system, in a West-European telecom company. We interviewed eleven stakeholders that were closely involved in the subject software releases. As a result, we listed a number of observations from the quantitative and qualitative analysis. We found that a release process that performs above average on cost and duration satisfies stakeholders through fast response and direct value, even when the reliability and availability of the actual system is weak.

The effects of perceived value and stakeholder satisfaction on software project impact

Abstract Context In this paper we present a multiple case study on the insights of software organizations into stakeholder satisfaction and (perceived) value of their software projects. Our study is based on the notion that quantifying and qualifying project size, cost, duration, defects, and estimation accuracy needs to be done in relation with stakeholder satisfaction and perceived value. Objectives We contrast project metrics such as cost, duration, number of defects and estimation accuracy with stakeholder satisfaction and perceived value. Method In order to find out whether our approach is practically feasible in an industrial setting, we performed two case studies; one in a Belgian telecom company and the other in a Dutch software company. Results In this study we evaluate 22 software projects that were delivered during one release in the Belgian telecom company, and 4 additional large software releases (representing an extension of 174% in project size) that were delivered in a Dutch software company. Eighty-three (83) key stakeholders of two companies provide stakeholder satisfaction and perceived value measurements in 133 completed surveys. Conclusions We conclude that a focus on shortening overall project duration, and improving communication and team collaboration on intermediate progress is likely to have a positive impact on stakeholder satisfaction and perceived value. Our study does not provide any evidence that steering on costs helped to improve these. As an answer to our research question - how do stakeholder satisfaction and perceived value relate to cost, duration, defects, size and estimation accuracy of software projects? – we found five take-away-messages.

Evidence-based software portfolio management: a tool description and evaluation

Context: In this paper we describe and evaluate a tool for Evidence-Based Software Portfolio Management (EBSPM) that we developed over time in close cooperation with software practitioners from The Netherlands and Belgium. Objectives: The goal of the EBSPM-tool is to measure, analyze, and benchmark the performance of interconnected sets of software projects in terms of size, cost, duration, and number of defects, in order to support innovation of a companys software delivery capability. The tool supports building and maintaining a research repository of finalized software projects from different companies, business domains, and delivery approaches. Method: The tool consists of two parts. First, a Research Repository, at this moment holding data of for now 490 finalized software projects, from four different companies. Second, a Performance Dashboard, built from a so-called Cost Duration Matrix. Results: We evaluated the tool by describing its use in two practical applications in case studies in industry. Conclusions: We show that the EBSPM-tool can be used successfully in an industrial context, especially regarding its benchmarking and visualization purposes.

A replicated study on correlating agile team velocity measured in function and story points

Since the rapid growth of agile development methods for software engineering, more and more organizations measure the size of iterations, releases, and projects in both function points and story points. In 2011 Santana et al. performed a case study on the relation between function points and story points, from data collected in a Brazilian Government Agency. In this paper we replicate this study, using data collected in a Dutch banking organization. Based on a statistical correlation test we find that a comparison between function points and story points as measured in our repository indicates a moderate negative linear relation, where Santana et al. concluded a strong positive linear relation between both size metrics in their case study. Based on the outcome of our study we conclude that it appears too early to make generic claims on the relation between function points and story points; in fact FSM-theory seems to underpin that such a relationship is a spurious one.

Strong agile metrics: mining log data to determine predictive power of software metrics for continuous delivery teams

ING Bank, a large Netherlands-based internationally operating bank, implemented a fully automated continuous delivery pipe-line for its software engineering activities in more than 300 teams, that perform more than 2500 deployments to production each month on more than 750 different applications. Our objective is to examine how strong metrics for agile (Scrum) DevOps teams can be set in an iterative fashion. We perform an exploratory case study that focuses on the classification based on predictive power of software metrics, in which we analyze log data derived from two initial sources within this pipeline. We analyzed a subset of 16 metrics from 59 squads. We identified two lagging metrics and assessed four leading metrics to be strong.