Magne Jørgensen

A Systematic Review of Software Development Cost Estimation Studies

This paper aims to provide a basis for the improvement of software-estimation research through a systematic review of previous work. The review identifies 304 software cost estimation papers in 76 journals and classifies the papers according to research topic, estimation approach, research approach, study context and data set. A Web-based library of these cost estimation papers is provided to ease the identification of relevant estimation research results. The review results combined with other knowledge provide support for recommendations for future software cost estimation research, including: 1) increase the breadth of the search for relevant studies, 2) search manually for relevant papers within a carefully selected set of journals when completeness is essential, 3) conduct more studies on estimation methods commonly used by the software industry, and 4) increase the awareness of how properties of the data sets impact the results when evaluating estimation methods

Evidence-based software engineering

Our objective is to describe how software engineering might benefit from an evidence-based approach and to identify the potential difficulties associated with the approach. We compared the organisation and technical infrastructure supporting evidence-based medicine (EBM) with the situation in software engineering. We considered the impact that factors peculiar to software engineering (i.e. the skill factor and the lifecycle factor) would have on our ability to practice evidence-based software engineering (EBSE). EBSE promises a number of benefits by encouraging integration of research results with a view to supporting the needs of many different stakeholder groups. However, we do not currently have the infrastructure needed for widespread adoption of EBSE. The skill factor means software engineering experiments are vulnerable to subject and experimenter bias. The lifecycle factor means it is difficult to determine how technologies will behave once deployed. Software engineering would benefit from adopting what it can of the evidence approach provided that it deals with the specific problems that arise from the nature of software engineering.

The Future of Empirical Methods in Software Engineering Research

We present the vision that for all fields of software engineering (SE), empirical research methods should enable the development of scientific knowledge about how useful different SE technologies are for different kinds of actors, performing different kinds of activities, on different kinds of systems. It is part of the vision that such scientific knowledge will guide the development of new SE technology and is a major input to important SE decisions in industry. Major challenges to the pursuit of this vision are: more SE research should be based on the use of empirical methods; the quality, including relevance, of the studies using such methods should be increased; there should be more and better synthesis of empirical evidence; and more theories should be built and tested. Means to meet these challenges include (1) increased competence regarding how to apply and combine alternative empirical methods, (2) tighter links between academia and industry, (3) the development of common research agendas with a focus on empirical methods, and (4) more resources for empirical research.

Experience with the accuracy of software maintenance task effort prediction models

The paper reports experience from the development and use of eleven different software maintenance effort prediction models. The models were developed applying regression analysis, neural networks and pattern recognition and the prediction accuracy was measured and compared for each model type. The most accurate predictions were achieved applying models based on multiple regression and on pattern recognition. We suggest the use of prediction models as instruments to support the expert estimates and to analyse the impact of the maintenance variables on the maintenance process and product. We believe that the pattern recognition based models evaluated, i.e., the prediction models based on the Optimized Set Reduction method, show potential for such use. >

Conducting realistic experiments in software engineering

An important goal of most empirical software engineering research is the transfer of research results to industrial applications. Two important obstacles for this transfer are the lack of control of variables of case studies, i.e., the lack of explanatory power, and the lack of realism of controlled experiments. While it may be difficult to increase the explanatory power of case studies, there is a large potential for increasing the realism of controlled software engineering experiments. To convince industry about the validity and applicability of the experimental results, the tasks, subjects and the environments of the experiments should be as realistic as practically possible. Such experiments are, however, more expensive than experiments involving students, small tasks and pen-and-paper environments. Consequently, a change towards more realistic experiments requires a change in the amount of resources spent on software engineering experiments. This paper argues that software engineering researchers should apply for resources enabling expensive and realistic software engineering experiments similar to how other researchers apply for resources for expensive software and hardware that are necessary for their research. The paper describes experiences from recent experiments that varied in size from involving one software professional for 5 days to 130 software professionals, from 9 consultancy companies, for one day each.

Estimating Software Development Effort Based on Use Cases-Experiences from Industry

Use case models are used in object-oriented analysis for capturing and describing the functional requirements of a system. Several methods for estimating software development effort are based on attributes of a use case model. This paper reports the results of three industrial case studies on the application of a method for effort estimation based on use case points. The aim of this paper is to provide guidance for other organizations that want to improve their estimation process applying use cases. Our results support existing claims that use cases can be used successfully in estimating software development effort. The results indicate that the guidance provided by the use case points method can support expert knowledge in the estimation process. Our experience is also that the design of the use case models has a strong impact on the estimates.

The impact of customer expectation on software development effort estimates

Abstract The results from the study described in this paper suggest that customer expectations of a projects total cost can have a very large impact on human judgment-based estimates (expert estimates) of the most likely use of software development effort. The information that the customer expectations did not constitute valid input data for making estimates did not remove the impact. Surprisingly, the estimators did not notice this impact or assessed it to be low. An implication of the results is that the provision of a realistic project estimate of most likely use of effort may require that the estimators do not know the customers expectations of the total cost of the project.

A comparison of software project overruns - flexible versus sequential development models

Flexible software development models, e.g., evolutionary and incremental models, have become increasingly popular. Advocates claim that among the benefits of using these models is reduced overruns, which is one of the main challenges of software project management. This paper describes an in-depth survey of software development projects. The results support the claim that projects which employ a flexible development model experience less effort overruns than do those which employ a sequential model. The reason for the difference is not obvious. We found, for example, no variation in project size, estimation process, or delivered proportion of planned functionality between projects applying different types of development model. When the managers were asked to provide reasons for software overruns and/or estimation accuracy, the largest difference was that more of flexible projects than sequential projects cited good requirement specifications-and good collaboration/communication with clients as contributing to accurate estimates.

Impact of effort estimates on software project work

Abstract This paper presents results from two case studies and two experiments on how effort estimates impact software project work. The studies indicate that a meaningful interpretation of effort estimation accuracy requires knowledge about the size and nature of the impact of the effort estimates on the software work. For example, we found that projects with high priority on costs and incomplete requirements specifications were prone to adjust the work to fit the estimate when the estimates were too optimistic, while too optimistic estimates led to effort overruns for projects with high priority on quality and well specified requirements. Two hypotheses were derived from the case studies and tested experimentally. The experiments indicate that: (1) effort estimates can be strongly impacted by anchor values, e.g. early indications on the required effort. This impact is present even when the estimators are told that the anchor values are irrelevant as estimation information; (2) too optimistic effort estimates lead to less use of effort and more errors compared with more realistic effort estimates on programming tasks.

Top-down and bottom-up expert estimation of software development effort

Abstract Expert estimation of software development effort may follow top-down or bottom-up strategies, i.e. the total effort estimate may be based on properties of the project as a whole and distributed over project activities (top-down) or calculated as the sum of the project activity estimates (bottom-up). The explorative study reported in this paper examines differences between these two strategies based on measurement and video recording of the discussions of seven estimation teams. Each estimation team applied a top-down estimation strategy on one project and a bottom-up estimation strategy on another. The main contribution of the study is the observation that the recall of very similar previously completed projects seemed to be a pre-condition for accurate top-down strategy based estimates, i.e. the abilities of the software estimators to transfer estimation experience from less similar projects was poor. This suggests that software companies should apply the bottom-up strategy unless the estimators have experience from, or access to, very similar projects.