Martin Höst

Guidelines for conducting and reporting case study research in software engineering

Case study is a suitable research methodology for software engineering research since it studies contemporary phenomena in its natural context. However, the understanding of what constitutes a case study varies, and hence the quality of the resulting studies. This paper aims at providing an introduction to case study methodology and guidelines for researchers conducting case studies and readers studying reports of such studies. The content is based on the authors’ own experience from conducting and reading case studies. The terminology and guidelines are compiled from different methodology handbooks in other research domains, in particular social science and information systems, and adapted to the needs in software engineering. We present recommended practices for software engineering case studies as well as empirically derived and evaluated checklists for researchers and readers of case study research.

Using Students as Subjects—A Comparative Study ofStudents and Professionals in Lead-Time Impact Assessment

In many studiesin software engineering students are used instead of professionalsoftware developers, although the objective is to draw conclusionsvalid for professional software developers. This paper presentsa study where the difference between the two groups is evaluated.People from the two groups have individually carried out a non-trivialsoftware engineering judgement task involving the assessmentof how ten different factors affect the lead-time of softwaredevelopment projects. It is found that the differences are onlyminor, and it is concluded that software engineering studentsmay be used instead of professional software developers undercertain conditions. These conditions are identified and describedbased on generally accepted criteria for validity evaluationof empirical studies.

Case Study Research in Software Engineering: Guidelines and Examples

Based on their own experiences of in-depth case studies of software projects in international corporations, in this bookthe authors present detailed practical guidelines on the preparation, conduct, design and reporting of case studies of software engineering. This is the first software engineering specific book on thecase study research method.

An industrial case study on distributed prioritisation in market-driven requirements engineering for packaged software

When developing packaged software, which is sold ‘off-the-shelf’ on a worldwide marketplace, it is essential to collect needs and opportunities from different market segments and use this information in the prioritisation of requirements for the next software release. This paper presents an industrial case study where a distributed prioritisation process is proposed, observed and evaluated. The stakeholders in the requirements prioritisation process include marketing offices distributed around the world. A major objective of the distributed prioritisation is to gather and highlight the differences and similarities in the requirement priorities of the different market segments. The evaluation through questionnaires shows that the stakeholders found the process useful. The paper also presents novel approaches to visualise the priority distribution among stakeholders, together with measures on disagreement and satisfaction. Product management found the proposed charts valuable as decision support when selecting requirements for the next release, as they revealed unforeseen differences among stakeholder priorities. Conclusions on stakeholder tactics are provided and issues of further research are identified, including ways of addressing identified challenges.

Empirical research methods in software engineering

Software engineering is not only about technical solutions. It is to a large extent also concerned with organizational issues, project management and human behaviour. For a discipline like software engineering, empirical methods are crucial, since they allow for incorporating human behaviour into the research approach taken. Empirical methods are common practice in many other disciplines. This chapter provides a motivation for the use of empirical methods in software engineering research. The main motivation is that it is needed from an engineering perspective to allow for informed and well-grounded decision. The chapter continues with a brief introduction to four research methods: controlled experiments, case studies, surveys and post-mortem analyses. These methods are then put into an improvement context. The four methods are presented with the objective to introduce the reader to the methods to a level that it is possible to select the most suitable method at a specific instance. The methods have in common that they all are concerned with quantitative data. However, several of them are also suitable for qualitative data. Finally, it is concluded that the methods are not competing. On the contrary, the different research methods can preferably be used together to obtain more sources of information that hopefully lead to more informed engineering decisions in software engineering.

Introducing an agile process in a software maintenance and evolution organization

The purpose of this paper is to present results of introducing an agile process based on extreme programming, XP, in an evolutionary and maintenance software development environment. The agile process was introduced to a large software development organization. The process was applied by a team during eight months. The conclusions indicate that it in this case is more difficult to introduce XP, in its original appearance, to the case environment than to less complex environments. The complexity of the organization made it necessary to redesign many of the practices in order for them to fit the needs of the software development team.

Exploring Bottlenecks in Market-Driven Requirements Management Processes with Discrete Event Simulation

This paper presents a study where a market-driven requirements management process is simulated. In market-driven software development, generic software packages are released to a market with many customers. New requirements are continuously issued, and the objective of the requirements management process is to elicit, manage, and prioritize the requirements. In the presented study, a specific requirements management process is modelled using discrete event simulation, and the parameters of the model are estimated based on interviews with people from the specific organisation where the process is used. Based on the results from simulations, conditions that result in an overload situation are identified. Simulations are also used to find process change proposals that can result in a non-overloaded process. The risk of overload can be avoided if the capacity of the requirements management process is increased, or if the number of incoming requirements is decreased, for example, through early rejection of low-priority requirements.

Checklists for Software Engineering Case Study Research

Case study is an important research methodology for software engineering. We have identified the need for checklists supporting researchers and reviewers in conducting and reviewing case studies. We derived checklists for researchers and reviewers respectively, using systematic qualitative procedures. Based on nine sources on case studies, checklists are derived and validated, and hereby presented for further use and improvement.

An experimental study of individual subjective effort estimations and combinations of the estimates

The required effort of a task can be estimated subjectively in interviews with experts in an organization in different ways. Interview techniques dealing with which type of questions to ask are evaluated and techniques for combining estimates from individuals into one estimate are compared in an experiment. The result shows that the interview technique is not as important as the combination technique. The estimate which is best with respect to mean value and standard deviation of the effort is based on an equal weighting of all individual estimates. The experiment is performed within the Personal Software Process (PSP).

Experimental context classification: incentives and experience of subjects

There is a need to identify factors that affect the result of empirical studies in software engineering research. It is still the case that seemingly identical replications of controlled experiments result in different conclusions due to the fact that all factors describing the experiment context are not clearly defined and hence controlled. In this article, a scheme for describing the participants of controlled experiments is proposed and evaluated. It consists of two main factors, the incentives for participants in the experiment and the experience of the participants. The scheme has been evaluated by classifying a set of previously conducted experiments from literature. It can be concluded that the scheme was easy to use and understand. It is also found that experiments that are classified in the same way to a large extent point at the same results, which indicates that the scheme addresses relevant factors.