Bente Anda

Conducting realistic experiments in software engineering

An important goal of most empirical software engineering research is the transfer of research results to industrial applications. Two important obstacles for this transfer are the lack of control of variables of case studies, i.e., the lack of explanatory power, and the lack of realism of controlled experiments. While it may be difficult to increase the explanatory power of case studies, there is a large potential for increasing the realism of controlled software engineering experiments. To convince industry about the validity and applicability of the experimental results, the tasks, subjects and the environments of the experiments should be as realistic as practically possible. Such experiments are, however, more expensive than experiments involving students, small tasks and pen-and-paper environments. Consequently, a change towards more realistic experiments requires a change in the amount of resources spent on software engineering experiments. This paper argues that software engineering researchers should apply for resources enabling expensive and realistic software engineering experiments similar to how other researchers apply for resources for expensive software and hardware that are necessary for their research. The paper describes experiences from recent experiments that varied in size from involving one software professional for 5 days to 130 software professionals, from 9 consultancy companies, for one day each.

Experiences from conducting semi-structured interviews in empirical software engineering research

Many phenomena related to software development are qualitative in nature. Relevant measures of such phenomena are often collected using semi-structured interviews. Such interviews involve high costs, and the quality of the collected data is related to how the interviews are conducted. Careful planning and conducting of the interviews are therefore necessary, and experiences from interview studies in software engineering should consequently be collected and analyzed to provide advice to other researchers. We have brought together experiences from 12 software engineering studies, in which a total of 280 interviews were conducted. Four areas were particularly challenging when planning and conducting these interviews; estimating the necessary effort, ensuring that the interviewer had the needed skills, ensuring good interaction between interviewer and interviewees, and using the appropriate tools and project artifacts. The paper gives advice on how to handle these areas and suggests what information about the interviews should be included when reporting studies where interviews have been used in data collection. Knowledge from other disciplines is included. By sharing experience, knowledge about the accomplishments of software engineering interviews is increased and hence, measures of high quality can be achieved

Quantifying the Effect of Code Smells on Maintenance Effort

Context: Code smells are assumed to indicate bad design that leads to less maintainable code. However, this assumption has not been investigated in controlled studies with professional software developers. Aim: This paper investigates the relationship between code smells and maintenance effort. Method: Six developers were hired to perform three maintenance tasks each on four functionally equivalent Java systems originally implemented by different companies. Each developer spent three to four weeks. In total, they modified 298 Java files in the four systems. An Eclipse IDE plug-in measured the exact amount of time a developer spent maintaining each file. Regression analysis was used to explain the effort using file properties, including the number of smells. Result: None of the 12 investigated smells was significantly associated with increased effort after we adjusted for file size and the number of changes; Refused Bequest was significantly associated with decreased effort. File size and the number of changes explained almost all of the modeled variation in effort. Conclusion: The effects of the 12 smells on maintenance effort were limited. To reduce maintenance effort, a focus on reducing code size and the work practices that limit the number of changes may be more beneficial than refactoring code smells.

Building Theories in Software Engineering

In mature sciences, building theories is the principal method of acquir- ing and accumulating knowledge that may be used in a wide range of settings. In software engineering, there is relatively little focus on theories. In particular, there is little use and development of empirically-based theories. We propose, and illustrate with examples, an initial framework for describing software engineering theories, and give advice on how to start proposing, testing, modifying and using theories to support both research and practise in software engineering.

Estimating Software Development Effort Based on Use Cases-Experiences from Industry

Use case models are used in object-oriented analysis for capturing and describing the functional requirements of a system. Several methods for estimating software development effort are based on attributes of a use case model. This paper reports the results of three industrial case studies on the application of a method for effort estimation based on use case points. The aim of this paper is to provide guidance for other organizations that want to improve their estimation process applying use cases. Our results support existing claims that use cases can be used successfully in estimating software development effort. The results indicate that the guidance provided by the use case points method can support expert knowledge in the estimation process. Our experience is also that the design of the use case models has a strong impact on the estimates.

Effort estimation of use cases for incremental large-scale software development

This paper describes an industrial study of an effort estimation method based on use cases, the use case points method. The original method was adapted to incremental development and evaluated on a large industrial system with modification of software from the previous release. We modified the following elements of the original method: a) complexity assessment of actors and use cases, and b) the handling of non-functional requirements and team factors that may affect effort. For incremental development, we added two elements to the method: c) counting both all and the modified actors and transactions of use cases, and d) effort estimation for secondary changes of software not reflected in use cases. We finally extended the method to: e) cover all development effort in a very large project. The method was calibrated using data from one release and it produced an estimate for the successive release that was only 17% lower than the actual effort. The study identified factors affecting effort on large projects with incremental development. It also showed how these factors can be calibrated for a specific context and produce relatively accurate estimates.

Experiences from introducing UML-based development in a large safety-critical project

UML and UML-based development methods have become de facto standards in industry, and there are many claims for the positive effects of modelling object-oriented systems using methods based on UML. However, there is no reported empirical evaluation of UML-based development in large, industrial projects. This paper reports a case study in ABB, a global company with 120,000 employees, conducted to identify immediate benefits as well as difficulties and their causes when introducing UML-based development in large projects. ABB decided to use UML-based development in the company’s system development projects as part of an effort to enable certification according to the IEC 61508 safety standard. A UML-based development method was first applied in a large, international project with 230 system developers, testers and managers. The goal of the project was to build a new version of a safety-critical process control system. Most of the software was embedded. The project members were mostly newcomers to the use of UML. Interviews with 16 system developers and project managers at their sites in Sweden and Norway were conducted to identify the extent to which the introduction of UML-based development had improved their development process. The interviewees had experienced improvements with traceability from requirements to code, design of the code, and development of test cases as well as in communication and documentation. These results thus support claims in the literature regarding improvements that may be obtained through the use of UML. However, the results also show that the positive effects of UML-based development were reduced due to (1) legacy code that it was not feasible to reverse engineer into UML, (2) the distribution of requirements to development teams based on physical units and not on functionality, (3) training that was not particularly adapted to this project and considered too expensive to give to project members not directly involved in development with UML, and (4) a choice of modelling tools with functionality that was not in accordance with the needs of the project. The results from this study should be useful in enabling other UML adopters to have more realistic expectations and a better basis for making project management decisions.

Towards an inspection technique for use case models

A use case model describes the functional requirements of a software system and is used as input to several activities in a software development project. The quality of the use case model therefore has an important impact on the quality of the resulting software product. Software inspection is regarded as one of the most efficient methods for verifying software documents. There are inspection techniques for most documents produced in a software development project, but no comprehensive inspection technique exists for use case models. This paper presents a taxonomy of typical defects in use case models and proposes a checklist-based inspection technique for detecting such defects. This inspection technique was evaluated in two studies with undergraduate students as subjects. The results from the evaluations indicate that inspections are useful for detecting defects in use case models and motivate further studies to improve the proposed inspection technique.

Quality and Understandability of Use Case Models

Use case models are used in object-oriented analysis for capturing and describing the functional requirements of a system. Use case models are also used in communication between stakeholders in development projects. It is therefore important that the use case models are constructed in such a way that they support the development process and promote a good understanding of the requirements among the stakeholders. Despite this, there are few guidelines on how to construct use case models.This paper describes an explorative study where three different sets of guidelines were used for constructing and documenting use case models. An experiment with 139 undergraduate students divided into 31 groups was conducted. Each group used one out of the three sets of guidelines when constructing a use case model from an informal requirements specification. After completing the use case model, each student answered a questionnaire.The results of the experiment indicate that guidelines based on templates support the construction of use case models that are easier to understand for the readers, than guidelines without specific details on how to document each use case. The guidelines based on templates were also considered as the most useful when constructing use cases. In addition to better understandability, our experiment indicates that the guidelines based on templates result in better use case models regarding also other quality attributes. Our results further indicate that it may be beneficial to combine the template guidelines with another set of guidelines that focus on the documentation of the flow of events of each use case.

Assessing Software System Maintainability using Structural Measures and Expert Assessments

Software maintenance is often expensive; hence, strategies for assessing the maintainability of complete software systems are important. Nevertheless, a software client usually has few means of assessing the maintainability of a software system as part of the acquisition process. Assessing the maintainability of complete systems is difficult due to the influence of many factors, such as the people, tasks and tools, in addition to the code. Furthermore, most research on maintainability focuses on individual classes of individual systems. This paper describes an empirical study in which the maintainability of four functionally equivalent systems developed in Java was assessed using both structural measures and expert assessments. The results suggest that such a combination may be useful. Although the assessment based on structural measures mostly corresponded with the expert assessments, there were several examples of potential problems regarding maintainability that were not captured by the structural measures.