Carolyn B. Seaman

Qualitative methods in empirical studies of software engineering

While empirical studies in software engineering are beginning to gain recognition in the research community, this subarea is also entering a new level of maturity by beginning to address the human aspects of software development. This added focus has added a new layer of complexity to an already challenging area of research. Along with new research questions, new research methods are needed to study nontechnical aspects of software engineering. In many other disciplines, qualitative research methods have been developed and are commonly used to handle the complexity of issues involving human behaviour. The paper presents several qualitative methods for data collection and analysis and describes them in terms of how they might be incorporated into empirical studies of software engineering, in particular how they might be combined with quantitative methods. To illustrate this use of qualitative methods, examples from real software engineering studies are used throughout.

Investigating and improving a COTS-based software development process

The work described in the paper is an investigation of COTS based software development within a particular NASA environment, with an emphasis on the processes used. Fifteen projects using a COTS based approach were studied and their actual process was documented. This process is evaluated to identify essential differences in comparison to traditional software development. The main differences, and the activities for which projects require more guidance, are requirements definition and COTS selection, high level design, integration and testing. Starting from these empirical observations, a new process and guidelines for COTS based development are developed and briefly presented. The new process is currently under experimentation.

Linking Software Development and Business Strategy Through Measurement

The GQM+Strategies approach extends the goal/question/metric paradigm for measuring the success or failure of goals and strategies, adding enterprise-wide support for determining action on the basis of measurement results. An organization can thus integrate its measurement program across all levels.

COTS-based software development: processes and open issues

The work described in this paper is an investigation of the COTS-based software development within a particular NASA environment, with an emphasis on the processes used. Fifteen projects using a COTS-based approach were studied and their actual process was documented. This process is evaluated to identify essential differences in comparison to traditional software development. The main differences, and the activities for which projects require more guidance, are requirements definition and COTS selection, high level design, integration and testing. Starting from these empirical observations, a new process and set of guidelines for COTS-based development are developed and briefly presented.

Communication and organization: an empirical study of discussion in inspection meetings

This paper describes an empirical study that addresses the issue of communication among members of a software development organization. In particular, data was collected concerning code inspections in one software development project. The question of interest is whether or not organizational structure (the network of relationships between developers) has an effect on the amount of effort expended on communication between developers. The independent variables in this study are various attributes of the organizational structure in which the inspection participants work. The dependent variables are measures of the communication effort expended in various parts of the code inspection process, focusing on the inspection meeting. Both quantitative and qualitative methods were used, including participant observation, structured interviews, generation of hypotheses from field notes, statistical tests of relationships, and interpretation of results with qualitative anecdotes. The study results show that past and present working relationships between inspection participants affect the amount of meeting time spent in different types of discussion, thus affecting the overall inspection meeting length. Reporting relationships and physical proximity also have an effect. The contribution of the study is a set of well-supported hypotheses for further investigation.

Measuring and Monitoring Technical Debt

Abstract Technical debt is a metaphor for immature, incomplete, or inadequate artifacts in the software development lifecycle that cause higher costs and lower quality in the long run. These artifacts remaining in a system affect subsequent development and maintenance activities, and so can be seen as a type of debt that the system developers owe the system. Incurring technical debt may speed up software development in the short run, but such benefit is achieved at the cost of extra work in the future, as if paying interest on the debt. In this sense, the technical debt metaphor characterizes the relationship between the short-term benefits of delaying certain software maintenance tasks or doing them quickly and less carefully, and the long-term cost of those delays. However, managing technical debt is more complicated than managing financial debt because of the uncertainty involved. In this chapter, the authors review the main issues associated with technical debt, and propose a technical debt management framework and a research plan for validation. The objective of our research agenda is to develop and validate a comprehensive technical debt theory that formalizes the relationship between the cost and benefit sides of the concept. Further, we propose to use the theory to propose mechanisms (processes and tools) for measuring and managing technical debt in software product maintenance. The theory and management mechanisms are intended ultimately to contribute to the improved quality of software and facilitate decision making in software maintenance.

Using technical debt data in decision making: potential decision approaches

The management of technical debt ultimately requires decision making - about incurring, paying off, or deferring technical debt instances. This position paper discusses several existing approaches to complex decision making, and suggests that exploring their applicability to technical debt decision making would be a worthwhile subject for further research.

A Balancing Act: What Software Practitioners Have to Say about Technical Debt

An interview study involving 35 practitioners from a variety of domains aimed to characterize technical debt at the ground level to find out how software practitioners perceive it. The study also aimed to understand the context in which technical debt occurs, including its causes, symptoms, and effects. In addition, the study focused on how practitioners currently deal with technical debt. This analysis paints a picture of a large, complex balancing act of various short- and long-term concerns. The Web Extra gives the interview questions used by Erin Lim, Nitin Taksande, and Carolyn Seaman.

Investigating the impact of design debt on software quality

Technical debt is a metaphor describing situations where developers accept sacrifices in one dimension of development (e.g. software quality) in order to optimize another dimension (e.g. implementing necessary features before a deadline). Approaches, such as code smell detection, have been developed to identify particular kinds of debt, e.g. design debt. What has not yet been understood is the impact design debt has on the quality of a software product. Answering this question is important for understanding how growing debt affects a software product and how it slows down development, e.g. though introducing rework such as fixing bugs. In this case study we investigate how design debt, in the form of god classes, affects the maintainability and correctness of software products by studying two sample applications of a small-size software development company. The results show that god classes are changed more often and contain more defects than non-god classes. This result complements findings of earlier research and suggests that technical debt has a negative impact on software quality, and should therefore be identified and managed closely in the development process.

A portfolio approach to technical debt management

Technical debt describes the effect of immature software artifacts on software maintenance - the potential of extra effort required in future as if paying interest for the incurred debt. The uncertainty of interest payment further complicates the problem of what debt should be incurred or repaid and when. To help software managers make informed decisions, a portfolio approach is proposed in this paper. The approach leverages the portfolio management theory in the finance domain to determine the optimal collection of technical debt items that should be incurred or held. We expect this approach could provide a new perspective for technical debt management.