Janice Singer

Selecting Empirical Methods for Software Engineering Research

Selecting a research method for empirical software engineering research is problematic because the benefits and challenges to using each method are not yet well catalogued. Therefore, this chapter describes a number of empirical methods available. It examines the goals of each and analyzes the types of questions each best addresses. Theoretical stances behind the methods, practical considerations in the application of the methods and data collection are also briefly reviewed. Taken together, this information provides a suitable basis for both understand- ing and selecting from the variety of methods applicable to empirical software engineering.

Hipikat: a project memory for software development

Sociological and technical difficulties, such as a lack of informal encounters, can make it difficult for new members of noncollocated software development teams to learn from their more experienced colleagues. To address this situation, we have developed a tool, named Hipikat that provides developers with efficient and effective access to the group memory for a software development project that is implicitly formed by all of the artifacts produced during the development. This project memory is built automatically with little or no change to existing work practices. After describing the Hipikat tool, we present two studies investigating Hipikats usefulness in software modification tasks. One study evaluated the usefulness of Hipikats recommendations on a sample of 20 modification tasks performed on the Eclipse Java IDE during the development of release 2.1 of the Eclipse software. We describe the study, present quantitative measures of Hipikats performance, and describe in detail three cases that illustrate a range of issues that we have identified in the results. In the other study, we evaluated whether software developers who are new to a project can benefit from the artifacts that Hipikat recommends from the project memory. We describe the study, present qualitative observations, and suggest implications of using project memory as a learning aid for project newcomers.

Guide to Advanced Empirical Software Engineering

This book gathers chapters from some of the top international empirical software engineering researchers focusing on the practical knowledge necessary for conducting, reporting and using empirical methods in software engineering. Topics and features include guidance on how to design, conduct and report empirical studies. The volume also provides information across a range of techniques, methods and qualitative and quantitative issues to help build a toolkit applicable to the diverse software development contexts

Studying Software Engineers: Data Collection Techniques for Software Field Studies

Software engineering is an intensively people-oriented activity, yet too little is known about how designers, maintainers, requirements analysts and all other types of software engineers perform their work. In order to improve software engineering tools and practice, it is therefore essential to conduct field studies, i.e. to study real practitioners as they solve real problems. To do so effectively, however, requires an understanding of the techniques most suited to each type of field study task. In this paper, we provide a taxonomy of techniques, focusing on those for data collection. The taxonomy is organized according to the degree of human intervention each requires. For each technique, we provide examples from the literature, an analysis of some of its advantages and disadvantages, and a discussion of how to use it effectively. We also briefly talk about field study design in general, and data analysis.

An examination of software engineering work practices

This paper presents work practice data of the daily activities of software engineers. Four separate studies are presented; one looking longitudinally at an individual SE; two looking at a software engineering group; and one looking at company-wide tool usage statistics. We also discuss the advantages in considering work practices in designing tools for software engineers, and include some requirements for a tool we have developed as a result of our studies.

How software engineers use documentation: the state of the practice

Software engineering is a human task, and as such we must study what software engineers do and think. Understanding the normative practice of software engineering is the first step toward developing realistic solutions to better facilitate the engineering process. We conducted three studies using several data-gathering approaches to elucidate the patterns by which software engineers (SEs) use and update documentation. Our objective is to more accurately comprehend and model documentation use, usefulness, and maintenance, thus enabling better decision making and tool design by developers and project managers. Our results confirm the widely held belief that SEs typically do not update documentation as timely or completely as software process personnel and managers advocate. However, the results also reveal that out-of-date software documentation remains useful in many circumstances.

Towards an Ontology of software maintenance

SUMMARY We suggest that empirical studies of maintenance are difficult to understand unless the context of the study is fully defined. We developed a preliminary ontology to identify a number of factors that influence maintenance. The purpose of the ontology is to identify factors that would affect the results of empirical studies. We present the ontology in the form of a UML model. Using the maintenance factors included in the ontology, we define two common maintenance scenarios and consider the industrial issues associated with them. Copyright

Awareness in the Wild: Why Communication Breakdowns Occur

Global software teams face challenges when collaborating over long distances, such as communicating changes in the project. During a four-month case study at IBM Ottawa Software Lab we observed the collaboration patterns of a multi-site development project team. In this period, we inspected project documentation, interviewed team leaders, attended project meetings, and spoke with developers to identify problems originated by the lack of awareness of changes related to the implementation of work items. Our observations show (1) that organizational culture has an effect on how developers are made aware; (2) that communication-based social networks revolving around particular work items are dynamic throughout development, and therefore awareness needs to be maintained in infrastructures of work; and (3) that information overload and communication breakdowns contributed to the generation of a broken integration build. We discuss these breakdowns in communication and implications for the design of collaboration tools that could mitigate these problems.

How do scientists develop and use scientific software

New knowledge in science and engineering relies increasingly on results produced by scientific software. Therefore, knowing how scientists develop and use software in their research is critical to assessing the necessity for improving current development practices and to making decisions about the future allocation of resources. To that end, this paper presents the results of a survey conducted online in October-December 2008 which received almost 2000 responses. Our main conclusions are that (1) the knowledge required to develop and use scientific software is primarily acquired from peers and through self-study, rather than from formal education and training; (2) the number of scientists using supercomputers is small compared to the number using desktop or intermediate computers; (3) most scientists rely primarily on software with a large user base; (4) while many scientists believe that software testing is important, a smaller number believe they have sufficient understanding about testing concepts; and (5) that there is a tendency for scientists to rank standard software engineering concepts higher if they work in large software development projects and teams, but that there is no uniform trend of association between rank of importance of software engineering concepts and project/team size.

Ethical issues in empirical studies of software engineering

The popularity of empirical methods in software engineering research is on the rise. Surveys, experiments, metrics, case studies, and field studies are examples of empirical methods used to investigate both software engineering processes and products. The increased application of empirical methods has also brought about an increase in discussions about adapting these methods to the peculiarities of software engineering. In contrast, the ethical issues raised by empirical methods have received little, if any, attention in the software engineering literature. This article is intended to introduce the ethical issues raised by empirical research to the software engineering research community and to stimulate discussion of how best to deal with these ethical issues. Through a review of the ethical codes of several fields that commonly employ humans and artifacts as research subjects, we have identified major ethical issues relevant to empirical studies of software engineering. These issues are illustrated with real empirical studies of software engineering.