Natalia Juristo

Basics of Software Engineering Experimentation

Basics of Software Engineering Experimentation is a practical guide to experimentation in a field which has long been underpinned by suppositions, assumptions, speculations and beliefs. It demonstrates to software engineers how Experimental Design and Analysis can be used to validate their beliefs and ideas. The book does not assume its readers have an in-depth knowledge of mathematics, specifying the conceptual essence of the techniques to use in the design and analysis of experiments and keeping the mathematical calculations clear and simple. Basics of Software Engineering Experimentation is practically oriented and is specially written for software engineers, all the examples being based on real and fictitious software engineering experiments.

Effectiveness of Requirements Elicitation Techniques: Empirical Results Derived from a Systematic Review

This paper reports a systematic review of empirical studies concerning the effectiveness of elicitation techniques, and the subsequent aggregation of empirical evidence gathered from those studies. The most significant results of the aggregation process are as follows: (I) interviews, preferentially structured, appear to be one of the most effective elicitation techniques; (2) many techniques often cited in the literature, like card sorting, ranking or thinking aloud, tend to be less effective than interviews; (3) analyst experience does not appear to be a relevant factor; and (4) the studies conducted have not found the use of intermediate representations during elicitation to have significant positive effects. It should be noted that, as a general rule, the studies from which these results were aggregated have not been replicated, and therefore the above claims cannot be said to be absolutely certain. However, they can be used by researchers as pieces of knowledge to be further investigated and by practitioners in development projects, always taking into account that they are preliminary findings

Reviewing 25 Years of Testing Technique Experiments

Mature knowledge allows engineering disciplines the achievement of predictable results. Unfortunately, the type of knowledge used in software engineering can be considered to be of a relatively low maturity, and developers are guided by intuition, fashion or market-speak rather than by facts or undisputed statements proper to an engineering discipline. Testing techniques determine different criteria for selecting the test cases that will be used as input to the system under examination, which means that an effective and efficient selection of test cases conditions the success of the tests. The knowledge for selecting testing techniques should come from studies that empirically justify the benefits and application conditions of the different techniques. This paper analyzes the maturity level of the knowledge about testing techniques by examining existing empirical studies about these techniques. We have analyzed their results, and obtained a testing technique knowledge classification based on their factuality and objectivity, according to four parameters.

The role of replications in Empirical Software Engineering

Replications play a key role in Empirical Software Engineering by allowing the community to build knowledge about which results or observations hold under which conditions. Therefore, not only can a replication that produces similar results as the original experiment be viewed as successful, but a replication that produce results different from those of the original experiment can also be viewed as successful. In this paper we identify two types of replications: exact replications, in which the procedures of an experiment are followed as closely as possible; and conceptual replications, in which the same research question is evaluated by using a different experimental procedure. The focus of this paper is on exact replications. We further explore them to identify two sub-categories: dependent replications, where researchers attempt to keep all the conditions of the experiment the same or very similar and independent replications, where researchers deliberately vary one or more major aspects of the conditions of the experiment. We then discuss the role played by each type of replication in terms of its goals, benefits, and limitations. Finally, we highlight the importance of producing adequate documentation for an experiment (original or replication) to allow for replication. A properly documented replication provides the details necessary to gain a sufficient understanding of the study being replicated without requiring the replicator to slavishly follow the given procedures.

How do personality, team processes and task characteristics relate to job satisfaction and software quality?

This article analyses the relationships between personality, team processes, task characteristics, product quality and satisfaction in software development teams. The data analysed here were gathered from a sample of 35 teams of students (105 participants). These teams applied an adaptation of an agile methodology, eXtreme Programming (XP), to develop a software product. We found that the teams with the highest job satisfaction are precisely the ones whose members score highest for the personality factors agreeableness and conscientiousness. The satisfaction levels are also higher when the members can decide how to develop and organize their work. On the other hand, the level of satisfaction and cohesion drops the more conflict there is between the team members. Finally, the teams exhibit a significant positive correlation between the personality factor extraversion and software product quality.

Guidelines for Eliciting Usability Functionalities

Like any other quality attribute, usability imposes specific constraints on software components. Features that raise the software systems usability have to be considered from the earliest development stages. But, discovering and documenting usability features is likely to be beyond the usability knowledge of most requirements engineers, developers, and users. We propose an approach based on developing specific guidelines that capitalize upon key elements recurrently intervening in the usability features elicitation and specification process. The use of these guidelines provides requirements analysts with a knowledge repository. They can use this repository to ask the right questions and capture precise usability requirements information.

Developing search strategies for detecting relevant experiments

Our goal is to analyze the optimality of search strategies for use in systematic reviews of software engineering experiments. Studies retrieval is an important problem in any evidence-based discipline. This question has not been examined for evidence-based software engineering as yet. We have run several searches exercising different terms denoting experiments to evaluate their recall and precision. Based on our evaluation, we propose using a high recall strategy when there are plenty of resources or the results need to be exhaustive. For any other case, we propose optimal, or even acceptable, search strategies. As a secondary goal, we have analysed trends and weaknesses in terminology used in articles reporting software engineering experiments. We have found that it is impossible for a search strategy to retrieve 100% of the experiments of interest (as happens in other experimental disciplines), because of the shortage of reporting standards in the community.

Assigning people to roles in software projects

This paper is based on the premise that peoples behavioural competencies or characteristics of professional conduct influence the effectiveness and efficiency with which they perform a predetermined role in the software process. We propose a capabilities‐oriented process model that includes traditional elements of the software process (activities, products, techniques, people and roles) and the original element of this paper (capabilities). With the aim of adding behavioural competencies to the process model, we define the capability–person and capability–role relationships involved in software development. Additionally, we propose two procedures that are based on each of these relationships: a procedure that can be used to determine the capabilities of the members of a development team; and a procedure that can be used to assign people to perform roles depending on their capabilities and the capabilities demanded by the roles. Finally, the person–capabilities–role relationship has been empirically validated. The results yielded by this experiment confirm the hypothesis that assigning people to roles according to their capabilities and the capabilities demanded by the role improves software development. Copyright

Systematic review and aggregation of empirical studies on elicitation techniques

We have located the results of empirical studies on elicitation techniques and aggregated these results to gather empirically grounded evidence. Our chosen surveying methodology was systematic review, whereas we used an adaptation of comparative analysis for aggregation because meta-analysis techniques could not be applied. The review identified 564 publications from the SCOPUS, IEEEXPLORE, and ACM DL databases, as well as Google. We selected and extracted data from 26 of those publications. The selected publications contain 30 empirical studies. These studies were designed to test 43 elicitation techniques and 50 different response variables. We got 100 separate results from the experiments. The aggregation generated 17 pieces of knowledge about the interviewing, laddering, sorting, and protocol analysis elicitation techniques. We provide a set of guidelines based on the gathered pieces of knowledge.

Is the European Industry Moving toward Solving Requirements Engineering Problems

This article gives an informal overview of the situation of European companies regarding requirements engineering. It reviews what progress the industry has made with respect to the requirements-related problems identified years ago and calls attention to the gap between RE practice and the solutions provided by research. The authors aim to determine whether the same problems still exist today, even though potential solutions have been provided in the literature for some time.