Robert Feldt

A systematic review of search-based testing for non-functional system properties

Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996-2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques.

Empirical evidence in global software engineering: a systematic review

Recognized as one of the trends of the 21st century, globalization of the world economies brought significant changes to nearly all industries, and in particular it includes software development. Many companies started global software engineering (GSE) to benefit from cheaper, faster and better development of software systems, products and services. However, empirical studies indicate that achieving these benefits is not an easy task. Here, we report our findings from investigating empirical evidence in GSE-related research literature. By conducting a systematic review we observe that the GSE field is still immature. The amount of empirical studies is relatively small. The majority of the studies represent problem-oriented reports focusing on different aspects of GSE management rather than in-depth analysis of solutions for example in terms of useful practices or techniques. Companies are still driven by cost reduction strategies, and at the same time, the most frequently discussed recommendations indicate a necessity of investments in travelling and socialization. Thus, at the same time as development goes global there is an ambition to minimize geographical, temporal and cultural separation. These are normally integral parts of cross-border collaboration. In summary, the systematic review results in several descriptive classifications of the papers on empirical studies in GSE and also reports on some best practices identified from literature.

Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review

BACKGROUND-Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE-This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD-The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies, and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS-Seven distinct evaluation strategies were identified, wherein the most common one, “Pre-Post Comparison,” was applied in 49 percent of the inspected papers. Quality was the most measured attribute (62 percent), followed by Cost (41 percent), and Schedule (18 percent). Looking at measurement perspectives, “Project” represents the majority with 66 percent. CONCLUSION-The evaluation validity of SPI initiatives is challenged by the scarce consideration of potential confounding factors, particularly given that “Pre-Post Comparison” was identified as the most common evaluation strategy, and the inaccurate descriptions of the evaluation context. Measurements to assess the short and mid-term impact of SPI initiatives prevail, whereas long-term measurements in terms of customer satisfaction and return on investment tend to be less used.

A systematic review on strategic release planning models

Context: Strategic release planning (sometimes referred to as road-mapping) is an important phase of the requirements engineering process performed at product level. It is concerned with selection and assignment of requirements in sequences of releases such that important technical and resource constraints are fulfilled. Objectives: In this study we investigate which strategic release planning models have been proposed, their degree of empirical validation, their factors for requirements selection, and whether they are intended for a bespoke or market-driven requirements engineering context. Methods: In this systematic review a number of article sources are used, including Compendex, Inspec, IEEE Xplore, ACM Digital Library, and Springer Link. Studies are selected after reading titles and abstracts to decide whether the articles are peer reviewed, and relevant to the subject. Results: Twenty four strategic release planning models are found and mapped in relation to each other, and a taxonomy of requirements selection factors is constructed. Conclusions: We conclude that many models are related to each other and use similar techniques to address the release planning problem. We also conclude that several requirement selection factors are covered in the different models, but that many methods fail to address factors such as stakeholder value or internal value. Moreover, we conclude that there is a need for further empirical validation of the models in full scale industry trials.

Links between the personalities, views and attitudes of software engineers

Context:: Successful software development and management depends not only on the technologies, methods and processes employed but also on the judgments and decisions of the humans involved. These, in turn, are affected by the basic views and attitudes of the individual engineers. Objective:: The objective of this paper is to establish if these views and attitudes can be linked to the personalities of software engineers. Methods:: We summarize the literature on personality and software engineering and then describe an empirical study on 47 professional engineers in ten different Swedish software development companies. The study evaluated the personalities of these engineers via the IPIP 50-item five-factor personality test and prompted them on their attitudes towards and basic views on their professional activities. Results:: We present extensive statistical analyses of their responses to show that there are multiple, significant associations between personality factors and software engineering attitudes. The tested individuals are more homogeneous in personality than a larger sample of individuals from the general population. Conclusion:: Taken together, the methodology and personality test we propose and the associated statistical analyses can help find and quantify relations between complex factors in software engineering projects in both research and practice.

Towards individualized software engineering: empirical studies should collect psychometrics

Even though software is developed by humans, research in software engineering primarily focuses on the technologies, methods and processes they use while disregarding the importance of the humans themselves. In this paper we argue that most studies in software engineering should give much more weight to human factors. In particular empirical software engineering studies involving human developers should always consider collecting psychometric data on the humans involved. We focus on personality as one important psychometric factor and present initial results from an empirical study investigating correlations between personality and attitudes to software engineering processes and tools. We discuss what are currently hindering a more wide-spread use of psychometrics and how overcoming these hurdles could lead to a more individualized software engineering.

Reporting Empirical Research in Global Software Engineering: A Classification Scheme

Increased popularity of global software engineering (GSE) has resulted in quite a number of research and industrial studies. As the area matures, an increased focus on empirically supported results leads to a greater potential impact on future research and industrial practice. However, since GSE scenarios are diverse, what works in one context might not directly apply in another. Thus it is necessary to understand, how GSE-related empirical findings should be reported to be useful for practitioners and researchers. Furthermore, itdasias important to summarize progress and get the big picture of published research to identify gaps and commonalities. In this paper we analyze differentiating factors of GSE scenarios and offer a classification scheme for describing the context of a GSE study. In addition, we report initial results of a systematic review on GSE-related empirical literature using papers from ICGSE 2006 and 2007, at the same time illustrating and evaluating the proposed scheme.

Challenges and practices in aligning requirements with verification and validation: a case study of six companies

Weak alignment of requirements engineering (RE) with verification and validation (VV) may lead to problems in delivering the required products in time with the right quality. For example, weak communication of requirements changes to testers may result in lack of verification of new requirements and incorrect verification of old invalid requirements, leading to software quality problems, wasted effort and delays. However, despite the serious implications of weak alignment research and practice both tend to focus on one or the other of RE or VV rather than on the alignment of the two. We have performed a multi-unit case study to gain insight into issues around aligning RE and VV by interviewing 30 practitioners from 6 software developing companies, involving 10 researchers in a flexible research process for case studies. The results describe current industry challenges and practices in aligning RE with VV, ranging from quality of the individual RE and VV activities, through tracing and tools, to change control and sharing a common understanding at strategy, goal and design level. The study identified that human aspects are central, i.e. cooperation and communication, and that requirements engineering practices are a critical basis for alignment. Further, the size of an organisation and its motivation for applying alignment practices, e.g. external enforcement of traceability, are variation factors that play a key role in achieving alignment. Our results provide a strategic roadmap for practitioners improvement work to address alignment challenges. Furthermore, the study provides a foundation for continued research to improve the alignment of RE with VV.

Quality Requirements in Industrial Practice—An Extended Interview Study at Eleven Companies

In order to create a successful software product and assure its quality, it is not enough to fulfill the functional requirements, it is also crucial to find the right balance among competing quality requirements (QR). An extended, previously piloted, interview study was performed to identify specific challenges associated with the selection, tradeoff, and management of QR in industrial practice. Data were collected through semistructured interviews with 11 product managers and 11 project leaders from 11 software companies. The contribution of this study is fourfold: First, it compares how QR are handled in two cases, companies working in business-to-business markets and companies that are working in business-to-consumer markets. These two are also compared in terms of impact on the handling of QR. Second, it compares the perceptions and priorities of QR by product and project management, respectively. Third, it includes an examination of the interdependencies among quality requirements perceived as most important by the practitioners. Fourth, it characterizes the selection and management of QR in downstream development activities.

Behavioral software engineering

Throughout the history of software engineering, the human aspects have repeatedly been recognized as important. Even though research that investigates them has been growing in the past decade, these aspects should be more generally considered.The main objective of this study is to clarify the research area concerned with human aspects of software engineering and to create a common platform for future research. In order to meet the objective, we propose a definition of the research area behavioral software engineering (BSE) and present results from a systematic literature review based on the definition.The result indicates that there are knowledge gaps in the research area of behavioral software engineering and that earlier research has been focused on a few concepts, which have been applied to a limited number of software engineering areas. The individual studies have typically had a narrow perspective focusing on few concepts from a single unit of analysis. Further, the research has rarely been conducted in collaboration by researchers from both software engineering and social science.Altogether, this review can help put a broader set of human aspects higher on the agenda for future software engineering research and practice. We present behavioral SE (BSE) and a systematic literature review.Result shows gaps in existing research; it has focused on only a few BSE concepts.BSE studies rarely consider multiple levels and focus on only a few concepts.Even though a majority of BSE research is empirical, few papers are experiments.This paper can help put a broader set of human aspects higher on the agenda.