Tony Gorschek

Empirical evidence in global software engineering: a systematic review

Recognized as one of the trends of the 21st century, globalization of the world economies brought significant changes to nearly all industries, and in particular it includes software development. Many companies started global software engineering (GSE) to benefit from cheaper, faster and better development of software systems, products and services. However, empirical studies indicate that achieving these benefits is not an easy task. Here, we report our findings from investigating empirical evidence in GSE-related research literature. By conducting a systematic review we observe that the GSE field is still immature. The amount of empirical studies is relatively small. The majority of the studies represent problem-oriented reports focusing on different aspects of GSE management rather than in-depth analysis of solutions for example in terms of useful practices or techniques. Companies are still driven by cost reduction strategies, and at the same time, the most frequently discussed recommendations indicate a necessity of investments in travelling and socialization. Thus, at the same time as development goes global there is an ambition to minimize geographical, temporal and cultural separation. These are normally integral parts of cross-border collaboration. In summary, the systematic review results in several descriptive classifications of the papers on empirical studies in GSE and also reports on some best practices identified from literature.

Requirements Abstraction Model

Software requirements arrive in different shapes and forms to development organizations. This is particularly the case in market-driven requirements engineering, where the requirements are on products rather than directed towards projects. This results in challenges related to making different requirements comparable. In particular, this situation was identified in a collaborative effort between academia and industry. A model, with four abstraction levels, was developed as a response to the industrial need. The model allows for placement of requirements on different levels and supports abstraction or break down of requirements to make them comparable to each other. The model was successfully validated in several steps at a company. The results from the industrial validation point to the usefulness of the model. The model will allow companies to ensure comparability between requirements, and hence it generates important input to activities such as prioritization and packaging of requirements before launching a development project.

A Model for Technology Transfer in Practice

Technology transfer, and thus industry-relevant research, involves more than merely producing research results and delivering them in publications and technical reports. It demands close cooperation and collaboration between industry and academia throughout the entire research process. During research conducted in a partnership between Blekinge Institute of Technology and two companies, Danaher Motion Saro AB (DHR) and ABB, we devised a technology transfer model that embodies this philosophy. We initiated this partnership to conduct industry-relevant research in requirements engineering and product management. Technology transfer in this context is a prerequisite: it validates academic research results in a real setting, and it provides a way to improve industry development and business processes

Evaluation and Measurement of Software Process Improvement—A Systematic Literature Review

BACKGROUND-Software Process Improvement (SPI) is a systematic approach to increase the efficiency and effectiveness of a software development organization and to enhance software products. OBJECTIVE-This paper aims to identify and characterize evaluation strategies and measurements used to assess the impact of different SPI initiatives. METHOD-The systematic literature review includes 148 papers published between 1991 and 2008. The selected papers were classified according to SPI initiative, applied evaluation strategies, and measurement perspectives. Potential confounding factors interfering with the evaluation of the improvement effort were assessed. RESULTS-Seven distinct evaluation strategies were identified, wherein the most common one, “Pre-Post Comparison,” was applied in 49 percent of the inspected papers. Quality was the most measured attribute (62 percent), followed by Cost (41 percent), and Schedule (18 percent). Looking at measurement perspectives, “Project” represents the majority with 66 percent. CONCLUSION-The evaluation validity of SPI initiatives is challenged by the scarce consideration of potential confounding factors, particularly given that “Pre-Post Comparison” was identified as the most common evaluation strategy, and the inaccurate descriptions of the evaluation context. Measurements to assess the short and mid-term impact of SPI initiatives prevail, whereas long-term measurements in terms of customer satisfaction and return on investment tend to be less used.

A method for evaluating rigor and industrial relevance of technology evaluations

One of the main goals of an applied research field such as software engineering is the transfer and widespread use of research results in industry. To impact industry, researchers developing technologies in academia need to provide tangible evidence of the advantages of using them. This can be done trough step-wise validation, enabling researchers to gradually test and evaluate technologies to finally try them in real settings with real users and applications. The evidence obtained, together with detailed information on how the validation was conducted, offers rich decision support material for industry practitioners seeking to adopt new technologies and researchers looking for an empirical basis on which to build new or refined technologies. This paper presents model for evaluating the rigor and industrial relevance of technology evaluations in software engineering. The model is applied and validated in a comprehensive systematic literature review of evaluations of requirements engineering technologies published in software engineering journals. The aim is to show the applicability of the model and to characterize how evaluations are carried out and reported to evaluate the state-of-research. The review shows that the model can be applied to characterize evaluations in requirements engineering. The findings from applying the model also show that the majority of technology evaluations in requirements engineering lack both industrial relevance and rigor. In addition, the research field does not show any improvements in terms of industrial relevance over time.

A systematic review on strategic release planning models

Context: Strategic release planning (sometimes referred to as road-mapping) is an important phase of the requirements engineering process performed at product level. It is concerned with selection and assignment of requirements in sequences of releases such that important technical and resource constraints are fulfilled. Objectives: In this study we investigate which strategic release planning models have been proposed, their degree of empirical validation, their factors for requirements selection, and whether they are intended for a bespoke or market-driven requirements engineering context. Methods: In this systematic review a number of article sources are used, including Compendex, Inspec, IEEE Xplore, ACM Digital Library, and Springer Link. Studies are selected after reading titles and abstracts to decide whether the articles are peer reviewed, and relevant to the subject. Results: Twenty four strategic release planning models are found and mapped in relation to each other, and a taxonomy of requirements selection factors is constructed. Conclusions: We conclude that many models are related to each other and use similar techniques to address the release planning problem. We also conclude that several requirement selection factors are covered in the different models, but that many methods fail to address factors such as stakeholder value or internal value. Moreover, we conclude that there is a need for further empirical validation of the models in full scale industry trials.

Software development in startup companies: A systematic mapping study

Context: Software startups are newly created companies with no operating history and fast in producing cutting-edge technologies. These companies develop software under highly uncertain conditions, tackling fast-growing markets under severe lack of resources. Therefore, software startups present a unique combination of characteristics which pose several challenges to software development activities. Objective: This study aims to structure and analyze the literature on software development in startup companies, determining thereby the potential for technology transfer and identifying software development work practices reported by practitioners and researchers. Method: We conducted a systematic mapping study, developing a classification schema, ranking the selected primary studies according their rigor and relevance, and analyzing reported software development work practices in startups. Results: A total of 43 primary studies were identified and mapped, synthesizing the available evidence on software development in startups. Only 16 studies are entirely dedicated to software development in startups, of which 10 result in a weak contribution (advice and implications (6); lesson learned (3); tool (1)). Nineteen studies focus on managerial and organizational factors. Moreover, only 9 studies exhibit high scientific rigor and relevance. From the reviewed primary studies, 213 software engineering work practices were extracted, categorized and analyzed. Conclusion: This mapping study provides the first systematic exploration of the state-of-art on software startup research. The existing body of knowledge is limited to a few high quality studies. Furthermore, the results indicate that software engineering work practices are chosen opportunistically, adapted and configured to provide value under the constrains imposed by the startup context.

Requirements engineering: In search of the dependent variables

When software development teams modify their requirements engineering process as an independent variable, they often examine the implications of these process changes by assessing the quality of the products of the requirements engineering process, e.g., a software requirements specification (SRS). Using the quality of the SRS as the dependent variable is flawed. As an alternative, this paper presents a framework of dependent variables that serves as a full range for requirements engineering quality assessment. In this framework, the quality of the SRS itself is just the first level. Other higher, and more significant levels, include whether the project was successful and whether the resulting product was successful. And still higher levels include whether or not the company was successful and whether there was a positive or negative impact on society as a whole.

Technology transfer decision support in requirements engineering research: a systematic review of REj

One of the main goals of an applied research field such as requirements engineering is the transfer of research results to industrial use. To promote industrial adoption of technologies developed in academia, researchers need to provide tangible evidence of the advantages of using them. This can be done through industry validation, enabling researchers to test and validate technologies in a real setting with real users and applications. The evidence obtained, together with detailed information on how the validation was conducted, offers rich decision support material for industrial practitioners seeking to adopt new technologies. This paper presents a comprehensive systematic literature review of all papers published in the Requirements Engineering journal containing any type of technology evaluation. The aim is to gauge the support for technology transfer, i.e., to what degree industrial practitioners can use the reporting of technology evaluations in the journal as decision support for adopting the technologies in industrial practice. Findings show that very few evaluations offer full technology transfer support, i.e., have a realistic scale, application or subjects. The major improvement potential concerning support for technology transfer is found to be the subjects used in the evaluations. Attaining company support, including support for using practitioners as subjects, is vital for technology transfer and for researchers seeking to validate technologies.

The Success Factors Powering Industry-Academia Collaboration

Collaboration between industry and academia supports improvement and innovation in industry and helps to ensure industrial relevance in academic research. This article presents an exploratory study of the factors for successful collaboration between industry and academia in software research.