Richard Torkar

A systematic review of search-based testing for non-functional system properties

Search-based software testing is the application of metaheuristic search techniques to generate software tests. The test adequacy criterion is transformed into a fitness function and a set of solutions in the search space are evaluated with respect to the fitness function using a metaheuristic search technique. The application of metaheuristic search techniques for testing is promising due to the fact that exhaustive testing is infeasible considering the size and complexity of software under test. Search-based software testing has been applied across the spectrum of test case design methods; this includes white-box (structural), black-box (functional) and grey-box (combination of structural and functional) testing. In addition, metaheuristic search techniques have also been applied to test non-functional properties. The overall objective of undertaking this systematic review is to examine existing work into non-functional search-based software testing (NFSBST). We are interested in types of non-functional testing targeted using metaheuristic search techniques, different fitness functions used in different types of search-based non-functional testing and challenges in the application of these techniques. The systematic review is based on a comprehensive set of 35 articles obtained after a multi-stage selection process and have been published in the time span 1996-2007. The results of the review show that metaheuristic search techniques have been applied for non-functional testing of execution time, quality of service, security, usability and safety. A variety of metaheuristic search techniques are found to be applicable for non-functional testing including simulated annealing, tabu search, genetic algorithms, ant colony methods, grammatical evolution, genetic programming (and its variants including linear genetic programming) and swarm intelligence methods. The review reports on different fitness functions used to guide the search for each of the categories of execution time, safety, usability, quality of service and security; along with a discussion of possible challenges in the application of metaheuristic search techniques.

A systematic review on strategic release planning models

Context: Strategic release planning (sometimes referred to as road-mapping) is an important phase of the requirements engineering process performed at product level. It is concerned with selection and assignment of requirements in sequences of releases such that important technical and resource constraints are fulfilled. Objectives: In this study we investigate which strategic release planning models have been proposed, their degree of empirical validation, their factors for requirements selection, and whether they are intended for a bespoke or market-driven requirements engineering context. Methods: In this systematic review a number of article sources are used, including Compendex, Inspec, IEEE Xplore, ACM Digital Library, and Springer Link. Studies are selected after reading titles and abstracts to decide whether the articles are peer reviewed, and relevant to the subject. Results: Twenty four strategic release planning models are found and mapped in relation to each other, and a taxonomy of requirements selection factors is constructed. Conclusions: We conclude that many models are related to each other and use similar techniques to address the release planning problem. We also conclude that several requirement selection factors are covered in the different models, but that many methods fail to address factors such as stakeholder value or internal value. Moreover, we conclude that there is a need for further empirical validation of the models in full scale industry trials.

Towards innovation measurement in the software industry

In todays highly competitive business environments with shortened product and technology life cycle, it is critical for software industry to continuously innovate. This goal can be achieved by developing a better understanding and control of the activities and determinants of innovation. Innovation measurement initiatives assess innovation capability, output and performance to help develop such an understanding. This study explores various aspects relevant to innovation measurement ranging from definitions, measurement frameworks and metrics that have been proposed in literature and used in practice. A systematic literature review followed by an online questionnaire and interviews with practitioners and academics were employed to identify a comprehensive definition of innovation that can be used in software industry. The metrics for the evaluation of determinants, inputs, outputs and performance were also aggregated and categorised. Based on these findings, a conceptual model of the key measurable elements of innovation was constructed from the findings of the systematic review. The model was further refined after feedback from academia and industry through interviews.

Links between the personalities, views and attitudes of software engineers

Context:: Successful software development and management depends not only on the technologies, methods and processes employed but also on the judgments and decisions of the humans involved. These, in turn, are affected by the basic views and attitudes of the individual engineers. Objective:: The objective of this paper is to establish if these views and attitudes can be linked to the personalities of software engineers. Methods:: We summarize the literature on personality and software engineering and then describe an empirical study on 47 professional engineers in ten different Swedish software development companies. The study evaluated the personalities of these engineers via the IPIP 50-item five-factor personality test and prompted them on their attitudes towards and basic views on their professional activities. Results:: We present extensive statistical analyses of their responses to show that there are multiple, significant associations between personality factors and software engineering attitudes. The tested individuals are more homogeneous in personality than a larger sample of individuals from the general population. Conclusion:: Taken together, the methodology and personality test we propose and the associated statistical analyses can help find and quantify relations between complex factors in software engineering projects in both research and practice.

Overcoming the Equivalent Mutant Problem: A Systematic Literature Review and a Comparative Experiment of Second Order Mutation

Context. The equivalent mutant problem (EMP) is one of the crucial problems in mutation testing widely studied over decades. Objectives. The objectives are: to present a systematic literature review (SLR) in the field of EMP; to identify, classify and improve the existing, or implement new, methods which try to overcome EMP and evaluate them. Method. We performed SLR based on the search of digital libraries. We implemented four second order mutation (SOM) strategies, in addition to first order mutation (FOM), and compared them from different perspectives. Results. Our SLR identified 17 relevant techniques (in 22 articles) and three categories of techniques: detecting (DEM); suggesting (SEM); and avoiding equivalent mutant generation (AEMG). The experiment indicated that SOM in general and JudyDiffOp strategy in particular provide the best results in the following areas: total number of mutants generated; the association between the type of mutation strategy and whether the generated mutants were equivalent or not; the number of not killed mutants; mutation testing time; time needed for manual classification. Conclusions . The results in the DEM category are still far from perfect. Thus, the SEM and AEMG categories have been developed. The JudyDiffOp algorithm achieved good results in many areas.

Towards individualized software engineering: empirical studies should collect psychometrics

Even though software is developed by humans, research in software engineering primarily focuses on the technologies, methods and processes they use while disregarding the importance of the humans themselves. In this paper we argue that most studies in software engineering should give much more weight to human factors. In particular empirical software engineering studies involving human developers should always consider collecting psychometric data on the humans involved. We focus on personality as one important psychometric factor and present initial results from an empirical study investigating correlations between personality and attitudes to software engineering processes and tools. We discuss what are currently hindering a more wide-spread use of psychometrics and how overcoming these hurdles could lead to a more individualized software engineering.

Quality Requirements in Industrial Practice—An Extended Interview Study at Eleven Companies

In order to create a successful software product and assure its quality, it is not enough to fulfill the functional requirements, it is also crucial to find the right balance among competing quality requirements (QR). An extended, previously piloted, interview study was performed to identify specific challenges associated with the selection, tradeoff, and management of QR in industrial practice. Data were collected through semistructured interviews with 11 product managers and 11 project leaders from 11 software companies. The contribution of this study is fourfold: First, it compares how QR are handled in two cases, companies working in business-to-business markets and companies that are working in business-to-consumer markets. These two are also compared in terms of impact on the handling of QR. Second, it compares the perceptions and priorities of QR by product and project management, respectively. Third, it includes an examination of the interdependencies among quality requirements perceived as most important by the practitioners. Fourth, it characterizes the selection and management of QR in downstream development activities.

Prioritization of quality requirements: State of practice in eleven companies

Requirements prioritization is recognized as an important but challenging activity in software product development. For a product to be successful, it is crucial to find the right balance among competing quality requirements. Although literature offers many methods for requirements prioritization, the research on prioritization of quality requirements is limited. This study identifies how quality requirements are prioritized in practice at 11 successful companies developing software intensive systems. We found that ad-hoc prioritization and priority grouping of requirements are the dominant methods for prioritizing quality requirements. The results also show that it is common to use customer input as criteria for prioritization but absence of any criteria was also common. The results suggests that quality requirements by default have a lower priority than functional requirements, and that they only get attention in the prioritizing process if decision-makers are dedicated to invest specific time and resources on QR prioritization. The results of this study may help future research on quality requirements to focus investigations on industry-relevant issues.

Review: On the application of genetic programming for software engineering predictive modeling: A systematic review

The objective of this paper is to investigate the evidence for symbolic regression using genetic programming (GP) being an effective method for prediction and estimation in software engineering, when compared with regression/machine learning models and other comparison groups (including comparisons with different improvements over the standard GP algorithm). We performed a systematic review of literature that compared genetic programming models with comparative techniques based on different independent project variables. A total of 23 primary studies were obtained after searching different information sources in the time span 1995-2008. The results of the review show that symbolic regression using genetic programming has been applied in three domains within software engineering predictive modeling: (i) Software quality classification (eight primary studies). (ii) Software cost/effort/size estimation (seven primary studies). (iii) Software fault prediction/software reliability growth modeling (eight primary studies). While there is evidence in support of using genetic programming for software quality classification, software fault prediction and software reliability growth modeling; the results are inconclusive for software cost/effort/size estimation.

Knowledge transfer challenges and mitigation strategies in global software development—A systematic literature review and industrial validation

Context: In this thesis we considered Knowledge Transfer (KT) in Global Software Development (GSD) from both the state of art and state of practice, in order to identify what are the challenges that hamper the success of KT in global software teams, as well as to find out what are the mitigation strategies that can be practiced to overcome these challenges. Objectives: The main objective of this research is to find an in-depth understanding of knowledge transfer challenges and mitigation strategies from both literature studies and industrial experienced employees. It also identifies the similarities and differences of challenges and strategies from literature studies and industrial experienced employees. The overall aim of this work is to provide a list of mitigation strategies to challenges, as guidelines to enable successful knowledge transfer in GSD. Methods: In order to fulfill the aim of the research, we collected the data through a Systematic Literature Review (SLR) and industrial interviews. Through SLR we found 35 articles relevant to our objectives. The data is extracted from those articles and conclusions are drawn. The relevant data is collected from databases such as Engineering village, ACM Digital Library, Science Direct, Wiley Inter Science, Scopus, ISI Web of Science and IEEE Xplore. We conducted 8 interviews from 8 different multinational companies. For analyzing the data we used grounded theory and qualitative comparative analysis. Results: In total, 72 different challenges and 107 mitigation strategies were identified from both SLR and interview results. In most of the studies, KT challenges in GSD are categorized into 3Cs (Communication, Control and Coordination). We also came up with a different view known as 2PT which conceptualizes the KT challenges and strategies into Personnel, Project and Technology factors. Conclusions: In future, researchers have to focus on the personnel, project and technology factors for implementing an effective KT process. From a practitioner‘s view, the results can be used to identify critical factors for effective KT. The challenges to KT show to what extent these results can be industrially applicable.