Tracy Hall

A Systematic Literature Review on Fault Prediction Performance in Software Engineering

Background: The accurate prediction of where faults are likely to occur in code can help direct test effort, reduce costs, and improve the quality of software. Objective: We investigate how the context of models, the independent variables used, and the modeling techniques applied influence the performance of fault prediction models. Method: We used a systematic literature review to identify 208 fault prediction studies published from January 2000 to December 2010. We synthesize the quantitative and qualitative results of 36 studies which report sufficient contextual and methodological information according to the criteria we develop and apply. Results: The models that perform well tend to be based on simple modeling techniques such as Naive Bayes or Logistic Regression. Combinations of independent variables have been used by models that perform well. Feature selection has been applied to these combinations when models are performing particularly well. Conclusion: The methodology used to build models seems to be influential to predictive performance. Although there are a set of fault prediction studies in which confidence is possible, more studies are needed that use a reliable methodology and which report their context, methodology, and performance comprehensively.

Key success factors for implementing software process improvement : a maturity-based analysis

Original article can be found at: http://www.sciencedirect.com/science/journal/01641212 Copyright Elsevier Inc. [Full text of this article is not available in the UHRA]

Implementing effective software metrics programs

Increasingly organisations are foregoing an ad hoc approach to metrics in favor of complete metrics programs. The authors identify consensus requirements for metric program success and examine how programs in two organisations measured up.

Software Process Improvement Problems in Twelve Software Companies: An Empirical Analysis

In this paper we discuss our study of the problems 12 software companies experienced in software development. In total we present qualitative data collected from 45 focus groups that involved over 200 software staff. We look at how different practitioner groups respond to software process improvement problems. We show our classification and analysis of this data using correspondence analysis. Correspondence analysis is a graphical data representation method new to software development research. The aim of the work we present is to develop a more holistic understanding of the problems practitioners are experiencing in their attempts to improve their software processes. Our main finding is that there is an association between a companys capability maturity and patterns of reported problems. Organizational problems are more associated with high maturity companies than with low maturity companies. Low maturity companies are closely linked to problems relating directly to projects such as documentation, timescales, tools and technology. Our findings also confirm differences in practitioner group problems. Senior managers cite problems with goals, culture and politics. Project managers are concerned with timescales, change management, budgets and estimates. Developers are experiencing problems with requirements, testing, documentation, communication, tools and technology. These associations are displayed graphically through correspondence analysis maps.

Models of motivation in software engineering

Motivation in software engineering is recognized as a key success factor for software projects, but although there are many papers written about motivation in software engineering, the field lacks a comprehensive overview of the area. In particular, several models of motivation have been proposed, but they either rely heavily on one particular model (the job characteristics model), or are quite disparate and difficult to combine. Using the results from our previous systematic literature review (SLR), we constructed a new model of motivation in software engineering. We then compared this new model with existing models and refined it based on this comparison. This paper summarises the SLR results, presents the important existing models found in the literature and explains the development of our new model of motivation in software engineering.

Researcher Bias: The Use of Machine Learning in Software Defect Prediction

Background. The ability to predict defect-prone software components would be valuable. Consequently, there have been many empirical studies to evaluate the performance of different techniques endeavouring to accomplish this effectively. However no one technique dominates and so designing a reliable defect prediction model remains problematic. Objective. We seek to make sense of the many conflicting experimental results and understand which factors have the largest effect on predictive performance. Method. We conduct a meta-analysis of all relevant, high quality primary studies of defect prediction to determine what factors influence predictive performance. This is based on 42 primary studies that satisfy our inclusion criteria that collectively report 600 sets of empirical prediction results. By reverse engineering a common response variable we build a random effects ANOVA model to examine the relative contribution of four model building factors (classifier, data set, input metrics and researcher group) to model prediction performance. Results. Surprisingly we find that the choice of classifier has little impact upon performance (1.3 percent) and in contrast the major (31 percent) explanatory factor is the researcher group. It matters more who does the work than what is done. Conclusion. To overcome this high level of researcher bias, defect prediction researchers should (i) conduct blind analysis, (ii) improve reporting protocols and (iii) conduct more intergroup studies in order to alleviate expertise issues. Lastly, research is required to determine whether this bias is prevalent in other applications domains.

Using an expert panel to validate a requirements process improvement model

In this paper we present components of a newly developed software process improvement model that aims to represent key practices in requirements engineering (RE). Our model is developed in response to practitioner needs highlighted in our empirical work with UK software development companies. We have now reached the stage in model development where we need some independent feedback as to how well our model meets our objectives. We perform this validation through involving a group of software process improvement and RE experts in examining our RE model components and completing a detailed questionnaire. A major part of this paper is devoted to explaining our validation methodology. There is very little in the literature that directly relates to how process models have been validated, therefore providing this transparency will benefit both the research community and practitioners. The validation methodology and the model itself contribute towards a better understanding of modelling RE processes.

Implementing Software Process Improvement: An Empirical study

In this paper we present survey data characterizing the implementation of SPI in 85 UK companies. We aim to provide SPI managers with more understanding of the critical success factors of implementing SPI. We present an analysis of the critical implementation factors identified in published case studies. We use a questionnaire to measure the use of these factors in ‘typical’ software companies. We found that many companies use SPI but the effectiveness of SPI implementation is variable. Many companies inadequately resource SPI and fail to evaluate the impact of SPI. On the other hand, companies show a good appreciation of the human factors associated with implementing SPI. Copyright

Code Bad Smells: a review of current knowledge

Fowler et al. identified 22 Code Bad Smells to direct the effective refactoring of code. These are increasingly being taken up by software engineers. However, the empirical basis of using Code Bad Smells to direct refactoring and to address ‘trouble’ in code is not clear, i.e., we do not know whether using Code Bad Smells to target code improvement is effective. This paper aims to identify what is currently known about Code Bad Smells. We have performed a systematic literature review of 319 papers published since Fowler et al. identified Code Bad Smells (2000 to June 2009). We analysed in detail 39 of the most relevant papers. Our findings indicate that Duplicated Code receives most research attention, whereas some Code Bad Smells, e.g., Message Chains, receive little. This suggests that our knowledge of some Code Bad Smells remains insufficient. Our findings also show that very few studies report on the impact of using Code Bad Smells, with most studies instead focused on developing tools and methods to automatically detect Code Bad Smells. This indicates an important gap in the current knowledge of Code Bad Smells. Overall this review suggests that there is little evidence currently available to justify using Code Bad Smells. Copyright

Defining a Requirements Process Improvement Model

Both software organisations and the academic community are aware that the requirements phase of software development is in need of further support. We address this problem by creating a specialised Requirements Capability Maturity Model (R-CMM1). The model focuses on the requirements engineering process as defined within the established Software Engineering Institute’s (SEI’s) software process improvement framework. Our empirical work with software practitioners is a primary motivation for creating this requirements engineering process improvement model. Although all organisations in our study were involved in software process improvement (SPI), they all showed a lack of control over many requirement engineering activities.This paper describes how the requirements engineering (RE) process is decomposed and prioritised in accordance with maturity goals set by the SEI’s Software Capability Maturity Model (SW CMM). Our R-CMM builds on the SEI’s framework by identifying and defining recommended RE sub-processes that meet maturity goals. This new focus will help practitioners to define their RE process with a view to setting realistic goals for improvement.