Ruchika Malhotra

Empirical validation of object-oriented metrics for predicting fault proneness models

Empirical validation of software metrics used to predict software quality attributes is important to ensure their practical relevance in software organizations. The aim of this work is to find the relation of object-oriented (OO) metrics with fault proneness at different severity levels of faults. For this purpose, different prediction models have been developed using regression and machine learning methods. We evaluate and compare the performance of these methods to find which method performs better at different severity levels of faults and empirically validate OO metrics given by Chidamber and Kemerer. The results of the empirical study are based on public domain NASA data set. The performance of the predicted models was evaluated using Receiver Operating Characteristic (ROC) analysis. The results show that the area under the curve (measured from the ROC analysis) of models predicted using high severity faults is low as compared with the area under the curve of the model predicted with respect to medium and low severity faults. However, the number of faults in the classes correctly classified by predicted models with respect to high severity faults is not low. This study also shows that the performance of machine learning methods is better than logistic regression method with respect to all the severities of faults. Based on the results, it is reasonable to claim that models targeted at different severity levels of faults could help for planning and executing testing by focusing resources on fault-prone parts of the design and code that are likely to cause serious failures.

Empirical Study of Object-Oriented Metrics

The increasing importance of software measurement has led to development of new software measures. Many metrics have been proposed related to various constructs like class, coupling, cohesion, inheritance, information hiding and polymorphism. But there is a little understanding of the empirical hypotheses and application of many of these measures. It is often difficult to determine which metric is more useful in which area. As a consequence, it is very difficult for project managers and practitioners to select measures for object-oriented systems. In this paper we investigate 22 metrics proposed by various researchers. The metrics are first defined and then explained using practical applications. They are applied on standard projects on the basis of which descriptive statistics, principal component analysis and correlation analysis is presented. Finally, a review of the empirical study concerning chosen metrics and subset of these measures that provide sufficient information is given and metrics providing overlapping information are excluded from the set.

A systematic review of machine learning techniques for software fault prediction

Reviews studies from 1991-2013 to assess application of ML techniques for SFP.Identifies seven categories of the ML techniques.Identifies 64 studies to answer the established research questions.Selects primary studies according to the quality assessment of the studies.Systematic literature review performs the following:Summarize ML techniques for SFP models.Assess performance accuracy and capability of ML techniques for constructing SFP models.Provide comparison between the ML and statistical techniques.Provide comparison of performance accuracy of different ML techniques.Summarize the strength and weakness of the ML techniques.Provides future guidelines to software practitioners and researchers. BackgroundSoftware fault prediction is the process of developing models that can be used by the software practitioners in the early phases of software development life cycle for detecting faulty constructs such as modules or classes. There are various machine learning techniques used in the past for predicting faults. MethodIn this study we perform a systematic review of studies from January 1991 to October 2013 in the literature that use the machine learning techniques for software fault prediction. We assess the performance capability of the machine learning techniques in existing research for software fault prediction. We also compare the performance of the machine learning techniques with the statistical techniques and other machine learning techniques. Further the strengths and weaknesses of machine learning techniques are summarized. ResultsIn this paper we have identified 64 primary studies and seven categories of the machine learning techniques. The results prove the prediction capability of the machine learning techniques for classifying module/class as fault prone or not fault prone. The models using the machine learning techniques for estimating software fault proneness outperform the traditional statistical models. ConclusionBased on the results obtained from the systematic review, we conclude that the machine learning techniques have the ability for predicting software fault proneness and can be used by software practitioners and researchers. However, the application of the machine learning techniques in software fault prediction is still limited and more number of studies should be carried out in order to obtain well formed and generalizable results. We provide future guidelines to practitioners and researchers based on the results obtained in this work.

Comparative analysis of regression and machine learning methods for predicting fault proneness models

Demand for quality software has undergone rapid growth during the last few years. This is leading to increase in development of machine learning techniques for exploring datasets which can be used in constructing models for predicting quality attributes such as Decision Tree (DT), Support Vector Machine (SVM) and Artificial Neural Network (ANN). This paper examines and compares Logistic Regression (LR), ANN (model predicted in an analogous study using the same dataset), SVM and DT methods. These two methods are explored empirically to find the effect of object-oriented metrics given by Chidamber and Kemerer on the fault proneness of object-oriented system classes. Data collected from Java applications is used in the study. The performance of the methods was compared by Receiver Operating Characteristic (ROC) analysis. DT modelling showed 84.7% of correct classifications of faulty classes and is a better model than the model predicted using LR, SVM and ANN method. The area under the ROC curve of LR, ANN, SVM and DT model is 0.826, 0.85, 0.85 and 0.87, respectively. The paper shows that machine learning methods are useful in constructing software quality models.

Software reuse metrics for object-oriented systems

The importance of software measurement is increasing leading to development of new measurement techniques. Reusing existing software components is a key feature in increasing software productivity. It is one of the key elements in object-oriented programming, which reduces the cost and increases the quality of the software. An important feature of C++ called templates support generic programming, which allows the programmer to develop reusable software modules such as functions, classes, etc. The need for software reusability metrics is particularly acute for an organization in order to measure the degree of generic programming included in the form of templates in code. This research addresses this need and introduces a new set of metrics for object-oriented software. Two metrics are proposed for measuring amount of genericty included in the code and then analytically evaluated against Weyukers set of nine axioms. This set of metrics is then applied to standard projects and accordingly ways in which project managers can use these metrics are suggested.

Fault Prediction Using Statistical and Machine Learning Methods for Improving Software Quality

Abstract —An understanding of quality attributes is relevant for the software organization to deliver high software reliability. An empirical assessment of metrics to predict the quality attributes is essential in order to gain insight about the quality of software in the early phases of software development and to ensure corrective actions. In this paper, we predict a model to estimate fault proneness using Object Oriented CK metrics and QMOOD metrics. We apply one statistical method and six machine learning methods to predict the models. The proposed models are validated using dataset collected from Open Source software. The results are analyzed using Area Under the Curve (AUC) obtained from Receiver Operating Characteristics (ROC) analysis. The results show that the model predicted using the random forest and bagging methods outperformed all the other models. Hence, based on these results it is reasonable to claim that quality models have a significant relevance with Object Oriented metrics and that machine learning methods have a comparable performance with statistical methods

Investigation of relationship between object-oriented metrics and change proneness

Software is the heartbeat of modern day technology. In order to keep up with the pace of modern day expansion, change in any software is inevitable. Defects and enhancements are the two main reasons for a software change. The aim of this paper is to study the relationship between object oriented metrics and change proneness. Software prediction models based on these results can help us identify change prone classes of a software which would lead to more rigorous testing and better results. In the previous research, the use of machine learning methods for predicting faulty classes was found. However till date no study determines the effectiveness of machine learning methods for predicting change prone classes. Statistical and machine learning methods are two different techniques for software quality prediction. We evaluate and compare the performance of these machine learning methods with statistical method (logistic regression). The results are based on three chosen open source software, written in java language. The performance of the predicted models was evaluated using receiver operating characteristic analysis. The study shows that machine learning methods are comparable to regression techniques. Testing based on change proneness of a software leads to better quality by targeting the most change prone classes. Thus, the developed models can be used to reduce the probability of defect occurrence and we commit ourselves to better maintenance.

Comparative analysis of statistical and machine learning methods for predicting faulty modules

Abstract The demand for development of good quality software has seen rapid growth in the last few years. This is leading to increase in the use of the machine learning methods for analyzing and assessing public domain data sets. These methods can be used in developing models for estimating software quality attributes such as fault proneness, maintenance effort, testing effort. Software fault prediction in the early phases of software development can help and guide software practitioners to focus the available testing resources on the weaker areas during the software development. This paper analyses and compares the statistical and six machine learning methods for fault prediction. These methods (Decision Tree, Artificial Neural Network, Cascade Correlation Network, Support Vector Machine, Group Method of Data Handling Method, and Gene Expression Programming) are empirically validated to find the relationship between the static code metrics and the fault proneness of a module. In order to assess and compare the models predicted using the regression and the machine learning methods we used two publicly available data sets AR1 and AR6. We compared the predictive capability of the models using the Area Under the Curve (measured from the Receiver Operating Characteristic (ROC) analysis). The study confirms the predictive capability of the machine learning methods for software fault prediction. The results show that the Area Under the Curve of model predicted using the Decision Tree method is 0.8 and 0.9 (for AR1 and AR6 data sets, respectively) and is a better model than the model predicted using the logistic regression and other machine learning methods.

An Adequacy Based Test Data Generation Technique Using Genetic Algorithms

As the complexity of software is increasing, generating an effective test data has become a necessity. This necessity has increased the demand for techniques that can generate test data effectively. This paper proposes a test data generation technique based on adequacy based testing criteria. Adequacy based testing criteria uses the concept of mutation analysis to check the adequacy of test data. In general, mutation analysis is applied after the test data is generated. But, in this work, we propose a technique that applies mutation analysis at the time of test data generation only, rather than applying it after the test data has been generated. This saves significant amount of time (required to generate adequate test cases) as compared to the latter case as the total time in the latter case is the sum of the time to generate test data and the time to apply mutation analysis to the generated test data. We also use genetic algorithms that explore the complete domain of the program to provide near-global optimum solution. In this paper, we first define and explain the proposed technique. Then we validate the proposed technique using ten real time programs. The proposed technique is compared with path testing technique (that use reliability based testing criteria) for these ten programs. The results show that the adequacy based proposed technique is better than the reliability based path testing technique and there is a significant reduce in number of generated test cases and time taken to generate test cases.

Soft Computing Approaches for Prediction of Software Maintenance Effort

The relationship between object oriented metrics and software maintenance effort is complex and non-linear. Therefore, there is considerable research interest in development and application of sophisticated techniques which can be used to construct models for predicting software maintenance effort. The aim of this paper is to evaluate and compare the application of different soft computing techniques – Artificial Neural Networks, Fuzzy Inference Systems and Adaptive Neuro-Fuzzy Inference Systems to construct models for prediction of Software Maintenance Effort. The maintenance effort data of two commercial software products is used in this study. The dependent variable in our study is maintenance effort. The independent variables are eight Object Oriented metrics . It is observed that soft computing techniques can be used for constructing accurate models for prediction of software maintenance effort and Adaptive Neuro Fuzzy Inference System technique gives the most accurate model.