Raed Shatnawi

An empirical study of the bad smells and class error probability in the post-release object-oriented system evolution

Bad smells are used as a means to identify problematic classes in object-oriented systems for refactoring. The belief that the bad smells are linked with problematic classes is largely based on previous metric research results. Although there is a plethora of empirical studies linking software metrics to errors and error proneness of classes in object-oriented systems, the link between the bad smells and class error probability in the evolution of object-oriented systems after the systems are released has not been explored. There has been no empirical evidence linking the bad smells with class error probability so far. This paper presents the results from an empirical study that investigated the relationship between the bad smells and class error probability in three error-severity levels in an industrial-strength open source system. Our research, which was conducted in the context of the post-release system evolution process, showed that some bad smells were positively associated with the class error probability in the three error-severity levels. This finding supports the use of bad smells as a systematic method to identify and refactor problematic classes in this specific context.

The effectiveness of software metrics in identifying error-prone classes in post-release software evolution process

Many empirical studies have found that software metrics can predict class error proneness and the prediction can be used to accurately group error-prone classes. Recent empirical studies have used open source systems. These studies, however, focused on the relationship between software metrics and class error proneness during the development phase of software projects. Whether software metrics can still predict class error proneness in a systems post-release evolution is still a question to be answered. This study examined three releases of the Eclipse project and found that although some metrics can still predict class error proneness in three error-severity categories, the accuracy of the prediction decreased from release to release. Furthermore, we found that the prediction cannot be used to build a metrics model to identify error-prone classes with acceptable accuracy. These findings suggest that as a system evolves, the use of some commonly used metrics to identify which classes are more prone to errors becomes increasingly difficult and we should seek alternative methods (to the metric-prediction models) to locate error-prone classes if we want high accuracy.

A Quantitative Investigation of the Acceptable Risk Levels of Object-Oriented Metrics in Open-Source Systems

Object-oriented metrics have been validated empirically as measures of design complexity. These metrics can be used to mitigate potential problems in the software complexity. However, there are few studies that were conducted to formulate the guidelines, represented as threshold values, to interpret the complexity of the software design using metrics. Classes can be clustered into low and high risk levels using threshold values. In this paper, we use a statistical model, derived from the logistic regression, to identify threshold values for the Chidamber and Kemerer (CK) metrics. The methodology is validated empirically on a large open-source system-the Eclipse project. The empirical results indicate that the CK metrics have threshold effects at various risk levels. We have validated the use of these thresholds on the next release of the Eclipse project-Version 2.1-using decision trees. In addition, the selected threshold values were more accurate than those were selected based on either intuitive perspectives or on data distribution parameters. Furthermore, the proposed model can be exploited to find the risk level for an arbitrary threshold value. These findings suggest that there is a relationship between risk levels and object-oriented metrics and that risk levels can be used to identify threshold effects.

An Investigation of Bad Smells in Object-Oriented Design

Bad smells are used to identify problematic classes in object-oriented design. Although intuitively making sense, the promise that bad smells can indicate the quality of design has not been validated by empirical evidence. This paper presents the results from an investigation that explored the relationship between the bad smells and the errors in an object-oriented system. The investigation found that some bad smells are positively associated with class errors

Deriving metrics thresholds using log transformation

Software metrics are surrogates of many software quality factors such as fault proneness, reusability, and maintenance effort. Software metrics are numbers collected from software code to assess and evaluate where problems are more probable to happen. These numbers are used to flag warnings of the problematic parts of software code using threshold values. However, the proposed techniques did not consider the data distribution and skewness in data. In this research, we aim to propose a methodology based on log transformation to improve the metrics quality. To explore the effect of log transformation on data analysis, we conduct analysis of using software metrics after transformation in identifying fault‐prone areas on multireleases of 11 products (41 releases). The results show that the log transformation can be used to derive threshold values for all metrics under investigation. The results of the transformation are then used to conduct fault‐proneness classification based on threshold values and compared against the results without transformation. The fault classification with transformation was more successful than without transformation. Copyright

Improving software fault-prediction for imbalanced data

Fault-proneness has been studied extensively as a quality factor. The prediction of fault-proneness of software modules can help software engineers to plan evolutions of the system. This plan can be compromised in case prediction models are biased or do not have high prediction performance. One major issue that can impact the prediction performance is the fault distributions such as the data imbalance, i.e., the majority of modules are faultless whereas the minority of modules is only faulty. In this paper, we propose to use the fault content (i.e., the number of faults in a module) to oversample the minority. We applied this technique on a large object-oriented system - Eclipse. The proposed oversampling is tested on three classifiers. The results have shown a better prediction performance than other traditional oversampling techniques. The oversampling technique is more convenient than other sampling techniques because its guided by information provided from the software history.

Empirical study of fault prediction for open-source systems using the Chidamber and Kemerer metrics

Software testers are usually provoked with projects that have faults. Predicting a classs fault-proneness is vital for minimising cost and improving the effectiveness of the software testing. Previous research on software metrics has shown strong relationships between software metrics and faults in object-oriented systems using a binary variable. However, these models do not consider the history of faults in classes. In this work, a dependent variable is proposed that uses fault history to rate classes into four categories (none, low risk, medium risk and high risk) and to improve the predictive capability of fault models. The study is conducted on many releases of four open-source systems. The study tests the statistical differences in seven machine learning algorithms to find whether the proposed variable can be used to build better prediction models. The performance of the classifiers using the four categories is significantly better than the binary variable. In addition, the results show improvements on the reliability of the prediction models as the software matures. Therefore the fault history improves the prediction of fault-proneness of classes in open-source systems.

An Empirical Study of the Effect of Power Law Distribution on the Interpretation of OO Metrics

Context. Software metrics are surrogates of software quality. Software metrics can be used to find possible problems or chances for improvements in software quality. However, software metrics are numbers that are not easy to interpret. Previous analysis of software metrics has shown fat tails in the distribution. The skewness and fat tails of such data are properties of many statistical distributions and more importantly the phenomena of the power law. These statistical properties affect the interpretation of software quality metrics. Objectives. The objective of this research is to validate the effect of power laws on the interpretation of software metrics. Method. To investigate the effect of power law properties on software quality, we study five open-source systems to investigate the distribution and their effect on fault prediction models. Results. Study shows that power law behavior has an effect on the interpretation and usage of software metrics and in particular the CK metrics. Many metrics have shown a power law behavior. Threshold values are derived from the properties of the power law distribution when applied to open-source systems. Conclusion. The properties of a power law distribution can be effective in improving the fault-proneness models by setting reasonable threshold values.

The application of ROC analysis in threshold identification, data imbalance and metrics selection for software fault prediction

Software engineers have limited resources and need metrics analysis tools to investigate software quality such as fault-proneness of modules. There are a large number of software metrics available to investigate quality. However, not all metrics are strongly correlated with faults. In addition, software fault data are imbalanced and affect quality assessment tools such as fault prediction or threshold values that are used to identify risky modules. Software quality is investigated for three purposes. First, the receiver operating characteristics (ROC) analysis is used to identify threshold values to identify risky modules. Second, the ROC analysis is investigated for imbalanced data. Third, the ROC analysis is considered for feature selection. This work validated the use of ROC to identify thresholds for four metrics (WMC, CBO, RFC and LCOM). The ROC results after sampling the data are not significantly different from before sampling. The ROC analysis selects the same metrics (WMC, CBO and RFC) in most datasets, while other techniques have a large variation in selecting metrics.

A guided oversampling technique to improve the prediction of software fault-proneness for imbalanced data

Fault-proneness is one of the most tackled quality factors in the field of software quality. Predicting the probability of the faulty classes is necessary information to guide developers in their endeavour to improve the software quality and to reduce the costs of testing and maintenance. The performance of the fault prediction models suffers greatly from the imbalance of fault distribution, i.e., the majority of modules are not faulty whereas the minority are only faulty. The imbalanced distribution of faults affects the efficiency of prediction models greatly. In this paper, we discuss many oversampling techniques that are used to improve the performance of prediction models. We propose to guide the oversampling process using the fault content (i.e., the number of faults in a module). This study is conducted on a large object-oriented system – Eclipse. The proposed oversampling is tested on ten classifiers. The results of this work shows that using fault content in sampling has better prediction performance than other traditional oversampling techniques. The decision trees and nearest neighbours have shown outstanding performance whereas other classifiers have shown acceptable performance.