Bora Caglayan

Merits of using repository metrics in defect prediction for open source projects

Many corporate code developers are the beta testers of open source software.They continue testing until they are sure that they have a stable version to build their code on. In this respect defect predictors play a critical role to identify defective parts of the software. Performance of a defect predictor is determined by correctly finding defective parts of the software without giving any false alarms. Having high false alarms means testers/ developers would inspect bug free code unnecessarily. Therefore in this research we focused on decreasing the false alarm rates by using repository metrics. We conducted experiments on the data sets of Eclipse project. Our results showed that repository metrics decreased the false alarm rates on the average to 23% from 32% corresponding up to 907 less files to inspect.

Defect prediction using social network analysis on issue repositories

People are the most important pillar of software development process. It is critical to understand how they interact with each other and how these interactions affect the quality of the end product in terms of defects. In this research we propose to include a new set of metrics, a.k.a. social network metrics on issue repositories in predicting defects. Social network metrics on issue repositories has not been used before to predict defect proneness of a software product. To validate our hypotheses we used two datasets, development data of IBM1 Rational ® Team Concert™ (RTC) and Drupal, to conduct our experiments. The results of the experiments revealed that compared to other set of metrics such as churn metrics using social network metrics on issue repositories either considerably decreases high false alarm rates without compromising the detection rates or considerably increases low prediction rates without compromising low false alarm rates. Therefore we recommend practitioners to collect social network metrics on issue repositories since people related information is a strong indicator of past patterns in a given team.

A Retrospective Study of Software Analytics Projects: In-Depth Interviews with Practitioners

Software analytics guide practitioners in decision making throughout the software development process. In this context, prediction models help managers efficiently organize their resources and identify problems by analyzing patterns on existing project data in an intelligent and meaningful manner. Over the past decade, the authors have worked with software organizations to build metric repositories and predictive models that address process-, product-, and people-related issues in practice. This article shares their experience over the years, reflecting the expectations and outcomes both from practitioner and researcher viewpoints.

Different strokes for different folks: a case study on software metrics for different defect categories

Defect prediction has been evolved with variety of metric sets, and defect types. Researchers found code, churn, and network metrics as significant indicators of defects. However, all metric sets may not be informative for all defect categories such that only one metric type may represent majority of a defect category. Our previous study showed that defect category sensitive prediction models are more successful than general models, since each category has different characteristics in terms of metrics. We extend our previous work, and propose specialized prediction models using churn, code, and network metrics with respect to three defect categories. Results show that churn metrics are the best for predicting all defects. The strength of correlation for code and network metrics varies with defect category: Network metrics have higher correlations than code metrics for defects reported during functional testing and in the field, and vice versa for defects reported during system testing.

Usage of multiple prediction models based on defect categories

Background: Most of the defect prediction models are built for two purposes: 1) to detect defective and defect-free modules (binary classification), and 2) to estimate the number of defects (regression analysis). It would also be useful to give more information on the nature of defects so that software managers can plan their testing resources more effectively. Aims: In this paper, we propose a defect prediction model that is based on defect categories. Method: We mined the version history of a large-scale enterprise software product to extract churn and static code metrics. and grouped them into three defect categories according to different testing phases. We built a learning-based model for each defect category. We compared the performance of our proposed model with a general one. We conducted statistical techniques to evaluate the relationship between defect categories and software metrics. We also tested our hypothesis by replicating the empirical work on Eclipse data. Results: Our results show that building models that are sensitive to defect categories is cost-effective in the sense that it reveals more information and increases detection rates (pd) by 10% keeping the false alarms (pf) constant. Conclusions: We conclude that slicing defect data and categorizing it for use in a defect prediction model would enable practitioners to take immediate actions. Our results on Eclipse replication showed that haphazard categorization of defects is not worth the effort.

Factors characterizing reopened issues: a case study

Background: Reopened issues may cause problems in managing software maintenance effort. In order to take actions that will reduce the likelihood of issue reopening the possible causes of bug reopens should be analysed. Aims: In this paper, we investigate potential factors that may cause issue reopening. Method: We have extracted issue activity data from a large release of an enterprise software product. We consider four dimensions, namely developer activity, issue proximity network, static code metrics of the source code changed to fix an issue, issue reports and fixes as possible factors that may cause issue reopening. We have done exploratory analysis on data. We build logistic regression models on data in order to identify key factors leading issue reopening. We have also conducted a survey regarding these factors with the QA Team of the product and interpreted the results. Results: Our results indicate that centrality in the issue proximity network and developer activity are important factors in issue reopening. We have also interpreted our results with the QA Team to point out potential implications for practitioners. Conclusions: Quantitative findings of our study suggest that issue complexity and developers workload play an important role in triggering issue reopening.

Dione: an integrated measurement and defect prediction solution

We present an integrated measurement and defect prediction tool: Dione. Our tool enables organizations to measure, monitor, and control product quality through learning based defect prediction. Similar existing tools either provide data collection and analytics, or work just as a prediction engine. Therefore, companies need to deal with multiple tools with incompatible interfaces in order to deploy a complete measurement and prediction solution. Dione provides a fully integrated solution where data extraction, defect prediction and reporting steps fit seamlessly. In this paper, we present the major functionality and architectural elements of Dione followed by an overview of our demonstration.

The effect of evolutionary coupling on software defects: an industrial case study on a legacy system

Evolutionary coupling is defined as the implicit relationship between two or more software artifacts that are frequently changed together. In this study we investigate the effect of evolutionary coupling on defect proneness of a large financial legacy software in an industrial software development environment. We collected historical data for 5 years from 3 different software repositories containing 150 thousand files on 274 modules. Our results indicate that there is a positive correlation between evolutionary coupling and defect measures. Furthermore, we built linear and logistic regression models by using evolutionary coupling measures in order to explain defects. Although regression analysis results show that evolutionary coupling measures can be useful to explain defects, especially for modules in which high correlation is detected, explanatory power decreases dramatically with the decreasing correlation.

Emergence of developer teams in the collaboration network

Developer teams may naturally emerge independent of managerial decisions, organizational structure, or work locations in large software. Such self organized collaboration teams of developers can be traced from the source code repositories. In this paper, we identify the developer teams in the collaboration network in order to present the work team evolution and the factors that affect team stability for a large, globally developed, commercial software. Our findings indicate that: a) Number of collaboration teams do not change over time, b) Size of the collaboration teams increases over time, c) Team activity is not related with team size, d) Factors related to team size, location and activity affect the stability of teams over time.

Merits of organizational metrics in defect prediction: an industrial replication

Defect prediction models presented in the literature lack generalization unless the original study can be replicated using new datasets and in different organizational settings. Practitioners can also benefit from replicating studies in their own environment by gaining insights and comparing their findings with those reported. In this work, we replicated an earlier study in order to investigate the merits of organizational metrics in building defect prediction models for large-scale enterprise software. We mined the organizational, code complexity, code churn and pre-release bug metrics of that large scale software and built defect prediction models for each metric set. In the original study, organizational metrics were found to achieve the highest performance. In our case, models based on organizational metrics performed better than models based on churn metrics but were outperformed by pre-release metric models. Further, we verified four individual organizational metrics as indicators for defects. We conclude that the performance of different metric sets in building defect prediction models depends on the projects characteristics and the targeted prediction level. Our replication of earlier research enabled assessing the validity and limitations of organizational metrics in a different context.