Sousuke Amasaki

A Bayesian belief network for assessing the likelihood of fault content

To predict software quality, we must consider various factors because software development consists of various activities, which the software reliability growth model (SRGM) does not consider. In this paper, we propose a model to predict the final quality of a software product by using the Bayesian belief network (BBN) model. By using the BBN, we can construct a prediction model that focuses on the structure of the software development process explicitly representing complex relationships between metrics, and handling uncertain metrics, such as residual faults in the software products. In order to evaluate the constructed model, we perform an empirical experiment based on the metrics data collected from development projects in a certain company. As a result of the empirical evaluation, we confirm that the proposed model can predict the amount of residual faults that the SRGM cannot handle.

On the effectiveness of weighted moving windows: Experiment on linear regression based software effort estimation

In construction of an effort estimation model, it seems effective to use a window of training data so that the model is trained with only recent projects. Considering the chronological order of projects within the window, and weighting projects according to their order within the window, may also affect estimation accuracy. In this study, we examined the effects of weighted moving windows on effort estimation accuracy. We compared weighted and non‐weighted moving windows under the same experimental settings. We confirmed that weighting methods significantly improved estimation accuracy in larger windows, although the methods also significantly worsened accuracy in smaller windows. This result contributes to understanding properties of moving windows. Copyright

The Evaluation of Weighted Moving Windows for Software Effort Estimation

In construction of an effort estimation model, it seems effective to use a window of training data so that the model is trained with only recent projects. Considering the chronological order of projects within the window, and weighting projects according to their order within the window, may also affect estimation accuracy. In this study, we examined the effects of weighted moving windows on effort estimation accuracy. We compared weighted and non-weighted moving windows under the same experimental settings. We confirmed that weighting methods significantly improved estimation accuracy in larger windows, though the methods also significantly worsened accuracy in smaller windows. This result contributes to understanding properties of moving windows.

The Effects of Moving Windows to Software Estimation: Comparative Study on Linear Regression and Estimation by Analogy

BACKGROUND: Models for estimating software development effort are constructed using a set of historical data for training. In construction, it seems effective to use a window of training data that consists of only recently finished projects. Two previous studies evaluated the use of a window with linear regression (LR) and estimation by analogy (EbA). However, these studies were based on different datasets and thus their findings could not be compared directly. OBJECTIVE: This study investigates the effect of using a window on estimation accuracy with EbA and LR. The difference between the results with the two modeling techniques was also investigated. METHOD: We compared the effectiveness of using a window with both LR and EbA, with the same experiment settings and data. RESULTS: There is a difference in accuracy between using a window and not using a window, with both software estimation methods. However, the effect of the use of a window is weaker with EbA than with LR. CONCLUSIONS: Windowing is effective with EbA and LR. However, the degree of effectiveness is weaker with EbA than with LR. The results contribute to understand how the windowing approach interrelates with software estimation models.

A New Challenge for Applying Time Series Metrics Data to Software Quality Estimation

In typical software development, a software reliability growth model (SRGM) is applied in each testing activity to determine the time to finish the testing. However, there are some cases in which the SRGM does not work correctly. That is, the SRGM sometimes mistakes quality for poor quality products. In order to tackle this problem, we focussed on the trend of time series data of software defects among successive testing phases and tried to estimate software quality using the trend. First, we investigate the characteristics of the time series data on the detected faults by observing the change of the number of detected faults. Using the rank correlation coefficient, the data are classified into four kinds of trends. Next, with the intention of estimating software quality, we investigate the relationship between the trends of the time series data and software quality. Here, software quality is defined by the number of faults detected during six months after shipment. Finally, we find a relationship between the trends and metrics data collected in the software design phase. Using logistic regression, we statistically show that two review metrics in the design and coding phase can determine the trend.

Improving Relevancy Filter Methods for Cross-Project Defect Prediction

Context: Cross-project defect prediction (CPDP)research has been popular. One of the techniques for CPDP isa relevancy filter which utilizes clustering algorithms to selecta useful subset of the cross-project data. Their performanceheavily relies on the quality of clustering, and using an advancedclustering algorithm instead of simple ones used in the past studiescan contribute to the performance improvement. Objective:To propose and examine a new relevancy filter method usingan advanced clustering method DBSCAN (Density-Based SpatialClustering). Method: We conducted an experiment that examinedthe predictive performance of the proposed method. Theexperiments compared three relevancy filter methods, namely,Burak-filter, Peters-filter, and the proposed method with 56project data and four prediction models. Results: The predictiveperformance measures supported the proposed method. It wasbetter than Burak-filter and Peters-filter in terms of AUC andg-measure. Conclusion: The proposed method achieved betterprediction than the conventional methods. The results suggestedthat exploring advanced clustering algorithms could contributeto cross-project defect prediction.

Improving Cross-Project Defect Prediction Methods with Data Simplification

Context: Cross-project defect prediction (CPDP) research has been popular and many CPDP methods were proposed. While these methods used cross-project data as is for their inputs, useless or noisy information in the cross-project data can cause the degradation of predictive and computation performance. Removing such information makes the cross-project data simple and it will affect the performance of CPDP methods. Objective: To identify and quantify the effects of the data simplification for CPDP methods. Method: We conducted experiments that compared the predictive performance between CPDP with and without the data simplification. We adopted a data simplification method based on an active learning method proposed for software effort estimation. The experiments adopted 44 versions of OSS projects, four prediction models, and two CPDP methods, namely, Burak-filter and cross-project selection. Results: The data simplification achieved significant improvement in predictive performance for the cross-project selection. It did not improve Burak-filter. Conclusion: The data simplification can be helpful for the cross-project selection in terms of predictive performance and size reduction of cross-project data.

Empirical Analysis of Change-Proneness in Methods Having Local Variables with Long Names and Comments

This paper focuses on the local variable names and comments that are major artifacts reflecting the programmers preference. It conducts an empirical analysis on the usefulness of those artifacts in assessing the software quality from the perspective of change-proneness in Java methods developed in six popular open source software products. The empirical results show: (1) a method having a longer named local variable is more change-prone, and (2) the presence of comments inside the method body strengthens the suspicions to be modified after the release. The above artifacts are worthy to find methods which can survive unscathed after the release.

The Effect of Moving Windows on Software Effort Estimation: Comparative Study with CART

BACKGROUND: Several studies in software effort estimation have found that it can be effective to use a window of recent projects as training data for building an effort estimation model. The previous studies evaluated the use of a window with popular estimation models: linear regression (LR) and estimation by analogy (EbA). Many effort estimation models have been proposed, and the generality of windowing approach still remains uncertain for other effort estimation models, especially for those based on different theory. OBJECTIVE: This study investigates the effect of using a window on estimation accuracy with Classification and regression trees (CART). CART was recently found as a good performance method, and is based on a different theory from LR and EbA. METHOD: We compared the estimation accuracy of a windowing approach and growing approach with the same data set and procedure as the past studies. RESULTS: There is a difference in the estimation accuracy between using a window and not using a window. However, the effctive range of using windows on CART is narrower than that on LR. CONCLUSIONS: Windowing is also effective with CART. However, the range of effectiveness is narrower. The results contribute to the generality of the effectiveness of windowing approach.

Performance Evaluation of Windowing Approach on Effort Estimation by Analogy

Background: In effort estimation model construction, it seems effective to window training project data so that only recently finished projects are used. This is because old projects might be less representative of an organization. The past study demonstrated windowing approach works with linear regression, which is one of global models. However, this approach has not been examined with local models. Local models use subset of historical data for model construction and thus windowing approach may influence on its performance more weakly. Aim: To investigate whether windowing approach works with local models. Method: We replicated the past study with EbA. Maxwell and CSC datasets were used for an experiment. Results: Windowing approach improved predictive performance. Although the difference was insignificant in any window size, the result indicated using windowing approach has positive effect on average. Conclusions: This result contributes to understand where windowing approach works well.