Zhihao Chen

Finding the right data for software cost modeling

Good software cost models can significantly help software project managers. With good models, project stakeholders can make informed decisions about how to manage resources, how to control and plan the project, or how to deliver the project on time, on schedule, and on budget. Real-world data sets, such as those coming from software engineering projects, often contain noisy, irrelevant, or redundant variables. We propose that cost modelers should perform data-pruning experiments after data collection and before model building. Such pruning experiments are simple and fast.

Validation methods for calibrating software effort models

COCONUT calibrates effort estimation models using an exhaustive search over the space of calibration parameters in a Cocomo I model. This technique is much simpler than other effort estimation method yet yields PRED levels comparable to those other methods. Also, it does so with less project data and fewer attributes (no scale factors). However, a comparison between COCONUT and other methods is complicated by differences in the experimental methods used for effort estimation. A review of those experimental methods concludes that software effort estimation models should be calibrated to local data using incremental holdout (not jack knife) studies, combined with randomization and hypothesis testing, repeated a statistically significant number of times.

Feature subset selection can improve software cost estimation accuracy

Cost estimation is important in software development for controlling and planning software risks and schedule. Good estimation models, such as COCOMO, can avoid insufficient resources being allocated to a project. In this study, we find that COCOMOs estimates can be improved via WRAPPER- a feature subset selection method developed by the data mining community. Using data sets from the PROMISE repository, we show WRAPPER significantly and dramatically improves COCOMOs predictive power.

Simple software cost analysis: safe or unsafe?

Delta estimation uses changes to old projects to estimate new projects. Delta estimation assumes that new costs can be extrapolated from old projects. In this study, we show that in certain real-world data sets. there exists attributes where this assumption does not hold. We define here an automatic method to find which attributes can be safely used for delta estimation.

Specialization and extrapolation of software cost models

Despite the widespread availability of software effort estimation models (e.g. COCOMO [2], Price-S [12], SEER-SEM [13], SLIM [14]), most managers still estimate new projects by extrapolating from old projects [3, 5, 7]. In this delta method, the cost of the next project is the cost of the last project multiplied by some factors modeling the difference between old and new projects [2].Delta estimation is simple, fast, and best of all, can take full advantage of local costing information. However delta estimation fails when the experience base (the old projects) can not be extrapolated to the new projects. Previously [10], we have shown that for a set of NASA projects, delta estimation would usually fail since most of the features and coefficients of the learned model vary wildly across sub-samples of the training data. In that prior work, no solution was offered for this problem.Here, we offer a solution and report the results of experiment with feature subset selection (FSS) and extrapolation. FSS methods are usually assessed via the mean change in model performance. However, as shown below, FSS can significantly reduce the variance as well. Hence, FSS should be routinely used in cost estimation.Our results should stop the trend in the effort modeling community of continually adding to the number of features in a model in order to improve estimation performance. Here we show that there are benefits in intelligently subtracting model features.

Evolving an experience base for software process research

Since 1996 the USC Center for Software Engineering has been accumulating a large amount of software process experience through many real-client project software engineering practices. Through the application of the Experience Factory approach, we have collected and evolved this experience into an experience base (eBASE) which has been leveraged successfully for empirically based software process research. Through eBASE we have realized tangible benefits in automating, organizational learning, and strategic advantages for software engineering research. We share our rationale for creating and evolving eBASE, give examples of how the eBASE has been used in recent process research, discuss current limitations and challenges with eBASE, and what we hope to do achieve in the future with it.