Daniel Port

Using the WinWin spiral model: a case study

Fifteen teams used the WinWin spiral model to prototype, plan, specify, and build multimedia applications for USCs Integrated Library System. The authors report lessons learned from this case study and how they extended the models utility and cost-effectiveness in a second round of projects.

Finding the right data for software cost modeling

Good software cost models can significantly help software project managers. With good models, project stakeholders can make informed decisions about how to manage resources, how to control and plan the project, or how to deliver the project on time, on schedule, and on budget. Real-world data sets, such as those coming from software engineering projects, often contain noisy, irrelevant, or redundant variables. We propose that cost modelers should perform data-pruning experiments after data collection and before model building. Such pruning experiments are simple and fast.

Validation methods for calibrating software effort models

COCONUT calibrates effort estimation models using an exhaustive search over the space of calibration parameters in a Cocomo I model. This technique is much simpler than other effort estimation method yet yields PRED levels comparable to those other methods. Also, it does so with less project data and fewer attributes (no scale factors). However, a comparison between COCONUT and other methods is complicated by differences in the experimental methods used for effort estimation. A review of those experimental methods concludes that software effort estimation models should be calibrated to local data using incremental holdout (not jack knife) studies, combined with randomization and hypothesis testing, repeated a statistically significant number of times.

Feature subset selection can improve software cost estimation accuracy

Cost estimation is important in software development for controlling and planning software risks and schedule. Good estimation models, such as COCOMO, can avoid insufficient resources being allocated to a project. In this study, we find that COCOMOs estimates can be improved via WRAPPER- a feature subset selection method developed by the data mining community. Using data sets from the PROMISE repository, we show WRAPPER significantly and dramatically improves COCOMOs predictive power.

Evaluating individual contribution toward group software engineering projects

It is widely acknowledged that group or team projects are a staple of undergraduate and graduate software engineering courses. Such projects provide students with experiences that better prepare them for their careers, so teamwork is often required or strongly encouraged by accreditation agencies. While there are a multitude of educational benefits of group projects, they also pose considerable challenge in fairly and accurately discerning individual contribution for evaluation purposes. Issues, approaches, and best practices for evaluating individual contribution are presented from the perspectives of the University of Kentucky, University of Ottawa, University of Southern California, and others. The techniques utilized within a particular course generally are a mix of (1) the group mark is everybodys mark, (2) everybody reports what they personally did, (3) other group members report the relative contributions of other group members, (4) pop quizzes on project details, and (5) cross-validating with the results of individual work.

Value-based processes for COTS-based applications

Economic imperatives are changing the nature of software development processes to reflect both the opportunities and challenges of using COTS products. Processes are increasingly moving away from the time-consuming composition of custom software from lines of code (although these processes still apply for developing the COTS products themselves) toward assessment, tailoring, and integration of COTS or other reusable components. Two factors are driving this change: COTS or other reusable components can provide significant user capabilities within limited costs and development time, and more COTS products are becoming available to provide needed user functions.

A stakeholder win–win approach to software engineering education

We have been applying the stakeholder win–win approach to software engineering education. The key stakeholders we are trying to simultaneously satisfy are the students; the industry recipients of our graduates; the software engineering community as parties interested in improved practices; and ourselves as instructors and teaching assistants. In order to satisfy the objectives or win conditions of these stakeholders, we have formed a strategic alliance with the USC Libraries to have software engineering student teams work with Library clients to define, develop, and transition USC digital library applications into operational use. This adds another set of key stakeholders: the Library clients of our class projects. This paper summarizes our experience in developing, conducting, and iterating the course. It concludes by evaluating the degree to which we have been able to meet the stakeholder-determined course objectives.

Not All CBS Are Created Equally: COTS-Intensive Project Types

COTS products affect development strategies and tactics, but not all CBS development efforts are equal. Based on our experiences with 20 large government and industry CBS projects assessed during our development of the COCOTS estimation model, and our hands-on experience with 52 small e-services CBS projects within USCs graduate level software engineering course, we have identified four distinct CBS activity areas: assessment intensive, tailoring intensive, glue-code intensive, and non-COTS intensive. The CBS activity type fundamentally affects the COTS related activity effort and project risks. In this work we define the three COTS activity intensive CBS types and discuss their strategic comparisons based on an empirical study of the spectrum of large and small CBS projects.

Simulating mixed agile and plan-based requirements prioritization strategies: proof-of-concept and practical implications

In this paper, we address the efficacy and pragmatics of mixing two primary strategies for requirements prioritization in order to incorporate the benefits of both plan-based (PB) and agile development methods while avoiding their drawbacks. As it is intractable to directly study the performance of strategies on real projects, we conducted a comprehensive empirically based simulation under a variety of conditions of requirements dynamism, project size, and duration. Simulation results suggest that a mixed strategy for requirements prioritization seems to work best in all but cost for typical levels of dynamism on average. Our findings also indicate that, as theorized, PB and agile strategies perform well within opposite extremes of dynamism. However, they do not outperform the mixed strategies even within their home grounds – that is large and complex systems with stable requirements for PB, and small and dynamic projects for agile methods. Given the unknown, unknowable, or variable nature of dynamism and the dramatic differences in effectiveness for agile and PB strategies under extreme development scenarios, a mixed strategy appears to yield the best results overall. We introduce two mixed strategies – simply adding cost–benefit (CB) to the agile approach, and a more sophisticated ‘hybrid’ (HY) approach that modulates development iteration size to maximize the expected CB for each iteration. We propose a step-by-step method to implement this HY strategy. We provide a structured analysis of the benefits and assumptions of agile and PB requirements prioritization methods (e.g., Pareto optimization), and outline a framework for analyzing and assessing the effectiveness of strategies including several new metrics. This research can furthermore serve as a framework for future validation of the proposed mixed strategies using actual software projects.

Educating software engineering students to manage risk

In 1996, the University of Southern California (USC) switched its core two-semester software engineering course from a hypothetical-project, homework-and-exam course based on the Bloom taxonomy of educational objectives (knowledge, comprehension, application, analysis, synthesis and evaluation). The revised course is a real-client team-project course based on the CRESST (Center for Research on Evaluation, Standards and Student Testing) model of learning objectives (content understanding, problem solving, collaboration, communication and self-regulation). We used the CRESST cognitive demands analysis to determine the necessary student skills required for software risk management and the other major project activities, and have been refining the approach over the last four years of experience, including revised versions for one-semester undergraduate and graduate project courses at Columbia University. This paper summarizes our experiences in evolving the risk management aspects of the project courses. These have helped us mature more general techniques, such as risk-driven specifications, domain-specific simplifier and complicator lists, and the SAIV (schedule as an independent variable) process model. The largely positive results in terms of review pass/fail rates, client evaluations, product adoption rates and hiring manager feedback are summarized as well.