B. Craig Meyers

Assessing the Quality of Large, Software-Intensive Systems: A Case Study

This paper presents a case study in carrying out an audit of a large, softwareintensive system. We discuss our experience in structuring the team for obtaining maximum effectiveness under a short deadline. We also discuss the goals of an audit, the methods of gathering and assimilating information, and specific lines of inquiry to be followed. We present observations on our approach in light of our experience and feedback from the customer. In the past decade, as engineers have attempted to build software-intensive systems of a scale not dreamed of heretofore, there have been extraordinary successes and failures. Those projects that have failed have often been spectacular and highly visible [3], particularly those commissioned with public money. Such failures do not happen all at once; like Brooks’ admonition that schedules slip one day at a time [2], failures happen incrementally. The symptoms of a failing project range from the subtle (a customer’s vague feelings of uneasiness) to the ridiculous (the vendor slips the schedule for the eighth time and promises that another

Systems-of-Systems Programmatics: Guidelines for Program Managers

30 million will fix everything). A project that has passed the “failure in progress” stage and gone on to full-fledged meltdown can be spotted by one sure symptom: the funding authority curtails payment and severely slows development. When that happens, the obvious question is asked by every involved party: “What now?” The answer is often an audit. This paper summarizes the experience of an audit undertaken by the Software Engineering Institute (SEI) in the summer of 1994 to examine a large, highly visible development effort exhibiting the meltdown symptom suggested above. The customer was a government agency in the process of procuring a large software-intensive system from a major contractor. The audit team included the authors of this paper, as well as members from other organizations. Members of the team had extensive backgrounds and expertise in software engineering, in large systems development,

Proceedings of the System of Systems Interoperability Workshop

Large, complex systems development has always been challenging, even when the “only” things a program manager had to worry about were cost, schedule, and performance for their program. The rampant growth in system complexity, greatly expanded use of joint and combined operations, and the emergence of such operational concepts as network-centric operations, has led to the prevalence of interoperable systems of systems as the preferred solution to providing operational capability. As difficult as program management is within a single program, it is significantly more complex in the context of a system of systems. The absence of any clear guidance for identifying and resolving conflicts between a systemcentric and system of systems perspectives is at the heart of several notable systems of systems failures. This paper provides some simple guidance to help program managers: the “laws of programmatics” for systems-of-systems.