James D. Smith

ImpACT: An Alternative to Technology Readiness Levels for Commercial-Off-The-Shelf (COTS) Software

The use of Technology Readiness Levels (TRLs) as a tool in assessing acquisition and development program risk has steadily increased over the past several years. There is considerable evidence to support the utility of using TRLs as part of a risk assessment, but there are some difficulties in using TRLs with software, especially Commercial-Off-The-Shelf (COTS) software technology and products. These difficulties take several forms, including “blurring-together” various aspects of COTS technology/product readiness; the absence of some important aspects of readiness; COTS product “decay;” and no mechanism to account for changes in the relative importance of the contributors to technology/product readiness over time. This paper briefly examines these issues, and proposes an alternate methodology—ImpACT—for assessing COTS software technology and product readiness which considers these factors.

Implementing Large-Scale COTS Reengineering within the United States Department of Defense

This paper reports on empirical research into the organizational and process changes necessary to implement large-scale reengineering using COTS. The research goal is to identify how these changes can be effected within the context of the existing United States Department of Defense acquisition management, requirements management, and planning, programming, and budgeting systems. While this effort is still in its early stages, initial results indicate that there are significant challenges in developing and defending budget inputs to support an evolutionary COTS-oriented acquisition model. At the same time, recent changes in acquisition constraints and other mitigating circumstances are being exploited to improve the likelihood of success.

Systems-of-Systems Programmatics: Guidelines for Program Managers

Large, complex systems development has always been challenging, even when the “only” things a program manager had to worry about were cost, schedule, and performance for their program. The rampant growth in system complexity, greatly expanded use of joint and combined operations, and the emergence of such operational concepts as network-centric operations, has led to the prevalence of interoperable systems of systems as the preferred solution to providing operational capability. As difficult as program management is within a single program, it is significantly more complex in the context of a system of systems. The absence of any clear guidance for identifying and resolving conflicts between a systemcentric and system of systems perspectives is at the heart of several notable systems of systems failures. This paper provides some simple guidance to help program managers: the “laws of programmatics” for systems-of-systems.

Best paper award 2005: Using earned value management for COTS-Based systems: issues and recommendations

Earned value management (EVM) has long been used by organizations to plan, monitor, and control the development and evolution of custom developed systems. EVM was developed for managing such projects, and assumes a waterfall development model. COTS-based systems (CBS), on the other hand, are formed and evolved through the selection and composition of pre-existing, off-the-shelf packages or components with potentially some number of custom components. Experience indicates that a spiral or iterative development process is a key to success with CBS. While EVM has been applied to CBS projects, the results have not been uniformly satisfying. This paper explores the fundamental challenges in using EVM with CBS, and proposes adaptations to some of the principals of EVM to render it more suitable for CBS development.