Debra L. Emmons

Using Historical NASA Cost and Schedule Growth to Set Future Program and Project Reserve Guidelines

By looking at historical schedule and cost growth, the past can be used to established reserve guidelines for future missions. This paper looks at recent NASA cost and schedule growth history, categorizes the reasons for growth, isolates growth due to external programmatic reasons versus internal technical reasons, assesses relationships for causality and provides guidance for the proper cost and schedule reserves to be carried at both the Program and Project levels. Mission cost and schedule growth history from both planetary and Earth-orbiting programs, such as Mars Exploration, Discovery, Medium and Small Explorer, Earth System Science Pathfinder, New Millennium and others, are investigated. The reserve guidelines developed are compared to industry standard guidelines and rules of thumb to determine if these standard practices continue to be valid.

Optimism in early conceptual designs and its effect on cost and schedule growth: An update

This paper updates a previous effort investigating how the evolution of initial concept designs are related to the cost and schedule growth of missions.12 The paper shows examples of the design evolution, and associated cost and schedule growth, for twenty historical NASA missions. Issues behind the cost and schedule growth of missions are varied, but in part may be attributed to systems that have changed substantially from those examined at initial concept through to launch. Historical resource growth is investigated for a variety of missions and mission types to provide guidelines and lessons learned to be used during the initial conceptual design stage for future missions. The data developed for the paper should help both project managers and cost and schedule estimators to develop more robust estimates earlier in the design process.

Explanation of change (EoC) study: Approach and findings

This study investigated thirty historical NASA science missions to explain the cost change experienced. The study included investigation of historical milestone and monthly status report documentation followed by interviews with key project personnel. Based on the information collected, the reasons for cost change were binned, at the highest level, into four separate categories: NASA External, Project External, Internal Planning, and Internal Execution. The results identified that roughly a third of the change is outside of the projects control, a third is due to assumptions made in project planning, and a third is due to the inherent difficulty of building highly complex, one-of-a-kind, cutting edge, Earth and space science missions. The different causes for growth are discussed.

Phase E Cost Analysis for NASA Science Missions

Phase E is the long awaited payoff for the development of NASA science missions as it encompasses both the mission operations to collect the science data required for success as well as the data analysis to turn the data into a final useable product. Phase E growth, however, can be problematic as increased Phase E cost can take funding away from the development of new science missions. There are two kinds of Phase E cost growth: 1) cost growth due to underestimation of Phase E cost and 2) cost growth due to extension of mission life for additional science. A recent study examined 20 NASA science missions and showed that Phase E lifecycle cost (LCC) growth, as planned from Phase B start to launch, on average exceeded 30%. The research for this paper has extended that study with expanded data collection and analysis and also addressed the actual Phase E cost as compared to LCC estimates. This additional research has found within the 20 missions LCC growth to be on average 49% to the end of mission life. The 20 mission data set also was examined for trends during the operational phase based on mission characteristics such as: Directed versus Competed acquisition, Earth orbiting versus Planetary, and number of instruments. Competed missions, Planetary missions, and missions with more than 5 instruments demonstrated the highest cost growth in Phase E. Data from an additional 26 missions was also collected to perform an expanded analysis of actual operations cost for trends based on science theme, mission class and cost per instrument. Larger missions were found to have a greater annual Phase E cost as well as greater cost per instrument. Heliophysics missions were found to have lower annual mission and per instrument cost than any other science theme. Astrophysics missions were found to have the highest per instrument cost of all science themes. Planetary missions were found to have lower cost per instrument than Astrophysics and Earth Science missions but higher annual costs due to a larger number of instruments. A rule of thumb was also derived that showed flagship mission annual operating costs are generally two to three times higher than non-flagship missions. Finally, a rough order of magnitude cost estimating relationship was derived using the 46 mission data set to provide some ability to estimate the annual Phase E cost given mission class, science theme and the number of instruments on a mission.

Quantitative approach to independent schedule estimates of NASA Science Missions

This paper examines a multi-level quantitative schedule analysis and estimating approach similar to one that is used when evaluating proposal costs, and cost risks. The Aerospace Corporation has developed a process for performing independent schedule estimates (ISE), which utilizes an analogy-based schedule assessment methodology, permitting greater insight into mission development times. The authors have used this tool in many proposal evaluations and independent assessments when estimating planetary and earth-orbiting mission schedules. Specific examples of how this tool has been used to evaluate overall development time and interim development milestone plans will be presented. Additionally, data will be shown for a past application of this tool in evaluating the schedule risk of a mission, and how the data compares to schedule reserves guidelines. This paper illustrates the progression in schedule estimating, and the importance of equal treatment as a programmatic discipline on par with cost estimating.

Affordability assessments to support strategic planning and decisions at NASA

NASAs goal to provide a human presence in space while contributing to the knowledge of the science of Earth, other planets, the solar system, and the universe requires a diverse set of scientific and exploration missions.12 Successful development and execution of these portfolios depends upon a sustainable and affordable long-term strategy. To provide a basis for an adequate annual funding profile to fit within NASAs budget, an objective affordability assessment of a missions technical baseline, associated risks, and cost and schedule is fundamental. For example, NASAs Science Mission Directorate lists close to 100 planned science missions for launch within a 20-year window in the Agency Mission Planning Manifest maintained by the NASA Office of Program Analysis and Evaluation. On the human space flight side, the NASA Exploration Systems Mission Directorate (ESMD) is in the process of trying to develop several multi-billion dollar systems (e.g. Orion capsule, Ares launch vehicle, etc.) in parallel to meet an initial operational capability within the next 10 years. While portfolios such as these might be defined such that the estimated life-cycle costs are covered by the expected budget, there are a variety of factors that might initiate a need to re-plan the portfolio, including budget cuts, new mission content, new Administration direction, different reserves strategies, and cost and schedule overruns. This can be seen in the 2009 Review of U.S. Human Space Flight (HSF) Plans Committee initiated by the White House, which examined the ongoing U.S. human space flight plans and programs, as well as alternatives to the current program of record. Using an affordability analysis process developed by The Aerospace Corporation, assessments were performed using the Sand Chart Tool (SCT) to support these portfolio analyses. SCT was developed to provide insight into the behavior of large portfolios and strategic issues associated with re-plans and execution of the specified content. It can run in two modes: Planning Mode, which applies a temporal stretch algorithm to fit portfolio costs to the budget (by stretching out schedule as necessary), and Evaluation Mode, which performs a probabilistic Monte Carlo analysis to determine the robustness of a given plan after considering cost-growth risks, schedule linkages, and other factors. This paper will 1) look at why affordability analysis is important in managing complex/high-value Agency portfolios, 2) provide a general overview of SCT and the affordability analysis process, and 3) examine recent examples of affordability assessments performed on the ESMD portfolio in support of the Review of HSF Committee.

Explanation of change (EoC) study: Considerations and implementation challenges

This paper discusses the implementation of considerations resulting from a study investigating the cost change experienced by historical NASA science missions. The study investigated historical milestone and monthly status report documentation followed by interviews with key project personnel. The reasons for cost change were binned as being external to NASA, external to the project and internal to the project relative to the projects planning and execution. Based on the results of the binning process and the synthesis of project meetings and interviews, nine considerations were made with the objective to decrease the potential for cost change in future missions. Although no one “magic bullet” consideration was discovered, the considerations taken as a whole should help reduce cost and schedule change in future NASA missions.

A dynamic deep space communication link analysis tool for the deep space network (DSN)

A dynamic deep space communication link analysis tool is described in this paper. This tool, developed by The Aerospace Corporation, provides the capability to analyze coverage and data throughput for communication links between a spacecraft and the Jet Propulsion Laboratorys (JPL) Deep Space Network (DSN). The tool determines the link margin and data throughput over time during the trajectory of a spacecraft as monitored by the DSN. The analysis takes into account several dynamic effects in calculating the link budget, including transmit and receive antenna gains and space loss. This tool consists of a database derived from JPLs DSMS Telecommunications Link Design Handbook, from which relevant link budget parameters are extracted for all of the 70-m and 34-m antennas. Some of the relevant DSN link budget parameters include receive and transmit gain, as well as system noise temperature. Since these parameters vary, depending on operating conditions, the database takes into account various conditions, such as cumulative weather distribution, month of operation, elevation angle, particular DSN antenna being used, and uplink and downlink frequency. By utilizing the corresponding DSN link budget parameters and the spacecrafts trajectory and link budget parameters, one is able to determine the amount of coverage and data throughput versus time

Contribution of schedule delays to cost growth: How to make peace with a marching army

Numerous research papers have shown that cost and schedule growth are interrelated for NASA space science missions. Although there has shown to be a strong correlation of cost growth with schedule growth, it is unclear what percentage of cost growth is caused by schedule growth and how schedule growth can be controlled. This paper attempts to quantify this percentage by looking at historical data and show detailed examples of how schedule growth influences cost growth. The paper also addresses a methodology to show an alternate approach for assessing and setting a robust baseline schedule and use schedule performance metrics to help assess if the project is performing to plan. Finally, recommendations are presented to help control schedule growth in order to minimize cost growth for NASA space science missions.

Findings and Considerations from the NASA Explanation of Change Study

The Explanation of Change This study investigated thirty historical NASA science missions to explain the cost change experienced. The study included investigation of historical milestone and monthly status report documentation followed by interviews with project personnel. Key events from the project’s development were identified and the associated cost change assessed. Based on the information collected, the reasons for cost change were binned, at the highest level, into four separate categories: NASA External, Project External, Internal Planning, and Internal Execution. The results identified that roughly a third of the change is outside of the project’s control, a third is due to assumptions made in project planning, and a third is due to the inherent difficulty of building highly complex, one-of-a-kind, cutting edge, Earth and space science missions. Additionally, this paper discusses the implementation of considerations. Based on the results of the binning process and the synthesis of project meetings and interviews, nine considerations were developed with the objective to decrease the potential for cost change in future missions. Although no one “magic bullet” consideration was discovered, the considerations taken as a whole should help reduce cost and schedule change in future NASA missions.