Steven L. Cornford

Experiences using Visualization Techniques to Present Requirements, Risks to Them, and Options for Risk Mitigation

For several years we have been employing a riskbased decision process to guide development and application of advanced technologies, and for research and technology portfolio planning. The process is supported by custom software, in which visualization plays an important role. During requirements gathering, visualization is used to help scrutinize the status (completeness, extent) of the information. During decision making based on the gathered information, visualization is used to help decisionmakers understand the space of options and their consequences. In this paper we summarize the visualization capabilities that we have employed, indicating when and how they have proven useful.

Scalable mechanisms for requirements interaction management

Capturing requirements, and managing tradeoffs among them, are critical yet complex activities. Well-designed computerized tools can effectively support these activities. A key challenge in construction of these support tools is how to scale them to handle a large volume of information. Particularly crucial are the ways in which large numbers of requirements and their interrelationships are presented to users. They need to be able to zoom in and out through the space of information so as to be able to see the big picture, and to locate and focus on specific details when needed. This paper describes a harmonious combination of techniques that support such scalability. The techniques have been embodied in a NASA tool, DDP, for defect detection and prevention. They have been exercised in uses of this tool for requirements/risk tradeoffs, and the use of this tool to capture institutional knowledge-bases of information.

Design and Development Assessment

An assessment methodology is described and illustrated. This methodology separates assessment into the following phases: (1) elicitation of requirements; (2) elicitation of failure modes and their impact (risk of loss of requirements); (3) elicitation of failure mode mitigations and their effectiveness (degree of reduction of failure modes); and (4) calculation of outstanding risk taking the mitigations into account. This methodology, with accompanying tool support, has been applied to assist in planning the engineering development of advanced technologies. Design assessment features prominently in these applications. The overall approach is also applicable to development assessment (of the development process to be followed to implement the design). Both design and development assessments are demonstrated on hypothetical scenarios based on the workshops TRMCS case study. TRMCS information has been entered into the assessment support tool, and serves as illustration throughout.

Quantitative methods for maturing and infusing advanced spacecraft technology

Today, the pace of spacecraft development has accelerated. Some of the pressure to accelerate comes from reduced budgets and government mandates to improve efficiency and rely on a greater extent on commercial systems and practices. More importantly, however, the accelerating pace is driven by new opportunities in science and technology. Future systems must respond to requirements to deliver higher resolution, greater responsiveness, and the increased need for interoperability. Additionally, NASA is being asked to field systems in less time and less cost without sacrificing mission reliability. To meet these often contradictory requirements, project managers must find new ways to infuse advanced technologies into spacecraft. As a result, most of NASAs science spacecraft incorporate an unprecedented amount of new technology. Candidate missions must identify new systems and components, analyze how the risks associated with new designs are to be mitigated, and identify methods for transferring resultant technology within and outside of NASA. Technology infusion in any application is a complex process. Incorporating an unproven new design into the development of an operational system presents significant cost, schedule, and technical risk. Historically, developers of operational systems have been cautious when incorporating new technology. New designs are often matured independent of operational systems and brought online only when they have proven their mettle. Demonstrator or precursor missions are used to test new designs before the commitment to a new technology is made. This is usually a very slow process; it can take years or decades to move a technology from the laboratory to fully operational status. The result is a stepwise evolution of capability - an approach that minimizes risk. Sometimes requirements evolve at a pace where demonstrator programs are not possible. When this occurs, project managers must accept the task of integrating complex new technologies into the mainstream development of an operational spacecraft. The tools used to assist the project manager with this task are surprisingly fragile. Techniques for measuring the readiness of a technology, for example, are highly qualitative. The importance of the language and culture surrounding the transfer of a technology from the laboratory to the application program is also generally underestimated. This paper focuses on these issues, examining the practice of maturing technology with special emphasis on methods that improve the integration of advanced designs during the development of operational spacecraft systems. Simplified practices are presented that could improve the accuracy and reduce the risk associated with estimating the readiness of a technology for use in space applications. While This work focuses on the challenges associated with building spacecraft, the authors believe the practices presented here could have broader application to other markets.

Risk based decision tool for space exploration missions

This paper presents an approach and corresponding tool to assess and analyze the risks involved in a mission during the pre-phase A design process. This approach is based on creating a risk template for each subsystem expert involved in the mission design process and defining appropriate interactions between the templates.

Combining the best attributes of qualitative and quantitative risk management tool support

Tools have been developed that support risk identification and management activities during different phases of a project lifecycle. For the earlier stages of the project lifecycle, a tool for the qualitative identification and manipulation of risk and risk mitigation data was developed. For the later stages of the lifecycle, a separate tool for the quantitative manipulation of requirements, risk and risk mitigation data was developed. These two tools were then combined into a single tool. The combination of these qualitative and quantitative risk management tools is the focus of this paper. The combination was first envisioned as simply a convenience, ensuring that the results from the early lifecycle risk management would flow smoothly into the later lifecycle management. However, it was found that the combination led to the possibility of extending many of the capabilities of each tool into the other tools phases. The net result is a combination that exhibits the best attributes of both qualitative and quantitative risk management tool support.

An integrated approach to risk assessment for concurrent design

This paper describes an approach to risk assessment and analysis suited to the early phase, concurrent design of a space mission. The approach integrates an agile, multi-user risk collection tool, a more in-depth risk analysis tool, and repositories of risk information. A JPL developed tool, named RAP, is used for collecting expert opinions about risk from designers involved in the concurrent design of a space mission. Another in-house developed risk assessment tool, named DDP, is used for the analysis. The risk model in DDP is generated by integrating the information collected in RAP, other design information available from the design sessions, and possibly risk and failure information available from other libraries and databases. The underlying software infrastructure for this transfer of information is based on translating the RAP data to XML, which in turn is interpreted by DDP and translated to DDP data. The advantage of the integration is its combination of the strengths of the components, while avoiding the need to construct a single monolithic all-encompassing tool and process. We briefly describe each of the RAP and DDP tools and demonstrate the integrated approach with an example generated from a study conducted at the Project Design Center (TeamX) at JPL.

Risk-based analysis and decision making in multi-disciplinary environments

A risk-based decision-making process conceived of and developed at JPL and NASA, has been used to help plan and guide novel technology applications for use on spacecraft. These applications exemplify key challenges inherent in multidisciplinary design of novel technologies deployed in mission-critical settings: 1) Cross-disciplinary concerns are numerous (e.g., spacecraft involve navigation, propulsion, telecommunications). These concerns are cross-coupled and interact in multiple ways (e.g., electromagnetic interference, heat transfer). 2) Time and budget pressures constrain development, operational resources constrain the resulting system (e.g., mass, volume, power). 3) Spacecraft are critical systems that must operate correctly the first time in only partially understood environments, with no chance for repair. 4) Past experience provides only a partial guide: New mission concepts are enhanced and enabled by new technologies, for which past experience is lacking. The decision-making process rests on quantitative assessments of the relationships between three classes of information-objectives (the things the system is to accomplish and constraints on its operation and development), risks (whose occurrence detracts from objectives), and mitigations (options for reducing the likelihood and/or severity of risks). The process successfully guides experts to pool their knowledge, using custom-built software to support information gathering and decision-making.Copyright

Model based mission assurance: NASA's assurance future

Model Based Systems Engineering (MBSE) is seeing increased application in planning and design of NASAs missions. This suggests the question: what will be the corresponding practice of Model Based Mission Assurance (MBMA)? Contemporaneously, NASAs Office of Safety and Mission Assurance (OSMA) is evaluating a new objectives-based approach to standards to ensure that the Safety and Mission Assurance disciplines and programs are addressing the challenges of NASAs changing missions, acquisition and engineering practices, and technology. MBSE is a prominent example of a changing engineering practice. We use NASAs objectives-based strategy for Reliability and Maintainability as a means to examine how MBSE will affect assurance. We surveyed MBSE literature to look specifically for these affects, and find a variety of them discussed (some are anticipated, some are reported from applications to date). Predominantly these apply to the early stages of design, although there are also extrapolations of how MBSE practices will have benefits for testing phases. As the effort to develop MBMA continues, it will need to clearly and unambiguously establish the roles of uncertainty and risk in the system model. This will enable a variety of uncertainty-based analyses to be performed much more rapidly than ever before and has the promise to increase the integration of CRM (Continuous Risk Management) and PRA (Probabilistic Risk Analyses) even more fully into the project development life cycle. Various views and viewpoints will be required for assurance disciplines, and an over-arching viewpoint will then be able to more completely characterize the state of the project/program as well as (possibly) enabling the safety case approach for overall risk awareness and communication.

Requirements based system level risk modeling

Space exploration missions are often characterized by multiple phases and each phase in turn satisfies some objective or requirement. The success of the mission is measured by the degree to which these requirements are satisfied. Missions either aim to demonstrate a new technology, or to obtain new science data or a combination of both of these. During the mission design process, numerous trade studies are conducted between cost, performance and risk. At a very high level, the goal is to maximize the probability of achieving the most science return (or demonstrating the most technology) at the least possible cost. We consider the problem of maximizing this probability by quantifying the degree of importance of each requirement and its probability of being satisfied. The probability of a requirement being satisfied, in turn, is assessed by finding the aggregate of the probability of all the possible events that could prevent it from being satisfied. We assume a complete list of the requirements, the relevant risk elements and their probability of occurrence and the quantified effect of the risk elements on the requirements. In order to assess the degree to which each requirement is satisfied, we need to determine the effect of the various risk elements on the requirement. The complexity arises due to the fact that various risk elements that effect a requirement in question are not necessarily independent. Moreover, in order to compute the weighted average of the requirements, its important to take into consideration their dependencies. Therefore we carefully define the relationships between the elements within each category (intra-category) and the elements between the two different categories of risk and requirements (inter-category).