Sally Godfrey

Defect categorization: making use of a decade of widely varying historical data

This paper describes our experience in aggregating a number of historical datasets containing inspection defect data using different categorization schemes. Our goal was to make use of the historical data by creating models to guide future development projects. We describe our approach to reconciling the different choices used in the historical datasets to categorize defects, and the challenges we faced. We also present a set of recommendations for others involved in classifying defects.

Using Sequence Diagrams to Detect Communication Problems between Systems

Many software systems are evolving complex system of systems (SoS) for which inter-system communication is both mission-critical and error-prone. Such communication problems ideally would be detected before deployment. In a NASA-supported Software Assurance Research Program (SARP) project, we are researching a new approach addressing such problems. In this paper, we show that problems in the communication between two systems can be detected by using sequence diagrams to model the planned communication and by comparing the planned sequence to the actual sequence. We identify different kinds of problems that can be addressed by modeling the planned sequence using different level of abstractions.

Connecting research and practice: an experience report on research infusion with software architecture visualization and evaluation

There are many technical challenges in ensuring high life-time quality of NASA’s systems. Some of NASA’s software-related challenges could potentially be addressed by the many powerful technologies that are being developed in software research laboratories. However, most such research technologies do not make the transition from the research lab to the software lab because research infusion and technology transfer is difficult. For example, there must be evidence that the technology works in the practitioner’s particular domain, and there must be a potential for great improvements and enhanced competitive edge for the practitioner, for such infusion to take place. NASA IV&V’s Research Infusion initiative strives to facilitate such infusion. In 2006, a research infusion project involving Johns Hopkins University Applied Physics Laboratory (JHU/APL) and the Fraunhofer Center for Experimental Software Engineering Maryland, was successfully completed infusing Fraunhofer’s software architecture visualization and evaluation (SAVE) tool. The infusion project helped improve JHU/APL’s software architecture and produced evidence that SAVE is applicable to software architecture problems in the aerospace domain, spawning a series of related research infusion projects. The project also led to the discovery of other needs that could not be addressed by current technologies and, therefore, spawned the research and development of a new technology that will be ready for infusion in the future. This paper describes the SAVE technology followed by a description of the infusion of SAVE at JHU/APL and the other projects that followed, as well as the newly started Dynamic SAVE research and development project. Lessons learned related to various aspects of research infusion conclude the paper.

Fully employing software inspections data

Software inspections provide a proven approach to quality assurance for software products of all kinds, including requirements, design, code, test plans, among others. Common to all inspections is the aim of finding and fixing defects as early as possible, and thereby providing cost savings by minimizing the amount of rework necessary later in the life cycle. Measurement data, such as the number and type of found defects and the effort spent by the inspection team, provide not only direct feedback about the software product to the project team, but are also valuable for process improvement activities. In this paper, we discuss NASA’s use of software inspections and the rich set of data that has resulted. In particular, we present results from analysis of inspection data that illustrate the benefits of fully utilizing that data for process improvement at several levels. Examining such data across multiple inspections or projects allows team members to monitor and trigger cross project improvements. Such improvements may focus on the software development processes of the whole organization as well as improvements to the applied inspection process itself.

Experiences in the implementation of a large Ada project

During the past several years, the Software Engineering Laboratory (SEL) of Goddard Space F l ight Center has been conducting an experiment in Ada [6],[8] to determine the cost ef fect iveness and feasibi l i ty of using Ada to develop f l ight dynamics software and to assess the effect of Ada on the f l ight dynamics environment. This experiment consists of near parallel developments of a dynamics simulator in both FORTRAN and Ada. A study team consisting of members from the SEL has monitored development progress and has collected data on both p ro jec ts throughout t h e i r development.

Developing an approach for analyzing and verifying system communication

Prominent characteristics of systems in the aerospace domain are that they are inherently complex, they must operate under tight resource constraints, and are often parts of a larger system of systems that must be reliable. These systems communicate with each other to exchange data and control information to together fulfill a larger task. In such a setup, the reliability of the communication channel plays a central role in the reliability of the entire system of systems and thus determines the success of fulfilling the larger task. Ensuring such a reliable communication is difficult due to several reasons: (1) the systems are developed independently by different teams at different locations, (2) the specification of the expected communication behavior is ambiguous, and (3) issues in the communication are often subtle and remain uncovered for a long time with the effect that bandwidth and other precious resources are wasted. We are proposing an approach called Dynamic Software Architecture Visualization and Evaluation (DynSAVE) to detect problems in the communication between systems by analyzing their communication behavior. The approach is divided into three main steps. The first step is the non-intrusive monitoring and recording of low level network traffic, the second step converts these raw communication records into meaningful messages, and the third step visualizes this abstracted information in such a way that issues can be detected. In this paper we discuss how the approach was applied to the Consultative Committee for Space Data Systems (CCSDS) File Delivery Protocol (CFDP), which is used for satellite communication by the JHU/APL Common Ground System. The approach has proven to be useful for understanding the communication behavior and uncovering subtle issues due to emerging system behaviors.

An Analysis Framework for Inter-system Interaction Behavior

Systems often collaborate to form a system-of-systems (SoS) and together fulfill some larger task. Correctness and performance issues in the interaction between participating systems are frequent occurrences and decrease the reliability of the entire SoS. We are currently developing an analysis framework to automatically compare a model of the desired interaction behavior (specification) to a model that is retrieved from the system execution and detect deviations between the two. The specification, the observed interaction behavior, and the evaluation result are presented in behavioral diagrams to be analyzed by the user.

Analyzing and detecting problems in Systems of Systems

Many software systems are evolving complex system of systems (SoS) for which intersystem communication is mission-critical. Evidence indicates that transmission failures and performance issues are not uncommon occurrences. In a NASA-supported Software Assurance Research Program (SARP) project, we are researching a new approach to addressing such problems. In this paper, we are presenting an approach for analyzing intersystem communications with the goal to uncover both transmission errors and performance problems. Our approach consists of a visualization and an evaluation component. While the visualization of the observed communication aims to facilitate understanding, the evaluation component automatically checks the conformance of an observed communication (actual) to a desired one (planned). The actual and the planned are represented as sequence diagrams. The evaluation algorithm checks the conformance of the actual to the planned diagram. We have applied our approach to the communication of aerospace systems and were successful in detecting and resolving even subtle and long existing transmission problems.