John Penix

Formal analysis of a space-craft controller using SPIN

The paper documents an application of the finite state model checker SPIN to formally analyze a multithreaded plan execution module. The plan execution module is one component of NASAs New Millennium Remote Agent, an artificial intelligence-based spacecraft control system architecture which launched in October of 1998 as part of the DEEP SPACE 1 mission. The bottom layer of the plan execution module architecture is a domain specific language, named ESL (Executive Support Language), implemented as an extension to multithreaded COMMON LISP. ESL supports the construction of reactive control mechanisms for autonomous robots and spacecraft. For the case study, we translated the ESL services for managing interacting parallel goal-and-event driven processes into the PROMELA input language of SPIN. A total of five previously undiscovered concurrency errors were identified within the implementation of ESL. According to the Remote Agent programming team, the effort has had a major impact, locating errors that would not have been located otherwise and, in one case, identifying a major design flaw. In fact, in a different part of the system, a concurrency bug identical to one discovered by this study escaped testing and caused a deadlock during an in-flight experiment, 96 million kilometers from Earth. The work additionally motivated the introduction of procedural abstraction in terms of inline procedures into SPIN.

Verification of time partitioning in the DEOS scheduler kernel

The paper describes an experiment to use the Spin model checking system to support automated verification of time partitioning in the Honeywell DEOS real time scheduling kernel. The goal of the experiment was to investigate whether model checking could be used to find a subtle implementation error that was originally discovered and fixed during the standard formal review process. To conduct the experiment, a core slice of the DEOS scheduling kernel was first translated without abstraction from C++ into Promela (the input language for Spin). We constructed an abstract test-driver environment and carefully introduced several abstractions into the system to support verification. Several experiments were run to attempt to verify that the system implementation adhered to the critical time partitioning requirements. During these experiments, the known error was rediscovered in the time partitioning implementation. We believe this case study provides several insights into how to develop cost-effective methods and tools to support the software design and implementation review process.

Efficient Specification-Based Component Retrieval

In this paper we present a mechanism for making specification-based component retrieval more efficient by limiting the amount of theorem proving required at query time. This is done by using a classification scheme to reduce the number of specification matching proofs that are required to process a query. Components are classified by assigning features that correspond to necessary conditions implied by the component specifications. We show how this method of feature assignment can be used to approximate reusability relationships between queries and library components. The set of possible classification features are formally defined, permitting automation of the classification process. The classification process itself is made efficient by using a specialized theorem proving tactic to prove feature implication. The retrieval mechanism was implemented and evaluated experimentally using a library of list manipulation components. The results indicate a better response time than existing formal approaches. The approach provides higher levels of consistency and automation than informal methods, with comparable retrieval performance.

Verifying Time Partitioning in the DEOS Scheduling Kernel

This paper describes an experiment to use the Spin model checking system to support automated verification of time partitioning in the Honeywell DEOS real-time scheduling kernel. The goal of the experiment was to investigate whether model checking with minimal abstraction could be used to find a subtle implementation error that was originally discovered and fixed during the standard formal review process. The experiment involved translating a core slice of the DEOS scheduling kernel from C++ into Promela, constructing an abstract “test-driver” environment and carefully introducing several abstractions into the system to support verification. Attempted verification of several properties related to time-partitioning led to the rediscovery of the known error in the implementation. The case study indicated several limitations in existing tools to support model checking of software. The most difficult task in the original DEOS experiment was constructing an adequate environment to close the system for verification. The fidelity of the environment was of crucial importance for achieving meaningful results during model checking. In this paper, we describe the initial environment modeling effort and a follow-on experiment with using semi-automated environment generation methods. Program abstraction techniques were also critical for enabling verification of DEOS. We describe an implementation scheme for predicate abstraction, an approach based on abstract interpretation, which was developed to support DEOS verification.

Amphion/NAV: deductive synthesis of state estimation software

Previous work on domain-specific deductive program synthesis described the Amphion/NAIF system for generating Fortran code from high-level graphical specifications describing problems in space system geometry. Amphion/NAIF specifications describe input-output functions that compute geometric quantities (e.g., the distance between two planets at a point in time, or the time when a radio communication path between a spacecraft and earth is occluded) by composing together Fortran subroutines from the NAIF subroutine library developed at the Jet Propulsion Laboratory. In essence, Amphion/NAIF synthesizes code for glueing together the NAIF components in a way such that the generated code implements the specification, with a concurrently generated proof that this implementation is correct. Amphion/NAIF demonstrated the success of domain-specific deductive program synthesis and is still in use today within the space science community. However, a number of questions remained open that we will attempt to answer in this paper.

Verification and Validation of AI Systems that Control Deep-Space Spacecraft

NASA is developing technology for the next generation of deep-space robotic spacecraft, with the aim of enabling new types of missions and radically reducing costs. One technology under development is Autonomy: highly capable spacecraft that perform significant scientific missions with little or no commanding and monitoring from Earth. Artificial Intelligence provides a basis for autonomy technology, but raises issues of verification and validation outside the scope of empirical testing technology for conventionally commanded spacecraft. This paper describes research towards extending formal methods verification techniques for the mathematical verification of AI systems controlling deep-space spacecraft. This paper first overviews a planned space mission called DS-1 which includes an AI-based autonomy experiment. It then describes part of this AI system called the executive, which includes an ‘intelligent’ operating system based on goal-oriented constructs. The paper then describes focused research on applying and extending model-checking technology for verifying both the core services of the executive and the concurrent task programs run by the executive.

Design for Verification with Dynamic Assertions

Completed design and implementation are often regarded as prerequisites of any verification. While recent development methods establish testability as a design criterion, there is no corresponding design support for other verification methods like model checking and static analysis. Since these methods have inherent scalability problems, their application becomes more difficult where it is most needed - for complex systems. Our design-for-verification (D4V) approach attempts to close this gap using a variety of techniques, such as design patterns, APIs and source annotations. This paper presents a overview of D4V, and introduces dynamic assertions as one of the proposed D4V techniques. Dynamic assertions are dedicated, non-intrusive check objects that are dynamically activated, evaluated and deactivated via assertions of their target objects. Since these check objects can have their own state, they can be used to verify a broad range of properties. Properties can be expressed in the target programming language, and checked in a testing environment. In addition, dynamic assertions can be configured via call contexts, making them suitable for connector-specific verification of component based systems

The second international workshop on automated program analysis, testing and verification

Program analysis, testing and verification are key techniques for building confidence in and increasing the quality of software systems. Such activities typically cost upwards of 50% of total development costs. Automation aims to allow both reduced costs and more thorough analysis, testing and verification and is vital to keep pace with increasing software complexity.

Deductive synthesis of event-based software architectures

Describes the application of the Rebound (REuse Based On UNDerstanding) framework to synthesize event-based software architectures. Within Rebound, deductive techniques are used to select components and wrap them for integration. The framework guides the selection of wrappers based on the problem specification and the components that are available for reuse. The wrapper specifications are used to generate matching conditions for component retrieval, creating a retrieval for adaptation scenario. The output of the synthesis activity is mapped to an event-based JavaBeans architecture.

Compositional specification of software architecture

This paper describes our experience using parameterized algebraic specifications to model properties of software architectures. The goal is to model the decomposition of requirements independent of the style used to implement the architecture. We begin by providing an overview of the role of architecture specification in software development. We then describe how architecture specifications are build up from component and connector specifi(ations and give an overview of insights gained from a case study used to validate the method.