Peter C. Mehlitz

Combining unit-level symbolic execution and system-level concrete execution for testing nasa software

We describe an approach to testing complex safety critical software that combines unit-level symbolic execution and system-level concrete execution for generating test cases that satisfy user-specified testing criteria. We have developed Symbolic Java PathFinder, a symbolic execution framework that implements a non-standard bytecode interpreter on top of the Java PathFinder model checking tool. The framework propagates the symbolic information via attributes associated with the program data. Furthermore, we use two techniques that leverage system-level concrete program executions to gather information about a units input to improve the precision of the unit-level test case generation. We applied our approach to testing a prototype NASA flight software component. Our analysis helped discover a serious bug that resulted in design changes to the software. Although we give our presentation in the context of a NASA project, we believe that our work is relevant for other critical systems that require thorough testing.

Symbolic PathFinder: integrating symbolic execution with model checking for Java bytecode analysis

Symbolic PathFinder (SPF) is a software analysis tool that combines symbolic execution with model checking for automated test case generation and error detection in Java bytecode programs. In SPF, programs are executed on symbolic inputs representing multiple concrete inputs and the values of program variables are represented by expressions over those symbolic inputs. Constraints over these expressions are generated from the analysis of different paths through the program. The constraints are solved with off-the-shelf solvers to determine path feasibility and to generate test inputs. Model checking is used to explore different symbolic program executions, to systematically handle aliasing in the input data structures, and to analyze the multithreading present in the code. SPF incorporates techniques for handling input data structures, strings, and native calls to external libraries, as well as for solving complex mathematical constraints. We describe the tool and its application at NASA, in academia, and in industry.

Model Based Analysis and Test Generation for Flight Software

We describe a framework for model-based analysis and test case generation in the context of a heterogeneous model-based development paradigm that uses and combines MathWorks and UML 2.0 models and the associated code generation tools. This paradigm poses novel challenges to analysis and test case generation that, to the best of our knowledge, have not been addressed before. The framework is based on a common intermediate representation for different modeling formalisms and leverages and extends model checking and symbolic execution tools for model analysis and test case generation, respectively. We discuss the application of our framework to software models for a NASA flight mission.

Model checking real time java using java pathfinder

The Real Time Specification for Java (RTSJ) is an augmentation of Java for real time applications of various degrees of hardness. The central features of RTSJ are real time threads; user defined schedulers; asynchronous events, handlers, and control transfers; a priority inheritance based default scheduler; non-heap memory areas such as immortal and scoped, and non-heap real time threads whose execution is not impeded by garbage collection. The Robust Software Systems group at NASA Ames Research Center has Java PathFinder(JPF) under development, a Java model checker. JPF at its core is a state exploring JVM which can examine alternative paths in a Java program (e.g., via backtracking) by trying all nondeterministic choices, including thread scheduling order. This paper describes our implementation of an RTSJ profile (subset) in JPF, including requirements, design decisions, and current implementation status. Two examples are analyzed: jobs on a multiprogramming operating system, and a complex resource contention example involving autonomous vehicles crossing an intersection. The utility of JPF in finding logic and timing errors is illustrated, and the remaining challenges in supporting all of RTSJ are assessed.

JPF-AWT: Model checking GUI applications

Verification of Graphical User Interface (GUI) applications presents many challenges. GUI applications are open systems that are driven by user events. Verification of such applications by means of model checking therefore requires a user model in order to close the state space.

Trust Your Model - Verifying Aerospace System Models with Java Pathfinder

Model Driven Development (MDD) is rapidly becoming a mainstream practice for the development of complex aerospace systems. UML has emerged as the de facto standard for modeling languages, supporting a wide range of modeling aspects and refinement levels. As a consequence, models can easily become too complex for manual verification and simple static analysis.

Design for Verification with Dynamic Assertions

Completed design and implementation are often regarded as prerequisites of any verification. While recent development methods establish testability as a design criterion, there is no corresponding design support for other verification methods like model checking and static analysis. Since these methods have inherent scalability problems, their application becomes more difficult where it is most needed - for complex systems. Our design-for-verification (D4V) approach attempts to close this gap using a variety of techniques, such as design patterns, APIs and source annotations. This paper presents a overview of D4V, and introduces dynamic assertions as one of the proposed D4V techniques. Dynamic assertions are dedicated, non-intrusive check objects that are dynamically activated, evaluated and deactivated via assertions of their target objects. Since these check objects can have their own state, they can be used to verify a broad range of properties. Properties can be expressed in the target programming language, and checked in a testing environment. In addition, dynamic assertions can be configured via call contexts, making them suitable for connector-specific verification of component based systems

Verification and validation of air traffic systems: Tactical separation assurance

The expected future increase in air traffic requires the development of innovative algorithms and software systems to automate safety critical functions such as separation assurance - the task of maintaining a safe distance between aircraft at all times. Extensive verification and validation (V&V) of such functions will be crucial for the acceptance of new air traffic management systems. This paper reports on work performed at the NASA Ames Research Center. We discuss how advanced V&V technologies can be used to create robust software prototypes for air traffic control software, and how conformance of production code with such prototypes can be assured. We present preliminary results of V&V efforts for a prototype of the Tactical Separation Assisted Flight Environment system (TSAFE).

Expecting the Unexpected: Radiation Hardened Software

Radiation induced Single Event Effects (SEEs) are a serious problem for spacecraft flight software, potentially leading to a complete loss of mission. Conventional risk mitigation has been focused on hardware, leading to slow, expensive and outdated on-board computing devices, increased power consumption and launch mass. Our approach is to look at SEEs from a software perspective, and to explicitly design flight software so that it can detect and correct the majority of SEES. Radiation hardened flight software will reduce the significant residual residual risk for critical missions and flight phases, and enable more use of inexpensive and fast COTS hardware.

Extending JPF to verify distributed systems

This paper presents our work on model checking distributed applications. We refer to distributed applications as a collection of communicating processes, regardless of their physical locations and the communication means. Our work targets applications written in Java. It relies on the multiprocess support included in Java Pathfinder (JPF) version 7 which allow for verifying the bytecode of distributed applications. The basic support for distributed applications in JPF does not account for communication between processes. In this work, we address this limitation. The work is implemented as a JPF extension which models inter- process communication (IPC) mechanisms. It uses a form of partial order reduction (POR) to explore all possible executions of a distributed Java application. Moreover, our approach provides a functionality to check the given distributed application against possible network failures which can occur at the operating system or the hardware layer.