Christoph Torens

Certification and Software Verification Considerations for Autonomous Unmanned Aircraft

Software verification for highly automatic unmanned aerial vehicles is not only a problem itself, it is furthermore constrained by certification standards and regulatory rules. These, however, are themselves still under development. As a top-level view, the current status of unmanned aerial vehicle verification, certification, and regulation is addressed and corresponding challenges are discussed. From a low-level view, this work presents the processes and tools that were established for the software development, verification, and validation of the unmanned rotorcraft software testbed ARTIS. Large efforts have been put into the software verification process to cope with the growing complexity of the autonomous system and the validation of the software behavior. Automated tests drive the development of the mission planning, mission management, and sensor fusion systems. High-level behavior is tested by complex simulation scenarios. To connect the aforementioned top- and low-level views, a comparison betwee...

Formal Requirements and Model-Checking for V&V Automation of a RPAS Mission Management System

The aerospace domain is a safety-critical domain. Therefore software has to be of high quality. Software development and testing in safety-critical domains is regulated by standards, such as DO-178B and DO-178C for the aerospace domain. However, the test approach in these standards is stochastic in nature, which means that errors in the code are tried to be identified by a large set of test cases. The trust in such software products and the overall quality is achieved by traceability to requirements and strict coverage criteria as well as general conformance to processes and rigorous documentation, but any absence of errors can usually not be proved. On the other hand, the use of formal methods for software, which is now standardized by the introduction of the Supplement DO-333, promises a true validation of certain safety-critical properties. This paper shows how formal requirements and model-checking were introduced to our test strategy for an automated planning and guidance software module. The requirements for an existing software artifact were elicited and then formalized into Linear Temporal Logic and Computation Tree Logic, which are two derivatives of temporal logic. A formal model for the software was developed and NuSMV was used as a model-checking tool to analyze the requirements in regards to the model. Furthermore, the corresponding certifcation considerations for this formal method are discussed according to the relevant standards.

Towards Intelligent System Health Management using Runtime Monitoring

System health management is an important feature of autonomy, enhancing consistency checks, overall system robustness and even some degree of self-awareness. Seemingly unrelated, debugging and analysis of such complex systems is another challenge during development that should not be underrated. We propose that the so-called runtime monitoring of relevant properties and system requirements is a viable technique to support both aforementioned concepts. A suitable monitoring approach for a cyber-physical system has to be efficient and capable of supervising various specifications, possibly relating different data sources and data history. We present a formal approach for log-analysis and monitoring for the DLR ARTIS framework using the stream-based specification language LOLA, currently developed at Saarland University, for the runtime monitoring of formal specifications. We have evaluated this approach by specifying relevant properties as LOLA stream equations. While we have identified a number of possible improvements in the specification language, we have demonstrated, even with the current language, that online and offline monitoring of relevant properties is indeed possible and gives engineers a powerful tool for debugging as well as implementing health management concepts.

Automated Verification and Validation of an Onboard Mission Planning and Execution System for UAVs

Automated mission planning is one of the key components of an autonomous UAV. The software validation and verification for such decisional autonomy functions is a challenging problem. The software component includes the closed loop vehicle control as well as environment perception. As this is a safety-critical software component, it is important that this software works safely and within projected performance boundaries. This paper discusses the verification and validation approach for the sampling-based mission planner of an unmanned rotorcraft. A layered test strategy is presented, which utilizes different testing methods that complement and build upon each other. Its strengths as well as the possible improvement directions of this approach are discussed. An emphasis is given on automated software-in-the-loop simulations. The approach additionally utilizes benchmarks to assess the implementation performance and real time properties. Finally, to be able to assess the overall test quality, a set of different scalable test abstractions (SUT size, test effort, level of automation, coverage, test complexity and feedback time) is used to analyze the presented strategy.

Software Verification Considerations for the ARTIS Unmanned Rotorcraft

This work presents the processes and tools that were installed and developed to validate the ARTIS software and achieve compliance of an unmanned rotorcraft testbed with corresponding standards. A brief introduction to the autonomous guidance and navigation capabilities of our unmanned aircraft is given in order to illustrate the software complexity and practical integration challenges introduced by such functionalities. Our software development process is presented which is aimed at a practical balance between exhaustive testing and the rapid integration of new features. It features a greedy integration procedure that is aimed at the preservation of existing features and performances. Automated tests drive the development of our mission planning, mission management and sensor fusion systems. New research code can be integrated such that side effects on existing systems are minimized.

Stream Runtime Monitoring on UAS

Unmanned Aircraft Systems (UAS) with autonomous decision-making capabilities are of increasing interest for a wide area of applications such as logistics and disaster recovery. In order to ensure the correct behavior of the system and to recognize hazardous situations or system faults, we applied stream runtime monitoring techniques within the DLR ARTIS (Autonomous Research Testbed for Intelligent System) family of unmanned aircraft. We present our experience from specification elicitation, instrumentation, offline log-file analysis, and online monitoring on the flight computer on a test rig. The debugging and health management support through stream runtime monitoring techniques have proven highly beneficial for system design and development. At the same time, the project has identified usability improvements to the specification language, and has influenced the design of the language.

Formal Monitoring of Risk-based Geo-fences

This work proposes a novel risk-based geo-fencing approach. Multiple, adjacent geo-fences are each assigned a specific risk level. An UAS with a low confidence level would be restricted to an area with no or very low risk. However, an UAS with a high confidence level could be allowed to exit such a geo-fence and cross over to another geo-fence with a different risk level. This approach enables different scenarios for the use of geo-fences and requirements for entering, flying, and leaving geo-fences of specific risk. Moreover, using runtime monitoring, the UAS can be assigned a dynamic confidence level, which represents the current and prior system health, system performance, or possibly environmental conditions. This results in a structured methodology for the independent assurance of geo-fences corresponding to specific and possibly dynamic confidence levels of an UAS. The geo-fencing problem as well as the risk-based approach is formalized in the specification language Lola, which provides a concise unambiguous mathematical foundation. Specified properties can be checked at runtime due to automatically generated monitors. This results in a trustworthy implementation, possibly enabling cost-effective UAS operation in accordance to upcoming regulations. Finally, a simulation is used as proof of concept to show the feasibility of the presented approach.

Towards Generic Requirements and Models for Automated Mission Tasks with RPAS

This work proposes an approach to create a set of generic mission automation requirements as well as a generic execution model for unmanned aircraft. Even at the highest requirement level, regulations for unmanned aircraft are dissimilar across different countries and thus regulation still comprises lack of requirements. Due to the comparably young domain of todays unmanned vehicles, the scientific and commercial communities have difficulties to define adequate safety requirements and system requirements. This is especially the case for highly automated software functions. There is no straightforward transition from existing automation in manned aviation’s pilot assistance systems into a timely and spatially contained autonomous function. Additionally pilot abilities must be transformed into software functions. However, for unmanned aircraft certification with today’s standards the corresponding validation and verification activities are expected to require a lot of resource intensive software activities like manual inspection efforts. To support requirement validation at early development stages with respect to acceptability, we present an approach that is based on a generic autonomous systems taxonomy of capabilities. Therefore, we propose a shared set of generic high-level requirements as well as a generic mission execution model for automated mission task elements that can be performed by many types of unmanned aircraft. This way, our approach tries to support the unmanned aircraft community by a common understanding of unmanned aircraft capabilities by using existing inherited terms and concepts. Moreover, the derived requirements and models are meant to facilitate the design of automated functions that are already in use by academia and industry research today.

Steps Towards Scalable and Modularized Flight Software for Unmanned Aircraft Systems

Unmanned aircraft (UA) applications impose a variety of computing tasks on the on-board computer system. From a research perspective, it is often more convenient to evaluate algorithms on bigger aircraft as they are capable of lifting heavier loads and thus more powerful computational units. On the other hand, smaller systems are often less expensive and operation is less restricted in many countries. This paper thus presents a conceptual design for flight software that can be evaluated on the UA of convenient size. The integration effort required to transfer the algorithm to different sized UA is significantly reduced. This scalability is achieved by using exchangeable payload modules and a flexible process distribution on different processing units. The presented approach is discussed using the example of the flight software of a 14 kg unmanned helicopter and an equivalent of 1.5 kg. The proof of concept is shown by means of flight performance in a hardware-in-the-loop simulation.

Considerations of Artificial Intelligence Safety Engineering for Unmanned Aircraft.

Unmanned aircraft systems promise to be useful for a multitude of applications such as cargo transport and disaster recovery. The research on increased autonomous decision-making capabilities is therefore rapidly growing and advancing. However, the safe use, certification, and airspace integration for unmanned aircraft in a broad fashion is still unclear. Standards for development and verification of manned aircraft are either only partially applicable or resulting safety and verification efforts are unrealistic in practice due to the higher level of autonomy required by unmanned aircraft. Machine learning techniques are hard to interpret for a human and their outcome is strongly dependent on the training data. This work presents the current certification practices in unmanned aviation in the context of autonomy and artificial intelligence. Specifically, the recently introduced categories of unmanned aircraft systems and the specific operation risk assessment are described, which provide means for flight permission not solely focusing on the aircraft but also incorporating the target operation. Exemplary, we show how the specific operation risk assessment might be used as an enabler for hard-to-certify techniques by taking the operation into account during system design.