Mirko Conrad

Testing-based translation validation of generated code in the context of IEC 61508

Production code generation with Model-Based Design has successfully replaced manual coding across various industries and application domains. Furthermore, code generated from executable graphical models is increasingly being deployed in high-integrity embedded applications.To validate the model-to-code translation process, generated software components and its precursory stages (i.e. models) should be subjected to an appropriate combination of quality assurance measures. For high-integrity applications, compliance with safety standards such as IEC 61508 needs to be demonstrated as well.On principle, translation validation of generated code could be carried out in the same manner as for manually written code. However, this would not leverage the advantages of Model-Based Design and w.r.t. process efficiency this would leave something to be desired. Therefore, engineering methods and tools for effective and efficient translation validation of generated code are highly desirable. As a step towards this goal, a workflow for verification and validation of models and generated code will be proposed and as far as possible mapped onto the objectives of IEC 61508-3. A cornerstone of this workflow is testing for numerical equivalence between models and generated code.

Software Tool Qualification According to ISO 26262

International standards that define requirements for the development of safety-related systems typically also define required confidence levels for the software tools used to develop those systems. The standards define—to a greater or lesser extent— procedures to classify, validate, certify, or qualify tools. To date, there is no common approach for tool validation, certification, and qualification across safety standards. Different standards attach different levels of importance to tool validation, certification, and qualification, and suggest different approaches to gain confidence in the tools used. With ISO 26262 ―Road Vehicles Functional Safety‖ on the horizon, automotive software practitioners will need to understand and implement the new software tool classification and qualification requirements laid out in this standard. ISO 26262 is the adaptation of IEC 61508 to comply with needs specific to the application sector of electric / electronic systems (E/E systems) within road vehicles. This adaptation applies to all activities during the safety lifecycle of systems composed of electrical, electronic, and software elements that provide safety-related functions. Clause 11 of ISO 26262-8 provides guidance on software tool classification and qualification. The clause applies, if the safety lifecycle incorporates using a software tool, such that (1) activities or tasks required by ISO 26262 rely on the correct functioning of that tool, and (2) relevant outputs of that tool are not fully examined or verified. This paper describes the tool classification and qualification approach of ISO/FDIS 26262 and summarizes the authors’ firsthand experiences with implementing this approach for development and verification tools. ISO/FDIS 26262 TOOL QUALIFICATION APPROACH This section provides a brief overview on the tool qualification approach as outlined in the final draft standard. International standards that define requirements for the development of safety-related systems typically also define the required level of confidence for the software tools used to develop these systems. To varying degrees, these standards define procedures to classify, validate, certify, or qualify tools. To date, there is no common approach for tool validation, certification, and qualification that can be applied to all safety standards. Different standards attach different levels of importance to these objectives and suggest different approaches to gain confidence in the tools used [CMR10]. ISO/FDIS 26262 ―Road Vehicles Functional Safety‖ [ISO/DIS 26262] is the adaptation of IEC 61508 [IEC 61508] to comply with needs specific to the application sector of electric / electronic systems (E/E systems) within road vehicles. This adaptation applies to all activities during the safety lifecycle of systems composed of electrical, electronic, and software elements that provide safety-related functions. As per ISO/FDIS 26262-8, 11, a software tool (or a software tool chain) used in the safety lifecycle, in a way that (1) activities or tasks required by ISO 26262 rely on the correct functioning of that tool, and (2) relevant outputs of that tool are not fully examined or verified, need to be assessed, classified, and potentially qualified. ISO/FDIS 26262-8 provides criteria to determine the required level

Verification, Validation, and Test with Model-Based Design

Model-Based Design with automatic code generation has long been employed for rapid prototyping and is increasing being used for mass production deployment. With the focus on production usage, comes the need to implement a comprehensive V&V strategy involving models and resulting code. A main principal of Model-Based Design is that generated code should behave like the simulation model. It should also be possible to verify that the model or design was fully implemented in the code. As a result, the transformation of models into generated code must be done in a way that facilitates traceability between the model and code. Also automated tests should be performed to determine that the code executes properly in its final software and hardware environments. For example in a typical commercial vehicle application, the control algorithm and plant model are simulated together in a system simulation environment. Once the system model satisfies the requirements, the control model is checked to ensure that it has been fully exercised or covered. Once checked, code is then generated for production applications. The code is analyzed, tested, and compared to the original model results. Common model and code verification activities include software-in-the-loop (SIL), processor-in-the-loop (PIL), and hardware-in-the-loop (HIL) testing. In addition to functional results, it is important especially for highintegrity systems, that the model and code have been checked and assessed to known standards. This paper describes recent advances in verification, validation, and test technologies involving Model-Based Design with production code generation. INTRODUCTION TO MODEL-BASED DESIGN A model represents a dynamic system whose response at any time is a mathematical function based on its inputs, current state, and current time. Historically, system engineers have used block diagrams as shown in Figure 1 to create models and design algorithms within numerous engineering areas such as Feedback Control and Signal Processing. In recent years, graphical modeling environments consisting of block diagrams and state machines have been used to analyze, simulate, prototype, specify, and deploy software algorithms within a variety of embedded systems and applications. ModelBased Design refers to the use of models and modeling environments as the basis for embedded system development. Figure 1: Feedback controller model. Systems developed using Model-Based Design include: • Commercial vehicle electronics • Power plant regulators • Digital motor controllers Model-Based Design is used throughout the system development life cycle and provides design flows that include continuous verification and validation of requirements, designs, and implementations. This approach is important for formal software processes such as IEC 61508 [1] and for other projects seeking error prevention and early error detection. The main development activities that occur during Model-Based Design include: • Modeling and simulation • Rapid prototyping • Production code generation and integration Verification and validation (V&V) occurs continuously during Model-Based Design and includes many activities. This article focus on two key activities: model checking and processor-in-the-loop testing. MODEL CHECKING An important group of model V&V activities comprise different static analyses and checking tools. Model advisors check a model for conditions and configuration settings that can result in inaccurate or slow simulation, problematic maintenance, or generation of inefficient production code. Reports are generated that list identified suboptimal conditions and settings. Advice is provided suggesting better modeling approaches and settings. There are several types of checks. Basic Model Checks: Automated model advice is provided using basic checks, requirements consistency checks, and industry model standard checks. Basic checks range from support for updating the model to be compatible with the current product release version, to identifying unconnected lines and ports, to checking the root model interfaces. To invoke the advisor, select the checks and then run them as shown in the leftand right-hand sides of Figure 2, respectively. Figure 2: Model advisor basic checks invocation After performing the checks, results are displayed as shown in Figure 3. Hyperlinks are provided in the report, automating navigating to the dialog or menu where the setting can be adjusted based on the reported advice. The basic model checks should be performed and reported deviations considered before other quality assurance measures such as peer reviews or industry model standard checks are done. Figure 3: Model advisor basic checks results Requirements Consistency Checks: If the model is linked with requirements in third-party requirement management tools or databases, these checks identify inconsistent, missing, or changed requirements. Requirements consistency checks can also identify and repair requirements with missing documents and inconsistent requirements descriptions. See Figure 4. Figure 4: Requirements consistency checks Modeling Standards Checks: Many projects use inhouse or industry specific software and modeling development standards. Model checks have already been developed for some industry standards including: • DO-178B • MAAB • IEC 61508 DO-178B is an aerospace standard that will not be discussed here. MathWorks Automotive Advisory Board (MAAB) checks facilitate designing and troubleshooting models for automotive applications. A new version of MAAB was made available in 2007, MAAB v2.0. MAAB checks include: • Prohibited blocks inside controllers • Port and signal name mismatches • Unconnected signals IEC 61508 is a generic, application-independent standard for electrical / electronic / programmable electronic safety-related systems (E/E/PES) that is supposed to ease the development of sector-specific norms for E/E/PES. It is applied transitionally in the development of E/E/PES in those areas for which a domain-specific norm does not yet exist. IEC 61508-3 is concerned with the requirements for software development. IEC 61508 can be considered as a prescriptive standard, which provides detailed lists of techniques and measures with recommendations. IEC 61508 model checks analyze the model and report on items such as model usage, model metrics, and configuration management information as shown in Figure 5. Figure 5: IEC 61508 checks report Model Complexity Measurement allows one to measure the complexity of the entire model as well as the individual subsystems. Cyclomatic model complexity is a measure of the structural complexity of a model. It is calculated with the IEC 61508 checks and approximates the McCabe complexity measure for code generated from the model. Model complexity measurement helps to achieve a modular approach on the model level and especially to maintain an appropriate module size limit. Finally, if the built-in checks are not sufficient, a model advisor API is available that facilitates the development of custom rule checks by engineers using Model-Based Design. PROCESSOR-IN-THE-LOOP TESTING Simulation of models is an early verification and validation (V&V) technique. Testing models via simulation is a more rigorous approach than the ad-hoc simulation runs used in early algorithm development. Model testing requires a systematic approach to test case creation and execution. Special blocks, such as signal builders and assertions, facilitate this type of rigorous test procedure. New capabilities for V&V on models now exist such as structural coverage analysis and test case generation. In-the-loop testing techniques allow one to reuse the model test cases and test environment for execution with the production application during various stages of integration. Software-in-the-loop (SIL) testing involves executing the production code for the controller within the modeling environment for non-real-time execution with the plant model and interaction with the user. The code executes on the same host platform that is used by the modeling environment. A code wrapper of the generated code provides the interface between the simulation and the generated code. See Figure 6. Figure 6: Software-in-the-loop testing. For hardware-in-the-loop (HIL) testing, the code is generated just for the plant model. It runs on a highly deterministic, real-time computer. Sophisticated signal conditioning and power electronics are needed to properly stimulate the ECU inputs (sensors) and receive the ECU outputs (actuator commands). Whereas rapid prototyping is often a development or design activity, HIL serves as a final lab test phase before final system integration and field tests commence. See Figure 7. Note that an on-target rapid prototyping and production code example using Model-Based Design based on case study by John Deere was presented at the SAE Commercial Vehicle conference in 2007 [2]. Figure 7: Hardware-in-the-loop testing. Processor-in-the-loop (PIL) testing occurs after SIL but before HIL testing. As with SIL, PIL exercises the production code for the controller in non-real-time. However the code executes on the actual embedded processor or an instruction set simulator, using the embedded cross-compiler. Thus, it verifies the embedded object code functional behavior. With PIL, the model does a single calculation iteration. The inputs are calculated and passed to a PIL block. The PIL block serves as a conduit and passes the model inputs to the code running on the embedded microprocessor, or emulated processor if an instruction set simulator is used. Once the target processor receives the model inputs, it executes a single time step and computes the output data. The outputs are then passed back to model using the PIL block. The model then continues to simulate while the target processor waits for new inputs. See Figure 8 for a PIL example using an Instruction Set Simulator (ISS). F

Adoption of Modeling Standards as a Part of Enterprise-Wide Deployment

The successful deployment of modeling packages across large organizations depends on the adoption of a uniform set of modeling standards within the organization. Further, the automation of modeling standards enforcement greatly facilitates their adoption. This paper examines several case studies of large-scale deployment of Model-Based Design and the areas where standards enforcement automation was employed. The paper further examines several areas where automation could be employed to improve the deployment process.