Hendrik Simon

Automatic test case generation for PLC programs using coverage metrics

This paper presents a method for automatic test case generation for PLC software following the IEC61131-3 standard. The core component is a model checker that iteratively creates program traces, each of them covering a part of the program in terms of a coverage metric. These test cases are translated into Structured Text, a programming languages defined in the IEC61131-3, to allow the execution on a soft-PLC or the actual hardware. Our approach is evaluated on a set of function blocks that are used in industry. We demonstrate that test cases can be automatically generated within few seconds in most cases.

Concolic test generation for PLC programs using coverage metrics

This paper presents a technique for fully automated generation of test cases for PLC programs adhering to the IEC 61131-3 standard. While previous methods strive for completeness and therefore struggle with the state explosion we pursue a symbolic execution based approach, dropping completeness but nevertheless achieving similar or even better results in practice. The core component is a symbolic execution engine which chooses the next state to execute, handles constraints emerging during the execution and derives respective test vectors leading to a state. To make for a high coverage of the generated tests, we adopt techniques from concolic testing, allow for use of heuristics to prioritise promising states but also merge states to alleviate the path explosion. We exploit peculiarities of PLC semantics to determine reasonable merge-points and unlike similar approaches even handle unreachable code. To examine the feasibility of our technique we evaluate it on function blocks used in industry.

Bone age assessment using support vector regression with smart class mapping

Bone age assessment on hand radiographs is a frequently and time consuming task to determine growth disturbances in human body. Recently, an automatic processing pipeline, combining content-based image retrieval and support vector regression (SVR), has been developed. This approach was evaluated based on 1,097 radiographs from the University of Southern California. Discretization of SVR continuous prediction to age classes has been done by (i) truncation. In this paper, we apply novel approaches in mapping of SVR continuous output values: (ii) rounding, where 0.5 is added to the values before truncation; (iii) curve, where a linear mapping curve is applied between the age classes, and (iv) age, where artificial age classes are not used at all. We evaluate these methods on the age range of 0-18 years, and 2-17 years for comparison with the commercial product BoneXpert that is using an active shape approach. Our methods reach root-mean-square (RMS) errors of 0.80, 0.76 and 0.73 years, respectively, which is slightly below the performance of the BoneXpert.

Bone Age Assessment Using Support Vector Machine Regression

Bone age assessment on hand radiographs is a costly and time consuming task in radiology. Recently, an automatic approach combining content-based image retrieval and support vector machines (SVM) has been developed. In this paper, we apply support vector regression (SVR) as a novel method, yielding a gain in performance. Our methods are designed to cope with the age range 0–18 years as compared to the age range 2–17 of the commercial product BoneXpert. On a standard data set from University of South Carolina, our approaches reach a rootmean-square error of 0.95 and 0.80 years for SVM and SVR, respectively. This is slightly below the performance of the commercial product using an active shape approach.

Mode-Aware Concolic Testing for PLC Software - Special Session "Formal Methods for the Design and Analysis of Automated Production Systems".

During the development of PLC software, standards usually require testing to consider certain coverage criteria. Since a manual generation of coverage tests is tedious and error-prone, automatic approaches as concolic testing are highly desirable. Approaches targeting non-reactive software usually cannot address their peculiarities, e. g. the cyclic execution combined with state-machine behaviour. Hence, we present a novel concolic testing technique to fill this gap. In particular, our technique utilises operation modes that typically describe the state machine semantics of single units in PLC programs, also called function blocks. This allows for guiding symbolic execution along paths that conform with the state-machine semantics and are likely to uncover new program behaviour. We show that our technique efficiently generates coverage tests for a variety of programs, outperforming existing approaches tailored to PLC software.

A priori test coverage estimation for automated production systems: Using generated behavior models for coverage calculation

Automated production systems need to be thoroughly tested. For this, the fully integrated system, consisting of software and hardware, is investigated for remaining bugs using system tests. Unlike unit tests, these tests involve human interaction with the system and are mostly performed manually, without any tool support for their simulation or adequacy examination. Thus, important functionality can be overlooked during testing, decreasing the overall system quality. Coverage measurement during testing can be a valuable support for assessing test adequacy, but requires instrumentation that might influence the real time behavior of the system. Further, test adequacy cannot be calculated until all tests have been executed. To tackle these problems, we propose an approach that aims at estimating coverage criteria prior to system test execution at the aPS and without instrumenting the code or defining hardware simulations. To this end, the nominal hardware behavior and human interaction models are derived from existing integration tests, enabling a priori estimation of the system test coverage via simulation of the software along with the deduced behavior.

Static analysis of Sequential Function Charts using abstract interpretation

In this work we present an abstract interpretation based static value set analysis method tailored for Sequential Function Charts (SFCs). Translation based approaches that transform SFCs into different presentations - and thus loosing the structure - have shown to be imprecise for this language. Our approach thus keeps the SFC structure and additionally provides value sets as entry-and exit information for each set of possibly active steps in the SFC. Further, analysis information for actions connected to the steps is provided, to allow for in-depth debugging. Beside value set information, our analysis does structural checks and, hence, is able to generate warnings for erroneous SFC behaviour, as specified in the IEC-61131-3 standard. As some semantic aspects for SFCs are left open in the standard specification, our analysis adopts the CODESYS semantics.