Ramsay Taylor

Inferring extended finite state machine models from software executions

The ability to reverse-engineer models of software behaviour is valuable for a wide range of software maintenance, validation and verification tasks. Current reverse-engineering techniques focus either on control-specific behaviour (e.g., in the form of Finite State Machines), or data-specific behaviour (e.g., as pre / post-conditions or invariants). However, typical software behaviour is usually a product of the two; models must combine both aspects to fully represent the software’s operation. Extended Finite State Machines (EFSMs) provide such a model. Although attempts have been made to infer EFSMs, these have been problematic. The models inferred by these techniques can be non-deterministic, the inference algorithms can be inflexible, and only applicable to traces with specific characteristics. This paper presents a novel EFSM inference technique that addresses the problems of inflexibility and non-determinism. It also adapts an experimental technique from the field of Machine Learning to evaluate EFSM inference techniques, and applies it to three diverse software systems.

Inferring Extended Finite State Machine models from software executions

The ability to reverse-engineer models of software behaviour is valuable for a wide range of software maintenance, validation and verification tasks. Current reverse-engineering techniques focus either on control-specific behaviour (e.g. in the form of Finite State Machines), or data-specific behaviour (e.g. as pre/post-conditions or invariants). However, typical software behaviour is usually a product of the two; models must combine both aspects to fully represent the softwares operation. Extended Finite State Machines (EFSMs) provide such a model. Although attempts have been made to infer EFSMs, these have been problematic. The models inferred by these techniques can be non deterministic, the inference algorithms can be inflexible, and only applicable to traces with specific characteristics. This paper presents a novel EFSM inference technique that addresses the problems of inflexibility and non determinism. It also adapts an experimental technique from the field of Machine Learning to evaluate EFSM inference techniques, and applies it to two open-source software projects.

Using Behaviour Inference to Optimise Regression Test Sets

Where a software component is updated or replaced regression testing is required. Regression test sets can contain considerable redundancy. This is especially true in the case where no formal regression test set exists and the new component must instead be compared against patterns of behaviour derived from in-use log data from the previous version. Previous work has applied search-based techniques such as Genetic Algorithms to minimise test sets, but these relied on code coverage metrics to select test cases. Recent work has demonstrated the advantage of behaviour inference as a test adequacy metric. This paper presents a multi-objective search-based technique that uses behaviour inference as the fitness metric. The resulting test sets are evaluated using mutation testing and it is demonstrated that a considerably reduced test set can be found that retains all of the fault finding capability of the complete set.

Automatic Inference of Erlang Module Behaviour

Previous work has shown the benefits of using grammar inference techniques to infer models of software behaviour for systems whose specifications are not available. However, this inference has required considerable direction from an expert user who needs to have familiarity with the system’s operation, and must be actively involved in the inference process. This paper presents an approach that can be applied automatically to infer a model of the behaviour of Erlang modules with respect to their presented interface. It integrates the automated learning system StateChum with the automated refactoring tool Wrangler to allow both interface discovery and behaviour inference to proceed without human involvement.

Smother: an MC/DC analysis tool for Erlang

This paper discusses an MC/DC analysis tool built for the Erlang programming language. Code coverage metrics are one way to measure the adequacy of a test suite, however, despite widespread industrial use in business-critical software, the only coverage metric readily available for Erlang is statement coverage, provided by the Cover tool that is distributed by the OTP standard libraries. An alternative to statement cover is Multiple Condition/Decision Coverage (MC/DC) -- one of the most extensive coverage metrics, which is mandated by many software safety standards. Here we describe the application of MC/DC analysis to Erlang programs, and include an extension to traditional MC/DC analysis that applies the underlying philosophy to the pattern matching decision structures of functional languages. We have implemented the approach in the Smother tool that we also describe here as well as its use by our industrial project partners.

Synapse: automatic behaviour inference and implementation comparison for Erlang

In the open environment of the world wide web, it is natural that there will be multiple providers of services, and that these service provisions — both specifications and implementations — will evolve. This multiplicity gives the user of these services a set of questions about how to choose between different providers, as well as how these choices work in an evolving environment. The challenge, therefore, is to concisely represent to the user the behaviour of a particular implementation, and the differences between this implementation and alternative versions. Inferred models of software behaviour – and automatically derived and graphically presented comparisons between them – serve to support effective decision making in situations where there are competing implementations of requirements. In this paper we use state machine models as the abstract representation of the behaviour of an implementation, and using these we build a tool by which one can visualise in an intuitive manner both the initial implementation and the differences between alternative versions. In this paper we describe our tool Synapse which implements this functionality by means of our grammar inference tool StateChum and a model-differencing algorithm. We describe the main functionality of Synapse, and demonstrate its usage by comparing different implementations of an example program from the existing literature.

Separation of Z Operations

Machine code and assembly language programs are structured using various branches and decision points, but between these they contain blocks of instructions that are simply sequentially composed. Most work on formal program analysis has focused on the behavior of the branch points -- primarily because composing the blocks of sequential code to determine their overal effect on the system is often intellectually trivial. This processs is also computationaly simple, but it is not computationally trivial. The aim of this work is to produce a system of rules that can be efficiently implemented and allow us to determine the overal behaviour of sequentially composed operations. To identify those sequential compositions that are trivial we will use techniques inspired by Separation logic[2, 1]. Separation logic itself is a very general, abstract collection of higher order logic statements but the simple observation at the heart of separation logic will be used: if two operations refer to completely disjoint parts of the state space they can be reasoned about independently.

Formalising Extended Finite State Machine Transition Merging

Model inference from system traces, e.g. for analysing legacy components or generating security tests for distributed components, is a common problem. Extended Finite State Machine (EFSM) models, managing an internal data state as a set of registers, are particularly well suited for capturing the behaviour of stateful components however existing inference techniques for (E)FSMs lack the ability to infer the internal state and its update functions.

Choreography-Based Analysis of Distributed Message Passing Programs

We report on the analysis of gen_server, a popular Erlang library to build client-server applications. Our analysis uses a tool based on choreographic models. We discuss how, once the library has been modelled in terms of communicating finite state machines, an automated analysis can be used to detect potential communication errors. The results of our analysis suggest how to properly use gen_server in order to guarantee the absence of communication errors.

mu2: A Refactoring-Based Mutation Testing Framework for Erlang

We present a mutation testing framework for the Erlang functional programming language. Mutation testing evaluates a test set by mutating the original System Under Test SUT and measuring the test sets ability to detect the change. Designing mutation operators can be difficult, since they must modify the original program in a way that is both semantically significant, and a realistic simulation of a potential fault either a fault with the system in its real context, or a common programmer error. The principal contribution of this work is the mu2 framework, which leverages the Wrangler refactoring API to allow users to specify their own mutation operators. The framework makes it possible to quickly and clearly define mutation operators that can have complex and subtle effects on the SUT. This allows users to define domain-specific operators that can simulate faults that are of particular relevance to their project, rather than relying on standard operators. The mutation testing framework was evaluated in an industrial setting and compared to code coverage test adequacy metrics. It was found to be a valuable compliment to code coverage techniques, since it was able to uncover additional testing limitations that could not be easily identified by coverage alone.