Matt Webster

Model checking agent programming languages

In this paper we describe a verification system for multi-agent programs. This is the first comprehensive approach to the verification of programs developed using programming languages based on the BDI (belief-desire-intention) model of agency. In particular, we have developed a specific layer of abstraction, sitting between the underlying verification system and the agent programming language, that maps the semantics of agent programs into the relevant model-checking framework. Crucially, this abstraction layer is both flexible and extensible; not only can a variety of different agent programming languages be implemented and verified, but even heterogeneous multi-agent programs can be captured semantically. In addition to describing this layer, and the semantic mapping inherent within it, we describe how the underlying model-checker is driven and how agent properties are checked. We also present several examples showing how the system can be used. As this is the first system of its kind, it is relatively slow, so we also indicate further work that needs to be tackled to improve performance.

Verifying autonomous systems

Exploring autonomous systems and the agents that control them.

Formal methods for the certification of autonomous unmanned aircraft systems

In this paper we assess the feasibility of using formal methods, and model checking in particular, for the certification of Unmanned Aircraft Systems (UAS) within civil airspace. We begin by modelling a basic UAS control system in PROMELA, and verify it against a selected subset of the CAAs Rules of the Air using the SPIN model checker. Next we build a more advanced UAS control system using the autonomous agent language Gwendolen, and verify it against the small subset of the Rules of the Air using the agent model checker AJPF. We introduce more advanced autonomy into the UAS agent and show that this too can be verified. Finally we compare and contrast the various approaches, discuss the paths towards full certification, and present directions for future research.

Detection of metamorphic computer viruses using algebraic specification

This paper describes a new approach towards the detection of metamorphic computer viruses through the algebraic specification of an assembly language. Metamorphic computer viruses are computer viruses that apply a variety of syntax-mutating, behaviour-preserving metamorphoses to their code in order to defend themselves against static analysis based detection methods. An overview of these metamorphoses is given. Then, in order to identify behaviourally equivalent instruction sequences, the syntax and semantics of a subset of the IA-32 assembly language instruction set is specified formally using OBJ – an algebraic specification formalism and theorem prover based on order-sorted equational logic. The concepts of equivalence and semi-equivalence are given formally, and a means of proving equivalence from semi-equivalence is given. The OBJ specification is shown to be useful for proving the equivalence or semi-equivalence of IA-32 instruction sequences by applying reductions – sequences of equational rewrites in OBJ. These proof methods are then applied to fragments of two different metamorphic computer viruses, Win95/Bistro and Win9x.Zmorph.A, in order to prove their (semi-)equivalence. Finally, the application of these methods to the detection of metamorphic computer viruses in general is discussed.

Toward Reliable Autonomous Robotic Assistants Through Formal Verification: A Case Study

It is essential for robots working in close proximity to people to be both safe and trustworthy. We present a case study on formal verification for a high-level planner/scheduler for the Care-O-bot, an autonomous personal robotic assistant. We describe how a model of the Care-O-bot and its environment was developed using Brahms, a multiagent workflow language. Formal verification was then carried out by automatically translating this model to the input language of an existing model checker. Four sample properties based on system requirements were verified. We then refined the environment model three times to increase its accuracy and the persuasiveness of the formal verification results. The first refinement uses a user activity log based on real-life experiments, but is deterministic. The second refinement uses the activities from the user activity log nondeterministically. The third refinement uses “conjoined activities” based on an observation that many user activities can overlap. The four samples properties were verified for each refinement of the environment model. Finally, we discuss the approach of environment model refinement with respect to this case study.

Detection of metamorphic and virtualization-based malware using algebraic specification

We present an overview of the latest developments in the detection of metamorphic and virtualization-based malware using an algebraic specification of the Intel 64 assembly programming language. After giving an overview of related work, we describe the development of a specification of a subset of the Intel 64 instruction set in Maude, an advanced formal algebraic specification tool. We develop the technique of metamorphic malware detection based on equivalence-in-context so that it is applicable to imperative programming languages in general, and we give two detailed examples of how this might be used in a practical setting to detect metamorphic malware. We discuss the application of these techniques within anti-virus software, and give a proof-of-concept system for defeating detection counter-measures used by virtualization-based malware, which is based on our Maude specification of Intel 64. Finally, we compare formal and informal approaches to malware detection, and give some directions for future research.

Towards Certification of Autonomous Unmanned Aircraft Using Formal Model Checking and Simulation

Unmanned aircraft are expected to increase in use in civil applications over the coming years, particularly for the so-called dull, dirty and dangerous missions. Unmanned aircraft will undoubtedly require some form of autonomy in order to ensure safe operations: communications failure could render a completely human-piloted unmanned aircraft dangerous to other airspace users. In order to be used for civil applications, unmanned aircraft must gain government regulatory approval in a process known as certification. This paper presents an approach to gathering evidence for certification of autonomous unmanned aircraft based on formal methods (in particular formal model checking) and flight simulation. In particular, rational agent-based autonomous systems are examined. Rational agents for unmanned aircraft can be model checked using implicit models of the aircraft’s physical environment specified in terms of the different sensor inputs the autonomous system may receive. However this presents difficulties when trying to model check the agents relative to physical quantities such as those found in regulatory documents like the CAA Air Navigation Order. It is shown how this can be remedied using an explicit physical model of the environment within the model checker, and how this explicit physical model can itself be verified through comparison with flight simulations. To conclude, an overview of related and future work is given.

Certification of a Civil UAS: A Virtual Engineering Approach

The use of Unmanned Autonomous Systems (UAS) is becoming an increasingly routine activity in military theatres of operation, particularly for the oft-cited ‘dull, dangerous and dirty’ missions. There is growing acceptance that UAS will find similar utility within the corresponding civilian missions and beyond. UAS technologies are maturing rapidly but the associated regulations to allow open access to civilian airspace are yet to be fully formulated. Current UK practice is therefore to allow UAS operation only in segregated airspace (airspace denied to all other potential users) or in non-segregated airspace but restricted to line-of-sight operations, below 400ft only. There is therefore a growing need to develop a means by which UAS can operate alongside existing airspace users, in all classes of nonsegregated UK airspace. The University of Liverpools Virtual engineering Centre, is developing tools and techniques that will allow both industry and regulators to establish a ‘design for certification’ ethos within the supply chain where safety-critical software and hardware is required. The processes will include requirements capture and validation phases, as well as a means of testing and evaluating whole UAS/sub-system virtual prototypes, with a view to being able to demonstrate compliance with the relevant airworthiness codes as early as possible in the design cycle.

Ethical Choice in Unforeseen Circumstances

For autonomous systems to be allowed to share environments with people, their manufacturers need to guarantee that the system behaves within acceptable legal, but also ethical, limits. Formal verification has been used to test if a system behaves within specified legal limits. This paper proposes an ethical extension to a rational agent controlling an Unmanned Aircraft(UA). The resulting agent is able to distinguish among possible plans and execute the most ethical choice it has. We implement a prototype and verify that when an agent does behave unethically, it does so because no more-ethical possibility is available.

Reproducer Classification Using the Theory of Affordances

We present a new approach to the classification of reproducers based on an affordance theory of reproductive behaviour. First, we define the notion of an affordance as an action that one object in an environment can perform for another object. Using this ontology we can classify the reproducer space according to the presence (or absence) of a self-description and/or reproductive machinery. We give examples of how various reproducers (both natural and artificial) can be categorised, and show how this ontology can be used to separate trivial from non-trivial examples of reproduction. With a worked example we show how we might use this approach to classify computer viruses, and gain insight into their reproductive reliance on external agency. Finally, we conjecture that reproduction requires a self-description and a reproductive mechanism, whether it is supplied from within or from an external agent