Ana R. Cavalli

New approaches for passive testing using an Extended Finite State Machine specification

This paper presents two new approaches for passive testing using an Extended Finite State Machine (EFSM) specification. The state of the art of passive testing shows us that all the methods for detection of errors based on EFSMs try to match the trace to the specification. Indeed, one searches a state succession in the specification machine that is able to generate the trace observed on the implementation. Using this approach, processing is performed on the specification and the trace remains in the background since no operation is applied to it. This made us realise that focusing our efforts on the trace could be beneficial and has given as result two approaches presented in this paper that extract information from the specification and then work on the trace. Thus, they take a different direction than the previous methods. We first present an approach to test traces by using invariants resulting from the specification. We formally define these invariants and we see how to extract them. We also discuss their ability to detect errors appearing in the implementation. This approach is able to test the data flow, but not in a very satisfactory way. This is the reason for a second approach seeking to apply a set of constraints to the trace. We develop in detail its principles. Both approaches are applied to a Simple Connection Protocol (SCP) and the results of preliminary experiments are presented. q 2003 Elsevier B.V. All rights reserved.

Hit-or-Jump: An algorithm for embedded testing with applications to IN services

This paper presents a new algorithm, Hit-or-Jump, for embedded testing of components of communication systems that can be modeled by communicating extended finite state machines. It constructs test sequences efficiently with a high fault coverage. It does not have state space explosion, as is often encountered in exhaustive search, and it quickly covers the system components under test without being “trapped”, as is experienced by random walks. Furthermore, it is a generalization and unification of both exhaustive search and random walks; both are special cases of Hit-or-Jump. The algorithm has been implemented and applied to embedded testing of telephone services in an Intelligent Network (IN) architecture, including the Basic Call Service and five supplementary services.

An EFSM-based intrusion detection system for ad hoc networks

Mobile ad hoc networks offer very interesting perspectives in wireless communications due to their easy deployment and their growing performances. However, due to their inherent characteristics of open medium, very dynamic topology, lack of infrastructure and lack of centralized management authority, MANET present serious vulnerabilities to security attacks. In this paper, we propose an intrusion detection scheme based on extended finite state machines (EFSM). We provide a formal specification of the correct behavior of the routing protocol and by the means of a backward checking algorithm, detect run-time violations of the implementation. We choose the standard proactive routing protocol OLSR as a case study and show that our approach allows to detect several kinds of attacks as well as conformance anomalies.

A formal approach for testing security rules

Nowadays, security policies are the key point of every modern infrastructure. The specification and the testing of such policies are the fundamental steps in the development of a secure system since any error in a set of rules is likely to harm the global security. To address both challenges, we propose a framework to specify security policies and test their implementation on a system. Our framework makes it possible to generate in an automatic manner, test sequences, in order to validate the conformance of a security policy. system behavior is specified using a formal description technique based on extended finite state machine (EFSM) [12]. The integration of security rules within the system specification is performed by specific algorithms. Then, the automatic tests generation is performed using a dedicated tool, called SIRIUS, developed in our laboratory. Finally, we briefly present a weblog system as a case study to demonstrate the reliability of our framework.

Network protocol system passive testing for fault management: A backward checking approach

Passive testing has proved to be a powerful technique for protocol system fault detection by observing its input/output behaviors yet without interrupting its normal operations. To improve the fault detection capabilities we propose a backward checking method that analyzes in a backward fashion the input/output trace from passive testing and its past. It effectively checks both the control and data portion of a protocol system, compliments the forward checking approaches, and detects more errors. We present our algorithm, study its termination and complexity, and report experiment results on the protocol SCP.

Passive testing and application to the GSM-MAP protocol

Abstract Passive testing is the process of collecting traces of messages exchanged between an operating implementation and its environment, in order to verify that these traces actually belong to the language accepted by the provided finite state machine specification. In this paper, we present an extension of the existing algorithms to consider an extended finite state machine as the specification. An algorithm is also introduced to take into account the number of transitions covered. These techniques are illustrated by the application to a real protocol, the GSM (global system for mobile communication)-MAP (mobile application part).

Two Complementary Tools for the Formal Testing of Distributed Systems with Time Constraints

The complexity and the variety of the deployed time dependent systems, as well as the high degree of reliability required for their global functioning, justify the care provided to the design of the best possible tests. Moreover,it is significant to automate these steps with an aim of reducing the time and the development cost and especially of increasing the reliability of the offered products. In this paper, we present two different tools to test systems with time constraints. The first one allows to automatically generate test cases based on model-based active testing techniques. Whereas the second tool is based on passive testing approach to check that the collected system traces respect a set of formal properties called Invariants.

Fast testing of critical properties through passive testing

We present a novel methodology to perform passive testing. The usual approach consists in recording the trace produced by the implementation under test and trying to find a fault by comparing this trace with the specification. We propose a more active approach to passive testing where the minimum set of (critical) properties required to a correct implementation may be explicitly indicated. In short, an invariant expresses that each time that the implementation under test performs a given sequence of input/output actions, then it must show a behavior reflected in the invariant. By using an adaptation of the classical pattern matching algorithms on strings, we obtain that the complexity of checking whether an invariant is fulfilled by the observed trace is in O(n ċ m), where n and m are the lengths of the trace and the invariant, respectively. If the length of the invariant is much smaller than the length of the trace then this complexity is almost linear with respect to the length of the trace. Actually, this is usually the case for most practical examples. In addition to our methodology, we present the case study that was the driving force for the development of our theory: The Wireless Application Protocol (WAP). We present a test architecture for WAP as well as the experimental results obtained from the application of our passive testing with invariants approach.

Timed Extended Invariants for the Passive Testing of Web Services

The service-oriented approach is becoming more and more popular to integrate highly heterogeneous systems. Web services are the natural evolution of conventional middleware technologies to support Web-based and enterprise level integration. Formal testing of such Web-based technology is a key point to guarantee its reliability. In this paper, we choose a non-intrusive approach based on monitoring to propose a conformance passive testing methodology to check that a composed Web service respects its functional requirements. This methodology is based on a set of formal invariants representing properties to be tested including data and time constraints. Passive testing of an industrial system (that uses a composition of Web services) is briefly presented to demonstrate the effectiveness of the proposed approach.

Experimental evaluation of FSM-based testing methods

The development of test cases is an important issue for testing software, communication protocols and other reactive systems. A number of methods are known for the development of a test suite based on a formal specification given in the form of a finite state machine. Well-known methods are called the W, Wp, UIO, UIOv, DS, H and HIS test derivation methods. These methods have been extensively used by research community in the last years; however no proper comparison has been made between them. In this paper, we experiment with these methods to assess their complexity, applicability, completeness, fault detection capability, length and derivation time of their test suites. The experiments are conducted on randomly generated specifications and on a realistic protocol called the simple connection protocol.