Zoltán Pap

Applying mutation analysis to SDL specifications

Mutation analysis is a fault based testing method used initially for code based software testing, and lately for specification based testing and validation as well. In this paper, the method is applied to SDL (Specification and Description Language) specifications. It is used to automate the process of conformance test generation and selection for telecommunications protocols. We present two algorithms for automatic test generation and selection. These provide the basis of the Test Selector tool developed at the Budapest University of Technology and Economics. We present the results of an empirical study using the tool.

A bounded incremental test generation algorithm for finite state machines

We propose a bounded incremental algorithm to generate test cases for deterministic finite state machine models. Our approach, in contrast to the traditional view, is based on the observation that system specifications are in most cases modified incrementally in practice as requirements evolve. We utilize an existing test set available for a previous version of the system to efficiently generate tests for the current - modified - system. We use a widely accepted framework to evaluate the complexity of the proposed incremental algorithm, and show that it is a function of the size of the change in the specification rather than the size of the specification itself. Thus, the method is very efficient in the case of small changes, and never performs worse than the relevant traditional algorithm - the HIS-method. We also demonstrate our algorithm through an example.

Using Change Impact Analysis to Select Tests for Extended Finite State Machines

A formal approach to select tests for regression testing of changes performed in a system evolution step is proposed. Systems are modeled as extended finite state machines (EFSMs) supporting several commonly used data types including booleans, numbers, arrays, queues and records. Tests are described using a sequence of input and expected output messages with concrete parameter values. Changes add/delete/replace one or more EFSM transitions. Transitions potentially executed by a test are automatically identified from its description. A simple structural invariant for a test description based on these transitions is introduced. It is shown that for a test description satisfying the invariant it can be accurately determined if a given change affects the test. Affected tests are selected for regression testing of the change. Failure of a description to meet the invariant is analyzed to identify non-observable regions in the description, which are then further analyzed using other system transitions to identify affected tests. We also describe a novel approach based on substitutability of tests to reduce the size of a regression test suite without affecting coverage. The effectiveness of the proposed approach is illustrated by applying it to several examples. Our experiments based on a well-known cost model for regression testing show that the proposed approach is economical for selective re-testing in these examples.

An Approach for Test Selection for EFSMs Using a Theorem Prover

This paper describes an automatic approach for selecting tests from a test suite to validate the changes made to an extended finite state machine (EFSM). EFSMs supporting variables over commonly used data types including booleans, numbers, arrays, queues, and records, and communicating with the environment using parameterized messages are considered. Changes to the EFSM add/delete/replace one or more transitions. Tests are described using a sequence of input and output messages with parameter values. We introduce a class of fully-observable tests. The description of a fully-observable test contains all the information to accurately determine the transitions executed by the test. Interaction among the EFSM transitions captured in terms of a compatibility relation is used along with a given test description to automatically identify fully-observable tests. A procedure is described for selecting a test for a given change based on accurately predicting if the test executes the change transition. We then describe how several tests can be simultaneously selected by grouping them based on overlap of their descriptions. The proposed approach has been implemented using a theorem prover and applied to several examples including protocols and web services with encouraging results.

RESERV: A Distributed, Load Balanced Information System for Grid Applications

Resource information systems are a key component of Computational Grids. Centralized information systems hamper scalability and reliability, and thus, completely distributed resource information systems, based on Distributed Hash Tables have been proposed. In some cases resource distribution might be highly uneven, load balancing of data becomes thus a crucial problem. However, current load balancing schemes cannot handle large amounts of data corresponding to a single resource type. In this paper we propose therefore RESERV, a distributed information system for Grid applications with a novel load balancing approach, able to handle extreme load unbalance.

Standardized languages for telecommunication systems

In this paper, we give a survey of the most important ITU-T languages used in the system modeling and development process. We discuss the goals, the most important features, and the basic syntactic and semantic rules of the languages and notations. We compare the capabilities of the ITU-T languages and the OMGs Unified Modeling Language, considering the modeling and development aspects they support. At the end of the paper, we give an informative example to demonstrate the use of the languages during the different phases of the development process.

Optimal string edit distance based test suite reduction for SDL specifications

We propose a test selection method that provides efficient test sets for systems based on SDL specifications. Our approach builds on previous results of Voung et al. and Feijs et al. on string edit distance based coverage metrics. The method reduces a set of test cases represented in the MSC (Message Sequence Chart) notation, while maintaining the highest possible distance between all pairs of traces defined by the given test set. The algorithm is tunable by a parameter representing the threshold distance for test redundancy. We show that the algorithm runs in polynomial time of the size of the input test set and that it is independent of the size of the system. We implemented and incorporated the algorithm into our SDL-based test selection framework, and evaluated against existing symbol coverage and fault coverage based test selection approaches by conducting experiments on the well-known INRES and Conference Protocol. Results indicate that the string edit distance based method yields similar results in terms of reduction-capability and coverage as the other approaches, but with significantly less complexity.

Analyzing the impact of protocol changes on tests

Protocols governing communication among system components evolve during design and maintenance and need to be re-tested. For faster testing turnaround time, it is important that the consistency of the testing infrastructure with the protocol be preserved across changes. In this paper, we propose a state exploration based approach to identify the impacts of protocol changes on a given set of protocol tests. Protocols are modeled as a network of communicating finite state machines exchanging messages over bounded queues. Each machine denotes the behavior of an individual protocol component (controller). A protocol test is modeled as a sequence of inputs from the environment to the protocol controllers in an execution starting from a stable protocol state. A notion of consistency of a test relative to a protocol is introduced. Conditions under which a protocol change requires changing a test to preserve the consistency of the test are identified. Changes consisting of multiple atomic updates are analyzed to remove redundancies and their impact on tests is studied. A by-product of the proposed approach is a classification of tests based on how they are impacted by protocol changes, which can help users in regression test selection.

Peer-to-peer based distributed file systems

In this paper, we present the problem of building a file system on top of a peer-to-peer network. While peer-to-peer networks are offering an increasing number of services, the scale of network file systems is increasing to that of distributed storage systems spanning the globe. The convergence of the two fields should give birth to decentralised, distributed and self-configuring storage networks. We show here the evolution of peer-to-peer networks and network file systems, and the difficulties that have prevented offering file system type services in peer-to-peer networks so far. We present CFS and Ivy, two DHT-based file systems that fail, however, to solve key problems, and outline possible solutions in the future.

On FSM-Based fault diagnosis

We study the problem of fault diagnosis, i.e., localization of difference(s) between an implementation and a specification in systems modelled by finite state machines. We show that even considering only a single fault in a finite state machine there are some situations when the exact diagnosis of the fault cannot be assured. We give an algorithm for fault diagnosis. If it is possible the procedure exactly locates a single fault, and in case exact localization is unfeasible it provides the set of all potential differences between the implementation and the specification.