Mariusz A. Fecko

A technique to generate feasible tests for communications systems with multiple timers

We present a new model for testing real-time protocols with multiple timers, which captures complex timing dependencies by using simple linear expressions involving timer-related variables. This new modeling technique, combined with the algorithms to eliminate inconsistencies, allows generation of feasible test sequences without compromising their fault coverage. The model is specifically designed for testing to avoid performing full reachability analysis, and to control the growth of the number of test scenarios. Based on extended finite state machines, it is applicable to languages such as SDL, VHDL, and Estelle. The technique models a realistic testing framework in which each I/O exchange takes a certain time to realize and timers can be arbitrarily started or stopped. A software tool implementing this technique is used to generate test cases for the U.S. Army wireless standard MIL-STD 188-220.

Test Generation in the Presence of Conflicting Timers

The UD’s and CCNY’s ongoing research to generate conformance tests for the Army network protocol MIL-STD 188–220 addressed test generation when multiple timers are running simultaneously. A test sequence may become unrealizable if there are conflicting conditions based on a protocol’s timers. This problem is handled in the hitherto generated tests by manually expanding a protocol’s extended FSM based on the set of conflicting timers, resulting in test sequences that are far from minimum-length. Similar inconsistencies, but based on arbitrary linear variables, are present in the extended FSMs modeling VHDL specifications. This paper presents an efficient solution to the conflicting timers problem that eliminates the redundancies of manual state expansion. CCNY’ s inconsistency removal algorithms are applied to a new model for testing protocols with multiple timers, in which complex timing dependencies are captured by simple linear expressions. This test generation technique is expected to significantly shorten the test sequences without compromising their fault coverage.

A success story of formal description techniques: Estelle specification and test generation for MIL-STD 188-220

This paper presents a success story of specifying a complex real-life protocol (MIL-STD 188-220) in Estelle and generating test sequences from the formal specification. 188-220 is being developed in the US Army, Navy and Marine Corps systems for mobile combat network radios. A key factor in this success story has been the collaboration among the researchers of the University of Delaware and the City College of the City University of New York, the developers of the US Army Communications-Electronics Command (CECOM), and the protocol designers in the Joint Combat Net Radio Working Group. Based on the research results, 188-220 test sequences are realizable without timer interruptions while providing a 200% increase in test coverage. The test cases are being installed at a CECOM test facility.

Combinatorial designs in multiple faults localization for battlefield networks

We present an application of combinatorial designs and variance analysis to correlating events in the midst of multiple network faults. The network fault model is based on the probabilistic dependency graph that accounts for the uncertainty about the state of network elements. Orthogonal arrays help reduce the exponential number of failure configurations to a small subset on which further analysis is performed. The preliminary results show that statistical analysis can pinpoint the probable causes of the observed symptoms with high accuracy and significant level of confidence. An example demonstrates how multiple soft link failures are localized in MIL-STD 188-220s datalink layer to explain the end-to-end connectivity problems in the network layer This technique can be utilized for the networks operating in an unreliable environment such as wireless and/or military networks.

Fault masking by multiple timing faults in timed EFSM models

Detection of multiple timing faults is a challenging task because these faults, although may be detectable individually, can mask each others faulty behavior, making a faulty implementation under test (IUT) indistinguishable from a non-faulty one during testing. This phenomenon, called fault masking, is formally defined in this paper. It is proven that graph augmentation algorithms proposed for timed Extended Finite State Machines (EFSMs) with multiple timers can detect pairwise occurrences of classes of timing faults in an IUT and, hence, detects fault masking.

Formal design and testing of MIL-STD 188-220A based on Estelle

This paper describes the Estelle specification of MIL-STD 188-280A intranet layer as well as a methodology for generating test sequences for checking the conformance of a protocol implementation to its specification. The methodology for deriving test cases from an Estelle specification, which serves as input to test generation techniques, is presented. A Chinese postman tour is used to determine a minimum-cost tour of the transition graph for various transition types. Finally, the paper discusses several controllability and optimization issues that need to be addressed in test cases generation for the intranet and datalink layers of MIL-STD 188-220A.

On stability analysis of virtual backbone in mobile ad hoc networks

Service discovery architectures and cluster-assisted routing protocols in mobile ad-hoc networks (MANETs) heavily use formation and maintenance of a virtual backbone (VB), where the most stable mobile nodes with higher node degree are dynamically selected as the backbone nodes. In this paper we present a novel analytic model for VB stability in MANETs. The model employs the dynamics of node movements, where link creation/failure is modeled via a random walk with probabilistic state-transition matrix. The backbone formation algorithm gives preference to the nodes with the smaller number of link changes and the higher degree. Therefore, the link arrivals and departures determine the probability (and thus the expected time) for a mobile node to leave, join, or remain in the backbone, i.e., the stability of a dynamic structure of VB.

Lessons learned from automating tests for an operations support system

We present experience gained in automating tests for an operations support system. A major portion of the effort was devoted to extending a commercial test tool so that testers could easily manipulate graphical user interface (GUI) objects on two implementations of the application. For this purpose, we developed a test automation library as support infrastructure for writing tests. The challenges and tradeoffs are discussed such as simplicity/complexity for a tester versus a library developer, hiding/exposing window hierarchy to the tester, providing common methods for different types of GUI objects, transparently manipulating custom GUI widgets, and coping with data‐dependent test cases. We discuss the requirements of test code reusability, maintainability, and portability, and describe the solutions we found. In addition, we offer observations about benefits and pitfalls of test automation, our recommendations for maximizing return on investment, and results from automating a variety of tests. Copyright

Testing protocols modeled as FSMs with timing parameters

An optimization method is introduced for generating minimum-length test sequences taking into account timing constraints for FSM models of communication protocols. Due to active timers in many of todays protocols, the number of consecutive self-loops that can be traversed in a given state before a timeout occurs is limited. A test sequence that does not consider timing constraints will likely be unrealizable in a test laboratory, thereby potentially resulting in the incorrect failing of valid implementations (or, vice versa). The solution uses a series of augmentations for a protocols directed graph representation. The resulting test sequence is proven to be of minimum-length while not exceeding the tolerable limit of consecutive self-loops at each state. Although UIO sequences are used for state verification method, the results also are applicable to test generation that uses distinguishing or characterizing sequences.

Issues in conformance testing: multiple semicontrollable interfaces

In a testing environment, where an IUT communicates with multiple entities, a tester may have differing degrees of controllability on the interactions between these entities and the IUT: directly controllable, semicontrollable, or uncontrollable. In this paper, a graph conversion algorithm is introduced that offers the testability of both the directly and semicontrollable inputs, while avoiding race conditions. Although, for the most general case, the graph conversion results in an exponentially large number of nodes, practical considerations make the converted graph size feasible. Currently, this methodology is being applied to generate tests for MIL-STD 188–220B, which increases the number of testable state transitions from approximately 200 to over 700.