Alan Hartman

Problems and algorithms for covering arrays

Covering arrays are combinatorial structures which extend the notion of orthogonal arrays and have applications in the realm of software testing. In this paper we raise several new problems motivated by these applications and discuss algorithms for their solution.

Software and Hardware Testing Using Combinatorial Covering Suites

In the 21st century our society is becoming more and more dependent on software systems. The safety of these systems and the quality of our lives is increasingly dependent on the quality of such systems. A key element in the manufacture and quality assurance process in software engineering is the testing of software and hardware systems. The construction of efficient combinatorial covering suites has important applications in the testing of hardware and software. In this paper we define the general problem, discuss the lower bounds on the size of covering suites, and give a series of constructions that achieve these bounds asymptotically. These constructions include the use of finite field theory, extremal set theory, group theory, coding theory, combinatorial recursive techniques, and other areas of computer science and mathematics. The study of these combinatorial covering suites is a fascinating example of the interplay between pure mathematics and the applied problems generated by software and hardware engineers. The wide range of mathematical techniques used, and the often unexpected applications of combinatorial covering suites make for a rewarding study.

The AGEDIS tools for model based testing

We describe the tools and interfaces created by the AGEDIS project, a European Commission sponsored project for the creation of a methodology and tools for automated model driven test generation and execution for distributed systems. The project includes an integrated environment for modeling, test generation, test execution, and other test related activities. The tools support a model based testing methodology that features a large degree of automation and also includes a feedback loop integrating coverage and defect analysis tools with the test generator and execution framework. Prototypes of the tools have been tried in industrial settings providing important feedback for the creation of the next generation of tools in this area.

Using a model-based test generator to test for standard conformance

In this paper we describe two experiments in the verification of software standard conformance. In our experiments, we use a model-based test generator to create a test suite for parts of the POSIX™ standard and another test suite for the specification of Java™ exception handling. We demonstrate that models derived from specifications produce better test suites than the suites specified by standards. In particular, our test suites achieved higher levels of code coverage with complete test requirements coverage. Moreover, the test suite for the Java study found code defects that were not exposed by other benchmark test suites. The effort involved in producing these models and test suites was comparable to the effort involved in developing a test suite by more conventional methods. We avoid the state space explosion problem by modeling only the external behavior of a specific feature of the standard, without modeling the details of any particular implementation.

Projected state machine coverage for software testing

Our research deals with test generation for software based on finite state machine (FSM) models of the program specification. We describe a set of coverage criteria and testing constraints for use in the automatic generation of test suites. We also describe the algorithms used to generate test suites based on these coverage criteria, and the implementation of these algorithms as an extension of the Murϕ model checker[4]. The coverage criteria are simple but powerful in that they generate test suites of high quality and moderate volume, without requiring the user to have a sophisticated grasp of the test generation technology. The testing constraints are used to combat the endemic problem of state explosion, typically encountered in FSM techniques. We illustrate our techniques on several well-known problems from the literature and describe two industrial trials, to demonstrate the validity of our claims.

The fundamental construction for 3-designs

Abstract We discuss a generalization of Wilsons fundamental construction for group divisible designs which is intended to produce 3-wise balanced designs, rather than pairwise balanced designs. The construction generalizes many known recursive constructions for Steiner quadruple systems and other related designs. The generalization of Wilsons construction is based on a structure which is a 3-design analogue of a group divisible design. We show that other such analogues which have appeared in the literature are special cases of our definition. We also give several new applications of these structures.

Optimal Parsing of Strings

Parsing of strings has been suggested by Lempel and Ziv, as a tool to define the complexity of strings [LZ1] and to compress them [ZL1, ZL2]. Recently, they have shown that the scheme may be modified to handle two dimensional data [LZ2].

2nd UML 2 semantics symposium: formal semantics for UML

The purpose of this symposium, held in conjunction with MoDELS 2006, was to present the current state of research of the UML 2 Semantics Project. Equally important to receiving feedback from an audience of experts was the opportunity to invite researchers in the field to discuss their own work related to a formal semantics for the Unified Modeling Language. This symposium is a follow-on to our first workshop, held in conjunction with ECMDA 2005.

A Model for Designing Generic Services

This paper describes a model for capturing service designs and the design of service product lines. The model promotes reuse of service artifacts, enables a balanced view of the service from different stakeholder viewpoints, and provides a foundation for design for service quality. The model leverages a phased approach to design, engineering and management of services. The model has applicability throughout the service lifecycle and provides a standardized vocabulary and structure for designing and maintaining generic services, irrespective of the domain of application.

Choosing a test modeling language: a survey

Deployment of model-based testing involves many difficulties that have slowed down its industrial adoption. The leap from traditional scripted testing to model-based testing seems as hard as moving from manual to automatic test execution. Two key factors in the deployment are the language used to define the test models, and the language used for defining the test objectives. Based on our experience, we survey the different types of languages and sketch solutions based on different approaches, considering the testing organization, the system under test, etc. The types of languages we cover include among others domain-specific, test-specific as well as generic design languages. We note that there are no best practices, but provide general guidelines for various cases.