Roger T. Alexander

Testing Web applications by modeling with FSMs

Researchers and practitioners are still trying to find effective ways to model and test Web applications. This paper proposes a system-level testing technique that combines test generation based on finite state machines with constraints. We use a hierarchical approach to model potentially large Web applications. The approach builds hierarchies of Finite State Machines (FSMs) that model subsystems of the Web applications, and then generates test requirements as subsequences of states in the FSMs. These subsequences are then combined and refined to form complete executable tests. The constraints are used to select a reduced set of inputs with the goal of reducing the state space explosion otherwise inherent in using FSMs. The paper illustrates the technique with a running example of a Web-based course student information system and introduces a prototype implementation to support the technique.

Design patterns and change proneness: an examination of five evolving systems

Design patterns are recognized, named solutions to common design problems. The use of the most commonly referenced design patterns should promote adaptable and reusable program code. When a system evolves, changes to code involving a design pattern should, in theory, consist of creating new concrete classes that are extensions or subclasses of previously existing classes. Changes should not, in theory, involve direct modifications to the classes in prior versions that play roles in a design patterns. We studied five systems, three proprietary systems and two open source systems, to identify the observable effects of the use of design patterns in early versions on changes that occur as the systems evolve. In four of the five systems, pattern classes are more rather than less change prone. Pattern classes in one of the systems were less change prone. These results held up after normalizing for the effect of class size - larger classes are more change prone in two of the five systems. These results provide insight into how design patterns are actually used, and should help us to learn to develop software designs that are more easily adapted.

A fault model for subtype inheritance and polymorphism

Although program faults are widely studied, there are many aspects of faults that we still do not understand, particularly about OO software. In addition to the simple fact that one important goal during testing is to cause failures and thereby detect faults, a full understanding of the characteristics of faults is crucial to several research areas. The power that inheritance and polymorphism brings to the expressiveness of programming languages also brings a number of new anomalies and fault types. This paper presents a model for the appearance and realization of OO faults and defines and discusses specific categories of inheritance and polymorphic faults. The model and categories can be used to support empirical investigations of object-oriented testing techniques, to inspire further research into object-oriented testing and analysis, and to help improve design and development of object-oriented software.

Distributing classes with woven concerns: an exploration of potential fault scenarios

Aspect-oriented programming (AOP) promises to benefit software engineering by providing a layer of abstraction that can modularize system-level concerns. AOP is still a very young area of research and has yet to receive mainstream acceptance in industry. As a result, the industry as a whole lacks experience and knowledge concerning long term maintenance issues with AOP in deployed commercial applications. Fault models that result from software maintenance in aspect-oriented software development (AOSD) are not nearly as well understood as they are for object-oriented software development (OOSD). This paper will explore some of the long-term maintenance issues that can occur with AspectJ, which is an implementation of AOP for the Java programming language. More specifically, the ability of AspectJ (as of version 1.2) to weave into existing bytecode that already contains woven aspects can create unexpected and potentially unsolvable problems. This will hopefully lead to further discussion in the research community that will result in a better understanding of the long-term maintenance issues inherent in AOSD.

Mutation of Java objects

Fault insertion based techniques have been used for measuring test adequacy and testability of programs. Mutation analysis inserts faults into a program with the goal of creating mutation-adequate test sets that distinguish the mutant from the original program. Software testability is measured by calculating the probability that a program will fail on the next test input coming from a predefined input distribution, given that the software includes a fault. Inserted faults must represent plausible errors. It is relatively easy to apply standard transformations to mutate scalar values such as integers, floats, and character data, because their semantics are well understood. Mutating objects that are instances of user defined types is more difficult. There is no obvious way to modify such objects in a manner consistent with realistic faults, without writing custom mutation methods for each object class. We propose a new object mutation approach along with a set of mutation operators and support tools for inserting faults into objects that instantiate items from common Java libraries heavily used in commercial software as well as user defined classes. Preliminary evaluation of our technique shows that it should be effective for evaluating real-world software testing suites.

Criteria for testing polymorphic relationships

The emphasis in object oriented programs is on defining abstractions that have both state and behavior. This emphasis causes a shift in focus from software units to the way software components are connected. Thus, we are finding that we need less emphasis on unit testing and more on integration testing. The compositional relationships of inheritance and aggregation, especially when combined with polymorphism, introduce new kinds of integration faults. The paper presents results from an ongoing research project that has the goal of improving the quality of object oriented software. New testing criteria are introduced that take the effects of inheritance and polymorphism into account. These criteria are based on the new analysis technique of quasi-interprocedural data flow analysis. These testing criteria can improve the quality of object oriented software by ensuring that integration tests are high quality.

Verifiable composition of access control and application features

Access control features are often spread across and tangled with other functionality in a design. This makes modifying and replacing these features in a design difficult. Aspect-oriented modeling (AOM) techniques can be used to support separation of access control concerns from other application design concerns. Using an AOM approach, access control features are described by aspect models and other application features are described by a primary model. Composition of aspect and primary models yields a design model in which access control features are integrated with other application features. In this paper, we present, through an example, an AOM approach that supports verifiable composition of behaviors described in access control aspect models and primary models. Given an aspect model, a primary model, and a specified property, the composition technique produces proof obligations as the behavioral descriptions in the aspect and primary models are composed. One has to discharge the proof obligations to establish that the composed model has the specified property.

A Candidate Fault Model for AspectJ Pointcuts

We present a candidate fault model for pointcuts in AspectJ programs. The fault model identifies faults that we believe are likely to occur when writing pointcuts in the AspectJ language. Categories of fault types are identified, and each individual fault type is described as categorized. We argue that a fault model that focuses on the unique constructs of the AspectJ language is needed for the systematic and effective testing of AspectJ programs. Our pointcut fault model is a first step towards such a model

Scalability issues with using FSMWeb to test web applications

Web applications are fast becoming more widespread, larger, more interactive, and more essential to the international use of computers. It is well understood that web applications must be highly dependable, and as a field we are just now beginning to understand how to model and test Web applications. One straightforward technique is to model Web applications as finite state machines. However, large numbers of input fields, input choices and the ability to enter values in any order combine to create a state space explosion problem. This paper evaluates a solution that uses constraints on the inputs to reduce the number of transitions, thus compressing the FSM. The paper presents an analysis of the potential savings of the compression technique and reports actual savings from two case studies.

The real costs of aspect-oriented programming

As a software developer, one of my principal objectives is to deliver the highest quality software possible. Unfortunately, as a human being, my abilities are also my limitations, and I always make mistakes of one kind or another. To remedy this, I use tools to help ameliorate as many of my limitations as possible. One of rny concerns is to understand how using a particular tool affects my ability to meet objectives. How does it help with the various software development tasks? What does it Cost me to use a particular tool? Do the benefits outweigh the costs? If so, what degree is the benefit? If not, then the tool is of little use. There must be a significant benefit gain to justify using a tool. The essence of aspect-oriented programming is that crosscutting concerns are factored into distinct abstractions (that is, aspects) that are solely responsible for a particular crosscutting concern. This raises the systems modularity and increases cohesion of the abstractions that are primary concerns. Researchers and software developers must understand and address the various significant weaknesses before AOPs benefits can be realized.