Avik Sinha

Text2Test: Automated Inspection of Natural Language Use Cases

The modularity and customer centric approach of use cases make them the preferred methods for requirement elicitation, especially in iterative software development processes as in agile programming. Numerous guidelines exist for use case style and content, but enforcing compliance to such guidelines in the industry currently requires specialized training and a strongly managed requirement elicitation process. However, often due to aggressive development schedules, organizations shy away from such extensive processes and end up capturing use cases in an ad-hoc fashion with little guidance. This results in poor quality use cases that are seldom fit for any downstream software activities. We have developed an approach for automated and “edittime”inspection of use cases based on the construction and analysis of models of use cases. Our models contain linguistic properties of the use case text along with the functional properties of the system under discussion. In this paper, we present a suite of model analysis techniques that leverage such models to validate uses cases simultaneously for their style and content. Such model analysis techniques can be combined with a robust NLP techniques to develop integrated development environments for use case authoring, as we do in Text2Test.When used in an industrial setting, Text2Test resulted in better compliance of use cases, in enhanced productivity

A linguistic analysis engine for natural language use case description and its application to dependability analysis in industrial use cases

We present 1) a novel linguistic engine made of configurable linguistic components for understanding natural language use case specification; and 2) results of the first of a kind large scale experiment of application of linguistic techniques to industrial use cases. Requirement defects are well known to have adverse effects on dependability of software systems. While formal techniques are often cited as a remedy for specification errors, natural language remains the predominant mode for specifying requirements. Therefore, for dependable system development, a natural language processing technique is required that can translate natural language textual requirements into validation ready computer models. In this paper, we present the implementation details of such a technique and the results of applying a prototype implementation of our technique to 80 industrial and academic use case descriptions. We report on the accuracy and effectiveness of our technique. The results of our experiment are very encouraging.

Model-based functional conformance testing of web services operating on persistent data

We propose a model based approach to functional conformance test generation for web services which operate in the presence of persistent data. Typically, web services are described in a standard notation called Web Services Description Language (WSDL). Unfortunately, WSDL standard does not allow behavioral specification (such as pre- and postconditions)of web services in the presence of persistent data. New standards which remedy this situation are being proposed (such as WSDL-S). In this paper, we propose the use of existing test generation techniques based on Extended Finite State Machine (EFSM) specification to address the generation of functional conformance testes for web services which operate on persistent data. The novel contribution of this paper is an algorithm which translates a WSDL-S behavioral specification of operations of a web service into an equivalent EFSM representation which can be exploited to generate an effective set of test cases.

Automated Functional Conformance Test Generation for Semantic Web Services

We present an automated approach to generate functional conformance tests for semantic Web services. The semantics of the Web services are defined using the inputs, outputs, preconditions, effects (IOPEs) paradigm. For each Web service, our approach produces testing goals which are refinements of the Web service preconditions using a set of fault models. A novel planner component accepts these testing goals, along with an initial state of the world and the Web service definitions to generate a sequence of Web service invocations as a test case. Another salient feature of our approach is generation of verification sequences to ensure that the changes to the world produced by an effect are implemented correctly. Lastly, a given application incorporating a set of semantic Web services may be accessible through several interfaces such as 1) direct invocation of the Web services, or 2) a graphical user interface (GUI). Our technique allows generation of executable test cases which can be applied through both interfaces. We describe the techniques used in our test generation approach. We also present results which compare two approaches: an existing manual approach without the formal IOPEs information and the IOPEs-based approach reported in this paper. These results indicate that the approach described here leads to substantial savings in effort with comparable results for requirements coverage and fault detection effectiveness.

Use Cases to Process Specifications in Business Process Modeling Notation

Use cases are a key technique to elicit software requirements from the point of view of the user of a system. Their prevalence is noticeable ever since the onset of agile programming techniques. Within SOA projects however, business process models are used for capability analysis and gap detection. Business process models present a global view of the system and hence are more suited for gap detection. Therefore, in practice both these forms of requirements continue to be useful and coexist. Often in big software projects and in distributed development environment such coexisting requirement specifications can grow out of synch. We present here a technique to semi-automatically transform use cases into business processes and to create mapping between them. By preserving the mapping between these forms one can enforce consistency between the two forms of requirements.

HOTTest: A model-based test design technique for enhanced testing of domain-specific applications

Model-based testing is an effective black-box test generation technique for applications. Existing model-based testing techniques, however, fail to capture implicit domain-specific properties, as they overtly rely on software artifacts such as design documents, requirement specifications, etc., for completeness of the test model. This article presents a technique, HOTTest, which uses a strongly typed domain-specific language to model the system under test. This allows extraction of type-related system invariants, which can be related to various domain-specific properties of the application. Thus, using HOTTest, it is possible to automatically extract and embed domain-specific requirements into the test models. In this article we describe HOTTest, its principles and methodology, and how it is possible to relate domain-specific properties to specific type constraints. HOTTest is described using the example of HaskellDB, which is a Haskell-based embedded domain-specific language for relational databases. We present an example application of the technique and compare the results to some other commonly used Model-based test automation techniques like ASML-based testing, UML-based testing, and EFSM-based testing.

Less is More: A Minimalistic Approach to UML Model-Based Conformance Test Generation

We present a minimalist approach to model-based test generation. Our approach leverages the information provided in the domain and behavioral models of an application under test to produce a small yet effective set of test cases without additional user input. The domain model consists of UML class diagram with invariants, while the behavioral model consists of UML use cases. Each use case flow has an associated guard condition and a set of updates (to the domain object diagram and the output parameters). We treat the model invariants to enable a novel specify once, test everywhere paradigm. Our approach frees the modeler from the responsibility of specifying appropriate alternate flows on use cases affected by each invariant - our analysis augments the specified use cases with appropriate alternate flows. Our approach then produces suitable testing goals which are refinements of the guard conditions on the augmented flows using a set of fault models. Another salient feature of our approach is generation of verification sequences to ensure that the object diagram updates associated with a given flow are implemented correctly. Our technique uses a novel set of fault models to mutate an object diagram and a novel algorithm which distinguishes between the original and the mutated object diagrams. We describe the techniques used in our test generation approach.

An experimental evaluation of a higher-ordered-typed-functional specification-based test-generation technique

HOTTest is a model based test automation technique of software systems based on models of the system described using HaskellDB. HaskellDB is an embedded domain specific language derived from Haskell. HOTTest enforces a systematic abstraction process and exploits system invariants for automatically producing test cases for domain specific requirements. Use of functional languages for system modeling is a new concept and hence HOTTest is subject to concerns of usability, like any other new technique. Also, the syntax and the declarative style of Haskell based languages make them difficult to learn. Similar concerns can be raised for HOTTest as it shares the same syntax with Haskell. In this paper we describe an experiment designed to study the usability of HOTTest and to compare it with existing model based test design techniques. The results show that HOTTest is more usable than the traditional technique and demonstrate that the test suites produced by HOTTest are more effective and efficient than those generated using the traditional model based test design technique.

A measurement framework for evaluating model-based test generation tools

This paper presents a measurement framework for evaluating model-based test generation (MBTG) tools. The proposed framework is derived by using the Goal Question Metric methodology, which helps formulate the metrics of interest: complexity, ease of learning, effectiveness, efficiency, and scalability. We demonstrate the steps involved in evaluating MBTG tools by describing a case study designed for this purpose. This case study involves the use of four MBTG tools that differ in their modeling techniques, test specification techniques, and test generation algorithms.

Requirements Modeling and Validation Using Bi-layer Use Case Descriptions

Extension of the modeling notations and formal languages for use case description are the commonly suggested solutions for adding precision to use case models. Practitioners have often argued against adoption of such techniques citing reasons like the steep learning curve for formal languages; and the quickness in using imprecise use case descriptions for communicating to different stake-holders of the system. In this paper we introduce the Archetest modeling environment, which through a unique bi-layer approach accepts use case descriptions in their imprecise form and then assists in adding precision through a wizard driven process. Thereby, it lends itself to both quick and precise modeling. Also the two forms of the use case models are self contained and cross-linked. This allows different modelers, the precise and the imprecise, to collaborate and also supports stake-holder specific feedbacks of the automated analysis. We describe the structure of Archetests use case models, and show how these models are amenable to automated processing. We present a case study which reports on typical modeling times using Archetest and demonstrates its scalability.