Keith C. Mander

Automated program flaw finding using simulated annealing

One of the major costs in a software project is the construction of test-data. This paper outlines a generalised test-case data generation framework based on optimisation techniques. The framework can incorporate a number of testing criteria, for both functional and non-functional properties. Application of the optimisation framework to testing specification failures and exception conditions is illustrated. The results of a number of small case studies are presented and show the efficiency and effectiveness of this dynamic optimisation-base approach to generating test-data.

An automated framework for structural test-data generation

Structural testing criteria are mandated in many software development standards and guidelines. The process of generating test data to achieve 100% coverage of a given structural coverage metric is labour-intensive and expensive. This paper presents an approach to automate the generation of such test data. The test-data generation is based on the application of a dynamic optimisation-based search for the required test data. The same approach can be generalised to solve other test-data generation problems. Three such applications are discussed-boundary value analysis, assertion/run-time exception testing, and component re-use testing. A prototype tool-set has been developed to facilitate the automatic generation of test data for these structural testing problems. The results of preliminary experiments using this technique and the prototype tool-set are presented and show the efficiency and effectiveness of this approach.

A search-based automated test-data generation framework for safety-critical systems

This paper presents the results of a three year research program to develop an automated test-data generation framework to support the testing of safety-critical software systems. The generality of the framework comes from the exploitation of domain independent search techniques, allowing new test criteria to be addressed by constructing functions that quantify the suitability of test-data against the test-criteria. The paper presents four applications of the framework - specification falsification testing, structural testing, exception condition testing and worst-case execution time testing. The results of three industrial scale case-studies are also presented to show that the framework offers useful support in the development safety-critical software systems.

Grand Challenges in Computing: Education---A Summary

The conference on grand challenges, held in Newcastle on 30 and 31 March 2004, occurred at a particularly opportune time. The strand on the educational aspects was particularly relevant and the idea innovative in the sense that this was the first occasion on which a grand challenge event with a focus on educational issues in computing had taken place. This paper provides some of the background and includes a distillation of the educational challenges that emerged from that event.

The SAZ Project: Integrating SSADM and Z.

This paper investigates the rationale for integrating a structured systems analysis method (SSADM version 4) and a formal notation (Z). It describes the integrated specification, and discusses the advantages and disadvantages of formal specification and development for information systems.

Strategies for lifecycle concurrency and iteration – A system dynamics approach

Abstract Increasingly fierce commercial pressures necessitate the use of advanced software lifecycle techniques to meet growing demands on both product time-to-market and business performance. Two significant methods of achieving such improved cycle-time capability are concurrent software engineering and staged-delivery. Concurrent software engineering exploits the potential for simultaneous performance of development activities between projects, product deliveries, development phases, and individual tasks. Staged-delivery enables lifecycle iteration to supply defined chunks of product functionality at pre-planned intervals. Used effectively, these techniques provide a powerful route to reduced cycle-times, increased product quality and, potentially, lower development costs. However, the degree and manner in which these techniques should be applied remains an area for active research.This paper identifies some of the issues and open problems of incremental lifecycle management by reference to the development of aeroengine control systems within Rolls-Royce plc. We explain why system dynamics is a promising technique for evaluating strategies for lifecycle concurrency and iteration.

Rigorous Specification using Structured Systems Analysis and Z

Abstract This paper describes the rationale for integrating a structured systems analysis method (SSADM, version 4) and a formal notation (Z). It introduces the SAZ Method and shows how this can be used to add rigour to the specification of the system state and processing in SSADM. The paper concludes by summarizing the results of using SAZ on a number of case studies.

Immune and evolutionary approaches to software mutation testing

We present an Immune Inspired Algorithm, based on CLONALG, for software test data evolution. Generated tests are evaluated using the mutation testing adequacy criteria, and used to direct the search for new tests. The effectiveness of this algorithm is compared against an elitist Genetic Algorithm, with effectiveness measured by the number of mutant executions needed to achieve a specific mutation score. Results indicate that the Immune Inspired Approach is consistently more effective than the Genetic Algorithm, generating higher mutation scoring test sets in less computational expense.

Software Quality Assurance using the SAZ Method

Quality assurance is a problem in the development of software systems. Structured methods use diagrammatic and text formats, which are difficult to assess for quality. Formal specifications are easier to check, but do not facilitate iterative capture of requirements. This paper describes the benefits of combining a widely-used structured method (SSADM version 4) with a formal notation (Z).

Software Vaccination: An Artificial Immune System Approach to Mutation Testing

Over time programming languages develop, paradigmsevolve, development teams change. The effect of this is that test suites wear out, therefore these also need to evolve. Mutation testing is an effective fault-based testing approach, but it is computationally expensive. Any evolutionary based approach to this process needs to simultaneously manage execution costs. In this conceptual paper we adopt immune systems as a metaphor for the basis of an alternative mutation testing system. It is envisaged that through monitoring of the development environment, a minimal set of effective mutations and test cases can be developed – a ‘vaccine’ – that can be applied to the software development process to protect it from errors – from infections.