John A. McDermid

An automated framework for structural test-data generation

Structural testing criteria are mandated in many software development standards and guidelines. The process of generating test data to achieve 100% coverage of a given structural coverage metric is labour-intensive and expensive. This paper presents an approach to automate the generation of such test data. The test-data generation is based on the application of a dynamic optimisation-based search for the required test data. The same approach can be generalised to solve other test-data generation problems. Three such applications are discussed-boundary value analysis, assertion/run-time exception testing, and component re-use testing. A prototype tool-set has been developed to facilitate the automatic generation of test data for these structural testing problems. The results of preliminary experiments using this technique and the prototype tool-set are presented and show the efficiency and effectiveness of this approach.

Analysis and synthesis of the behaviour of complex programmable electronic systems in conditions of failure

This paper introduces a new method for safety analysis which modifies, automates and integrates a number of classical safety analysis techniques to address some of the problems currently encountered in complex safety assessments. The method enables the analysis of a complex programmable electronic system from the functional level through to low levels of its hardware and software implementation. In the course of the assessment, the method integrates design and safety analysis and harmonises hardware safety analysis with the hazard analysis of software architectures. It also introduces an algorithm for the synthesis of fault trees, which mechanises and simplifies a large and traditionally problematic part of the assessment, the development of fault trees. In this paper, we present the method and discuss its application on a prototypical distributed brake-by-wire system for cars. We argue that the method can help us rationalise and simplify an inherently creative and difficult task and therefore gain a consistent and meaningful picture of how a complex programmable system behaves in conditions of failure. q 2001 Elsevier Science Ltd. All rights reserved.

Hierarchically Performed Hazard Origin and Propagation Studies

This paper introduces a new method for safety analysis called HiPHOPS (Hierarchically Performed Hazard Origin and Propagation Studies). HiP-HOPS originates from a number of classical techniques such as Functional Failure Analysis, Failure Mode and Effects Analysis and Fault Tree Analysis. However, it extends, automates and integrates these techniques in order to address some of the problems currently encountered in complex safety assessments. The method enables integrated assessment of a complex system from the functional level through to the low level of component failure modes. It mechanises and simplifies a large part of the analysis, the development of fault trees, and can guarantee the consistency of results. HiP-HOPS is currently supported by a tool called the Safety Argument Manager (SAM). In this paper we introduce the method and we show how it has helped us analyse and improve the safety of a distributed brake-by-wire system for cars.

Towards integrated safety analysis and design

There are currently many problems with the development and assessment of software intensive safety-critical systems. In this paper we describe the problems, and introduce a novel approach to their solution, based around goal-structuring concepts, which we believe will ameliorate some of the difficulties. We discuss the use of modified and new forms of safety assessment notations to provide evidence of safety, and the use of data derived from such notations as a means of providing quantified input into the design assessment process. We then show how the design assessment can be partially automated, and from this develop some ideas on how we might move from analytical to synthetic approaches, using safety criteria and evidence as a fitness function for comparing alternative automatically-generated designs.

Large-scale complex IT systems

The reductionism behind todays software-engineering methods breaks down in the face of systems complexity.

A search-based automated test-data generation framework for safety-critical systems

This paper presents the results of a three year research program to develop an automated test-data generation framework to support the testing of safety-critical software systems. The generality of the framework comes from the exploitation of domain independent search techniques, allowing new test criteria to be addressed by constructing functions that quantify the suitability of test-data against the test-criteria. The paper presents four applications of the framework - specification falsification testing, structural testing, exception condition testing and worst-case execution time testing. The results of three industrial scale case-studies are also presented to show that the framework offers useful support in the development safety-critical software systems.

Safety Case Construction and Reuse Using Patterns

This paper presents an approach to the reuse of common structures in safety case arguments through their documentation as ’Safety Case Patterns’. Problems with the existing, informal and ad-hoc approaches to safety case material reuse are highlighted. We argue that through explicit capture and documentation of reusable safety case elements as patterns, the process of safety case construction and reuse can be made more systematic. For the description of patterns a safety case pattern language and a graphical pattern notation (based on the Goal Structuring Notation) are presented. Using this framework we briefly describe a number of example argument patterns. A fully documented example pattern is included as an appendix to this paper.

Experience with the application of HAZOP to computer-based systems

This paper summarises the experience gained from application of Hazard and Operability Studies (HAZOP) and related techniques to four computer-based systems. Emphasis is placed on working practices and the integration of HAZOP-style analysis into a safety-oriented lifecycle. Two of the case studies are described in some detail. An industrial study is used to investigate working practices, highlighting a number of areas of concern with the traditional team approach. A second example is described using an alternative process known as Software Hazard Analysis and Resolution in Design (SHARD), showing its effectiveness on a technology demonstrator case study. This example also demonstrates the integration of our approach with other techniques such as our Failure Propagation and Transformation Notation (FPTN) and Software Fault Trees.

An integrated tool set for software safety analysis

Abstract Traditional methods for assessing software safety suffer from poor integration (from methodological, operational and semantic points of view) both with each other, and with the rest of the develpment life cycle of safety-critical systems. Our goal is to develop a set of methods and tools that addresses these weaknesses; this article describes our current research in these areas. We describe an integrated approach to software safety analysis based on the techniques of fault tree analysis and failure modes, effects, and criticality analysis, together with a prototype tool set to implement these techniques. Issues pertaining to the integration of safety analysis into a broader development life cycle are also discussed. Our approach emphasizes pragmatism and simplicity—we aim to create a set of tools and methods that are robust, and straightforward, and directly usable by industrial practitioners in the field of software safety.

Safety Case Development: Current Practice, Future Prospects

Safety-critical and safety-related systems are becoming more highly integrated and continue to increase in complexity. In parallel with this, certification standards for such systems are becoming more stringent, requiring more extensive and more detailed analyses. Safety cases, therefore, are themselves growing in size and complexity and are becoming increasingly costly to produce. It has become necessary to re-examine how and why safety cases are built in order that we might provide a means for managing their inherent complexity and reduce production costs.