Jose Luis de la Vara

An extended systematic literature review on provision of evidence for safety certification

Abstract Context Critical systems in domains such as aviation, railway, and automotive are often subject to a formal process of safety certification. The goal of this process is to ensure that these systems will operate safely without posing undue risks to the user, the public, or the environment. Safety is typically ensured via complying with safety standards. Demonstrating compliance to these standards involves providing evidence to show that the safety criteria of the standards are met. Objective In order to cope with the complexity of large critical systems and subsequently the plethora of evidence information required for achieving compliance, safety professionals need in-depth knowledge to assist them in classifying different types of evidence, and in structuring and assessing the evidence. This paper is a step towards developing such a body of knowledge that is derived from a large-scale empirically rigorous literature review. Method We use a Systematic Literature Review (SLR) as the basis for our work. The SLR builds on 218 peer-reviewed studies, selected through a multi-stage process, from 4963 studies published between 1990 and 2012. Results We develop a taxonomy that classifies the information and artefacts considered as evidence for safety. We review the existing techniques for safety evidence structuring and assessment, and further study the relevant challenges that have been the target of investigation in the academic literature. We analyse commonalities in the results among different application domains and discuss implications of the results for both research and practice. Conclusion The paper is, to our knowledge, the largest existing study on the topic of safety evidence. The results are particularly relevant to practitioners seeking a better grasp on evidence requirements as well as to researchers in the area of system safety. As a major finding of the review, the results strongly suggest the need for more practitioner-oriented and industry-driven empirical studies in the area of safety certification.

Business Process Modelling and Purpose Analysis for Requirements Analysis of Information Systems

Although requirements analysis is acknowledged as a critical success factor of information system development for organizations, problems related to the requirements stage are frequent. Some of these problems are lack of understanding of the business by system analysts, lack of focus on the purpose of the system, and miscommunication between business people and system analysts. As a result, an information system may not fulfil organizational needs. To try to prevent these problems, this paper describes an approach based on business process modelling and purpose analysis through BPMN and the goal/strategy Map approach. The business environment is modelled in the form of business process diagrams. The diagrams are validated by end-users, and the purpose of the system is then analyzed in order to agree on the effect that the information system should have on the business processes. Finally, requirements are specified by means of the description of the business process tasks to be supported by the system.

Evidence management for compliance of critical systems with safety standards: A survey on the state of practice

Abstract Context Demonstrating compliance of critical systems with safety standards involves providing convincing evidence that the requirements of a standard are adequately met. For large systems, practitioners need to be able to effectively collect, structure, and assess substantial quantities of evidence. Objective This paper aims to provide insights into how practitioners deal with safety evidence management for critical computer-based systems. The information currently available about how this activity is performed in the industry is very limited. Method We conducted a survey to determine practitioners’ perspectives and practices on safety evidence management. A total of 52 practitioners from 15 countries and 11 application domains responded to the survey. The respondents indicated the types of information used as safety evidence, how evidence is structured and assessed, how evidence evolution is addressed, and what challenges are faced in relation to provision of safety evidence. Results Our results indicate that (1) V&V artefacts, requirements specifications, and design specifications are the most frequently used safety evidence types, (2) evidence completeness checking and impact analysis are mostly performed manually at the moment, (3) text-based techniques are used more frequently than graphical notations for evidence structuring, (4) checklists and expert judgement are frequently used for evidence assessment, and (5) significant research effort has been spent on techniques that have seen little adoption in the industry. The main contributions of the survey are to provide an overall and up-to-date understanding of how the industry addresses safety evidence management, and to identify gaps in the state of the art. Conclusion We conclude that (1) V&V plays a major role in safety assurance, (2) the industry will clearly benefit from more tool support for collecting and manipulating safety evidence, and (3) future research on safety evidence management needs to place more emphasis on industrial applications.

SafetyMet: A Metamodel for Safety Standards

In domains such as automotive, avionics, and railway, critical systems must comply with safety standards to allow their operation in a given context. Safety compliance can be an extremely demanding activity as practitioners have to show fulfilment of the safety criteria specified in the standards and thus that a system can be deemed safe. This is usually both costly and time consuming, and becomes even more challenging when, for instance, a system changes or aims to be reused in another project or domain. This paper presents SafetyMet, a metamodel for safety standards targeted at facilitating safety compliance. The metamodel consists of entities and relationships that abstract concepts common to different safety standards from different domains. Its use can help practitioners to show how they have followed the recommendations of a standard, and particularly in evolutionary or cross-domain scenarios. We discuss the benefits of the use of the metamodel, its limitations, and open issues in order to clearly present the aspects of safety compliance that are facilitated and those that are not addressed.

Business processes contextualisation via context analysis

Context-awareness has emerged as a new perspective for business process modelling. Even though some works have studied it, many challenges have not been addressed yet. There is a clear need for approaches that (i) facilitate the identification of the context properties that influence a business process and (ii) provide guidance for correct modelling of contextualised business processes. This paper addresses this need by defining an approach for business process contextualisation via context analysis, a technique that supports reasoning about context and discovery of its relevant properties. The approach facilitates adequate specification of context variants and of business process execution for them. As a result, we obtain business processes that fit their context and are correct.

Classification, Structuring, and Assessment of Evidence for Safety -- A Systematic Literature Review

Safety assurance and certification are amongst the most expensive and time-consuming tasks in the development of safety-critical systems. Demonstration of compliance with safety standards involves providing evidence that the standards safety criteria are met. To handle large collections of evidence effectively, safety professionals need knowledge of how to classify different types of evidence, how to structure the evidence, and how to assess it. This paper takes a step towards developing such a body of knowledge by conducting a Systematic Literature Review (SLR). Specifically, the SLR identifies and classifies the information and artefacts considered as evidence for safety, examines existing techniques for evidence structuring and assessment, and summarizes the challenges noted in the literature in relation to safety evidence. The paper, to our knowledge, is the first systematic review on the topic of safety evidence. The results we present are particularly relevant to practitioners seeking to better understand the evidence requirements for safety certification, as well as to researchers conducting research in this area.

A review of traceability research at the requirements engineering conference re@21

Traceability between development artefacts and mainly from and to requirements plays a major role in system lifecycle, supporting activities such as system validation, change impact analysis, and regulation compliance. Many researchers have been working on this topic and have published their work throughout the editions of the Requirements Engineering Conference. This paper aims to analyse the research on traceability published in the past 20 years of this conference and to provide insights into its contribution to the traceability area. We have selected and reviewed 70 papers in the proceedings of the conference and summarised several aspects of traceability that have been addressed and by whom. The paper also discusses the evolution of the topic at the conference, compares the results with those reported in other publications, and proposes aspects on which further research should be conducted.

Towards a model-based evolutionary chain of evidence for compliance with safety standards

Compliance with safety standards can greatly increase the development cost and time of critical systems. Major problems arise when evolutions to a system entail reconstruction of the body of safety evidence. When changes occur in the development or certification processes, identification of the new evidence to provide, the evidence that is no longer adequate, or the evidence that can be reused poses some challenges. Therefore, practitioners need support to identify how a chain of evidence evolves as a result of the changes. Otherwise, execution of the above activities can be very costly, and it can even result in abandonment of certification efforts. This paper outlines a solution to deal with these challenges. The solution is based on the use of model-driven engineering technology, which has already been applied for safety certification but not from an evolutionary chain of evidence-based perspective. The paper also sets the background for developing the solution, describes real situations in which the solution can help industry, and discusses possible challenges for developing it. The solution will be developed as part of OPENCOSS, a research project on cross-domain evolutionary certification.

Improving Requirements Analysis through Business Process Modelling: A Participative Approach

Although requirements analysis is acknowledged as a critical success factor of information system development for organizations, mistakes are frequent at the requirements stage. Two of these mistakes are the lack of understanding of the business by requirements engineers and the miscommunication between business people and systems analysts. As a result of these problems, information systems may not fulfill organizational needs. To prevent these problems, this paper describes an approach based on business process modeling. The business environment is modeled in the form of BPMN diagrams. The diagrams are validated by end-users and are then analyzed by systems analysts in order to reach an agreement on the effect that the information system will have on the organization. Finally, requirements are specified by means of the description of the business process tasks to be supported by the information system.

Safety Evidence Traceability: Problem Analysis and Model

[Context and motivation] Safety evidence plays an important role in gaining confidence in the safe operation of a system in a given context. For a large system, it is necessary to provide information about thousands of artefacts that might be used as evidence and about the relationships among themselves and also with other safety assurance assets. [Question/problem] Past research has only addressed some needs of traceability in safety-critical systems and thus has not provided a complete picture of safety evidence traceability. Lack of knowledge and awareness of these needs can result in poor evidence management and lead to certification risks. [Principal ideas/results] This paper aims to provide a broad overview of safety evidence traceability needs for practice and its associated challenges. We also propose a safety evidence traceability model, which has been validated with data from real-world critical systems. [Contribution] We discuss the motivation and challenges for safety evidence traceability, and present the various traces that need to be captured and maintained. This information can help researchers to shape future research based on industry needs and can help practitioners to gain a deeper understanding and a wider knowledge of safety evidence traceability, thereby facilitating safety assurance and certification.