Nektarios Karanikas

Critical review of safety performance metrics

Various tools for safety performance measurement have been introduced in order to fulfil the need for safety monitoring in organisations, which is tightly related to their overall performance and achievement of their business goals. Such tools include accident rates, benchmarking, safety culture and climate assessments, cost-effectiveness studies, etc. The current work reviews the most representative methods for safety performance evaluation that have been suggested and applied by a variety of organisations, safety authorities and agencies. This paper discusses several viewpoints of the applicability, feasibility and appropriateness of such tools, based on the viewpoints of managers and safety experts involved in a relevant research that was conducted in a large aviation organisation. The extensive literature cited, the discussion topics, along with the conclusions and recommendations derived, might be considered by any organisation that seeks a realistic safety performance assessment and establishment of effective measurement tools.

Combining soft system methodology and Pareto analysis in the assessment of safety management performance: an aviation case

Although reengineering is strategically advantageous for organisations in order to keep functional and sustainable, safety must remain a priority and respective efforts need to be maintained. This paper suggests the combination of soft system methodology (SSM) and Pareto analysis on the scope of safety management performance evaluation, and presents the results of a survey, which was conducted in order to assess the effectiveness, efficacy and ethicality of the individual components of an organisations safety program. The research employed quantitative and qualitative data and ensured a broad representation of functional managers and safety professionals, who collectively hold the responsibility for planning, implementing and monitoring safety practices. The results showed that SSM can support the assessment of safety management performance by revealing weaknesses of safety initiatives, and Pareto analysis can underwrite the prioritisation of the remedies required. The specific methodology might be adapted by any organisation that requires a deep evaluation of its safety management performance, seeks to uncover the mechanisms that affect such performance, and, under limited resources, needs to focus on the most influential deficiencies.

Hazard Analysis and Safety Requirements for Small Drone Operations: To What Extent Do Popular Drones Embed Safety?

Currently, published risk analyses for drones refer mainly to commercial systems, use data from civil aviation, and are based on probabilistic approaches without suggesting an inclusive list of hazards and respective requirements. Within this context, this article presents: (1) a set of safety requirements generated from the application of the systems theoretic process analysis (STPA) technique on a generic small drone system; (2) a gap analysis between the set of safety requirements and the ones met by 19 popular drone models; (3) the extent of the differences between those models, their manufacturers, and the countries of origin; and (4) the association of drone prices with the extent they meet the requirements derived by STPA. The application of STPA resulted in 70 safety requirements distributed across the authority, manufacturer, end user, or drone automation levels. A gap analysis showed high dissimilarities regarding the extent to which the 19 drones meet the same safety requirements. Statistical results suggested a positive correlation between drone prices and the extent that the 19 drones studied herein met the safety requirements generated by STPA, and significant differences were identified among the manufacturers. This work complements the existing risk assessment frameworks for small drones, and contributes to the establishment of a commonly endorsed international risk analysis framework. Such a framework will support the development of a holistic and methodologically justified standardization scheme for small drone flights.

Safety Culture Development : The Gap Between Industry Guidelines and Literature, and the Differences Amongst Industry Sectors.

Reason’s typology of safety culture (i.e. Just, Informative, Learning, Flexible and Reporting cultures) is widely used in the industry and academia. Through literature review we developed a framework including 36 markers that reflect the operationalization of Reason’s sub-cultures and general organizational prerequisites. We used the framework to assess to what extent safety culture development guidelines of seven industry sectors (i.e. aviation, railway, oil and gas, nuclear, healthcare, defense and maritime) incorporate academic references, and are similar to each other. Gap analysis and statistics showed that the guidelines include 53–69 % of the safety culture markers, with significant differences across subcultures and industry sectors. The results suggested that there is a gap between the industry guidelines and literature, as well as variant approaches to safety culture across the industry. The framework suggested in the study might be used as reference for completing existing safety culture development plans and constructing safety culture assessment instruments.

Safety Metrics Based on Utilisation of resources

The objective of the study described in this paper is to define safety metrics that are based on the utilisation of resources. The background of this research is a specific need of the aviation industry where small and medium-sized enterprises lack large amounts of safety-related data to measure and demonstrate their safety performance proactively. The research department of the Aviation Academy has initiated a 4-year study, which will test the possibility to develop new safety indicators that will be able to represent safety levels proactively without the benefit of large data sets. The research team has reviewed the academic and professional literature about safety performance indicators and has performed surveys into 13 companies in order to explore what, how, and why safety performance indicators are used and whether there is a statistically linear relation between SMS process metrics and safety outcomes. The preliminary results showed that companies do not use data from all SMS processes in the development of safety performance indicators, they do not ground the selection of indicators on specific criteria, they implement SMS process in different ways, but they are eager to use alternative metrics, including ones potentially to be derived on the basis of contemporary safety models and views. As part of the development of alternative safety metrics, safety performance indicators were defined that are based on the difference between required resources and available resources. Resources are people, time, equipment and budget. This work is inspired by the general notion that a large gap between ‘work as imagined’ and ‘work as done’ has a negative influence on the level of safety. Work as imagined in this context is represented by available resources and work as done by required resources. The metrics were defined by a combination of literature research and semi-structured interviews with operational practitioners in the aviation industry. The suitability of the metrics will subsequently be tested in pilot studies within the aviation industry.

The Balance Between Safety and Productivity and its Relationship with Human Factors and Safety Awareness and Communication in Aircraft Manufacturing

Background This paper presents the findings of a pilot research survey which assessed the degree of balance between safety and productivity, and its relationship with awareness and communication of human factors and safety rules in the aircraft manufacturing environment. Methods The study was carried out at two Australian aircraft manufacturing facilities where a Likert-scale questionnaire was administered to a representative sample. The research instrument included topics relevant to the safety and human factors training provided to the target workforce. The answers were processed in overall, and against demographic characteristics of the sample population. Results The workers were sufficiently aware of how human factors and safety rules influence their performance and acknowledged that supervisors had adequately communicated such topics. Safety and productivity seemed equally balanced across the sample. A preference for the former over the latter was associated with a higher awareness about human factors and safety rules, but not linked with safety communication. The size of the facility and the length and type of employment were occasionally correlated with responses to some communication and human factors topics and the equilibrium between productivity and safety. Conclusion Although human factors training had been provided and sufficient bidirectional communication was present across the sample, it seems that quality and complexity factors might have influenced the effects of those safety related practices on the safety–productivity balance for specific parts of the population studied. Customization of safety training and communication to specific characteristics of employees may be necessary to achieve the desired outcomes.

Exploring the line between the preventive and punitive character of measures in the frame of a just culture

Abstract This paper presents the results of a survey which aimed at exploring the perceptions of the employees of a large aviation organization regarding the punitive or preventive character of possible measures that management could take in cases of errors and violations. The analysis of the quantitative and qualitative data collected showed that the viewpoints of the staff were sufficiently aligned only for half of the measures and that all measures inflicting a short- or long-term dissociation of the end-users from their current working place and function were highly unfavourable and linked to castigation. Also, statistically significant differences were observed across groups of specialties and years of service regarding the appropriateness of specific measures especially in the case of errors. The findings of this study in combination with literature references, suggest that the establishment of a just culture structure with agreed lines between the punitive and preventive character of measures and its endorsement by the employees is achievable, but it requires a bottom-up approach and periodical revision. Similar exploratory research, complemented possibly by explanatory studies, is recommended to be carried out by organizations prior or during the development of their just culture policy and related measures.

Exploiting data from safety investigations and processes to assess performance of safety management aspects

Abstract This paper presents an alternative way to use records from safety investigations as a means to support the evaluation of safety management (SM) aspects. Datasets from safety investigation reports and progress records of an aviation organization were analyzed with the scope of assessing safety management’s role, speed of safety communication, timeliness of safety investigation processes and realization of safety recommendations, and the extent of convergence among SM and investigation teams. The results suggested an interfering role of the safety department, severe delays in safety investigations, timely implementation of recommendations, quick dissemination of investigation reports to the end-users, and a low ratio of investigation team recommendations included in the final safety investigation reports. The results were attributed to non-scalable safety investigation procedures, ineffective resource management, lack of consistent bidirectional communication, lack of investigators’ awareness about the overall organizational context, and a weak commitment of other departments to the realization of safety recommendations. The set of metrics and the combination of quantitative and qualitative methods presented in this paper can support organizations to the transition towards a performance-based evaluation of safety management.

Measuring Safety Through the Distance Between System States with the RiskSOAP Indicator

Modern engineering systems are complex socio-technical structures with a mission to offer services of high quality, while in parallel ensuring profitability for their owners. However, practice has shown that accidents are inevitable, and the need for the use of systems-theoretic tools to support safety-driven design and operation has been acknowledged. As indicated in accident investigation reports, the degradation of risk situation awareness (SA) usually leads to safety issues. However, the literature lacks a methodology to compare existing systems with their ideal composition, which is likely to enhance risk SA. To fill this gap, the risk SA provision (RiskSOAP) is a comparison-based methodology and goes through three stages: (1) determine the desired/ideal system composition, (2) identify the as-is one(s), (3) employ a comparative strategy to depict the distance between the compared units. RiskSOAP embodies three methods: STPA (System Theoretic Process Analysis), EWaSAP (Early Warning Sign Analysis) and dissimilarity measures. The practicality, applicability and generality of RiskSOAP is demonstrated through its application to three case studies. The purpose of this work is to suggest the RiskSOAP indicator as a measure for safety in terms of the gap between system design and operation, thus increasing system’s risk SA. RiskSOAP can serve as a criterion for planning system modifications or selecting between alternative systems, and can support the design, development, operation and maintenance of safe systems.

Meeting report: The 3rd European STAMP Workshop, Amsterdam, The Netherlands

Traditional system safety approaches are being challenged by the introduction of new technology and the increasing complexity of the systems we build and operate. System Theoretic Accident Model and Processes (STAMP) is a new systems thinking approach that w as introduced in Nancy Leveson’s book “Engineering a Safer World” (MIT Press, 2012). While relatively new, the STAMP approach and its respective tools and methods are already being used in many industry sectors such as space, aviation, automotive, software, security medical, defence and nuclear. In more than 150 published articles, the potential of the STAMP family of tools to identify more hazards and causal factors than traditional tools (e.g, FTA, HAZOP, FMEA) has been demonstrated...