Martha Grabowski

Risk Mitigation in Large-Scale Systems: Lessons from High Reliability Organizations

This article argues that we should turn our attention from single organizations to systems of organizations if we are to manage in a way that reduces the potential for catastrophic outcomes in organizations. Risk mitigation measures for large-scale systems are derived from research on high reliability organizations (HROs). The article focuses on characteristics similar to both types of systems—which include simultaneous autonomy and interdependence, intended and unintended consequences of behavior, long incubation periods during which problems can arise, and risk migration—and shows how risk mitigation principles that evolved from HRO research can be applied to large-scale systems.

Using system simulation to model the impact of human error in a maritime system

Human error is cited as the predominant cause of transportation accidents. This paper describes the modeling of human error related accident event sequences in a risk assessment of maritime oil transportation in Prince William Sound, Alaska. The risk analysts were confronted with incomplete and misleading data that made it difficult to use theoretical frameworks. They were required, therefore, to make significant modeling assumptions in order to produce valid and useful results. A two stage human error framwork was developed for the Prince William Sound Risk Assessment based on interviews with maritime experts. Conditional probabilities implied by this framework were elicited from system experts (tanker masters, mates, engineers, and state pilots) and used within a dynamic simulation to produce the risk analysis base case results discussed. The ability to quantify the effectiveness of proposed risk reduction interventions aimed at reducing human and organizational error were limited by the level of detail described by the taxonomy of human error.

The Prince William Sound Risk Assessment

After the grounding of the Exxon Valdez and its subsequent oil spill, all parties with interests in Prince William Sound (PWS) were eager to prevent another major pollution event. While they implemented several measures to reduce the risk of an oil spill, the stakeholders disagreed about the effectiveness of these measures and the potential effectiveness of further proposed measures. They formed a steering committee to represent all the major stakeholders in the oil industry, in the government, in local industry, and among the local citizens. The steering committee hired a consultant team, which created a detailed model of the PWS system, integrating system simulation, data analysis, and expert judgment. The model was capable of assessing the current risk of accidents involving oil tankers operating in the PWS and of evaluating measures aimed at reducing this risk. The risk model showed that actions taken prior to the study had reduced the risk of oil spill by 75 percent, and it identified measures estimated to reduce the accident frequency by an additional 68 percent, including improving the safety-management systems of the oil companies and stationing an enhanced-capability tug, called the Gulf Service, at Hinchinbrook Entrance. In all, various stakeholders made multimillion dollar investments to reduce the risk of further oil spills based on the results of the risk assessment.

A Risk Management Procedure for the Washington State Ferries

The state of Washington operates the largest passenger vessel ferry system in the United States. In part due to the introduction of high-speed ferries, the state of Washington established an independent blue-ribbon panel to assess the adequacy of requirements for passenger and crew safety aboard the Washington state ferries. On July 9, 1998, the Blue Ribbon Panel on Washington State Ferry Safety engaged a consultant team from The George Washington University and Rensselaer Polytechnic Institute/Le Moyne College to assess the adequacy of passenger and crew safety in the Washington state ferry (WSF) system, to evaluate the level of risk present in the WSF system, and to develop recommendations for prioritized risk reduction measures, which, once implemented, can improve the level of safety in the WSF system. The probability of ferry collisions in the WSF system was assessed using a dynamic simulation methodology that extends the scope of available data with expert judgment. The potential consequences of collisions were modeled in order to determine the requirements for onboard and external emergency response procedures and equipment. The methodology was used to evaluate potential risk reduction measures and to make detailed risk management recommendations to the blue-ribbon panel and the Washington State Transportation Commission.

Risk modeling in distributed, large-scale systems

Risk is inherent in distributed, large-scale systems. The paper explores the challenges of risk modeling in such systems, and suggests a risk modeling approach that is responsive to the requirements of complex, distributed, large-scale systems. An example of the use of the approach in the marine transportation system is given. The paper concludes with a discussion of limitations of the approach and of future work.

Accident precursors and safety nets: Leading indicators of tanker operations safety

Leading indicators, one type of accident precursor, are conditions, events or measures that precede an undesirable event and that have some value in predicting the arrival of the event, whether it is an accident, incident, near miss, or undesirable safety state. Leading indicators are associated with proactive activities that identify hazards and assess, eliminate, minimize and control risk. An empirical analysis of leading indicators of safety for an international energy transportation company was undertaken, utilizing a previously validated research model. Quantitative safety performance and qualitative safety culture data were obtained from 943 participants on 37 vessels from three fleets in the organization. Organizational, vessel and individual safety factors and leading indicators were identified and an analysis of fleet, vessel, and individual safety cultures was undertaken. The results indicate that individual and vessel-level leading indicators can provide important input to an organizations continuous safety measuring and monitoring systems.

Human performance and embedded intelligent technology in safety-critical systems

Information technology continues to evolve rapidly. We see this particularly in the evolution of embedded intelligent systems--knowledge-based systems deployed in larger hosts with real-time response requirements, which provide real-time advice, guidance, information, recommendations and explanations to their users. These systems have recently been deployed in safety-critical large-scale systems, where humans and technology are jointly responsible for executing tasks, monitoring operations, and providing system safety. Thus, human interaction with intelligent technology in safety-critical systems has important implications. Those interactions can enhance or reduce system efficiency, enhance or compromise safety, and augment or negate the other benefits that technology provides. In this paper, we focus on interactions between human operators and embedded intelligent systems. We first consider the role of technology in safety-critical systems, and discuss studies of the impact of technology on human operators in such systems. We then describe embedded intelligent systems, and studies of their impacts on human operators. To illustrate these points, we consider the case of embedded intelligent technology introduction in one such setting, and the results of an empirical investigation of the impact of the technology on human performance in that system. We conclude with a discussion of the implications of the study and of the importance of understanding the impact of embedded intelligent technology on human operators in safetycritical systems.

Sailing on Friday: Developing the Link Between Safety Culture and Performance in Safety-Critical Systems

As large-scale organizations and systems become more complex and difficult to understand, and with the notable failure of organizational practices to prevent disaster in safety-critical systems, interest in identifying leading indicators (LI) of adverse events in safety-critical settings has increased. This paper examines the question of how to provide early warning of adverse events to groups in complex safety-critical settings by understanding the connection between safety culture and performance. Significant safety factors that were correlated with safety performance were identified as LIs of safety, using safety culture survey data from three industry partners gathered over a two-year period. Complementary techniques suggest further use of multiple analysis techniques in order to understand the structure, variance, and power of the identified LIs, as well as to provide validation methods for the results.

Evaluation of Embedded Intelligent Real‐time Systems*

Over the past two decades, questions have surfaced about the effectiveness and contribution of intelligent systems to decision makers in a variety of settings. This paper focuses on the evaluation challenges associated with intelligent real-time software systems that are embedded in larger host systems. With the proliferation of such systems in operational settings such as aerospace, medical, manufacturing, and transportation systems, increased attention to evaluations of such systems, and to resulting software safety, is warranted. This paper describes one such evaluation and proposes a set of evaluation criteria for embedded intelligent real-time systems (EIRTS). Implications of the evaluation and the evaluation criteria are discussed.

Social Media and Warning Response Impacts in Extreme Events: Results from a Naturally Occurring Experiment

Our understanding of the impacts of social media on individuals who receive warnings of extreme events is limited. There is to date no uniform approach to integrating social media as part of emergency management strategies. This research addresses the question of the role of social media in the effectiveness of the warning response process in the context of a naturally occurring experiment. The results of the experiment contribute to our understanding of how social media complements as well as facilitates the warning response process.