Rudy Darken

Virtual Environments and the Enhancement of Spatial Behavior: Towards a Comprehensive Research Agenda

There is currently much research activity involving virtual environments (VEs) and spatial behavior (spatial perception, cognition, and performance). After some initial remarks describing and categorizing the different types of research being conducted on VEs and spatial behavior, discussion in this Forum paper focuses on one specific type, namely, research concerned with the use of VE technology for training spatial behavior in the real world. We initially present an overview of issues and problems relevant to conducting research in this area, and then, in the latter portion of the paper, present an overview of the research that we believe needs to be done in this area. We have written this paper for the forum section of Presence because, despite its length, it is essentially an opinion piece. Our aim here is not to report the results of research in our own laboratory nor to review the literature, as other available papers already serve these goals. Rather, the primary purpose of this paper is to stimulate open discussion about needed future research. In general, we believe that such a discussion can serve the research establishment as much as reports of completed work.

Critical Infrastructure as Complex Emergent Systems

The United States Department of Homeland Security (DHS) is charged with “build[ing] a safer, more secure, and more resilient America by enhancing protection of the Nation’s Critical infrastructure and key resources (CI/KR) ...” using an all-hazards approach. The effective implementation of this strategy hinges upon our understanding of catastrophes and their potential effect on the functioning of our infrastructure. Unfortunately, there has been no unifying theory of catastrophe to guide decisionmaking, preparedness, or response. We do not know, for example, why some catastrophes are “worse” than others, or if the rate of catastrophes is increasing or decreasing. Furthermore, DHS has adopted a risk-informed decision-making process, but has done so without defining key terms, such as “risk”, or quantifying the primary elements of risk – definitions that are badly needed before setting a course of action and allocating resources. We present a framework, based upon network science and normal accident theory that can be used to guide policy decisions for homeland security. We show that exceedance probability, which is commonly used by the insurance industry to set hazard insurance premiums, provides a unifying policy framework for homeland security investments. Furthermore, since the exceedance probability for catastrophic consequences obeys a power law, we define resilience, explicitly, as the exponent of that power law. This allows a mathematical definition of resilience that resonates with our innate sense of resilience. That is, the more resilient a given system, the larger it’s resiliency exponent. Such an approach also allows one to classify hazards as ‘high’ or ‘low’ risk, according to the resiliency exponent, and to guide investments towards prevention or response. This framework provides a more rigorous foundation for Federal investment decisions and a rational basis for policies to best protect the Nation’s infrastructure. A strategy without a theory The United States Department of Homeland Security (DHS) is charged with the responsibility of “build[ing] a safer, more secure, and more resilient America by enhancing protection of the Nation’s Critical infrastructure and key resources (CI/KR) to prevent, deter, neutralize, or mitigate the effects of deliberate efforts by terrorists to destroy, incapacitate, or exploit them; and to strengthen national preparedness, timely response, and rapid recovery in the event of an attack, natural disaster, or other emergency.” The homeland security strategy is considered all-hazards because it embraces both natural and human-made catastrophes such as Hurricane Katrina, and the 9/11 Terrorist attacks. The effective implementation of the all-hazards strategy hinges upon our understanding of catastrophes: earthquakes and wild fires in Southern California; hurricanes in Florida; terrorist attacks on infrastructure; and pandemic threats such as the H1N1 influenza. Unfortunately, there has been no unifying theory of catastrophe to guide decisionmaking, preparedness, or response. We do not know, for example, why some catastrophes are “worse” than others, or if the rate of catastrophes is increasing or decreasing. Moreover, we do not know what properties of a human or natural system contribute to fragility or resilience. This lack of understanding has led to organizational confusion (what is the goal?), duplication of effort (different agencies doing the same thing), and poor utilization of limited resources (inadequate identification of the most at-risk assets, maximal return on investment, and resourcing of adequate response capability). DHS has adopted a riskinformed decision-making process, but has done so without defining key terms such as “risk” or quantifying the primary elements of risk: “threat”, “vulnerability”, “resilience”, and “consequence” – terms used throughout DHS policy and strategy documents. Riskinformed decisions are difficult to make without operational definitions of risk and resiliency! For example, the National Strategy for the Physical Protection of Critical Infrastructure and Key Assets recommends, “the first objective of this strategy is to identify and assure the protection of those assets, systems, and functions that we deem most ‘critical’ in terms of national-level public health and safety, governance, economic and national security, and public confidence. We must develop a comprehensive, prioritized assessment of facilities, systems, and functions of national-level criticality and monitor their preparedness across infrastructure sectors.” This is a laudable objective, but since 2003 DHS has not been able to define ‘critical’, ‘prioritization’, or ‘preparedness’ – definitions that are badly needed before setting a course of action and allocating precious resources. The authors claim this malady will continue to persist until a suitable theory of catastrophe is developed and turned into practice. We propose a theory of all-hazards catastrophe, the results of which can be used to guide policy decisions for homeland security. Our theory is based on network science and normal accident theory. In a related approach, Ramo borrows on ideas taken from physical science to explain how political disasters happen. Ramo’s ideas were previously explored and illustrated by Buchanan in a broader context. Similarly, Taleb’s highly popular book on randomness lays the foundation for some of the ideas expressed in the author’s theory of catastrophe – specifically addressing the claim that many catastrophes are the result of random processes, rather than deterministic cause-andeffects. While Taleb focuses on “black swans” – highly unlikely, highly consequential, unpredictable events, we argue that black swans are statistically predictable and follow a power law exceedence probability distribution. Lewis applied the theory of complex systems to critical infrastructure and showed the relationship between power laws, black swans, and normal accident theory to critical infrastructure systems. Thus, power laws appear to be fundamental to catastrophe theory, which raises the question of “why”? Our answer: catastrophic events, including black swans, are normal accidents that increase with increasing self-organization.

Deriving Training Strategies for Spatial Knowledge Acquisition From Behavioral, Cognitive, and Neural Foundations

While much has been made of the potential uses for virtual environment (VE) technologies as training aids, there are few guidelines and strategies to inform system development from the user’s perspective. Assumptions are that a human factors-based evaluation will ensure optimal performance, transferring training from virtual to real worlds; however, there are complex, yet unexplored, issues surrounding system optimization and employment. A comprehensive investigation into the foundations of training, traversing levels of performance analysis, from overt behavioral responses to the less explicit neuronal patterns, is proposed from which optimal training strategies can be inferred and system development guidelines deduced.

This year in the MOVES institute

The MOVES Institutes mission is research, application,and education in the grand challenges of modeling, virtualenvironments, and simulation. Specialties are 3D visualsimulation, networked virtual environments, computer-generatedautonomy, human-performance engineering,immersive technologies, defense /entertainmentcollaboration, and evolving operational modeling.

When is VE Training Effective? A Framework and Two Case Studies

There are a number of studies that suggest that virtual environment (VE) systems can facilitate transfer of training (Darken & Peterson, 2002; Péruch, Belingard, & Thinus-Blanc, 2000; Rose et al., 2001). Yet this benefit is not universal, as Brooks et al. (2002) found that VE training was not beneficial for a recognition task. Thus, VE training may facilitate some types of training tasks, while not benefiting others. As technology pushes training more frequently into highly immersive environments, it is important to delineate and examine characteristics of VE trainers (such as egocentric perspective or multimodal interactivity) and consider the impact these characteristics have on training tasks and desired outcomes. Towards this end, a preliminary framework is herein presented which suggests that specific characteristics of VE systems impact specific training outcomes. This effort presents two operationally-based case studies that begin to examine different parts of the proposed framework. The first study examines how egocentric perspective was introduced into helicopter navigator training to impact spatial knowledge acquisition; the second study investigates how a VE was used to introduce interactivity into training to improve training outcomes for procedural knowledge. Given the low number of participants, no substantive conclusions can be made; rather, the studies are framed as an initial approach to VE training effectiveness within a theoretical model.

Remembering Nat Durlach

In the early 1990s, I was a graduate student at George Washington University and working at the Naval Research Laboratory in Washington, D.C. in one of the first VE labs established there. Through a mutual colleague, I met Terry Allard, who was then a program officer with the Office of Naval Research (ONR). Terry took an interest in my dissertation research, which had to do with spatial orientation and navigation in large virtual spaces. Shortly thereafter, there was interest at ONR in creating a research program based on some of those ideas. Terry contacted me and said that he wanted to pair me up with his mentor from when he was a graduate student at MIT, and that we should get together and brainstorm what such a research program might be about. That mentor, of course, was Nat Durlach. I was a very green researcher at the time, still a year or two away from completing my doctorate. I knew of Nat’s research in spatialized audio but I had not crossed paths with him up to that point. What did I know about Nat Durlach? I found out. Reflecting back, it was one of those moments when you realize how little you really know and how small your world actually is. I contacted Nat, we talked on the phone, and he invited me to spend a couple of days at MIT with him and his team. I arrived there the morning of our first face-to-face meeting and was shown into a small conference room. In walked Nat with a pad of paper and a fistful of black pens. I should have realized that was a hint of what I was in for. After introductions and a few opening comments, we set to work. Mike Zyda often uses the term ‘‘clear thinker’’ to refer to a person who can quickly and efficiently organize vast amounts of complex information into something comprehensible. By that definition, Nat was the clearest thinker I ever met. He probed me for hours about what I knew, what I thought I knew, and what I didn’t know. We argued about what topics were foundational knowledge and which were not. We formulated theoretic approaches and associated research questions and talked about prioritization. He was deeply concerned about measurement, both developing valid measures and ensuring that they were reliable and reproducible. I was schooled on what it meant to be a scientist and on what constituted good science. Nat scribbled furiously with his black pens, drafting page after page, methodically working through the problems we identified. Everyone in the room participated. There was real energy there. We were excited about what we were doing. He had the rare ability to organize and relate many ideas simultaneously in his mind because I recall that our notes from the meeting were surprisingly structured and readable. In the end, we had crafted a document that was an important contribution to Terry’s new program. In subsequent years, Nat and I were given the opportunity to explore some of the ideas that we had proposed. A number of experiments resulted and I was able to see how he executed a study and presented the results in a way that had meaning and practical utility. He became a leader in the larger ONR-funded VETT (Virtual Environment Training Technology) program that investigated the use of VEs for training in a much wider context. In that role, he had a unique capacity to drive a discussion to a specific point, and just when you thought you had the issue resolved, he’d say something that made you realize how big the problem actually was and that you’d only been looking at one small part of it.

Network–based risk assessment of the US crude pipeline infrastructure

The purpose of this project was to assess the nation’s US crude oil pipeline network and identify the hubs that would have the greatest financial impact if their operation were disrupted by a terrorist attack, natural disaster, or other catastrophic event. The resilience of the crude oil pipeline network was analysed using the model-based risk analysis (MBRA) software tool developed by the Naval Post Graduate School Center for Homeland Defense and Security. Results identified five critical hubs, with the most critical hub being the Cushing, OK Trading Hub (COTH). A disruption in operation of the COTH or any of the five critical hubs would have far reaching negative consequences, creating long lasting political, psychological, and economic impacts. Based on the results of this assessment, it is recommended that the critical hubs be considered critical national assets and protected as such.