Jana L. Schwartz

ALHAT System Architecture and Operational Concept

An autonomous lunar landing system applicable to a wide variety of crewed and robotic lunar descent vehicles is under development as part of the ALHAT (autonomous precision landing and hazard detection and avoidance technology) project. This system, referred to as the ALHAT system module (ASM) is a highly advanced integrated sensor suite that enables landing a lunar descent vehicle within tens of meters of a certified and designated landing location anywhere on the Moon, under any lighting condition. This paper describes the basic ASM architecture and its novel concept of operations, and matures this architecture through description of top level lunar landing requirements. Working closely with NASA primary stakeholders, a fully developed ASM design will enable global lunar access for exploration of unique and challenging areas on the lunar surface never before visited.

Modeling strategic use of human computer interfaces with novel hidden Markov models

Immersive software tools are virtual environments designed to give their users an augmented view of real-world data and ways of manipulating that data. As virtual environments, every action users make while interacting with these tools can be carefully logged, as can the state of the software and the information it presents to the user, giving these actions context. This data provides a high-resolution lens through which dynamic cognitive and behavioral processes can be viewed. In this report, we describe new methods for the analysis and interpretation of such data, utilizing a novel implementation of the Beta Process Hidden Markov Model (BP-HMM) for analysis of software activity logs. We further report the results of a preliminary study designed to establish the validity of our modeling approach. A group of 20 participants were asked to play a simple computer game, instrumented to log every interaction with the interface. Participants had no previous experience with the games functionality or rules, so the activity logs collected during their naïve interactions capture patterns of exploratory behavior and skill acquisition as they attempted to learn the rules of the game. Pre- and post-task questionnaires probed for self-reported styles of problem solving, as well as task engagement, difficulty, and workload. We jointly modeled the activity log sequences collected from all participants using the BP-HMM approach, identifying a global library of activity patterns representative of the collective behavior of all the participants. Analyses show systematic relationships between both pre- and post-task questionnaires, self-reported approaches to analytic problem solving, and metrics extracted from the BP-HMM decomposition. Overall, we find that this novel approach to decomposing unstructured behavioral data within software environments provides a sensible means for understanding how users learn to integrate software functionality for strategic task pursuit.

A unified framework for capturing concept development methods

Concept development in engineering is a challenging exercise whose execution varies greatly with the context of the problem being solved. Engineers have a plethora of non-domain specific methods available to aid different aspects of concept development and problem solving. As engineering has evolved, design processes, which are assemblies of some of these different methods, have been used to identify how methods relate and can be used to define and solve a larger problem. Yet, because methods are constantly being updated and added by the research community, design processes may not incorporate the latest methods. Furthermore, processes are generally tied to the notion of development in phases, which may not be the appropriate discriminators for determining a methods applicability.

Operationalizing Engagement with Multimedia as User Coherence with Context

Traditional approaches for assessing user engagement within multimedia environments rely on methods that are removed from the human computer interaction itself, such as surveys, interviews and baselined physiology. We propose a context coherence approach that operationalizes engagement as the amount of independent user variation that covaries in time with multimedia contextual events during unscripted interactions. This can address questions about the features of multimedia users are most engaged and how engaged users are without the need for prescribed interactions or baselining. We assessed the validity of this approach in a psychophysiological study. Forty participants played interactive video games. Intake and post-stimulus questionnaires collected subjective engagement reports that provided convergent and divergent validity criteria to evaluate our approach. Estimates of coherence between physiological variation and in-game contextual events predicted subjective engagement and added information beyond physiological metrics computed from baselines taken outside of the multimedia context. Our coherence metric accounted for task-dependent engagement, independent of predispositions; this was not true of a baselined physiological approach that was used for comparison. Our findings show compelling evidence that a context-sensitive approach to measuring engagement overcomes shortcomings of traditional methods by making best use of contextual information sampled from multimedia in time-series analyses.

Concept development of a multi-vehicle system for an operationally responsive mission

Draper Laboratory has developed a framework that describes engineering problem solving for conceptual design. The framework serves as a guide for conducting the difficult and important early stages of design, and does so regardless of concept domain. Further, it provides the organizational structure for a catalog of methods that can be collaboratively used to find solutions to complex problems. This paper documents the efficacy of the framework and method catalog against an Operationally Responsive System (ORS) problem.

Multi-modal Interruptions on Primary Task Performance

In this paper we have investigated a range of multi-modal displays (visual, auditory, haptic) to understand the effects of interruptions across various modalities on response times. Understanding these effects is particularly relevant in complex tasks that require perceptual attention, where pertinent information needs to be delivered to a user, e.g., driving. Multi-modal signal presentation, based on the Multiple Resource Theory framework, is a potential solution. To explore this solution, we conducted a study in which participants perceived and responded to a secondary task while conducting a visual, auditory, and haptic vigilance task during a driving scenario. We analyzed response times, errors, misses, and subjective responses and our results indicated that haptic interruptions of a primarily haptic task can be responded to the fastest, and visual interruptions are not the preferred modality in a driving scenario. With the results of this study, we can define logic for a context-based framework to better determine how to deliver incoming information in a driving scenario.

Augmenting Smart Buildings and Autonomous Vehicles with Wearable Thermal Technology

Smart buildings and autonomous vehicles are expected to see rapid growth and adoption in the coming decades. Americans spend over 90% of their lives in buildings or automobiles, meaning that 90% of their lives could be spent interfacing with intelligent environments. EMBR Labs has developed EMBR WaveTM, a wearable thermoelectric system, for introducing thermal sensation as a connected mode of interaction between smart environments and their occupants. In this paper we highlight applications of wearable thermal technology for passengers in autonomous vehicles and occupants of smart buildings. Initial findings, collected through partnerships with Draper and UC Berkeley, respectively, are presented that illustrate the potential for wearable thermal technology to improve the situational awareness of passengers in autonomous vehicles and improve personal comfort in smart buildings.

A Systems Approach for Augmented Reality Design

Effective ways of presenting digital data are needed to augment a users experience in the real world without distracting or overloading them. We propose a system of systems approach for the design, development, and evaluation of information presentation devices, particularly augmented reality devices. We developed an evaluation environment that enables the synchronized presentation of multimodal stimuli and collection of user responses in an immersive environment. We leveraged visual, audio, thermal, and tactile information presentation modalities during a navigation and threat identification task. Twelve participants completed the task while response time and accuracy data were collected. Results indicated variability among devices and pairs of devices, and suggested that information presented by some pairs of devices was more effective and easily acted upon than that presented by others. The results of this work provided important guidance regarding future design decisions and suggest the utility of our system of systems approach. Implications and future directions are discussed.