Samuel W. Hincks

Dynamic difficulty using brain metrics of workload

Dynamic difficulty adjustments can be used in human-computer systems in order to improve user engagement and performance. In this paper, we use functional near-infrared spectroscopy (fNIRS) to obtain passive brain sensing data and detect extended periods of boredom or overload. From these physiological signals, we can adapt a simulation in order to optimize workload in real-time, which allows the system to better fit the task to the user from moment to moment. To demonstrate this idea, we ran a laboratory study in which participants performed path planning for multiple unmanned aerial vehicles (UAVs) in a simulation. Based on their state, we varied the difficulty of the task by adding or removing UAVs and found that we were able to decrease error by 35% over a baseline condition. Our results show that we can use fNIRS brain sensing to detect task difficulty in real-time and construct an interface that improves user performance through dynamic difficulty adjustment.

This is your brain on interfaces: enhancing usability testing with functional near-infrared spectroscopy

This project represents a first step towards bridging the gap between HCI and cognition research. Using functional near-infrared spectroscopy (fNIRS), we introduce tech-niques to non-invasively measure a range of cognitive workload states that have implications to HCI research, most directly usability testing. We present a set of usability experiments that illustrates how fNIRS brain measurement provides information about the cognitive demands placed on computer users by different interface designs.

Designing Implicit Interfaces for Physiological Computing: Guidelines and Lessons Learned Using fNIRS

A growing body of recent work has shown the feasibility of brain and body sensors as input to interactive systems. However, the interaction techniques and design decisions for their effective use are not well defined. We present a conceptual framework for considering implicit input from the brain, along with design principles and patterns we have developed from our work. We also describe a series of controlled, offline studies that lay the foundation for our work with functional near-infrared spectroscopy (fNIRS) neuroimaging, as well as our real-time platform that serves as a testbed for exploring brain-based adaptive interaction techniques. Finally, we present case studies illustrating the principles and patterns for effective use of brain data in human--computer interaction. We focus on signals coming from the brain, but these principles apply broadly to other sensor data and in domains such as aviation, education, medicine, driving, and anything involving multitasking or varying cognitive workload.

Brain-based target expansion

The bubble cursor is a promising cursor expansion technique, improving a users movement time and accuracy in pointing tasks. We introduce a brain-based target expansion system, which improves the efficacy of bubble cursor by increasing the expansion of high importance targets at the optimal time based on brain measurements correlated to a particular type of multitasking. We demonstrate through controlled experiments that brain-based target expansion can deliver a graded and continuous level of assistance to a user according to their cognitive state, thereby improving task and speed-accuracy metrics, even without explicit visual changes to the system. Such an adaptation is ideal for use in complex systems to steer users toward higher priority goals during times of increased demand.

Phylter: A System for Modulating Notifications in Wearables Using Physiological Sensing

As wearable computing becomes more mainstream, it holds the promise of delivering timely, relevant notifications to the user. However, these devices can potentially inundate the user, distracting them at the wrong times and providing the wrong amount of information. As physiological sensing also becomes consumer-grade, it holds the promise of helping to control these notifications. To solve this, we build a system Phylter that uses physiological sensing to modulate notifications to the user. Phylter receives streaming data about a user’s cognitive state, and uses this to modulate whether the user should receive the information. We discuss the components of the system and how they interact.

Using fNIRS for Real-Time Cognitive Workload Assessment

In this paper, we evaluate the possibility of detecting continuous changes in the users cognitive workload using functional near-infrared spectroscopy fNIRS. We dissect the source of meaning in a large collection of n-backs and argue that the problem of controlling the content of a participants mind poses a major problem for calibrating an algorithm using black box machine learning. We therefore suggest that the field simplify its task, and begin to focus on building algorithms that work on specialized subjects, before adapting these to a wider audience.

Building implicit interfaces for wearable computers with physiological inputs: zero shutter camera and phylter

We propose implicit interfaces that use passive physiological input as additional communication channels between wearable devices and wearers. A defining characteristic of physiological input is that it is implicit and continuous, distinguishing it from conventional event-driven action on a keyboard, for example, which is explicit and discrete. By considering the fundamental differences between the two types of inputs, we introduce a core framework to support building implicit interface, such that the framework follows the three key principles: Subscription, Accumulation, and Interpretation of implicit inputs. Unlike a conventional event driven system, our framework subscribes to continuous streams of input data, accumulates the data in a buffer, and subsequently attempts to recognize patterns in the accumulated data -- upon request from the application, rather than directly in response to the input events. Finally, in order to embody the impacts of implicit interfaces in the real world, we introduce two prototype applications for Google Glass, Zero Shutter Camera triggering a camera snapshot and Phylter filtering notifications the both leverage the wearers physiological state information.