Megan Strait

What we can and cannot (yet) do with functional near infrared spectroscopy.

Functional near infrared spectroscopy (NIRS) is a relatively new technique complimentary to EEG for the development of brain-computer interfaces (BCIs). NIRS-based systems for detecting various cognitive and affective states such as mental and emotional stress have already been demonstrated in a range of adaptive human–computer interaction (HCI) applications. However, before NIRS-BCIs can be used reliably in realistic HCI settings, substantial challenges oncerning signal processing and modeling must be addressed. Although many of those challenges have been identified previously, the solutions to overcome them remain scant. In this paper, we first review what can be currently done with NIRS, specifically, NIRS-based approaches to measuring cognitive and affective user states as well as demonstrations of passive NIRS-BCIs. We then discuss some of the primary challenges these systems would face if deployed in more realistic settings, including detection latencies and motion artifacts. Lastly, we investigate the effects of some of these challenges on signal reliability via a quantitative comparison of three NIRS models. The hope is that this paper will actively engage researchers to acilitate the advancement of NIRS as a more robust and useful tool to the BCI community.

The design, development, and deployment of a tabletop interface for collaborative exploration of genomic data

In this paper, we reflect on the design, development, and deployment of G-nome Surfer; a multi-touch tabletop user interface for collaborative exploration of genomic data. G-nome Surfer lowers the threshold for using advanced bioinformatics tools, reduces the mental workload associated with manipulating genomic information, and fosters effective collaboration. We describe our two-year-long effort from design strategy to iterations of design, development, and evaluation. This paper presents four main contributions: (1) a set of design requirements for supporting collaborative exploration in data-intensive domains, (2) the design, implementation, and validation of a multi-touch tabletop interface for collaborative exploration, (3) a methodology for evaluating the strengths and limitations of tabletop interaction for collaborative exploration, and (4) empirical evidence for the feasibility and value of integrating tabletop interaction in college-level education.

Let me tell you! investigating the effects of robot communication strategies in advice-giving situations based on robot appearance, interaction modality and distance

Recent proposals for how robots should talk to people when they give advice suggest that the same strategies humans employ with other humans are effective for robots as well. However, the evidence is exclusively based on people’s observation of robot giving advice to other humans. Hence, it is not clear whether the results still apply when people actually participate in real interactions with robots. We address this shortcoming in a novel systematic mixed-methods study where we employ both survey-based subjective and brain-based objective measures (using functional near infrared spectroscopy). The results show that previous results from observation conditions do not transfer automatically to interaction conditions, and that robot appearance and interaction distance are important modulators of human perceptions of robot behavior in advice-giving contexts.

Some Correlates of Agency Ascription and Emotional Value and Their Effects on Decision-Making

The prefrontal cortex (PFC) has been investigated extensively with functional magnetic resonance imaging (fMRI) and identified as a neural correlate of emotion regulation and decision-making, particularly in the context of moral utilitarian dilemmas. However, there are two limitations of previous work: (1) fMRI requires strict constraints on the physical experimental environment and (2) experimental manipulations have yet to consider the role of agency on the dilemma outcome and the corresponding neural activity. In this paper, we extend previous work by first evaluating an alternative neuroimaging technique, functional near infrared spectroscopy (NIRS), for observing decision-making processes in a less-constrained environment. We then examine the role of agency in deciding emotional (moral) and non-emotional dilemmas through a 2-part, 20-subject preliminary investigation. Our findings are two-fold: they suggest (1) NIRS is a potential alternative to fMRI in this decision-making context and (2) agency shows some influence on prefrontal neural activity, making NIRS a promising method for objective evaluation of agency and emotional value in human-agent interactions.

Measuring users' responses to humans, robots, and human-like robots with functional near infrared spectroscopy

The Uncanny Valley Hypothesis (UVH) describes the sudden change in a persons affect from affinity to aversion that is evoked by robots that border a human-like appearance. The portion of the human-likeness spectrum in which such aversion is posited to occur is referred to as the “uncanny valley”. However, evidence in support of the UVH is primarily based on subjectively assessed evaluations. Thus it remains an open question as to whether there are behavioral or neurophysiological manifestations of uncanny valley effects. To address this gap in literature, we investigated the activation of the anterior prefrontal cortex (PFC) - a region of the brain associated with emotion regulation - in response to a series of robots with varying human-likeness. We hypothesized that highly human-like robots - which have been found to receive negative subjective attributions - will also elicit increased activity in the PFC versus humans or robots with lesser degrees of human-likeness in accordance with the UVH. Our results show a “valley” in brain activity in the PFC corresponding to the valley observed via subjective measures alone, thus suggesting one neural manifestation (the PFC) of uncanny valley effects and further supporting the affective response (aversion) posited to occur by the UVH. However, the results also reveal a second “uncanny valley” in prefrontal hemodynamics, which suggests that the effects (and the contributing factors) are more complex than previously understood.

Reliability of NIRS-based BCIs: a placebo-controlled replication and reanalysis of brainput

Previously, we contributed to the development of a brain-computer interface (BCI), Brainput, using functional near infrared spectroscopy (NIRS). Initially Brainput was found to improve performance on a human-robot team task by adapting a robots autonomy using NIRS-based classifications of the users multitasking states [15, 16]. However, the failure to find any performance improvements in a follow-up study prompted reinvestigation of the original system via a reanalysis of Brainputs signal processing on a larger NIRS dataset and a placebo-controlled replication using random (instead of NIRS-based) state classifications. This reinvestigation revealed confounds in the original study responsible for the initial performance improvements, thus indicating that further work in signal processing is necessary to achieve reliable NIRS-based BCIs.

Using functional near infrared spectroscopy to measure moral decision-making: effects of agency, emotional value, and monetary incentive

The prefrontal cortex (PFC) has been investigated extensively with functional magnetic resonance imaging (fMRI) and identified as a neural substrate central to emotion regulation and decision-making, particularly in the context of utilitarian moral dilemmas. However, there are two important limitations to prior work: (1) fMRI imposes strict constraints on the physical environment of the participant and (2) experimental manipulations have yet to consider the role of agency and personal incentive on both brain-based and behavioral correlates. To address the first limitation, we investigated functional near infrared spectroscopy (NIRS), which showed it was a potential alternative to fMRI for observing the decision-making processes in a less-constrained environment [1]. To address the second, we examined the role of agency in deciding moral and non-moral dilemmas and whether the influences can be further modulated by way of monetary incentives. Our findings show that all three factors exert influences on both...

NIRS-Based BCIs: Reliability and Challenges

Previously we contributed to the development of a brain-computer interface (Brainput) using functional near infrared spectroscopy (NIRS). This NIRS-based BCI was designed to improve performance on a human-robot team task by dynamically adapting a robot’s autonomy based on the person’s multitasking state. Two multitasking states (corresponding to low and high workload) were monitored in real-time using an SVM-based model of the person’s hemodynamic activity in the prefrontal cortex. In the initial evaluation of Brainput’s efficacy, the NIRS-based adaptivity was found to significantly improve performance on the human-robot team task (from a baseline success rate of 45% to a rate of 82%). However, failure to find any performance improvements in an extension of the original evaluation prompted a reinvestigation of the system via: (1) a reanalysis of Brainput’s signal processing on a larger NIRS dataset and (2) a placebo-controlled replication using random (instead of NIRS-based) state classifications [1].