Christopher G. Courtney

Optimal Arousal Identification and Classification for Affective Computing Using Physiological Signals: Virtual Reality Stroop Task

A closed-loop system that offers real-time assessment and manipulation of a users affective and cognitive states is very useful in developing adaptive environments which respond in a rational and strategic fashion to real-time changes in user affect, cognition, and motivation. The goal is to progress the user from suboptimal cognitive and affective states toward an optimal state that enhances user performance. In order to achieve this, there is need for assessment of both 1) the optimal affective/cognitive state and 2) the observed user state. This paper presents approaches for assessing these two states. Arousal, an important dimension of affect, is focused upon because of its close relation to a users cognitive performance, as indicated by the Yerkes-Dodson Law. Herein, we make use of a Virtual Reality Stroop Task (VRST) from the Virtual Reality Cognitive Performance Assessment Test (VRCPAT) to identify the optimal arousal level that can serve as the affective/cognitive state goal. Three stimuli presentations (with distinct arousal levels) in the VRST are selected. We demonstrate that when reaction time is used as the performance measure, one of the three stimuli presentations can elicit the optimal level of arousal for most subjects. Further, results suggest that high classification rates can be achieved when a support vector machine is used to classify the psychophysiological responses (skin conductance level, respiration, ECG, and EEG) in these three stimuli presentations into three arousal levels. This research reflects progress toward the implementation of a closed-loop affective computing system.

Validity of the Virtual Reality Stroop Task (VRST) in active duty military

Virtual environments provide the ability to systematically deliver test stimuli in simulated contexts relevant to real world behavior. The current study evaluated the validity of the Virtual Reality Stroop Task (VRST), which presents test stimuli during a virtual reality military convoy with simulated combat threats. Active duty Army personnel (N = 49) took the VRST, a customized version of the Automated Neuropsychological Assessment Metrics (ANAM)–Fourth Edition TBI Battery (2007) that included the addition of the ANAM Stroop and Tower tests, and traditional neuropsychological measures, including the Delis-Kaplan Executive Function System version of the Color–Word Interference Test. Preliminary convergent and discriminant validity was established, and performance on the VRST was significantly associated with computerized and traditional tests of attention and executive functioning. Valid virtual reality cognitive assessments open new lines of inquiry into the impact of environmental stimuli on performance and offer promise for the future of neuropsychological assessments used with military personnel.

Virtual Reality Stroop Task for Neurocognitive Assessment

Given the prevalence of traumatic brain injury (TBI), and the fact that many mild TBIs have no external marker of injury, there is a pressing need for innovative assessment technology. The demand for assessment that goes beyond traditional paper-and-pencil testing has resulted in the use of automated cognitive testing for increased precision and efficiency; and the use of virtual environment technology for enhanced ecological validity and increased function-based assessment. To address these issues, a Virtual Reality Stroop Task (VRST) that involves the subject being immersed in a virtual Humvee as Stroop stimuli appear on the windshield was developed. This study is an initial validation of the VRST as an assessment of neurocognitive functioning. When compared to the paper-and-pencil, as well as Automated Neuropsychological Assessment Metrics versions of the Stroop, the VRST appears to have enhanced capacity for providing an indication of a participants reaction time and ability to inhibit a prepotent response while immersed in a military relevant simulation that presents psychophysiologically arousing high and low threat stimuli.

Virtual reality Stroop task for assessment of supervisory attentional processing

The current project is a preliminary attempt at assessing the potential of a three-dimensional virtual reality Stroop task (VRST) for investigation of supervisory attentional processing. The VRST was compared with paper-and-pencil and computer automated (two-dimensional) versions of the Stroop. Psychophysiological measures were also used to assess varying levels of user arousal. The three versions of the Stroop task were conducted with a sample of 50 healthy university students, between the ages of 18 and 28 years, which included equivalent distributions of men and women from ethnically diverse populations. No significant differences were found for age, gender, or education. Results indicated that the typical Stroop effect pattern found in the Delis–Kaplan Executive Function System (D-KEFS) and Automated Neuropsychological Assessment Metrics (ANAM) occurs in the VRST; the complexity condition of the VRST offers potential for assessment of exogenous and endogenous attentional processing; the two single-item presentation formats of the Stroop are comparable; there is a psychophysiological difference in arousal between low- and high-threat zones; and while there is a learning effect present in the VRST across conditions, it is limited by the arousal conditions in the high-threat zones. Additionally, no negative side effects were associated with use of the virtual environment. We conclude that the VRST has the potential to offer a novel approach to assessment of supervisory attentional processing within an ecologically relevant environment.

Neurocognitive Workload Assessment Using the Virtual Reality Cognitive Performance Assessment Test

The traditional approach to assessing neurocognitive performance makes use of paper and pencil neuropsychological assessments. This received approach has been criticized as limited in the area of ecological validity. While virtual reality environments provide increased ecological validity, they are often done without taking seriously the demands of rigorous research design and control for potentially confounding variables. The newly developed Virtual Reality Cognitive Performance Assessment Test (VRCPAT) focuses upon enhanced ecological validity using virtual environment scenarios to assess neurocognitive processing. After an assessment for potential confounds (i.e. appropriate level of immersion and performance on neuropsychological measures), the VRCPAT batterys Attention Module (i.e. Humvee scenario) was administered to a sample of healthy adults. Findings suggest that increase in stimulus complexity and stimulus intensity can manipulate attention performance within the Attention Module.

An initial validation of the Virtual Reality Paced Auditory Serial Addition Test in a college sample

BACKGROUND Numerous studies have demonstrated that the Paced Auditory Serial Addition Test (PASAT) has utility for the detection of cognitive processing deficits. While the PASAT has demonstrated high levels of internal consistency and test-retest reliability, administration of the PASAT has been known to create undue anxiety and frustration in participants. As a result, degradation of performance may be found on the PASAT. The difficult nature of the PASAT may subsequently decrease the probability of their return for follow up testing. NEW METHOD This study is a preliminary attempt at assessing the potential of a PASAT embedded in a virtual reality environment. The Virtual Reality PASAT (VR-PASAT) was compared with a paper-and-pencil version of the PASAT as well as other standardized neuropsychological measures. The two modalities of the PASAT were conducted with a sample of 50 healthy university students, between the ages of 19 and 34 years. Equivalent distributions were found for age, gender, education, and computer familiarity. RESULTS Moderate relationships were found between VR-PASAT and other putative attentional processing measures. The VR-PASAT was unrelated to indices of learning, memory, or visuospatial processing. COMPARISON WITH EXISTING METHOD(S) Comparison of the VR-PASAT with the traditional paper-and-pencil PASAT indicated that both versions require the examinee to sustain attention at an increasingly demanding, externally determined rate. CONCLUSIONS Results offer preliminary support for the construct validity (in a college sample) of the VR-PASAT as an attentional processing measure and suggest that this task may provide some unique information not tapped by traditional attentional processing tasks.

Age-related affective modulation of the startle eyeblink response: older adults startle most when viewing positive pictures.

Previous studies reveal age by valence interactions in attention and memory, such that older adults focus relatively more on positive and relatively less on negative stimuli than younger adults. In the current study, eyeblink startle response was used to measure differences in emotional reactivity to images that were equally arousing to both age groups. Viewing positive and negative pictures from the International Affective Picture System had opposite effects on startle modulation for older and younger adults. Younger adults showed the typical startle blink pattern, with potentiated startle when viewing negative pictures compared to positive pictures. Older adults, on the other hand, showed the opposite pattern, with potentiated startle when viewing positive pictures compared to viewing negative and neutral pictures. Potential underlying mechanisms for this interaction are evaluated. This pattern suggests that, compared with younger adults, older adults are more likely to spontaneously suppress responses to negative stimuli and process positive stimuli more deeply.

Visuospatial processing and learning effects in virtual reality based mental rotation and navigational tasks

Visuospatial function and performance in interactions between humans and computers involve the human identification and manipulation of computer generated stimuli and their location. The impact of learning on mental rotation has been demonstrated in studies relating everyday spatial activities and spatial abilities. An aspect of visuospatial learning in virtual environments that has not been widely studied is the impact of threat on learning in a navigational task. In fact, to our knowledge, the combined assessment of learning during mental rotation trials and learning in an ecologically valid virtual reality-based navigational environment (that has both high and low threat zones) has not been adequately studied. Results followed expectation: 1) learning occurred in the virtual reality based mental rotation test. Although there was a relation between route learning and practice, a primacy effect was observed as participants performed more poorly when going from the first zone to the last.

Psychophysiology to assess impact of varying levels of simulation fidelity in a threat environment

There are many virtual environments found in the serious game community that simulate real world scenarios. There is a broad range of fidelity and experimental controls among these serious games. An important component to most evaluations is the extent to which level of fidelity impacts the persons immersed in the serious game. While a great deal of virtual environment and serious game research has assessed the subjective state or feeling of the participant (e.g., the participants sense of presence) through the use of questionnaires, the current study examines participant experience by examining psychophysiological responses of participants to their surroundings. The primary goal in this study was evaluative: will a virtual environment with arousing contents result in increased sensory arousal if it is presented in a highly immersive configuration? A secondary goal of this study was to investigate the utility of our environment to offer varying levels of stimulus threat to impact the users experience of the virtual environment. Increased simulation fidelity in an arousing environment resulted in faster heart rates and increased startle eyeblink amplitudes, suggesting that higher fidelity scenarios had great efficacy related to sensory arousal.

Can you give me a hand? A comparison of hands and feet as optimal anatomical sites for skin conductance recording

The fingers and feet have long been accepted as optimal anatomical recording sites for electrodermal activity. The available literature suggests that the feet are more responsive than the fingers. The present report compared skin conductance level (SCL) and responses (SCRs) from the left foot and the distal phalanges of the fingers on the nondominant hand among 19 participants. The principal results were (a) SCRs recorded from the fingers were significantly larger and more frequent with shorter latencies than SCRs from the foot, (b) SCL from the fingers was significantly higher than from the foot, (c) the fingers exhibited significantly greater discrimination conditioning than the foot, and (d) skin conductance measures recorded from the fingers and foot were significantly positively correlated. Specifically, our results demonstrate that the distal phalanges of the fingers are electrodermally more responsive than the abductor hallucis area of the foot.