Sandra P. Marshall

The Index of Cognitive Activity: measuring cognitive workload

The Index of Cognitive Activity is an innovative technique that provides an objective psychophysiological measurement of cognitive workload. As users operate in increasingly complex environments, it is essential that the designers of these environments understand the cognitive demands placed on the users. The Index of Cognitive Activity (ICA) provides an important estimate of the levels of cognitive effort of the user. The ICA is based on changes in pupil dilation that occur as a user interacts with a visual display. The Index is described, and several applications are presented.

Eye metrics as an objective assessment of surgical skill.

Objective:Currently, surgical skills assessment relies almost exclusively on subjective measures, which are susceptible to multiple biases. We investigate the use of eye metrics as an objective tool for assessment of surgical skill. Summary Background Data:Eye tracking has helped elucidate relationships between eye movements, visual attention, and insight, all of which are employed during complex task performance (Kowler and Martins, Science. 1982;215:997–999; Tanenhaus et al, Science. 1995;268:1632–1634; Thomas and Lleras, Psychon Bull Rev. 2007;14:663–668; Thomas and Lleras, Cognition. 2009;111:168–174; Schriver et al, Hum Factors. 2008;50:864–878; Kahneman, Attention and Effort. 1973). Discovery of associations between characteristic eye movements and degree of cognitive effort have also enhanced our appreciation of the learning process. Methods:Using linear discriminate analysis (LDA) and nonlinear neural network analyses (NNA) to classify surgeons into expert and nonexpert cohorts, we examine the relationship between complex eye and pupillary movements, collectively referred to as eye metrics, and surgical skill level. Results:Twenty-one surgeons participated in the simulated and live surgical environments. In the simulated surgical setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 91.9% and 92.9% accuracy, respectively. In the live operating room setting, LDA and NNA were able to correctly classify surgeons as expert or nonexpert with 81.0% and 90.7% accuracy, respectively. Conclusions:We demonstrate, in simulated and live-operating environments, that eye metrics can reliably distinguish nonexpert from expert surgeons. As current medical educators rely on subjective measures of surgical skill, eye metrics may serve as the basis for objective assessment in surgical education and credentialing in the future. Further development of this potential educational tool is warranted to assess its ability to both reliably classify larger groups of surgeons and follow progression of surgical skill during postgraduate training.

Integrating psychophysiological measures of cognitive workload and eye movements to detect strategy shifts

This paper presents a case study to introduce a new technique for identifying and comparing cognitive strategies. The technique integrates eye movements with the Index of Cognitive Activity, a psychophysiological measurement of cognitive workload derived from changes in pupil dilation. The first part of the paper describes the technique and its constituent elements. The second part describes the task used in this study. The task has been used extensively in previous studies with electrophysiological recordings of EEG, EOG, and EMG and is know to elicit different levels of cognitive workload. The third part of the paper shows the results and provides detail about three separate strategies that emerged in the subject performance. The strategies are first identified from differences in the Index of Cognitive Activity and corroborated through detailed analyses of the participant eye movements as he performed the task.

Brief Report: Eye-Movement Patterns During an Embedded Figures Test in Children with ASD

The present study examined fixation frequency and duration during an Embedded Figures Test (EFT) in an effort to better understand the attentional and perceptual processes by which individuals with autism spectrum disorder (ASD) achieve accelerated EFT performance. In particular, we aimed to elucidate differences in the patterns of eye-movement in ASD and typically developing (TD) children, thus providing evidence relevant to the competing theories of weak central coherence (WCC) and enhanced perceptual functioning. Consistent with prior EFT studies, we found accelerated response time (RT) in children with ASD. No group differences were seen for fixation frequency, but the ASD group made significantly shorter fixations compared to the TD group. Eye-movement results indicate that RT advantage in ASD is related to both WCC and enhanced perceptual functioning.

Predictors of adherence to treatment for latent tuberculosis infection in high-risk Latino adolescents: a behavioral epidemiological analysis

The objective was to test whether theoretical variables predict adherence to treatment for latent tuberculosis infection in high-risk Latino adolescents. 286 Latino adolescents, age 13-18 years, were recruited from 10 middle/high schools in San Diego County, San Diego, USA. Participants completed a baseline interview and up to 9 monthly interviews. The cumulative number of pills consumed in 9 months was regressed on 16 independent variables, entered hierarchically in seven blocks. The final model accounted for 25% of the variance in adherence to isoniazid (INH), F (16, 230)=4.69, p<0.001. Adherence counseling (+), age (-), grades (+), being bicultural (+), and risk behaviors (-) were significantly related to adherence. Learning theories presume that adherence to medical regimens requires social support and freedom from physical and social barriers. Results support these theories. Future studies should explore additional precepts in order to identify additional predictors and to maximize adherence to INH among Latino adolescents and other high-risk populations. Doing so should decrease the risk of active TB among high-risk racial/ethnic and foreign-born populations.

Extracting Explicit and Implict Information from Complex Visualizations

How do experienced users extract information from a complex visualization? We examine this question by presenting experienced weather forecasters with visualizations that did not show the needed information explicitly and examining their eye movements. We replicated Carpenter & Shah (1998) when the information was explicitly available on the visualization. However, when the information was not explicitly available, we found that forecasters used spatial reasoning in the form of spatial transformations. We also found a strong imagerial component for constructing meteorological information.

Culture-Fair Cognitive Ability Assessment: Information Processing and Psychophysiological Approaches.

Valid assessment with diverse populations requires tools that are not influenced by cultural elements. This study investigated the relationships between culture, information processing efficiency, and general cognitive capacities in samples of Caucasian and Mexican American college students. Consistent with the neural efficiency hypothesis, pupillary responses (indexing mental effort) and detection accuracy scores on a visual backward-masking task were both significantly related to the Wechsler Adult Intelligence Scale-Revised (WAIS-R) Full Scale scores. These measures of information processing efficiency were similar in the two groups. However, they were related only to Caucasian American, but not to a comparable sample of Mexican American, students’ WAIS-R scores. Therefore, the differential validity in prediction suggests that the WAIS-R test may contain cultural influences that reduce the validity of the WAIS-R as a measure of cognitive ability for Mexican American students. Information processing and psychophysiological approaches may be helpful in developing culture-fair cognitive ability measures.

Content Effects on Word Problem Performance: A Possible Source of Test Bias?

Gender differences in mathematics test performance that favor males are rarely found on tests of computation or other mathematical symbol manipulations. These differences appear primarily in tests that are labeled as tests of “mathematical reasoning” and consist largely of word problems. The content of the word problems’ cover stories is a possible source of gender bias. Some have suggested that students are discouraged from solving problems for affective reasons when the content of the problem is sex-typed for the opposite sex; cognitive science research on problem-solving processes suggests that familiarity of content would be likely to affect problem-solving performance. To test these hypotheses, an experiment was conducted in which underlying mathematics problems were clothed in four different cover stories: masculine, feminine, neutral familiar, and neutral unfamiliar. No effect of sex-typing was found; there was a highly significant but small effect of familiarity. Ratings of problem characteristics were also collected, primarily to guide and confirm the realization of the design intentions.

Measuring cognitive workload across different eye tracking hardware platforms

As pertinent technologies continue to evolve, eye tracking hardware options grow more diverse. Consequently, it is important that researchers verify that new systems and parameters used in testing meet data collection quality standards. The current study evaluated hardware from four manufacturers: SR Research, Seeing Machines, SensoMotoric Instruments and Tobii Technology. The eye trackers included different system types and different sampling rates. The purpose of this research was to determine whether or not the pupil recording of each system was precise enough to effectively utilize the Index of Cognitive Activity, a validated cognitive workload metric. Results indicated that each system effectively captured Index of Cognitive Activity data. System factors such as system type sampling rate did not affect the metric. To maintain the integrity of data collected by succeeding generations of eye tracker, it is important that this type of quality-control research continues.

What the Eyes Reveal: Measuring the Cognitive Workload of Teams

This paper describes the measurement of cognitive workload using the Networked Evaluation System (NES). NES is a unique network of coordinated eye-tracking systems that allows monitoring of groups of decision makers working together in a single environment. Two implementations are described. The first is a military application with teams of officers working together on a simulated joint relief mission, and the second is a fatigue study with teams of individuals working together in a simulated lunar search and recovery mission.