Kurtulus Izzetoglu

Optical brain monitoring for operator training and mental workload assessment

An accurate measure of mental workload in human operators is a critical element of monitoring and adaptive aiding systems that are designed to improve the efficiency and safety of human-machine systems during critical tasks. Functional near infrared (fNIR) spectroscopy is a field-deployable non-invasive optical brain monitoring technology that provides a measure of cerebral hemodynamics within the prefrontal cortex in response to sensory, motor, or cognitive activation. In this paper, we provide evidence from two studies that fNIR can be used in ecologically valid environments to assess the: 1) mental workload of operators performing standardized (n-back) and complex cognitive tasks (air traffic control--ATC), and 2) development of expertise during practice of complex cognitive and visuomotor tasks (piloting unmanned air vehicles--UAV). Results indicate that fNIR measures are sensitive to mental task load and practice level, and provide evidence of the fNIR deployment in the field for its ability to monitor hemodynamic changes that are associated with relative cognitive workload changes of operators. The methods reported here provide guidance for the development of strategic requirements necessary for the design of complex human-machine interface systems and assist with assessments of human operator performance criteria.

Functional near-infrared spectroscopy

The purpose of the this article is to describe an emerging neuroimaging technology, functional near-infrared spectroscopy (fNIRs), which has several attributes that make it possible to conduct neuroimaging studies of the cortex in clinical offices and under more realistic, ecologically valid parameters. fNIRs use near-infrared light to measure changes in the concentration of oxygenated and deoxygenated hemoglobin in the cortex. Although fNIR imaging is limited to the outer cortex, it provides neuroimaging that is safe, portable, and very affordable relative to other neuroimaging technologies. It is also relatively robust to movement artifacts and can readily be integrated with other technologies such as EEG

fNIRS Study of Walking and Walking While Talking in Young and Old Individuals

BACKGROUND Evidence suggests that gait is influenced by higher order cognitive and cortical control mechanisms. However, less is known about the functional correlates of cortical control of gait. METHODS Using functional near-infrared spectroscopy, the current study was designed to evaluate whether increased activations in the prefrontal cortex (PFC) were detected in walking while talking (WWT) compared with normal pace walking (NW) in 11 young and 11 old participants. Specifically, the following two hypotheses were evaluated: (a) Activation in the PFC would be increased in WWT compared with NW. (b) The increase in activation in the PFC during WWT as compared with NW would be greater in young than in old participants. RESULTS Separate linear mixed effects models with age as the two-level between-subject factor, walking condition (NW vs WWT) as the two-level repeated within-subject factor, and HbO2 levels in each of the 16 functional near-infrared spectroscopy channels as the dependent measure revealed significant task effects in 14 channels, indicating a robust bilateral increased activation in the PFC in WWT compared with NW. Furthermore, the group-by-task interaction was significant in 11 channels with young participants showing greater WWT-related increase in HbO2 levels compared with the old participants. CONCLUSIONS This study provided the first evidence that oxygenation levels are increased in the PFC during WWT compared with NW in young and old individuals. This effect was modified by age suggesting that older adults may under-utilize the PFC in attention-demanding locomotion tasks.

Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks

A symbiotic relation between the operator and the operational environment can be realized by an advanced computing platform designed to understand and adapt to the cognitive and the physiological state of the user, especially during sensitive and cognitively demanding operations. The success of such a complex system depends not only on the efficacy of the individual components, but also on the efficient and appropriate integration of its parts. Because near infrared technology allows the design of portable, safe, affordable, and negligibly intrusive monitoring systems, the functional near infrared (fNIR) monitoring of brain hemodynamics can be of value in this type of complex system, particularly in helping to understand the cognitive and emotional state of the user during mentally demanding operations. This article presents the deployment and statistical analysis of fNIR spectroscopy for the purpose of cognitive state assessment while the user performs a complex task. This article is based on data collected during the Augmented Cognition-Technical Integration Experiment session. The experimental protocol for this session used a complex task, resembling a video game, called the Warship Commander Task (WCT). The WCT was designed to approximate naval air warfare management. Task difficulty and task load were manipulated by changing the following: (a) the number of airplanes that had to be managed at a given time, (b) the number of unknown (vs. known) airplane identities, and (c) the presence or absence of an auditory memory task. The fNIR data analysis explored the following: (a) the relations among cognitive workload, the participants performance, and changes in blood oxygenation levels of the dorsolateral prefrontal cortex; and (b) the effect of divided attention as manipulated by the secondary component of the WCT (the auditory task). The primary hypothesis was that blood oxygenation in the prefrontal cortex, as assessed by fNIR, would rise with increasing task load and would demonstrate a positive correlation with performance measures. The results indicated that the rate of change in blood oxygenation was significantly sensitive to task load changes and correlated fairly well with performance variables.

Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development

Functional near infrared spectroscopy (fNIRS) is a non-invasive, safe, and portable optical neuroimaging method that can be used to assess brain dynamics during skill acquisition and performance of complex work and everyday tasks. In this paper we describe neuroergonomic studies that illustrate the use of fNIRS in the examination of training-related brain dynamics and human performance assessment. We describe results of studies investigating cognitive workload in air traffic controllers, acquisition of dual verbal-spatial working memory skill, and development of expertise in piloting unmanned vehicles. These studies used conventional fNIRS devices in which the participants were tethered to the device while seated at a workstation. Consistent with the aims of mobile brain imaging (MoBI), we also describe a compact and battery-operated wireless fNIRS system that performs with similar accuracy as other established fNIRS devices. Our results indicate that both wired and wireless fNIRS systems allow for the examination of brain function in naturalistic settings, and thus are suitable for reliable human performance monitoring and training assessment.

Functional brain imaging using near-infrared technology

0739-5175/07/

Registering fNIR Data to Brain Surface Image using MRI templates

25.00©2007IEEE I n the last decade, functional near-infrared spectroscopy (fNIR) has been introduced as a new neuroimaging modality with which to conduct functional brain imaging studies [1]–[24]. fNIR technology uses specific wavelengths of light, irradiated through the scalp, to enable the noninvasive measurement of changes in the relative ratios of deoxygenated hemoglobin (deoxy-Hb) and oxygenated hemoglobin (oxy-Hb) during brain activity. This technology allows the design of portable, safe, affordable, noninvasive, and minimally intrusive monitoring systems. These qualities make fNIR suitable for the study of hemodynamic changes due to cognitive and emotional brain activity under many working and educational conditions, as well as in the field. Functional imaging is typically conducted in an effort to understand the activity in a given brain region in terms of its relationship to a particular behavioral state or its interactions with inputs from another region’s activity. The program of research in cognitive neuroscience conducted by our optical brain imaging group has utilized the current-generation fNIR system to investigate brain activity, primarily in the dorsolateral and inferior frontal cortex [20]–[24]. To date, the fNIR studies of cognition and emotion have focused on functions associated with Brodman’s areas BA9, BA10, BA46, BA45, BA47, and BA44. Recent positron emission tomography (PET) and functional magnetic resonance (fMRI) studies have shown that these areas play a critical role in sustained attention, both the short-term storage and the executive process components of working memory, episodic memory, problem solving, response inhibition, and the perception of smell (for a recent review, see [25] and [26]). In addition, word recognition and the storage of verbal materials activate Broca’s area and left hemisphere supplementary and premotor areas [25], [27], [28]. To date, studies utilizing fNIR have shown results consistent with fMRI and PET findings for working memory and sustained attention [21]–[23]. In this article, we will describe the working principles of fNIR and how the hemodynamic signals are extracted from the raw fNIR measurements using the modified Beer-Lambert Law. We will also introduce the fNIR system that we have developed and used in our studies. Current results from the augmented cognition research conducted in our laboratory are also presented, and the merits of optical imaging in augmented cognition are summarized. Working Principles Typically, an optical apparatus consists of a light source by which the tissue is radiated and a light detector that receives light after it has interacted with the tissue. Photons that enter tissue undergo two different types of interaction, namely absorption (loss of energy to the medium) and scattering [4], [5], [19]. Most biological tissues are relatively transparent to light in the near-infrared range between 700 to 900 nm, which is usually called the “optical window.” This is mainly due to the fact that within this optical window, the absorbance of the main constituents in the human tissue (i.e., water, oxy-Hb, and deoxy-Hb) is small, allowing the light to penetrate the tissue (see Figure 1). Among the main absorbers (chromophores) in the tissue, oxyand deoxy-Hb are strongly linked to tissue oxygenation and metabolism. Fortunately, in the optical window, the absorption spectra of oxyand deoxy-Hb remain significantly different than each other, allowing spectroscopic separation of these compounds to be possible using only a few sample wavelengths. fNIR technology employs specified wavelengths of light within the optical window. Once the photons are introduced into the human head, they are either scattered by extraand intracellular boundaries of different layers of the head (skin, skull, cerebrospinal fluid, brain, etc.) or absorbed mainly by oxyand deoxy-Hb. A photodetector placed a certain distance away from the light source can collect the photons that are not absorbed and those that traveled along the “banana shaped path” between the source and detector due to scattering [9], [29] as shown Figure 2. In functional optical brain imaging studies, the attenuation (reduction in the amount of photons detected by the photodetectors) due to scattering is assumed to be constant since the amount of scatterers within different layers of the head does not change due to cognitive activity. The change in the attenuation measured as a result of cognitive activity is hence due to the changes in absorption resulting from the variation in the concentrations of oxyand deoxy-Hb in the brain tissue. This relationship is not surprising, since cerebral hemodynamic changes are related to functional brain activity through a mechanism that is called neurovascular coupling [8], [30]. In fact, this physiological relationship and the ability of fNIR Functional Brain Imaging Using Near-Infrared Technology

Monitoring expertise development during simulated UAV piloting tasks using optical brain imaging

Functional near-infrared spectroscopy (fNIR) measures changes in the relative levels of oxygenated and deoxygenated hemoglobin and has increasingly been used to assess neural functioning in the brain. In addition to the ongoing technological developments, investigators have also been conducting studies on functional mapping and refinement of data analytic strategies in order to better understand the relationship between the fNIR signal and brain activity. However, since fNIR is a relatively new functional brain imaging modality as compared to positron emission tomography (PET) and functional magnetic resonance imaging (fMRI), it still lacks brain-mapping tools designed to allow researchers and clinicians to easily interact with their data. The aim of this study is to develop a registration technique for the fNIR measurements using anatomical landmarks and structural magnetic resonance imaging (MRI) templates in order to visualize the brain activation when and where it happens. The proposed registration technique utilizes chain-code algorithm and depicts activations over respective locations based on sensor geometry. Furthermore, registered data locations have been used to create spatiotemporal visualization of fNIR measurements

THE EVOLUTION OF FIELD DEPLOYABLE fNIR SPECTROSCOPY FROM BENCH TO CLINICAL SETTINGS

An accurate assessment of mental workload and expertise level would help improve operational safety and efficacy of human computer interaction for aerospace applications. The current study utilized functional near-infrared spectroscopy (fNIR) to investigate the relationship of the hemodynamic response in the anterior prefrontal cortex to changes in mental workload, level of expertise, and task performance during learning of simulated unmanned aerial vehicle (UAV) piloting tasks. Results indicated that fNIR measures are correlated to task performance and subjective self-reported measures; and contained additional information that allowed categorizing learning phases. Level of expertise does appear to influence the hemodynamic response in the dorsolateral/ventrolateral prefrontal cortices. Since fNIR allows development of portable and wearable instruments, it has the potential to be deployed in future learning environments to personalize the training regimen and/or assess the effort of human operators in critical multitasking settings.

Cognitive Workload and Learning Assessment During the Implementation of a Next-Generation Air Traffic Control Technology Using Functional Near-Infrared Spectroscopy

In the late 1980s and early 1990s, Dr. Britton Chance and his colleagues, using picosecond-long laser pulses, spearheaded the development of time-resolved spectroscopy techniques in an effort to obtain quantitative information about the optical characteristics of the tissue. These efforts by Chance and colleagues expedited the translation of near-infrared spectroscopy (NIRS)-based techniques into a neuroimaging modality for various cognitive studies. Beginning in the early 2000s, Dr. Britton Chance guided and steered the collaboration with the Optical Brain Imaging team at Drexel University toward the development and application of a field deployable continuous wave functional near-infrared spectroscopy (fNIR) system as a means to monitor cognitive functions, particularly during attention and working memory tasks as well as for complex tasks such as war games and air traffic control scenarios performed by healthy volunteers under operational conditions. Further, these collaborative efforts led to various clinical applications, including traumatic brain injury, depth of anesthesia monitoring, pediatric pain assessment, and brain–computer interface in neurology. In this paper, we introduce how these collaborative studies have made fNIR an excellent candidate for specified clinical and research applications, including repeated cortical neuroimaging, bedside or home monitoring, the elicitation of a positive effect, and protocols requiring ecological validity. This paper represents a token of our gratitude to Dr. Britton Chance for his influence and leadership. Through this manuscript we show our appreciation by contributing to his commemoration and through our work we will strive to advance the field of optical brain imaging and promote his legacy.