Carol L. Mackersie

Hearing impairment and cognitive energy: the Framework for Understanding Effortful Listening (FUEL)

The Fifth Eriksholm Workshop on “Hearing Impairment and Cognitive Energy” was convened to develop a consensus among interdisciplinary experts about what is known on the topic, gaps in knowledge, the use of terminology, priorities for future research, and implications for practice. The general term cognitive energy was chosen to facilitate the broadest possible discussion of the topic. It goes back to Titchener (1908) who described the effects of attention on perception; he used the term psychic energy for the notion that limited mental resources can be flexibly allocated among perceptual and mental activities. The workshop focused on three main areas: (1) theories, models, concepts, definitions, and frameworks; (2) methods and measures; and (3) knowledge translation. We defined effort as the deliberate allocation of mental resources to overcome obstacles in goal pursuit when carrying out a task, with listening effort applying more specifically when tasks involve listening. We adapted Kahneman’s seminal (1973) Capacity Model of Attention to listening and proposed a heuristically useful Framework for Understanding Effortful Listening (FUEL). Our FUEL incorporates the well-known relationship between cognitive demand and the supply of cognitive capacity that is the foundation of cognitive theories of attention. Our FUEL also incorporates a motivation dimension based on complementary theories of motivational intensity, adaptive gain control, and optimal performance, fatigue, and pleasure. Using a three-dimensional illustration, we highlight how listening effort depends not only on hearing difficulties and task demands but also on the listener’s motivation to expend mental effort in the challenging situations of everyday life.

Subjective and psychophysiological indexes of listening effort in a competing-talker task.

BACKGROUND The effects of noise and other competing backgrounds on speech recognition performance are well documented. There is less information, however, on listening effort and stress experienced by listeners during a speech-recognition task that requires inhibition of competing sounds. PURPOSE The purpose was (a) to determine if psychophysiological indexes of listening effort were more sensitive than performance measures (percentage correct) obtained near ceiling level during a competing speech task, (b) to determine the relative sensitivity of four psychophysiological measures to changes in task demand, and (c) to determine the relationships between changes in psychophysiological measures and changes in subjective ratings of stress and workload. RESEARCH DESIGN A repeated-measures experimental design was used to examine changes in performance, psychophysiological measures, and subjective ratings in response to increasing task demand. STUDY SAMPLE Fifteen adults with normal hearing participated in the study. The mean age of the participants was 27 (range: 24-54). DATA COLLECTION AND ANALYSIS Psychophysiological recordings of heart rate, skin conductance, skin temperature, and electromyographic (EMG) activity were obtained during listening tasks of varying demand. Materials from the Dichotic Digits Test were used to modulate task demand. The three levels of task demand were single digits presented to one ear (low-demand reference condition), single digits presented simultaneously to both ears (medium demand), and a series of two digits presented simultaneously to both ears (high demand). Participants were asked to repeat all the digits they heard, while psychophysiological activity was recorded simultaneously. Subjective ratings of task load were obtained after each condition using the National Aeronautics and Space Administration Task Load Index questionnaire. Repeated-measures analyses of variance were completed for each measure using task demand and session as factors. RESULTS Mean performance was higher than 96% for all listening tasks. There was no significant change in performance across listening conditions for any listener. There was, however, a significant increase in mean skin conductance and EMG activity as task demand increased. Heart rate and skin temperature did not change significantly. There was no strong association between subjective and psychophysiological measures, but all participants with mean normalized effort ratings of greater than 4.5 (i.e., effort increased by a factor of at least 4.5) showed significant changes in skin conductance. CONCLUSIONS Even in the absence of substantial performance changes, listeners may experience changes in subjective and psychophysiological responses consistent with the activation of a stress response. Skin conductance appears to be the most promising measure for evaluating individual changes in psychophysiological responses during listening tasks.

The effect of compression ratio and release time on the categorical rating of sound quality

Two experiments were carried out to determine how manipulating the compression ratio and release time of a single-band wide dynamic range hearing aid affects sound quality. In experiment I, compression ratio was varied over the range from linear to 10:1 (low compression threshold, attack time = 5 ms, release time = 200 ms). In experiment II, compression ratios of 1.5, 2, and 3:1 were combined with release times of 60, 200, and 1000 ms (attack time = 5 ms). Twenty listeners with sensorineural hearing loss rated the clarity, pleasantness, background noise, loudness, and the overall impression of speech-in-noise (Ventilation, Apartment, Cafeteria) processed through a compression hearing aid. Results revealed that increasing compression ratio caused decreases in ratings on all scales. Increasing release time caused ratings of pleasantness to increase, and ratings of background noise and loudness to decrease. At the 3:1 compression ratio, increasing the release time caused increases in ratings of clarity, pleasantness, and overall impression, and a decrease in background noise. Significant correlations were found between scales. Regression analysis revealed that the contributions of the scales of clarity, pleasantness, background noise, and loudness to the prediction of overall impression differed as a function of the competing noise condition.

Effect of release time in compression hearing aids: Paired‐comparison judgments of quality

Paired-comparison judgments of quality were obtained from 20 hearing-impaired listeners for speech processed through simulated compression hearing aids varying in release time (60, 200, 1000 ms) at three different compression ratios (1.5, 2, 3:1) and for three different background noises (ventilation, apartment, cafeteria). Analysis revealed that the main effect of release time did not have a significant effect on perceived quality. The interaction between release time and noise type was found to be significant. While no significant difference in preference for release times was evident for the ventilation noise, the longer release times (200 and 1000 ms) were preferred for the higher level noises (apartment noise, cafeteria noise). Post hoc testing revealed that the mean preference scores for the 200- and 1000-ms release time were significantly greater than that of the 60-ms release time with the competing cafeteria noise (p < 0.05). Analysis of individual subject data revealed statistically significant preferences that differed from the group mean, suggesting that individualized fitting of this parameter of a compression hearing aid might be warranted.

Effects of hearing loss on heart rate variability and skin conductance measured during sentence recognition in noise.

Objective: The purpose of the study was to determine the effects of hearing loss and noise on (1) two autonomic nervous system measures associated with stress (skin conductance and heart rate variability) and on (2) subjective ratings of workload/stress. The authors hypothesized that hearing loss would increase psychophysiological and subjective reactivity to noise during speech recognition tasks. Both psychophysiological and subjective indicators of workload/stress were expected to increase with a reduction in signal-to-noise ratio. Design: Sentence recognition in the presence of babble noise was assessed in 15 adults with clinically normal hearing and 18 adults with sensorineural hearing loss. Mean sentence recognition was equalized for the two groups using an adaptive procedure to estimate 80% recognition of words in sentences. Sentences were then presented in quiet and at four fixed signal-to-noise ratios: −6, −3, 0, and +3 dB relative to the individually determined signal-to-noise thresholds. Electrocardiography and skin conductance recordings were obtained during each listening condition. The high-frequency spectral component of heart rate variability was extracted from the electrocardiographic recordings as a measure of parasympathetic nervous system activity. Subjective ratings of effort, mental demand, stress, and perceived performance were obtained after each listening condition using the National Aeronautics and Space Administration Task Load Index. Results: Recognition scores referenced to the adaptive threshold were similar for the two groups. Participants with hearing loss showed a decrease in high-frequency heart rate variability at lower signal-to-noise ratios, whereas those with normal hearing did not. Skin conductance levels were not sensitive to changes in signal-to-noise ratio. However, overall skin conductance reactivity to noise (relative to quiet) was higher for those with hearing loss than for those with normal hearing. In contrast to the psychophysiological findings, there were no significant differences between subjective ratings for the two groups. Conclusions: Listeners with hearing loss show greater autonomic nervous system reactivity to babble noise during speech recognition than do listeners with normal hearing, when recognition performance is equal. The findings are consistent with the conclusion that listeners with hearing loss experience increased effort and/or stress during speech recognition in noise.

A comparison of response time and word recognition measures using a word-monitoring and closed-set identification task.

OBJECTIVES The primary purpose of this study was to investigate the possibility of improving speech recognition testing sensitivity by incorporating response time measures as a metric. Two different techniques for obtaining response time were compared: a word-monitoring task and a closed-set identification task. DESIGN Recordings of the Modified Rhyme Test were used to test 12 listeners with normal hearing. Data were collected using a word-monitoring and a closed-set identification task. Response times and percent correct scores were obtained for each task using signal to noise ratios (SNRs) of -3, 0, +3, +6, +9, and +12 dB. RESULTS Both response time and percent correct measures were sensitive to changes in SNR, but greater sensitivity was found with the percent correct measures. Individual subject data showed that combining response time measures with percent correct scores improved test sensitivity for the monitoring task, but not for the closed-set identification task. CONCLUSIONS The best test sensitivity was obtained by combining percent correct and response time measures for the monitoring task. Such an approach may hold promise for future clinical applications.

Autonomic Nervous System Reactivity During Speech Repetition Tasks: Heart Rate Variability and Skin Conductance.

Cognitive and emotional challenges may elicit a physiological stress response that can include arousal of the sympathetic nervous system (fight or flight response) and withdrawal of the parasympathetic nervous system (responsible for recovery and rest). This article reviews studies that have used measures of electrodermal activity (skin conductance) and heart rate variability (HRV) to index sympathetic and parasympathetic activity during auditory tasks. In addition, the authors present results from a new study with normal-hearing listeners examining the effects of speaking rate on changes in skin conductance and high-frequency HRV (HF-HRV). Sentence repetition accuracy for normal and fast speaking rates was measured in noise using signal to noise ratios that were adjusted to approximate 80% accuracy (+3 dB fast rate; 0 dB normal rate) while monitoring skin conductance and HF-HRV activity. A significant increase in skin conductance level (reflecting sympathetic nervous system arousal) and a decrease in HF-HRV (reflecting parasympathetic nervous system withdrawal) were observed with an increase in speaking rate indicating sensitivity of both measures to increased task demand. Changes in psychophysiological reactivity with increased auditory task demand may reflect differences in listening effort, but other person-related factors such as motivation and stress may also play a role. Further research is needed to understand how psychophysiological activity during listening tasks is influenced by the acoustic characteristics of stimuli, task demands, and by the characteristics and emotional responses of the individual.

Effects of fundamental frequency and vocal-tract length cues on sentence segregation by listeners with hearing loss

The purpose was to determine the effect of hearing loss on the ability to separate competing talkers using talker differences in fundamental frequency (F0) and apparent vocal-tract length (VTL). Performance of 13 adults with hearing loss and 6 adults with normal hearing was measured using the Coordinate Response Measure. For listeners with hearing loss, the speech was amplified and filtered according to the NAL-RP hearing aid prescription. Target-to-competition ratios varied from 0 to 9 dB. The target sentence was randomly assigned to the higher or lower values of F0 or VTL on each trial. Performance improved for F0 differences up to 9 and 6 semitones for people with normal hearing and hearing loss, respectively, but only when the target talker had the higher F0. Recognition for the lower F0 target improved when trial-to-trial uncertainty was removed (9-semitone condition). Scores improved with increasing differences in VTL for the normal-hearing group. On average, hearing-impaired listeners did not benefit from VTL cues, but substantial inter-subject variability was observed. The amount of benefit from VTL cues was related to the average hearing loss in the 1-3-kHz region when the target talker had the shorter VTL.

Response time and word recognition using a Modified-Rhyme monitoring task: list equivalency and time-order effects.

OBJECTIVE The purpose of this study was to assess list equivalency and time-order effects of word recognition scores and response time measures obtained using a digital recording of the Modified Rhyme Test (MRT) with a response time monitoring task (Mackersie, Neuman, & Levitt, 1999). DESIGN Response times and percent correct measures were obtained from listeners with normal hearing using the MRT materials presented at a signal to noise ratio of +3 dB. Listeners were tested using a word-monitoring task in which six alternatives were presented in series and listeners pushed a button when they heard the target word (as displayed on the computer monitor). Listeners were tested in two sessions. During each session each of the six MRT lists was administered once. Time-order effects were examined both between and within test sessions. RESULTS All lists were equivalent for both speech recognition accuracy and response time except List 1, which showed slightly higher percent correct scores than the other lists. Varied patterns of systematic change over time were observed in 75% of the listeners for the response time measures and for 33% of the listeners for the percent correct measures. CONCLUSIONS Lists 2 through 6 of this version of the MRT are equivalent, with List 1 producing slightly higher word recognition scores. Systematic changes over time in response time data for the majority of listeners suggest the need for careful implementation of the test to avoid time-order effects.

Research on hearing-aid self-adjustment by adults

The purpose of the work is to develop a protocol for user self-adjustment of hearing aids and to determine its efficacy and candidacy. An initial study involved 26 adults with hearing loss. Control of overall volume, high-frequency boost, and low-frequency cut employed prerecorded and preprocessed sentence stimuli. Participants took a speech-perception test after an initial self-adjustment and then had the opportunity to repeat the adjustment. The final self-selected outputs were not significantly different from those prescribed by a widely used threshold-based method (NAL-NL2)—regardless of prior hearing-aid experience. All but one participant attained a speech-intelligibility index of 60%. Previous users of hearing aids, however, did not meet this criterion until after taking the speech-perception test. This work is continuing with the UCSD Open-source Speech-processing Platform which provides real-time, six-band, processing of microphone input from ear-level transducer assemblies. This system provides a more realistic participant experience, finer control of level and spectrum, and places no limits on the speech materials used for self-adjustment and outcome assessment. Ongoing work investigates the need for the speech-perception test as part of the self-adjustment protocol and the importance of the level and spectrum from which users make their initial adjustments.The purpose of the work is to develop a protocol for user self-adjustment of hearing aids and to determine its efficacy and candidacy. An initial study involved 26 adults with hearing loss. Control of overall volume, high-frequency boost, and low-frequency cut employed prerecorded and preprocessed sentence stimuli. Participants took a speech-perception test after an initial self-adjustment and then had the opportunity to repeat the adjustment. The final self-selected outputs were not significantly different from those prescribed by a widely used threshold-based method (NAL-NL2)—regardless of prior hearing-aid experience. All but one participant attained a speech-intelligibility index of 60%. Previous users of hearing aids, however, did not meet this criterion until after taking the speech-perception test. This work is continuing with the UCSD Open-source Speech-processing Platform which provides real-time, six-band, processing of microphone input from ear-level transducer assemblies. This system provides ...