Mickael L. D. Deroche

Not Just the Norm: Exemplar-Based Models Also Predict Face Aftereffects

The face recognition literature has considered two competing accounts of how faces are represented within the visual system: Exemplar-based models assume that faces are represented via their similarity to exemplars of previously experienced faces, while norm-based models assume that faces are represented with respect to their deviation from an average face, or norm. Face identity aftereffects have been taken as compelling evidence in favor of a norm-based account over an exemplar-based account. After a relatively brief period of adaptation to an adaptor face, the perceived identity of a test face is shifted toward a face with attributes opposite to those of the adaptor, suggesting an explicit psychological representation of the norm. Surprisingly, despite near universal recognition that face identity aftereffects imply norm-based coding, there have been no published attempts to simulate the predictions of norm- and exemplar-based models in face adaptation paradigms. Here, we implemented and tested variations of norm and exemplar models. Contrary to common claims, our simulations revealed that both an exemplar-based model and a version of a two-pool norm-based model, but not a traditional norm-based model, predict face identity aftereffects following face adaptation.

Speech recognition against harmonic and inharmonic complexes: Spectral dips and periodicity

Speech recognition in a complex masker usually benefits from masker harmonicity, but there are several factors at work. The present study focused on two of them, glimpsing spectrally in between masker partials and periodicity within individual frequency channels. Using both a theoretical and an experimental approach, it is demonstrated that when inharmonic complexes are generated by jittering partials from their harmonic positions, there are better opportunities for spectral glimpsing in inharmonic than in harmonic maskers, and this difference is enhanced as fundamental frequency (F0) increases. As a result, measurements of masking level difference between the two maskers can be reduced, particularly at higher F0s. Using inharmonic maskers that offer similar glimpsing opportunity to harmonic maskers, it was found that the masking level difference between the two maskers varied little with F0, was influenced by periodicity of the first four partials, and could occur in low-, mid-, or high-frequency regions. Overall, the present results suggested that both spectral glimpsing and periodicity contribute to speech recognition under masking by harmonic complexes, and these effects seem independent from one another.

Roles of the target and masker fundamental frequencies in voice segregation

Intelligibility of a target voice improves when its fundamental frequency (F0) differs from that of a masking voice, but it remains unclear how this masking release (MR) depends on the two relative F0s. Three experiments measured speech reception thresholds (SRTs) for a target voice against different maskers. Experiment 1 evaluated the influence of target F0 itself. SRTs against white noise were elevated by at least 2 dB for a monotonized target voice compared with the unprocessed voice, but SRTs differed little for F0s between 50 and 150 Hz. In experiments 2 and 3, a MR occurred when there was a steady difference in F0 between the target voice and a stationary speech-shaped harmonic complex or a babble. However, this MR was considerably larger when the F0 of the masker was 11 semitones above the target F0 than when it was 11 semitones below. In contrast, for a fixed masker F0, the MR was similar whether the target F0 was above or below. The dependency of these MRs on the masker F0 suggests that a spectral mechanism such as glimpsing in between resolved masker partials may account for an important part of this phenomenon.

Deficits in the pitch sensitivity of cochlear-implanted children speaking English or Mandarin

Sensitivity to complex pitch is notoriously poor in adults with cochlear implants (CIs), but it is unclear whether this is true for children with CIs. Many are implanted today at a very young age, and factors related to brain plasticity (age at implantation, duration of CI experience, and speaking a tonal language) might have strong influences on pitch sensitivity. School-aged children participated, speaking English or Mandarin, having normal hearing (NH) or wearing a CI, using their clinically assigned settings with envelope-based coding strategies. Percent correct was measured in three-interval three-alternative forced choice tasks, for the discrimination of fundamental frequency (F0) of broadband harmonic complexes, and for the discrimination of sinusoidal amplitude modulation rate (AMR) of broadband noise, with reference frequencies at 100 and 200 Hz to focus on voice pitch processing. Data were fitted using a maximum-likelihood technique. CI children displayed higher thresholds and shallower slopes than NH children in F0 discrimination, regardless of linguistic background. Thresholds and slopes were more similar between NH and CI children in AMR discrimination. Once the effect of chronological age was extracted from the variance, the aforementioned factors related to brain plasticity did not contribute significantly to the CI childrens sensitivity to pitch. Unless different strategies attempt to encode fine structure information, potential benefits of plasticity may be missed.

Voice emotion perception and production in cochlear implant users

Voice emotion is a fundamental component of human social interaction and social development. Unfortunately, cochlear implant users are often forced to interface with highly degraded prosodic cues as a result of device constraints in extraction, processing, and transmission. As such, individuals with cochlear implants frequently demonstrate significant difficulty in recognizing voice emotions in comparison to their normal hearing counterparts. Cochlear implant-mediated perception and production of voice emotion is an important but relatively understudied area of research. However, a rich understanding of the voice emotion auditory processing offers opportunities to improve upon CI biomedical design and to develop training programs benefiting CI performance. In this review, we will address the issues, current literature, and future directions for improved voice emotion processing in cochlear implant users.

Phase effects in masking by harmonic complexes: Speech recognition

Harmonic complexes that generate highly modulated temporal envelopes on the basilar membrane (BM) mask a tone less effectively than complexes that generate relatively flat temporal envelopes, because the non-linear active gain of the BM selectively amplifies a low-level tone in the dips of a modulated masker envelope. The present study examines a similar effect in speech recognition. Speech reception thresholds (SRTs) were measured for a voice masked by harmonic complexes with partials in sine phase (SP) or in random phase (RP). The maskers fundamental frequency (F0) was 50, 100 or 200 Hz. SRTs were considerably lower for SP than for RP maskers at 50-Hz F0, but the two converged at 100-Hz F0, while at 200-Hz F0, SRTs were a little higher for SP than RP maskers. The results were similar whether the target voice was male or female and whether the maskers spectral profile was flat or speech-shaped. Although listening in the masker dips has been shown to play a large role for artificial stimuli such as Schroeder-phase complexes at high levels, it contributes weakly to speech recognition in the presence of harmonic maskers with different crest factors at more moderate sound levels (65 dB SPL).

Mandarin Tone Identification in Cochlear Implant Users Using Exaggerated Pitch Contours.

Objective: To determine whether exaggerating the variations in fundamental frequency (F0) contours of Mandarin-based pitch fluctuations could improve tone identification by cochlear implant (CI) users. Methods: Twelve normal-hearing (NH) listeners and 11 CI users were tested for their ability to recognize F0 contours modeled after Mandarin tones, in 4- or 5-alternatives forced-choice paradigms. Two types of stimuli were used: computer-generated complex tones and voice recordings. Four contours were tested with voice recordings: flat, rise, fall, and dip. A fifth contour, peak, was added for complex tones. The F0 range of each contour was varied in an adaptive manner. A maximum-likelihood technique was used to fit a psychometric function to the performance data and extract threshold at 70% accuracy. Results: As F0 range increased, performance in tone identification improved but did not reach 100% for some CI users, suggesting that confusions between contours could always be made even with extremely exaggerated contours. Compared with NH participants, CI users required substantially larger F0 ranges to identify tones, on the order of 9.3 versus 0.4 semitones. CI users achieved better performance for complex tones than for voice recordings, whereas the reverse was true for NH participants. Confusion matrices showed that the “flat” tone was often a default option when the tone contours F0 range presented was too narrow for participants to respond correctly. Conclusion: These results demonstrate markedly impaired ability for CI users to identify tonal contours, but suggest that the use of exaggerated pitch contours may be helpful for tonal language perception.

Voice emotion recognition and production by individuals with normal hearing and with cochlear implants

Children and adults with normal hearing (NH) as well as those who use cochlear implants (CIs) were tested on a voice emotion recognition task. The materials were child-directed sentences, and acoustic analyses showed that features such as voice pitch range were exaggerated relative to earlier reports in the literature with adult-directed materials. The NH participants achieved ceiling-level performance in the task, while the children and adults with CIs had lower, and highly variable, scores. In a parallel study, we have also collected complex pitch discrimination thresholds in many of these participants. In a new preliminary study, we are analyzing the acoustic features of voice emotion production by these populations. The task involves reading simple sentences in a happy and a sad way. In this presentation, we will report on relationships between the perceptual data on voice emotion recognition and complex pitch discrimination by child and adult NH and CI participants. In addition, we will report on our...

Voice segregation by difference in fundamental frequency: Effect of masker type

Speech reception thresholds were measured for a voice against two different maskers: Either two concurrent voices with the same fundamental frequency (F0) or a harmonic complex with the same long-term excitation pattern and broadband temporal envelope as the masking sentences (speech-modulated buzz). All sources had steady F0s. A difference in F0 of 2 or 8 semitones provided a 5-dB benefit for buzz maskers, whereas it provided a 3- and 8-dB benefit, respectively, for masking sentences. Whether intelligibility of a voice increases abruptly with small ΔF0s or gradually toward larger ΔF0s seems to depend on the nature of the masker.

The intelligibility of speech in a harmonic masker varying in fundamental frequency contour, broadband temporal envelope, and spatial location

Abstract Differences in fundamental frequency (F0), modulations in the masker envelope, and differences in spatial location between a speech target and a masker can improve speech intelligibility in cocktail‐party situations. These cues have been thoroughly investigated independently and associated with unmasking mechanisms: F0 segregation, temporal dip listening and spatial unmasking, respectively. Two experiments were conducted to examine whether F0 segregation interacts with spatial unmasking (experiment 1) or temporal modulations in the masker envelope (experiment 2) by measuring speech reception thresholds for a monotonized or an intonated voice against eight types of harmonic complex masker. In experiment 1, the masker varied in F0 contour (monotonized or intonated), mean F0 (0 or 3 semitones above that of the target) and spatial location (co‐located or separated from the target). In experiment 2, the masker varied in F0 contour, mean F0 and broadband temporal envelope (stationary or 1‐voice modulated). The benefits associated with spatial separation and F0 differences added up linearly in almost all conditions, whereas modulations in the masker envelope improved speech intelligibility only in the presence of intonated maskers. In addition, in both experiments F0 segregation benefited considerably from natural variations in the F0 pattern of the target voice, but was largely disrupted by those of the masker. HighlightsSpeech Reception Thresholds were measured in the presence of harmonic maskers.F0 contours, spatial location, and temporal envelope were manipulated orthogonally.F0 segregation interacted with dip listening but little with spatial unmasking.F0 fluctuations from a naturally intonated voice led to masking release.F0 fluctuations from a naturally intonated masker almost abolished masking release.