Nicolas Dumay

Sleep-Associated Changes in the Mental Representation of Spoken Words

The integration of a newly learned spoken word form with existing knowledge in the mental lexicon is characterized by the word forms ability to compete with similar-sounding entries during auditory word recognition. Here we show that although the mere acquisition of a spoken form is swift, its engagement in lexical competition requires an incubation-like period that is crucially associated with sleep. Words learned at 8 p.m. do not induce (inhibitory) competition effects immediately, but do so after a 12-hr interval including a nights sleep, and continue to induce such effects after 24 hr. In contrast, words learned at 8 a.m. do not show such effects immediately or after 12 hr ofwakefulness, but show the effects only after 24 hr, after sleep has occurred. This time-course dissociation is best accommodated by connectionist and neural models of learning in which sleep provides an opportunity for hippocampal information to be fed into long-term neocortical memory.

Lexical competition and the acquisition of novel words.

Three experiments examined the involvement of newly learnt words in lexical competition. Adult participants were familiarized with novel nonsense sequences that overlapped strongly with existing words (e.g. cathedruke, derived from cathedral) through repeated presentation in a phoneme-monitoring task. Experiment 1 looked at the immediate effects of exposure to these sequences, with participants showing familiarity with the form of the novel sequences in a two-alternative forced choice task. The effect of this exposure on lexical competition was examined by presenting the existing words (e.g. cathedral) in a lexical decision task. The immediate effect of the exposure was facilitatory, suggesting that the novel words had activated the representation of the closest real word rather than developing their own lexical representations. In Experiment 2, inhibitory lexical competition effects emerged over the course of 5 days for offset-diverging (e.g. cathedruke-cathedral) but not onset-diverging (e.g. yothedral-cathedral) novel words. Experiment 3 disentangled the roles of time and level-of-exposure in the lexicalization process and assessed the generality of the observed lexical inhibition using pause detection. A single, concentrated exposure session was used, which resulted in good recognition performance soon after. Lexicalization effects were absent immediately after exposure but emerged 1 week later, despite no intervening exposure to the novel items. These results suggest that integrating a novel word into the mental lexicon can be an extended process: phonological information is learnt swiftly, but full integration with existing items develops at a slower rate.

Behavioral and Electrophysiological Study of Phonological Priming between Bisyllabic Spoken Words

Phonological priming between bisyllabic (CV.CVC) spoken items was examined using both behavioral (reaction times, RTs) and electrophysiological (event-related potentials, ERPs) measures. Word and pseudoword targets were preceded by pseudoword primes. Different types of final phonological overlap between prime and target were compared. Critical pairs shared the last syllable, the rime or the coda, while unrelated pairs were used as controls. Participants performed a target shadowing task in Experiment 1 and a delayed lexical decision task in Experiment 2. RTs were measured in the first experiment and ERPs were recorded in the second experiment. The RT experiment was carried out under two presentation conditions. In Condition 1 both primes and targets were presented auditorily, while in Condition 2 the primes were presented visually and the targets auditorily. Priming effects were found in the unimodal condition only. RTs were fastest for syllable overlap, intermediate for rime overlap, and slowest for coda overlap and controls that did not differ from one another. ERPs were recorded under unimodal auditory presentation. ERP results showed that the amplitude of the auditory N400 component was smallest for syllable overlap, intermediate for rime overlap, and largest for coda overlap and controls that did not differ from one another. In both experiments, the priming effects were larger for word than for pseudoword targets. These results are best explained by the combined influences of nonlexical and lexical processes, and a comparison of the reported effects with those found in monosyllables suggests the involvement of rime and syllable representations.

The role of the syllable in lexical segmentation in French: word-spotting data.

Three word-spotting experiments assessed the role of syllable onsets and offsets in lexical segmentation. Participants detected CVC words embedded initially or finally in bisyllabic nonwords with aligned (CVC.CVC) or misaligned (CV.CCVC) syllabic structure. A misalignment between word and syllable onsets (Experiment 1) produced a greater perceptual cost than a misalignment between word and syllable offsets (Experiments 2 and 3). These results suggest that listeners rely on syllable onsets to locate the beginning of words. The implications for theories of lexical access in continuous speech are discussed.

Exploring phonological encoding through repeated segments

Five experiments explored the influence of repeated phonemes on the production of short utterances. In Experiment 1 coloured object naming showed faster latencies when colour and object started with the same phoneme (‘green goat’) than when they did not; the opposite was found when colour and object were named on consecutive trials (‘green’ – ‘goat’). Experiments 2 and 3 focused on adjective-noun phrases and showed no effect of repeated phonemes on either acoustical duration of speeded responses, or latencies in a delayed variant of the task, suggesting a higher-level – rather than articulatory – locus of the effect. Experiments 4 and 5 demonstrated that the facilitation induced by repeated segments is not specific to word onset (‘green chain’) and is independent of whether or not the repeated phonemes occupy the same within-word position (‘green flag’). These results indicate that in the production of multiple words, word forms are concurrently activated and evoke phonological segments represented in a position-nonspecific manner.

Do words go to sleep? Exploring consolidation of spoken forms through direct and indirect measures

We address the notion of integration of new memory representations and the potential dependence of this phenomenon on sleep, in light of recent findings on the lexicalization of spoken words. A distinction is introduced between measures tapping directly into the strength of the newly acquired knowledge and indirect measures assessing the influence of this knowledge on spoken word identification.

A Word-Order Constraint in Single-Word Production?: Failure to Replicate Janssen, Alario, and Caramazza (2008)

Janssen, Alario, and Caramazza (2008) reported finding that word order constrains the activation of lexical and phonological representations even in the production of isolated words. In their experiments, French or U.S. English speakers named the color or the object depicted in colored-picture displays; in some cases, the color name started with the same sound as the object name. French speakers named colors faster in the phonologically congruent condition (e.g., stimulus: robe rouge, “red dress”; correct response: “rouge”) than in the incongruent condition (e.g., stimulus: vache rouge, “red cow”; correct response: “rouge”; see Navarrete & Costa, 2005, for similar results in Spanish), but showed no difference between conditions for object naming (e.g., stimulus: robe rouge; correct response: “robe”). Intriguingly, English speakers showed the reverse pattern: Phonological congruency sped up their naming of objects, but not their naming of colors. Janssen et al. interpreted their findings in terms of language-specific syntactic constraints: Given that nouns typically precede color adjectives in French, activation of the noun’s phonology is initially favored over that of the adjective, and, as a result, activation of the object name influences retrieval of the color name, whereas the reverse is not possible. In English, by contrast, because modifying adjectives systematically occur before their nouns, the exact mirror pattern is observed. The proposed account, however, does not fit with recent data reported by Kuipers and La Heij (2009) for speakers of Dutch, which like English uses only prenominal adjectives. Kuipers and La Heij found that color naming rather than object naming benefited from overlap of the initial sounds—the same pattern Janssen et al. (2008) observed in French speakers. The inconsistency between the English and Dutch findings cannot be explained simply by invoking the fact that in Dutch, as in French, adjectives inherit the gender of their nouns. Gender agreement could indeed create the conditions of phonological facilitation from the noun to the prenominal adjective; however, as Janssen et al. themselves pointed out (p. 219), there is no reason why this process should eliminate, or even reduce, the phonological effect in object naming. The data reported by Janssen et al. (2008) are also incompatible with our own independent attempt to find an influence of color on single-word object naming in U.K. English speakers. To test this possible influence, we used 20 colored line drawings of common objects with monosyllabic names (CELEX frequency: 18 per million; length: 3.5 phonemes; Baayen, Piepenbrock, & Gulikens, 1995). These drawings are known to induce phonological facilitation in colored-object naming experiments in which participants are asked to name both the color and the object in each drawing (adjective plus noun: e.g., “red rat” vs. “blue rat”; Damian & Dumay, 2007; see also Damian & Dumay, 2009; Dumay, Damian, Stadthagen-Gonzalez, & Perez, 2009). To maximize the chance of color affecting noun retrieval in the present experiments, we rendered color more salient by coloring the entire drawings instead of just the lines. As in the experiments of Janssen et al., participants had to name either the color or the object depicted in colored-picture displays; task (color or object naming) was manipulated between participants, and phonological congruency was manipulated within participants. Forty-eight native U.K. English speakers were assigned randomly to one or the other task. Each object was presented once in a congruent trial, once in an incongruent trial, and four times in filler trials (which used two colors that were phonologically unrelated to the objects). Trial order was pseudorandomized for each participant so that neither the same color nor the same object appeared on consecutive trials. The first author, who was blind to the experimental condition, hand-measured all latencies directly from the response spectrograms. As in Janssen et al. (2008), latencies longer than 3 standard deviations above each participant’s conditional mean (1.3%) were excluded. As shown in Figure 1a, we found an asymmetry exactly opposite to that observed by Janssen et al.: Phonological congruency sped up color naming by 25 ms on average, F1(1, 23) = 15.46, p < .001, ηp 2 = .402; F2(1, 19) = 9.54, p < .007, ηp 2 = .334, but had no effect on object naming (the

Generalization from newly learned words reveals structural properties of the human reading system.

Connectionist accounts of quasiregular domains, such as spelling–sound correspondences in English, represent exception words (e.g., pint) amid regular words (e.g., mint) via a graded “warping” mechanism. Warping allows the model to extend the dominant pronunciation to nonwords (regularization) with minimal interference (spillover) from the exceptions. We tested for a behavioral marker of warping by investigating the degree to which participants generalized from newly learned made-up words, which ranged from sharing the dominant pronunciation (regulars), a subordinate pronunciation (ambiguous), or a previously nonexistent (exception) pronunciation. The new words were learned over 2 days, and generalization was assessed 48 hr later using nonword neighbors of the new words in a tempo naming task. The frequency of regularization (a measure of generalization) was directly related to degree of warping required to learn the pronunciation of the new word. Simulations using the Plaut, McClelland, Seidenberg, and Patterson (1996) model further support a warping interpretation. These findings highlight the need to develop theories of representation that are integrally tied to how those representations are learned and generalized.

Improved memory for information learnt before alcohol use in social drinkers tested in a naturalistic setting

Alcohol is known to facilitate memory if given after learning information in the laboratory; we aimed to investigate whether this effect can be found when alcohol is consumed in a naturalistic setting. Eighty-eight social drinkers were randomly allocated to either an alcohol self-dosing or a sober condition. The study assessed both retrograde facilitation and alcohol induced memory impairment using two independent tasks. In the retrograde task, participants learnt information in their own homes, and then consumed alcohol ad libitum. Participants then undertook an anterograde memory task of alcohol impairment when intoxicated. Both memory tasks were completed again the following day. Mean amount of alcohol consumed was 82.59 grams over the evening. For the retrograde task, as predicted, both conditions exhibited similar performance on the memory task immediately following learning (before intoxication) yet performance was better when tested the morning after encoding in the alcohol condition only. The anterograde task did not reveal significant differences in memory performance post-drinking. Units of alcohol drunk were positively correlated with the amount of retrograde facilitation the following morning. These findings demonstrate the retrograde facilitation effect in a naturalistic setting, and found it to be related to the self-administered grams of alcohol.

Look more carefully: Even your data show sleep makes memories more accessible. A reply to Schreiner and Rasch (in press)

Experiments on declarative memory (i.e., about facts and episodes that we can consciously recollect) usually involvemore than one item (e.g., pictures, words, locations, voices, etc. or associations between these elements). In other words, the net performance in these experiments by construction hides the presence of two opposing forces at work at the item-level: ‘forgetting’ (i.e., ‘oblivescence’), which is the inability to express previous knowledge, and ‘reminiscence’, which refers to the gained, or regained, ability to express knowledge that was previously inaccessible (e.g., Erdelyi, 2010). Until recently, studies looking at how sleep influences this type of memory have never really tracked these undercurrents. As a result, because the benefit of sleep, over being awake, in most experiments comes about as a smaller decline in net performance between the immediate test and the retest, the default view has been that sleep only stabilizes the learnt information, missing out on the possibility that it also promotes reminiscence. In Dumay (2016), I examined this possibility by reanalysing two data sets (Dumay & Gaskell, 2007; Tamminen, Payne, Stickgold, Wamsley, & Gaskell, 2010) that showed a benefit of sleep on both free recall (i.e., Now, tell me what you know) and old/new recognition (i.e., Have you heard this before?) of made-up words, distinguishing between items remembered at the 0-h test and not forgotten (i.e., ‘maintained’) by the 12h retest, and items not remembered at the test, but eventually recovered (i.e., ‘gained’) at the retest (i.e., the reminiscence cases). For free recall, this ‘item-fate’ analysis confirmed the anti-forgetting effect of sleep, with sleep increasing the likelihood of still remembering an item at retest. But it also showed a significant contribution of sleep to reminiscence: recovering an item thatwas inaccessible at the immediate test was more likely after sleep than after an equivalent period of wake. The recognition data confirmed the effect of sleep on reminiscence, but failed to reproduce the benefit for maintained items, yet revealed by free recall. I concluded that sleep protects our memories against forgetting, but at the same time also makes them more accessible. Given the presence of significant interactions between group (sleep vs. wake) and item-fate in favour of gained items, I also claimed that the benefit of sleep to memory was primarily due to increased reminiscence. The dissociation between the two effects across tasks (i.e., recognition shows no sleep effect on protection against loss, but only an effect on reminiscence) suggests that reminiscence may be driven partly by an overnight increase in stimulus familiarity, useful only for recognition, whereas protection against loss may not. This separation of the two effects was corroborated by the presence of correlations between the probabilities of reminiscence and forgetting only for the wake group. In view of these elements, I suggested that stronger reminiscence after sleep and protection against loss reflected only partially overlapping mechanisms (i.e., they should not be taken exactly as the two sides of the same coin). Stronger reminiscence after sleep is hard to explain by reduced retroactive interference in the hippocampus,