Wonil Choi

The neural substrates of natural reading: a comparison of normal and nonword text using eyetracking and fMRI

Most previous studies investigating the neural correlates of reading have presented text using serial visual presentation (SVP), which may not fully reflect the underlying processes of natural reading. In the present study, eye movements and BOLD data were collected while subjects either read normal paragraphs naturally or moved their eyes through “paragraphs” of pseudo-text (pronounceable pseudowords or consonant letter strings) in two pseudo-reading conditions. Eye movement data established that subjects were reading and scanning the stimuli normally. A conjunction fMRI analysis across natural- and pseudo-reading showed that a common eye-movement network including frontal eye fields (FEF), supplementary eye fields (SEF), and intraparietal sulci was activated, consistent with previous studies using simpler eye movement tasks. In addition, natural reading versus pseudo-reading showed different patterns of brain activation: normal reading produced activation in a well-established language network that included superior temporal gyrus/sulcus, middle temporal gyrus (MTG), angular gyrus (AG), inferior frontal gyrus, and middle frontal gyrus, whereas pseudo-reading produced activation in an attentional network that included anterior/posterior cingulate and parietal cortex. These results are consistent with results found in previous single-saccade eye movement tasks and SVP reading studies, suggesting that component processes of eye-movement control and language processing observed in past fMRI research generalize to natural reading. The results also suggest that combining eyetracking and fMRI is a suitable method for investigating the component processes of natural reading in fMRI research.

Language structure in the brain: A fixation-related fMRI study of syntactic surprisal in reading.

How is syntactic analysis implemented by the human brain during language comprehension? The current study combined methods from computational linguistics, eyetracking, and fMRI to address this question. Subjects read passages of text presented as paragraphs while their eye movements were recorded in an MRI scanner. We parsed the text using a probabilistic context-free grammar to isolate syntactic difficulty. Syntactic difficulty was quantified as syntactic surprisal, which is related to the expectedness of a given words syntactic category given its preceding context. We compared words with high and low syntactic surprisal values that were equated for length, frequency, and lexical surprisal, and used fixation-related (FIRE) fMRI to measure neural activity associated with syntactic surprisal for each fixated word. We observed greater neural activity for high than low syntactic surprisal in two predicted cortical regions previously identified with syntax: left inferior frontal gyrus (IFG) and less robustly, left anterior superior temporal lobe (ATL). These results support the hypothesis that left IFG and ATL play a central role in syntactic analysis during language comprehension. More generally, the results suggest a broader cortical network associated with syntactic prediction that includes increased activity in bilateral IFG and insula, as well as fusiform and right lingual gyri.

Neural correlates of fixation duration during real-world scene viewing: Evidence from fixation-related fire fmri

During active scene perception, our eyes move from one location to another via saccadic eye movements, with the eyes fixating objects and scene elements for varying amounts of time. Much of the variability in fixation duration is accounted for by attentional, perceptual, and cognitive processes associated with scene analysis and comprehension. For this reason, current theories of active scene viewing attempt to account for the influence of attention and cognition on fixation duration. Yet almost nothing is known about the neurocognitive systems associated with variation in fixation duration during scene viewing. We addressed this topic using fixation-related fMRI, which involves coregistering high-resolution eye tracking and magnetic resonance scanning to conduct event-related fMRI analysis based on characteristics of eye movements. We observed that activation in visual and prefrontal executive control areas was positively correlated with fixation duration, whereas activation in ventral areas associated with scene encoding and medial superior frontal and paracentral regions associated with changing action plans was negatively correlated with fixation duration. The results suggest that fixation duration in scene viewing is controlled by cognitive processes associated with real-time scene analysis interacting with motor planning, consistent with current computational models of active vision for scene perception.

Neural correlates of fixation duration in natural reading: Evidence from fixation-related fMRI.

A key assumption of current theories of natural reading is that fixation duration reflects underlying attentional, language, and cognitive processes associated with text comprehension. The neurocognitive correlates of this relationship are currently unknown. To investigate this relationship, we compared neural activation associated with fixation duration in passage reading and a pseudo-reading control condition. The results showed that fixation duration was associated with activation in oculomotor and language areas during text reading. Fixation duration during pseudo-reading, on the other hand, showed greater involvement of frontal control regions, suggesting flexibility and task dependency of the eye movement network. Consistent with current models, these results provide support for the hypothesis that fixation duration in reading reflects attentional engagement and language processing. The results also demonstrate that fixation-related fMRI provides a method for investigating the neurocognitive bases of natural reading.

See before you jump: full recognition of parafoveal words precedes skips during reading.

Serial attention models of eye-movement control during reading were evaluated in an eye-tracking experiment that examined how lexical activation combines with visual information in the parafovea to affect word skipping (where a word is not fixated during first-pass reading). Lexical activation was manipulated by repetition priming created through prime-target pairs embedded within a sentence. The boundary technique (Rayner, 1975) was used to determine whether the target word was fully available during parafoveal preview or whether it was available with transposed letters (e.g., Herman changed to Hreman). With full parafoveal preview, the target word was skipped more frequently when it matched the earlier prime word (i.e., was repeated) than when it did not match the earlier prime word (i.e., was new). With transposed-letter (TL) preview, repetition had no effect on skipping rates despite the great similarity of the TL preview string to the target word and substantial evidence that TL strings activate the words from which they are derived (Perea & Lupker, 2003). These results show that lexically based skipping is based on full recognition of the letter string in parafoveal preview and does not involve using the contextual constraint to compensate for the reduced information available from the parafovea. These results are consistent with models of eye-movement control during reading in which successive words in a text are processed 1 at a time (serially) and in which word recognition strongly influences eye movements.

Coordination of word recognition and oculomotor control during reading: the role of implicit lexical decisions.

The coordination of word-recognition and oculomotor processes during reading was evaluated in eye-tracking experiments that examined how word skipping, where a word is not fixated during first-pass reading, is affected by the lexical status of a letter string in the parafovea and ease of recognizing that string. Ease of lexical recognition was manipulated through target-word frequency (Experiment 1) and through repetition priming between prime-target pairs embedded in a sentence (Experiment 2). Using the gaze-contingent boundary technique the target word appeared in the parafovea either with full preview or with transposed-letter (TL) preview. The TL preview strings were nonwords in Experiment 1 (e.g., bilnk created from the target blink), but were words in Experiment 2 (e.g., sacred created from the target scared). Experiment 1 showed greater skipping for high-frequency than low-frequency target words in the full preview condition, but not in the TL preview (nonword) condition. Experiment 2 showed greater skipping for target words that repeated an earlier prime word than for those that did not, with this repetition priming occurring both with preview of the full target and with preview of the targets TL neighbor word. However, time to progress from the word after the target was greater following skips of the TL preview word, whose meaning was anomalous in the sentence context, than following skips of the full preview word whose meaning fit sensibly into the sentence context. Together, the results support the idea that coordination between word-recognition and oculomotor processes occurs at the level of implicit lexical decisions.

Neural correlates of active vision: An fMRI comparison of natural reading and scene viewing.

Theories of eye movement control during active vision tasks such as reading and scene viewing have primarily been developed and tested using data from eye tracking and computational modeling, and little is currently known about the neurocognition of active vision. The current fMRI study was conducted to examine the nature of the cortical networks that are associated with active vision. Subjects were asked to read passages for meaning and view photographs of scenes for a later memory test. The eye movement control network comprising frontal eye field (FEF), supplementary eye fields (SEF), and intraparietal sulcus (IPS), commonly activated during single-saccade eye movement tasks, were also involved in reading and scene viewing, suggesting that a common control network is engaged when eye movements are executed. However, the activated locus of the FEF varied across the two tasks, with medial FEF more activated in scene viewing relative to passage reading and lateral FEF more activated in reading than scene viewing. The results suggest that eye movements during active vision are associated with both domain-general and domain-specific components of the eye movement control network.

Word recognition during reading: The interaction between lexical repetition and frequency

Memory studies utilizing long-term repetition priming have generally demonstrated that priming is greater for low-frequency than for high-frequency words and that this effect persists if words intervene between the prime and the target. In contrast, word-recognition studies utilizing masked short-term repetition priming have typically shown that the magnitude of repetition priming does not differ as a function of word frequency and does not persist across intervening words. We conducted an eyetracking-while-reading experiment to determine which of these patterns more closely resembles the relationship between frequency and repetition during the natural reading of a text. Frequency was manipulated using proper names that were either high-frequency (e.g., Stephen) or low-frequency (e.g., Dominic). The critical name was later repeated in the sentence, or a new name was introduced. First-pass reading times and skipping rates on the critical name revealed robust repetition-by-frequency interactions, such that the magnitude of the repetition-priming effect was greater for low-frequency than for high-frequency names. In contrast, measures of later processing showed effects of repetition that did not depend on lexical frequency. These results are interpreted within a framework that conceptualizes eye-movement control as being influenced in different ways by lexical- and discourse-level factors.

Individual differences in the perceptual span during reading: Evidence from the moving window technique

We report the results of an eye tracking experiment that used the gaze-contingent moving window technique to examine individual differences in the size of readers’ perceptual span. Participants read paragraphs while the size of the rightward window of visible text was systematically manipulated across trials. In addition, participants completed a large battery of individual-difference measures representing two cognitive constructs: language ability and oculomotor processing speed. Results showed that higher scores on language ability measures and faster oculomotor processing speed were associated with faster reading times and shorter fixation durations. More interestingly, the size of readers’ perceptual span was modulated by individual differences in language ability but not by individual differences in oculomotor processing speed, suggesting that readers with greater language proficiency are more likely to have efficient mechanisms to extract linguistic information beyond the fixated word.

Toward Semantics in the Wild: Activation to Manipulable Nouns in Naturalistic Reading

The neural basis of language processing, in the context of naturalistic reading of connected text, is a crucial but largely unexplored area. Here we combined functional MRI and eye tracking to examine the reading of text presented as whole paragraphs in two experiments with human subjects. We registered high-temporal resolution eye-tracking data to a low-temporal resolution BOLD signal to extract responses to single words during naturalistic reading where two to four words are typically processed per second. As a test case of a lexical variable, we examined the response to noun manipulability. In both experiments, signal in the left anterior inferior parietal lobule and posterior inferior temporal gyrus and sulcus was positively correlated with noun manipulability. These regions are associated with both action performance and action semantics, and their activation is consistent with a number of previous studies involving tool words and physical tool use. The results show that even during rapid reading of connected text, where semantics of words may be activated only partially, the meaning of manipulable nouns is grounded in action performance systems. This supports the grounded cognition view of semantics, which posits a close link between sensory–motor and conceptual systems of the brain. On the methodological front, these results demonstrate that BOLD responses to lexical variables during naturalistic reading can be extracted by simultaneous use of eye tracking. This opens up new avenues for the study of language and reading in the context of connected text. SIGNIFICANCE STATEMENT The study of language and reading has traditionally relied on single word or sentence stimuli. In fMRI, this is necessitated by the fact that time resolution of a BOLD signal much lower than that of cognitive processes that take place during natural reading of connected text. Here, we propose a method that combines eye tracking and fMRI, and can extract word-level information from the BOLD signal using high-temporal resolution eye tracking. In two experiments, we demonstrate the method by analyzing the activation of manipulable nouns as subjects naturally read paragraphs of text in the scanner, showing the involvement of action/motion perception areas. This opens up new avenues for studying neural correlates of language and reading in more ecologically realistic contexts.