Eiling Yee

Eye Movements to Pictures Reveal Transient Semantic Activation during Spoken Word Recognition.

Two experiments explore the activation of semantic information during spoken word recognition. Experiment 1 shows that as the name of an object unfolds (e.g., lock), eye movements are drawn to pictorial representations of both the named object and semantically related objects (e.g., key). Experiment 2 shows that objects semantically related to an uttered words onset competitors become active enough to draw visual attention (e.g., if the uttered word is logs, participants fixate on key because of partial activation of lock), despite that the onset competitor itself is not present in the visual display. Together, these experiments provide detailed information about the activation of semantic information associated with a spoken word and its phonological competitors and demonstrate that transient semantic activation is sufficient to impact visual attention.

Neural correlates of lexicon and grammar: Evidence from the production, reading, and judgment of inflection in aphasia

Are the linguistic forms that are memorized in the mental lexicon and those that are specified by the rules of grammar subserved by distinct neurocognitive systems or by a single computational system with relatively broad anatomic distribution? On a dual-system view, the productive -ed-suffixation of English regular past tense forms (e.g., look-looked) depends upon the mental grammar, whereas irregular forms (e.g., dig-dug) are retrieved from lexical memory. On a single-mechanism view, the computation of both past tense types depends on associative memory. Neurological double dissociations between regulars and irregulars strengthen the dual-system view. The computation of real and novel, regular and irregular past tense forms was investigated in 20 aphasic subjects. Aphasics with non-fluent agrammatic speech and left frontal lesions were consistently more impaired at the production, reading, and judgment of regular than irregular past tenses. Aphasics with fluent speech and word-finding difficulties, and with left temporal/temporo-parietal lesions, showed the opposite pattern. These patterns held even when measures of frequency, phonological complexity, articulatory difficulty, and other factors were held constant. The data support the view that the memorized words of the mental lexicon are subserved by a brain system involving left temporal/temporo-parietal structures, whereas aspects of the mental grammar, in particular the computation of regular morphological forms, are subserved by a distinct system involving left frontal structures.

Lexical-semantic activation in broca's and wernicke's aphasia: Evidence from eye movements

Lexical processing requires both activating stored representations and selecting among active candidates. The current work uses an eye-tracking paradigm to conduct a detailed temporal investigation of lexical processing. Patients with Brocas and Wernickes aphasia are studied to shed light on the roles of anterior and posterior brain regions in lexical processing as well as the effects of lexical competition on such processing. Experiment 1 investigates whether objects semantically related to an uttered word are preferentially fixated, for example, given the auditory target hammer, do participants fixate a picture of a nail? Results show that, like normal controls, both groups of patients are more likely to fixate on an object semantically related to the target than an unrelated object. Experiment 2 explores whether Brocas and Wernickes aphasics show competition effects when words share onsets with the uttered word, for instance, given the auditory target hammer, do participants fixate a picture of a hammock? Experiment 3 investigates whether these patients activate words semantically related to onset competitors of the uttered word, for example, given the auditory target hammock, do participants fixate a nail due to partial activation of the onset competitor hammer? Results of Experiments 2 and 3 show pathological patterns of performance for both Brocas and Wernickes aphasics under conditions of lexical onset competition. However, the patterns of deficit differed, suggesting different functional and computational roles for anterior and posterior areas in lexical processing. Implications of the findings for the functional architecture of the lexical processing system and its potential neural substrates are considered.

Theories of spoken word recognition deficits in Aphasia: Evidence from eye-tracking and computational modeling

We used eye-tracking to investigate lexical processing in aphasic participants by examining the fixation time course for rhyme (e.g., carrot-parrot) and cohort (e.g., beaker-beetle) competitors. Brocas aphasic participants exhibited larger rhyme competition effects than age-matched controls. A re-analysis of previously reported data (Yee, Blumstein, & Sedivy, 2008) confirmed that Wernickes aphasic participants exhibited larger cohort competition effects. Individual-level analyses revealed a negative correlation between rhyme and cohort competition effect size across both groups of aphasic participants. Computational model simulations were performed to examine which of several accounts of lexical processing deficits in aphasia might account for the observed effects. Simulation results revealed that slower deactivation of lexical competitors could account for increased cohort competition in Wernickes aphasic participants; auditory perceptual impairment could account for increased rhyme competition in Brocas aphasic participants; and a perturbation of a parameter controlling selection among competing alternatives could account for both patterns, as well as the correlation between the effects. In light of these simulation results, we discuss theoretical accounts that have the potential to explain the dynamics of spoken word recognition in aphasia and the possible roles of anterior and posterior brain regions in lexical processing and cognitive control.

Manual Experience Shapes Object Representations

How do people represent object meaning? It is now uncontentious that thinking about manipulable objects (e.g., pencils) activates brain regions underlying action. But is this activation part of the meaning of these objects, or is it merely incidental? The research we report here shows that when the hands are engaged in a task involving motions that are incompatible with those used to interact with frequently manipulated objects, it is more difficult to think about those objects—but not harder to think about infrequently manipulated objects (e.g., bookcases). Critically, the amount of manual experience with the object determines the amount of interference. These findings show that brain activity underlying manual action is part of, not peripheral to, the representation of frequently manipulated objects. Further, they suggest that people’s ability to think about an object changes dynamically on the basis of the match between their (experience-based) mental representation of its meaning and whatever they are doing at that moment.

Colorless Green Ideas (Can) Prime Furiously

That similar words can prime one another is not news. However, this phenomenon can be exploited to make inferences about the organization of conceptual representations. What types of similarity matter? Although there is evidence that similarity of function, shape, and even manner of manipulation is reflected in semantic memory, evidence for organization on the basis of color similarity is sparse. This lack of evidence is surprising: Intuition suggests that color is a prominent feature of many object concepts. The research reported here clarifies this puzzle and illustrates the dynamic nature of conceptual representations. Our research demonstrates color-based priming (e.g., “emerald” primes “cucumber”) in participants who completed a Stroop color-naming task before a priming task. Notably, the size of the Stroop effect predicted the size of the priming effect. When the order of tasks was reversed, priming effects were eliminated. By demonstrating that both extrinsic and intrinsic factors can influence conceptual activation, our findings have implications for theories of semantic memory.

Putting concepts into context

At first glance, conceptual representations (e.g., our internal notion of the object “lemon”) seem static; we have the impression that there is something that the concept lemon “means” (a sour, yellow, football-shaped citrus fruit) and that this meaning does not vary. Research in semantic memory has traditionally taken this “static” perspective. Consequently, only effects demonstrated across a variety of contexts have typically been considered informative regarding the architecture of the semantic system. In this review, we take the opposite approach: We review instances of context-dependent conceptual activation at many different timescales—from long-term experience, to recent experience, to the current task goals, to the unfolding process of conceptual activation itself—and suggest that the pervasive effects of context across all of these timescales indicate that rather than being static, conceptual representations are constantly changing and are inextricably linked to their contexts.

Function Follows Form: Activation of Shape and Function Features During Object Identification

Most theories of semantic memory characterize knowledge of a given object as comprising a set of semantic features. But how does conceptual activation of these features proceed during object identification? We present the results of a pair of experiments that demonstrate that object recognition is a dynamically unfolding process in which function follows form. We used eye movements to explore whether activating one objects concept leads to the activation of others that share perceptual (shape) or abstract (function) features. Participants viewed 4-picture displays and clicked on the picture corresponding to a heard word. In critical trials, the conceptual representation of 1 of the objects in the display was similar in shape or function (i.e., its purpose) to the heard word. Importantly, this similarity was not apparent in the visual depictions (e.g., for the target Frisbee, the shape-related object was a triangular slice of pizza, a shape that a Frisbee cannot take); preferential fixations on the related object were therefore attributable to overlap of the conceptual representations on the relevant features. We observed relatedness effects for both shape and function, but shape effects occurred earlier than function effects. We discuss the implications of these findings for current accounts of the representation of semantic memory.

Looking for meaning: Eye movements are sensitive to overlapping semantic features, not association

Theories of semantic memory differ in the extent to which relationships among concepts are captured via associative or via semantic relatedness. We examined the contributions of these two factors, using a visual world paradigm in which participants selected the named object from a four-picture display. We controlled for semantic relatedness while manipulating associative strength by using the visual world paradigm’s analogue to presenting asymmetrically associated pairs in either their forward or backward associative direction (e.g., ham—eggs vs. eggs—ham). Semantically related objects were preferentially fixated regardless of the direction of presentation (and the effect size was unchanged by presentation direction). However, when pairs were associated but not semantically related (e.g., iceberg—lettuce), associated objects were not preferentially fixated in either direction. These findings lend support to theories in which semantic memory is organized according to semantic relatedness (e.g., distributed models) and suggest that association by itself has little effect on this organization.

fMRI-adaptation evidence of overlapping neural representations for objects related in function or manipulation

Sensorimotor-based theories of semantic memory contend that semantic information about an object is represented in the neural substrate invoked when we perceive or interact with it. We used fMRI adaptation to test this prediction, measuring brain activation as participants read pairs of words. Pairs shared function (flashlight-lantern), shape (marble-grape), both (pencil-pen), were unrelated (saucer-needle), or were identical (drill-drill). We observed adaptation for pairs with both function and shape similarity in left premotor cortex. Further, degree of function similarity was correlated with adaptation in three regions: two in the left temporal lobe (left medial temporal lobe, left middle temporal gyrus), which has been hypothesized to play a role in mutimodal integration, and one in left superior frontal gyrus. We also found that degree of manipulation (i.e., action) and function similarity were both correlated with adaptation in two regions: left premotor cortex and left intraparietal sulcus (involved in guiding actions). Additional considerations suggest that the adaptation in these two regions was driven by manipulation similarity alone; thus, these results imply that manipulation information about objects is encoded in brain regions involved in performing or guiding actions. Unexpectedly, these same two regions showed increased activation (rather than adaptation) for objects similar in shape. Overall, we found evidence (in the form of adaptation) that objects that share semantic features have overlapping representations. Further, the particular regions of overlap provide support for the existence of both sensorimotor and amodal/multimodal representations.