Michael J. Spivey

Competing activation in bilingual language processing: Within- and between-language competition

Two eye-tracking experiments examined spoken language processing in Russian-English bilinguals. The proportion of looks to objects whose names were phonologically similar to the name of a target object in either the same language (within-language competition), the other language (between-language competition), or both languages at the same time (simultaneous competition) was compared to the proportion of looks in a control condition in which no objects overlapped phonologically with the target. Results support previous findings of parallel activation of lexical items within and between languages, but suggest that the magnitude of the between-language competition effect may vary across first and second languages and may be mediated by a number of factors such as stimuli, language background, and language mode.

Cross Talk Between Native and Second Languages: Partial Activation of an Irrelevant Lexicon

Bilingualism provides a unique opportunity for exploring hypotheses about how the human brain encodes language. For example, the “input switch” theory states that bilinguals can deactivate one language module while using the other. A new measure of spoken language comprehension, headband-mounted eyetracking, allows a firm test of this theory. When given spoken instructions to pick up an object, in a monolingual session, late bilinguals looked briefly at a distractor object whose name in the irrelevant language was initially phonetically similar to the spoken word more often than they looked at a control distractor object. This result indicates some overlap between the two languages in bilinguals, and provides support for parallel, interactive accounts of spoken word recognition in general.

Syntactic Ambiguity Resolution in Discourse: Modeling the Effects of Referential Context and Lexical Frequency

Sentences with temporarily ambiguous reduced relative clauses (e.g., The actress selected by the director believed that...) were preceded by discourse contexts biasing a main clause or a relative clause. Eye movements in the disambiguating region (by the director) revealed that, in the relative clause biasing contexts, ambiguous reduced relatives were no more difficult to process than unambiguous reduced relatives or full (unreduced) relatives. Regression analyses demonstrated that the effects of discourse context at the point of ambiguity (e.g., selected) interacted with the past participle frequency of the ambiguous verb. Reading times were modeled using a constraint-based competition framework in which multiple constraints are immediately integrated during parsing and interpretation. Simulations suggested that this framework reconciles the superficially conflicting results in the literature on referential context effects on syntactic ambiguity resolution.

Shared and separate systems in bilingual language processing: Converging evidence from eyetracking and brain imaging

The neurological and cognitive aspects of bilingual language processing were examined in late Russian-English bilinguals using headband-mounted eyetracking and functional neuroimaging. A series of three eyetracking studies suggested that, at early stages of word recognition, bilinguals can activate both languages in parallel, even when direct linguistic input is in one language only. A functional neuroimaging study suggested that, although the same general structures are active for both languages, differences within these general structures are present across languages and across levels of processing. For example, different centers of activation were associated with first versus second language processing within the left Inferior Frontal Gyrus, but not within the Superior Temporal Gyrus. We suggest that parallel activation (as found with eyetracking) and shared cortical structures (as found with fMRI) may be characteristic of early stages of language processing (such as phonetic processing), but the two languages may be using separate structures at later stages of processing (such as lexical processing).

Graded motor responses in the time course of categorizing atypical exemplars

The time course of categorization was investigated in four experiments, which revealed graded competitive effects in a categorization task. Participants clicked one of two categories (e.g.,mammal orfish) in response to atypical or typical exemplars (e.g.,whale orcat) in the form of words (Experiments 1 and 2) or pictures (Experiments 3 and 4). Streamingx, y coordinates of mouse movement trajectories were recorded. Normalized mean trajectories revealed a graded competitive process: Atypical exemplars produced trajectories with greater curvature toward the competing category than did typical exemplars. The experiments contribute to recent examination of the time course of categorization and carry implications for theories of representation in cognitive science.

Bilingual and monolingual processing of competing lexical items

Performance of bilingual Russian–English speakers and monolingual English speakers during auditory processing of competing lexical items was examined using eye tracking. Results revealed that both bilinguals and monolinguals experienced competition from English lexical items overlapping phonetically with an English target item (e.g., spear and speaker). However, only bilingual speakers experienced competition from Russian competitor items overlapping crosslinguistically with an English target (e.g., spear and spichki, Russian for matches). English monolinguals treated the Russian competitors as they did any other filler items. This difference in performance between bilinguals and monolinguals tested with exactly the same sets of stimuli suggests that eye movements to a crosslinguistic competitor are due to activation of the other language and to between-language competition rather than being an artifact of stimulus selection or experimental design. With the majority of the world’s population speaking more than one language (Romaine, 1995), studying bilingualism and multilingualism can provide valuable insights into human cognition and language. The capability of one cognitive system to successfully manage two languages is striking. Do bilinguals use the two languages independently, alternating between them by turning them on and off, or do they constantly keep both languages active and process the two in parallel at all times? The traditional language switch hypothesis, according to which bilinguals are able to selectively activate and deactivate their two languages (Gerard & Scarborough, 1989; MacNamara & Kushnir, 1971), has been challenged by a number of recent findings. Parallel activation has been inferred from early studies using the bilingual Stroop task (Chen & Ho, 1986; Preston & Lambert, 1969), code switching (Grainger, 1993; Grainger & Dijkstra, 1992; Li, 1996; Soares & Grosjean, 1984), interlingual homographs (Dijkstra, Timmermans, & Schriefers, 1997; Dijkstra, van Jaarsveld, & ten Brinke, 1998), cog

Continuous Dynamics in Real-Time Cognition

Real-time cognition is best described not as a sequence of logical operations performed on discrete symbols but as a continuously changing pattern of neuronal activity. The continuity in these dynamics indicates that, in between describable states of mind, mental activity does not lend itself to the linguistic labels relied on by much of psychology. We discuss eye-tracking and mouse-tracking evidence for this temporal continuity and provide geometric visualizations of mental activity, depicting it as a continuous trajectory through a state space (a multidimensional space in which locations correspond to mental states). When the state of the system travels toward a frequently visited region of that space, the destination may constitute recognition of a particular word or a particular object; but on the way there, the majority of the mental trajectory is in intermediate regions of that space, revealing graded mixtures of mental states.

Action Dynamics Reveal Parallel Competition in Decision Making

When deciding between two alternatives, such as whether toorder the pasta or the chicken, or whether to pursue a career inacademia or industry, a person may feel torn—as if the optionsliterally pull him or her in two directions. This metaphor mayhavesomesurprisingliteraltruth.Ifasked,forexample,whether‘‘murder is sometimes justified,’’ individuals may be inclined toboth agree and disagree with the statement. Here, we document,for the first time, the pull toward contrasting responses duringevaluative thinking, reporting the results of a study examiningthe trajectory of participants’ reaching movements toward dif-ferent response options.Our results suggest that a decision process is not necessarilycompleted inthe brain’scognitivesubsystems before itissharedwith other subsystems, as has been traditionally assumed.Rather,simultaneous‘‘pull’’frommultipleresponsealternativesseemstoinfluencetheexecutionofmovementitself.Thisfindingsuggests that a dynamic approach to mental processing—anapproach that has already provided descriptions of perception,attention,andcategorization (e.g.,AbramsBGoldSGratton,Coles,Sirevaag,Eriksen,D Hovland & Sears, 1938; McClelland & Rogers, 2003;Spivey, 2007; Tipper, Howard, & Houghton, 1999)—may shednew light on high-level cognition (Roe, Busemeyer, & Town-send, 2001; Townsend & Busemeyer, 1989).

The mechanics of embodiment: a dialog on embodiment and computational modeling

Embodied theories are increasingly challenging traditional views of cognition by arguing that conceptual representations that constitute our knowledge are grounded in sensory and motor experiences, and processed at this sensorimotor level, rather than being represented and processed abstractly in an amodal conceptual system. Given the established empirical foundation, and the relatively underspecified theories to date, many researchers are extremely interested in embodied cognition but are clamoring for more mechanistic implementations. What is needed at this stage is a push toward explicit computational models that implement sensorimotor grounding as intrinsic to cognitive processes. In this article, six authors from varying backgrounds and approaches address issues concerning the construction of embodied computational models, and illustrate what they view as the critical current and next steps toward mechanistic theories of embodiment. The first part has the form of a dialog between two fictional characters: Ernest, the “experimenter,” and Mary, the “computational modeler.” The dialog consists of an interactive sequence of questions, requests for clarification, challenges, and (tentative) answers, and touches the most important aspects of grounded theories that should inform computational modeling and, conversely, the impact that computational modeling could have on embodied theories. The second part of the article discusses the most important open challenges for embodied computational modeling.

Linguistically Mediated Visual Search

During an individuals normal interaction with the environment and other humans, visual and linguistic signals often coincide and can be integrated very quickly. This has been clearly demonstrated in recent eyetracking studies showing that visual perception constrains on-line comprehension of spoken language. In a modified visual search task, we found the inverse, that real-time language comprehension can also constrain visual perception. In standard visual search tasks, the number of distractors in the display strongly affects search time for a target defined by a conjunction of features, but not for a target defined by a single feature. However, we found that when a conjunction target was identified by a spoken instruction presented concurrently with the visual display, the incremental processing of spoken language allowed the search process to proceed in a manner considerably less affected by the number of distractors. These results suggest that perceptual systems specialized for language and for vision interact more fluidly than previously thought.