Bruno Galantucci

The motor theory of speech perception reviewed.

More than 50 years after the appearance of the motor theory of speech perception, it is timely to evaluate its three main claims that (1) speech processing is special, (2) perceiving speech is perceiving gestures, and (3) the motor system is recruited for perceiving speech. We argue that to the extent that it can be evaluated, the first claim is likely false. As for the second claim, we review findings that support it and argue that although each of these findings may be explained by alternative accounts, the claim provides a single coherent account. As for the third claim, we review findings in the literature that support it at different levels of generality and argue that the claim anticipated a theme that has become widespread in cognitive science.

Brain-to-brain coupling: a mechanism for creating and sharing a social world.

Cognition materializes in an interpersonal space. The emergence of complex behaviors requires the coordination of actions among individuals according to a shared set of rules. Despite the central role of other individuals in shaping ones mind, most cognitive studies focus on processes that occur within a single individual. We call for a shift from a single-brain to a multi-brain frame of reference. We argue that in many cases the neural processes in one brain are coupled to the neural processes in another brain via the transmission of a signal through the environment. Brain-to-brain coupling constrains and shapes the actions of each individual in a social network, leading to complex joint behaviors that could not have emerged in isolation.

Experimental Semiotics: A New Approach For Studying Communication As A Form Of Joint Action

In the last few years, researchers have begun to investigate the emergence of novel forms of human communication in the laboratory. I survey this growing line of research, which may be called experimental semiotics, from three distinct angles. First, I situate the new approach in its theoretical and historical context. Second, I review a sample of studies that exemplify experimental semiotics. Third, I present an empirical study that illustrates how the new approach can help us understand the socio-cognitive underpinnings of human communication. The main conclusion of the paper will be that, by reproducing micro samples of historical processes in the laboratory, experimental semiotics offers new powerful tools for investigating human communication as a form of joint action.

Experimental Semiotics: A Review

In the last few years a new line of research has appeared in the literature. This line of research, which may be referred to as experimental semiotics (ES; Galantucci, 2009; Galantucci and Garrod, 2010), focuses on the experimental investigation of novel forms of human communication. In this review we will (a) situate ES in its conceptual context, (b) illustrate the main varieties of studies thus far conducted by experimental semioticians, (c) illustrate three main themes of investigation which have emerged within this line of research, and (d) consider implications of this work for cognitive neuroscience.

Joint Action: Current Perspectives

In recent years researchers have begun to investigate how the perceptual, motor and cognitive activities of two or more individuals become organized into coordinated action. In the first part of this introduction we identify three common threads among the ten papers of this special issue that exemplify this new line of research. First, all of the papers are grounded in the experimental study of online interactions between two or more individuals. Second, albeit at different levels of analysis, the contributions focus on the mechanisms supporting joint action. Third, many of the papers investigate empirically the pre-requisites for the highly sophisticated forms of joint action that are typical of humans. In the second part of the introduction, we summarize each of the papers, highlighting more specific connections among them.

How communication changes when we cannot mime the world: Experimental evidence for the effect of iconicity on combinatoriality.

Communication systems are exposed to two different pressures: a pressure for transmission efficiency, such that messages are simple to produce and perceive, and a pressure for referential efficiency, such that messages are easy to understand with their intended meaning. A solution to the first pressure is combinatoriality--the recombination of a few basic meaningless forms to express an infinite number of meanings. A solution to the second is iconicity--the use of forms that resemble what they refer to. These two solutions appear to be incompatible with each other, as iconic forms are ill-suited for use as meaningless combinatorial units. Furthermore, in the early stages of a communication system, when basic referential forms are in the process of being established, the pressure for referential efficiency is likely to be particularly strong, which may lead it to trump the pressure for transmission efficiency. This means that, where iconicity is available as a strategy, it is likely to impede the emergence of combinatoriality. Although this hypothesis seems consistent with some observations of natural language, it was unclear until recently how it could be soundly tested. This has changed thanks to the development of a line of research, known as Experimental Semiotics, in which participants construct novel communication systems in the laboratory using an unfamiliar medium. We conducted an Experimental Semiotic study in which we manipulated the opportunity for iconicity by varying the kind of referents to be communicated, while keeping the communication medium constant. We then measured the combinatoriality and transmission efficiency of the communication systems. We found that, where iconicity was available, it provided scaffolding for the construction of communication systems and was overwhelmingly adopted. Where it was not available, however, the resulting communication systems were more combinatorial and their forms more efficient to produce. This study enriches our understanding of the fundamental design principles of human communication and contributes tools to enrich it further.

Perceptuomotor Compatibility Effects in Speech

Kerzel and Bekkering (2000) found perceptuomotor compatibility effects between spoken syllables and visible speech gestures and interpreted them as evidence in favor of the distinctive claim of the motor theory of speech perception that the motor system is recruited for perceiving speech. We present three experiments aimed at testing this interpretation. In Experiment 1, we replicated the original findings by Kerzel and Bekkering but with audible syllables. In Experiments 2 and 3, we tested the results of Experiment 1 under more stringent conditions, with different materials and different experimental designs.In all of our experiments, we found the same result: Perceiving syllables affects uttering syllables. The result is consistent both with the results of a number of other behavioral and neural studies related to speech and with more general findings of perceptuomotor interactions. Taken together, these studies provide evidence in support of the motor theory claim that the motor system is recruited for perceiving speech.

Naming a Structured World: A Cultural Route to Duality of Patterning

The lexicons of human languages organize their units at two distinct levels. At a first combinatorial level, meaningless forms (typically referred to as phonemes) are combined into meaningful units (typically referred to as morphemes). Thanks to this, many morphemes can be obtained by relatively simple combinations of a small number of phonemes. At a second compositional level of the lexicon, morphemes are composed into larger lexical units, the meaning of which is related to the individual meanings of the composing morphemes. This duality of patterning is not a necessity for lexicons and the question remains wide open regarding how a population of individuals is able to bootstrap such a structure and the evolutionary advantages of its emergence. Here we address this question in the framework of a multi-agents model, where a population of individuals plays simple naming games in a conceptual environment modeled as a graph. We demonstrate that errors in communication as well as a blending repair strategy, which crucially exploits a shared conceptual representation of the environment, are sufficient conditions for the emergence of duality of patterning, that can thus be explained in a pure cultural way. Compositional lexicons turn out to be faster to lead to successful communication than purely combinatorial lexicons, suggesting that meaning played a crucial role in the evolution of language.

The emergence of duality of patterning: Insights from the laboratory

Abstract The concept of duality of patterning (henceforth DP) has recently begun to undergo new scrutiny. In particular, the fact that Al-Sayyid Bedouin Sign Language (ABSL) does not appear to exhibit a layer of meaningless units (Sandler et al. 2011) casts doubt on the universality of DP as a defining feature of natural language. Why, then, do the vast majority of the worlds languages exhibit DP? Two hypotheses have been suggested. The first is that DP is a necessary solution to the problem of conveying a large number of meanings; the second is that DP arises as a consequence of conventionalization. We tested these hypotheses in an experimental-semiotics study. Our results supported the hypothesis based on conventionalization but were inconclusive with regard to the hypothesis based on the number of meanings. At the same time, the task of measuring DP in an experimental-semiotics study presented interesting challenges, suggesting that the concept of DP may need some overhauling.

The locus and time course of long-term morphological priming

Two experiments investigated the effects of long-term morphological priming in the fragment completion task. Completions for some of the fragments were presented visually during a preceding task, and others were presented auditorily. In addition, some of the target completions were morphologically related to words that were presented visually during the study task, while still others were unrelated to any of the study words. Fragments were most likely to be completed if either the completion or one of its morphological relatives was presented visually during the study task. Analyses of response latencies also indicated that the time course of morphological priming was similar to that of visual identity priming and that both morphological and visual identity priming had earlier influences than auditory identity priming. Overall, the results indicate that morphological priming includes a modality-specific component that reflects the operation of processes that occur relatively early in the time course of processing.