Jon Sprouse

A validation of Amazon Mechanical Turk for the collection of acceptability judgments in linguistic theory

Amazon’s Mechanical Turk (AMT) is a Web application that provides instant access to thousands of potential participants for survey-based psychology experiments, such as the acceptability judgment task used extensively in syntactic theory. Because AMT is a Web-based system, syntacticians may worry that the move out of the experimenter-controlled environment of the laboratory and onto the user-controlled environment of AMT could adversely affect the quality of the judgment data collected. This article reports a quantitative comparison of two identical acceptability judgment experiments, each with 176 participants (352 total): one conducted in the laboratory, and one conducted on AMT. Crucial indicators of data quality—such as participant rejection rates, statistical power, and the shape of the distributions of the judgments for each sentence type—are compared between the two samples. The results suggest that aside from slightly higher participant rejection rates, AMT data are almost indistinguishable from laboratory data.

Assessing the reliability of textbook data in syntax: Adger's Core Syntax 1

There has been a consistent pattern of criticism of the reliability of acceptability judgment data in syntax for at least 50 years (e.g., Hill 1961), culminating in several high-profile criticisms within the past ten years (Edelman & Christiansen 2003, Ferreira 2005, Wasow & Arnold 2005, Gibson & Fedorenko 2010, in press). The fundamental claim of these critics is that traditional acceptability judgment collection methods, which tend to be relatively informal compared to methods from experimental psychology, lead to an intolerably high number of false positive results. In this paper we empirically assess this claim by formally testing all 469 (unique, US-English)

Revisiting Satiation: Evidence for an Equalization Response Strategy

This reply revisits the topic of syntactic satiation as first discussed in Snyder 2000. I argue that the satiation effect reported in Snyder 2000 is the result of a response strategy in which participants attempt to equalize the number of yes and no responses, a strategy enabled by the design features of Snyders original experiment. Four predictions differentiate the response strategy from a true satiation effect. Nine experiments are presented to test these predictions. The results are discussed with respect to the nature of satiation, the stability of acceptability judgments, and the consequences for linguistic methodology.

Syntactic Islands and Learning Biases: Combining Experimental Syntax and Computational Modeling to Investigate the Language Acquisition Problem

The induction problems facing language learners have played a central role in debates about the types of learning biases that exist in the human brain. Many linguists have argued that some of the learning biases necessary to solve these language induction problems must be both innate and language-specific (i.e., the Universal Grammar (UG) hypothesis). Though there have been several recent high-profile investigations of the necessary learning bias types for different linguistic phenomena, the UG hypothesis is still the dominant assumption for a large segment of linguists due to the lack of studies addressing central phenomena in generative linguistics. To address this, we focus on how to learn constraints on long-distance dependencies, also known as syntactic island constraints. We use formal acceptability judgment data to identify the target state of learning for syntactic island constraints and conduct a corpus analysis of child-directed data to affirm that there does appear to be an induction problem when learning these constraints. We then create a computational learning model that implements a learning strategy capable of successfully learning the pattern of acceptability judgments observed in formal experiments, based on realistic input. Importantly, this model does not explicitly encode syntactic constraints. We discuss learning biases required by this model in detail as they highlight the potential problems posed by syntactic island effects for any theory of syntactic acquisition. We find that, although the proposed learning strategy requires fewer complex and domain-specific components than previous theories of syntactic island learning, it still raises difficult questions about how the specific biases required by syntactic islands arise in the learner. We discuss the consequences of these results for theories of acquisition and theories of syntax.

Experimental syntax and island effects

1. Experimental syntax and island effects: toward a comprehensive theory of islands Jon Sprouse and Norbert Hornstein Part I. Global Issues in the Investigation of Island Effects: 2. Deriving competing predictions from grammatical approaches and reductionist approaches to island effects Jon Sprouse, Matthew W. Wagers and Colin Phillips 3. Islands in the grammar? Standards of evidence Philip Hofmeister, Laura Staum Casasanto and Ivan A. Sag 4. On the nature of island constraints. I: Language processing and reductionist accounts Colin Phillips 5. Computational models of acquisition for islands Lisa Pearl and Jon Sprouse 6. On the nature of island constraints. II: Language learning and innateness Colin Phillips Part II. Specific Issues in the Investigation of Island Effects: 7. Memory mechanisms for wh-dependency formation and their implications for islandhood Matthew W. Wagers 8. Whats negative about negative islands? A re-evaluation of extraction from weak island contexts Robert Kluender and Simone Gieselman 9. On the structural nature of island constraints Brian Dillon and Norbert Hornstein 10. Backgrounded constituents cannot be extracted Adele E. Goldberg 11. Microvariation in islands Dave Kush, Akira Omaki and Norbert Hornstein 12. Subject islands in German revisited Johannes Jurka 13. Subject islands are different Maria Polinsky, Carlos G. Gallo, Peter Graff, Ekaterina Kravtchenko, Adam Milton Morgan and Anne Sturgeon 14. What vs. who and which: kind-denoting fillers and the complexity of whether-islands Theodora Alexopoulou and Frank Keller 15. Resumption in English Maria Polinsky, Lauren Eby Clemens, Adam Milton Morgan, Ming Xiang and Dustin Heestand 16. The island (in)sensitivity of sluicing and sprouting Masaya Yoshida, Jiyeon Lee and Michael Walsh Dickey.

The empirical status of data in syntax: A reply to Gibson and Fedorenko

This is a commentary in response to The need for quantitative methods in syntax and semantics research, by Edward Gibson and Evelina Fedorenko.

The Differential Sensitivity of Acceptability Judgments to Processing Effects

THE DIFFERENTIAL SENSITIVITY OF ACCEPTABILITY JUDGMENTS TO PROCESSING EFFECTS Jon Sprouse University of California, Irvine Rooij, Robert van. 2003. Questioning to resolve decision problems. Linguistics and Philosophy 26:727–763. Sharvit, Yael. 2002. Embedded questions and ‘de dicto’ readings. Natural Language Semantics 10:97–123. Szabolcsi, Anna. 1997. Background notions in lattice theory and generalized quantifiers. In Ways of scope taking, ed. by Anna Szabolcsi, 1–29. Dordrecht: Kluwer.

Experimental Syntax and Island Effects: Deriving competing predictions from grammatical approaches and reductionist approaches to island effects

Marr (1982) famously proposed that our theories of information-processing devices can be usefully stated at multiple levels: the computational level, the representational-algorithmic level, and the implementational level. Marr described the computational level as an answer to the question “What problem must this device solve?” He argued that the computational level would specify the properties of the problem that must be solved by the device and the computations that the device must perform in service of that goal, in a way that abstracts away from the exigencies of actually solving the problem in practice. Marr used a cash register as an example: the computational-level description of a cash register comprises the theory of addition, including properties such as commutativity and associativity. However, at the computational level there is no statement of the procedure the device follows or the series of states it occupies to carry out addition. A theory at that level of description is a representationalalgorithmic theory. For a cash register this could be the addition algorithm that we all learn in school, implemented in base 10: start from the right, and “carry over the ones”; or it could be implemented in base 2, which a digital device would use. Finally, Marr described the implementational level as a theory of how the operations of the algorithmic level are implemented in the hardware of the device. For a cash register, there are several hardware options that can implement this level, from the spinning drums in mechanical cash registers to the electronic processors in computers. Extending the Marr framework to sentence-level language phenomena is relatively straightforward, at least in theory. Grammatical theories tend to be computational-level descriptions, as they describe the properties of the final grammatical structures that must be built, as well as the properties of the structure-building operations that are required to build them, but abstract away from the requirements of real-time sentence processing. Parsing theories tend to be algorithmic-level theories, as they describe the specific parsing