Bruce Tesar

Constraints in Phonological Acquisition: Learning phonotactic distributions

Many essentials of a language’s phonology can be learned from distributional evidence, in the absence of detailed morphological analysis. But distributional learning from positive evidence encounters the subset problem in full force. Here we explore an approach in which the learning algorithm, based on the errordriven variant of Recursive Constraint Demotion (RCD: Tesar 1995, Tesar & Smolensky 1998), is persistently biased to place markedness constraints as high as possible, but aims to place faithfulness constraint as low as possible. The learner seeks only to reproduce the output from identical input, avoiding all concern with nontrivial underlying forms; under the M?F bias this results in significant learning. (Hayes 1999 independently develops a similar approach from the same basic assumptions.) We introduce an explicit measure of the degree to which a hierarchy possesses M?F structure, and we investigate the consequences of trying to maximize this measure by low placement of F in suitably biased versions of RCD. We argue that existing proposals by which M?F structure is carried over from an initial state through a learning procedure blind to the M/F distinction, as in the different conceptions of Smolensky 1996 and Ito & Mester 1999a, cannot accomplish this goal successfully, as they are currently understood. We conclude that Biased Constraint Demotion (BCD) must be used by the learner at each step. The key issue is deciding which F to rank when there is more than one F constraint to choose from. We suggest that the main desideratum is the ’freeing up’ of further M constraints for ranking, though we also show that such decisions have further consequences downstream for the resultant hierarchy that may motivate a certain kind of ‘look ahead’ in the decision-making process. We also consider the issue of the ranking of specialgeneral pairs of faithfulness constraints, arguing that the matter cannot be resolved by examining the structure of constraints in isolation. We show that special/general relations can be derived mid-hierarchy, on the one hand, and on the other, can arise between constraints that appear to be independent. We note that in sharp contrast to the faithfulness situation, special/general relations between markedness constraints are handled automatically by BCD; this provides learning-theoretic motivation for resolving the positional markedness vs. positional faithfulness controversy (Beckman 1998, Zoll 1998) and for deeper scrutiny of faithfulness theory as a whole. Learning Phonotactic Distributions Alan Prince & Bruce Tesar Department of Linguistics Rutgers Center for Cognitive Science Rutgers University, New Brunswick 10/8/1999

The Learnability of Optimality Theory: An Algorithm and Some Basic Complexity Results

If Optimality Theory (Prince & Smolensky 1991, 1993) is correct, Universal Grammar provides a set of universal constraints which are highly general, inherently conflicting, and consequently rampantly violated in the surface forms of languages. A language’s grammar ranks the universal constraints in a dominance hierarchy, higher-ranked constraints taking absolute priority over lower-ranked constraints, so that violations of a constraint occur in well-formed structures when, and only when, they are necessary to prevent violation of higher-ranked constraints. Languages differ principally in how they rank the universal constraints in their language-specific dominance hierarchies. The surface forms of a given language are structural descriptions of inputs which are optimal in the following sense: they satisfy the universal constraints, or, when these constraints are brought into conflict by an input, they satisfy the highest-ranked constraints possible. This notion of optimality is partly language-specific, since the ranking of constraints is language-particular, and partly universal, since the constraints which evaluate well-formedness are (at least to a considerable extent) universal. In many respects, ranking of universal constraints in Optimality Theory plays a role analogous to parameter-setting in principles-and-parameters theory. Evidence in favor of this Optimality-Theoretic characterization of Universal Grammar is provided elsewhere; most work to date addresses phonology: see Prince & Smolensky 1993 (henceforth, ‘P&S’) and the several dozen works cited therein, notably McCarthy & Prince 1993; initial work addressing syntax includes Grimshaw 1993 and Legendre, Raymond & Smolensky 1993. Here, we investigate the learnability of grammars in Optimality Theory. Under the assumption of innate knowledge of the universal constraints, the primary task of the learner is the determination of the dominance ranking of these constraints which is particular to the target language. We will present a simple and efficient algorithm for solving this problem, assuming a given set of hypothesized underlying forms. (Concerning the problem of acquiring underlying forms, see the discussion of ‘optimality in the lexicon’ in P & S 1993:§9). The fact that surface forms are optimal means that every positive example entails a great number of implicit negative examples: for a given input, every candidate output other than the correct form is ill-formed.1 As a consequence, even a single positive example can greatly constrain the possible grammars for a target language, as we will see explicitly. In §1 we present the relevant principles of Optimality Theory and discuss the special nature of the learning problem in that theory. Readers familiar with the theory may wish to proceed directly to §1.3. In §2 we present the first version of our learning algorithm, initially, through a concrete example; we also consider its (low) computational complexity. Formal specification of the first version of the algorithm and proof of its correctness are taken up in the Appendix. In §3 we generalize the algorithm, identifying a more general core called Constraint Demotion(‘CD’) and then a family of CD algorithms which differ in how they apply this core to the acquisition data. We sketch a proof of the correctness and convergence of the CD algorithms, and of a bound on the number of examples needed to complete learning. In §4 we briefly consider the issue of ties in the ranking of constraints and the case of inconsistent data. Finally, we observe that the CD algorithm entails a Superset Principle for acquisition: as the learner refines the grammar, the set of well-formed structures shrinks.

An iterative strategy for language learning

Abstract One of the major challenges of language acquisition is the fact that the auditory signal received by a child underdetermines the structural description of the utterance. This paper approaches the problem by capitalizing on the optimizing structure of Optimality Theory. The learner uses a hypothesized grammar to make a best guess at the full structural description of an observed overt form, filling in the hidden structure not apparent. The learner uses it to modify their grammar, despite the fact that the full description is based in part on the previous (most likely wrong) grammar. The claim is that the learner can go back and forth between estimating the hidden structure and estimating the grammar, eventually converging on the correct grammar. The results of some simulations are presented in support of the claim.

Surgery in Language Learning

The architecture of generative phonology brings with it a difficult challenge for any learner: underlying forms must be acquired at the same time as the phonology – the system of rules or constraint-rankings. Yet, each depends on the other, and neither is known in advance. If the learner had prior knowledge of the underlying forms, then the constraint-ranking could be determined by familiar procedures. If the learner knew the ranking, then the range of viable underlying forms would be greatly limited, simplifying the process of finding them. In addition, the learner must identify a phonology which is maximally restrictive, so that distributional restrictions implicit in the data are enforced, rather than being portrayed as accidental gaps in the lexicon. Since it is impossible to explore all possible lexicon-phonology pairings, an effective learner must use an incremental strategy which goes back and forth between hypotheses about the lexicon and hypotheses about the phonology, testing and improving each until a satisfactory match is found. A principal issue in designing any such procedure is deciding what to do when the mapping fails: should the lexicon be changed or should the phonology be changed?

Using Inconsistency Detection to Overcome Structural Ambiguity

The Inconsistency Detection Learner (IDL) is an algorithm for language learning that addresses the problem of structural ambiguity.If an overt form is structurally ambiguous, the learner must be capable of inferring which interpretation of the overt form is correct by reference to other overt data of the language.The IDL does this by attempting to construct grammars for combinations of interpretations of the overt forms, and discarding those combinations that are inconsistent. The potential of this algorithm for overcoming the combinatorial growth in combinations of interpretations is supported by computational results from an implementation of the IDL using an optimality-theoretic system of metrical stress grammars.

Learning optimality-theoretic grammars☆

We present evidence that Optimality Theorys account of Universal Grammar has manifold implications for learning. The general principles of Optimality Theory (OT; Prince and Smolensky, 1993) are reviewed and illustrated with Grimshaw and Samek-Lodovicis (1995) OT theory of clausal subjects. The optimization structure OT provides grammar is used to derive a principled decomposition of the learning problem into the problem of assigning hidden structure to primary learning data and the problem of learning the grammar governing that hidden structure. Methods are proposed for analyzing both sub-problems, and their combination is illustrated for the problem of learning a stress system from data lacking metrical constituent boundaries. We present general theorems showing that the proposed solution to the grammar learning sub-problem exploits the special structure imposed by OT on the space of human grammars to correctly and efficiently home in on a target grammar.

Learning Phonological Grammars for Output-Driven Maps

A candidate is an input, an output, and a correspondence relation between them. An input for a word is constructed from the underlying forms for the morphemes of the word. A candidate has a set of (zero or more) disparities. A disparity is a difference between the input and the output of a candidate, for example when corresponding segments differ in the value of a feature. The candidate shown in (1) has two disparities; the subscripts are IO correspondence indices. The corresponding segments with index 2 disagree in stress: the input segment is unstressed, while the output segment is stressed. The corresponding segments with index 4 disagree in length: the input segment is long, while the output segment is short.

Faithful Contrastive Features in Learning

This article pursues the idea of inferring aspects of phonological underlying forms directly from surface contrasts by looking at optimality theoretic linguistic systems (Prince & Smolensky, 1993/2004). The main result proves that linguistic systems satisfying certain conditions have the faithful contrastive feature property: Whenever 2 distinct morphemes contrast on the surface in a particular environment, at least 1 of the underlying features on which the 2 differ must be realized faithfully on the surface. A learning procedure exploiting the faithful contrastive feature property, contrast analysis, can set the underlying values of some features, even where featural minimal pairs do not exist, but is nevertheless fundamentally limited in what it can set. This work suggests that observation of surface contrasts between pairs of words can contribute to the learning of underlying forms, while still supporting the view that interaction with the phonological mapping will be necessary to fully determine underlying forms.

Computing Optimal Descriptions for Optimality Theory Grammars with Context-Free Position Structures

This paper describes an algorithm for computing optimal structural descriptions for Optimality Theory grammars with context-free position structures. This algorithm extends Tesars dynamic programming approach (Tesar, 1994) (Tesar, 1995a) to computing optimal structural descriptions from regular to context-free structures. The generalization to context-free structures creates several complications, all of which are overcome without compromising the core dynamic programming approach. The resulting algorithm has a time complexity cubic in the length of the input, and is applicable to grammars with universal constraints that exhibit context-free locality.

Enforcing Grammatical Restrictiveness Can Help Resolve Structural Ambiguity

Two major issues in formal language learnability are the problem of learning restrictive distributions (sometimes known as the “subset problem”), and the problem of structural ambiguity. While substantial progress has been made in addressing each of these problems in isolation, a complication can arise when a learner is faced with a learning situation that exhibits both problems. It is possible for the two problems to interact: allowing grammars of differing restrictiveness can complicate efforts to contend with structural ambiguity. The main result of this paper is a demonstration that a construction already proposed for learning restrictive grammars, the r-measure, can be used to contend with the complications in structural ambiguity that result from the existence of grammars of differing restrictiveness.