Emmanuel Roche

Two parsing algorithms by means of finite state transducers

We present a new approach, illustrated by two algorithms, for parsing not only Finite State Grammars but also Context Free Grammars and their extension, by means of finite state machines. The basis is the computation of a fixed point of a finite-state function, i.e. a finite-state transducer. Using these techniques, we have built a program that parses French sentences with a grammar of more than 200,000 lexical rules with a typical response time of less than a second. The first algorithm computes a fixed point of a non-deterministic finite-state transducer and the second computes a fixed point of a deterministic bidirectional device called a bimachine. These two algorithms point out a new connection between the theory of parsing and the theory of representation of rational transductions.

Text disambiguation by finite state automata, an algorithm and experiments on corpora

Consulting a dictionary for the words of a given text provides multiple solutions, that is, ambiguities; thus, the sequence of words pilot studies could lead for example to:pilot: N singular, V infinitive, V (conjugated)studies: N plural, V (conjugated)pilot studies: N plural (compound).These informations could be organized in the form of a finite automaton such as:[see pdf for figure]The exploration of the context should provide clues that eliminate the non-relevant solutions. For this purpose we use local grammar constraints represented by finite automata. We have designed and implemented an algorithm which performs this task by using a large variety of linguistic constraints. Both the texts and the rules (or constraints) are represented in the same formalism, that is finite automata. Performing subtraction operations between text automata and constraint automata reduce the ambiguities. Experiments were performed on French texts with large scale dictionaries (one dictionary of 600.000 simple inflected forms and one dictionary of 150.000 inflected compunds). Syntactic patterns represented by automata, including shapes of compound nouns such as Noun followed by an Adjective (in gender-number agreement) (Cf 5.1), can be matched in texts.This process is thus an extension of the classic matching procedures because of the on-line dictionary consultation and because of the grammar constraints. It provides a simple and efficient indexing tool.

Smaller representations for finite-state transducers and finite-state automata

Finite-state transducers and finite-state automata are efficient and natural representations for a large variety of problems. We describe a new algorithm for turning a finite-state transducer into the composition of two deterministic finite-state transducers such that the combined size of the derived transducers can be exponentially smaller than other known deterministic constructions. As a consequence, this can also be used to build deterministic representations of finite-state automata smaller than the minimal finite-state automata computed by the classic determinization and minimization algorithms. We also report experimental results on large scale dictionaries and rule-based systems.

Transducer parsing of free and frozen sentences

In language processing, finite state models are not a lesser evil that bring simplicity and efficiency at the cost of accuracy. On the contrary, they provide a very natural framework to describe complex linguistic phenomena. We present here one aspect of parsing with finite state transducers and show that this technique can be applied to complex linguistic situations.