Sameer S. Pradhan

Support Vector Learning for Semantic Argument Classification

The natural language processing community has recently experienced a growth of interest in domain independent shallow semantic parsing—the process of assigning a Who did What to Whom, When, Where, Why, How etc. structure to plain text. This process entails identifying groups of words in a sentence that represent these semantic arguments and assigning specific labels to them. It could play a key role in NLP tasks like Information Extraction, Question Answering and Summarization. We propose a machine learning algorithm for semantic role parsing, extending the work of Gildea and Jurafskyxa0(2002), Surdeanu et al.xa0(2003) and others. Our algorithm is based on Support Vector Machines which we show give large improvement in performance over earlier classifiers. We show performance improvements through a number of new features designed to improve generalization to unseen data, such as automatic clustering of verbs. We also report on various analytic studies examining which features are most important, comparing our classifier to other machine learning algorithms in the literature, and testing its generalization to new test set from different genre. On the task of assigning semantic labels to the PropBank (Kingsbury, Palmer, & Marcus,xa02002) corpus, our final system has a precision of 84% and a recall of 75%, which are the best results currently reported for this task. Finally, we explore a completely different architecture which does not requires a deep syntactic parse. We reformulate the task as a combined chunking and classification problem, thus allowing our algorithm to be applied to new languages or genres of text for which statistical syntactic parsers may not be available.

Semantic Role Labeling Using Different Syntactic Views

Semantic role labeling is the process of annotating the predicate-argument structure in text with semantic labels. In this paper we present a state-of-the-art baseline semantic role labeling system based on Support Vector Machine classifiers. We show improvements on this system by: i) adding new features including features extracted from dependency parses, ii) performing feature selection and calibration and iii) combining parses obtained from semantic parsers trained using different syntactic views. Error analysis of the baseline system showed that approximately half of the argument identification errors resulted from parse errors in which there was no syntactic constituent that aligned with the correct argument. In order to address this problem, we combined semantic parses from a Minipar syntactic parse and from a chunked syntactic representation with our original baseline system which was based on Charniak parses. All of the reported techniques resulted in performance improvements.

Semantic Role Chunking Combining Complementary Syntactic Views

This paper describes a semantic role labeling system that uses features derived from different syntactic views, and combines them within a phrase-based chunking paradigm. For an input sentence, syntactic constituent structure parses are generated by a Charniak parser and a Collins parser. Semantic role labels are assigned to the constituents of each parse using Support Vector Machine classifiers. The resulting semantic role labels are converted to an IOB representation. These IOB representations are used as additional features, along with flat syntactic chunks, by a chunking SVM classifier that produces the final SRL output. This strategy for combining features from three different syntactic views gives a significant improvement in performance over roles produced by using any one of the syntactic views individually.

University of Colorado dialog systems for travel and navigation

This paper presents recent improvements in the development of the University of Colorado CU Communicator and CU-Move spoken dialog systems. First, we describe the CU Communicator system that integrates speech recognition, synthesis and natural language understanding technologies using the DARPA Hub Architecture. Users are able to converse with an automated travel agent over the phone to retrieve up-to-date travel information such as flight schedules, pricing, along with hotel and rental car availability. The CU Communicator has been under development since April of 1999 and represents our test-bed system for developing robust human-computer interactions where reusability and dialogue system portability serve as two main goals of our work. Next, we describe our more recent work on the CU Move dialog system for in-vehicle route planning and guidance. This work is in joint collaboration with HRL and is sponsored as part of the DARPA Communicator program. Specifically, we will provide an overview of the task, describe the data collection environment for in-vehicle systems development, and describe our initial dialog system constructed for route planning.

Parsing arguments of nominalizations in English and Chinese

In this paper, we use a machine learning framework for semantic argument parsing, and apply it to the task of parsing arguments of eventive nominalizations in the FrameNet database. We create a baseline system using a subset of features introduced by Gildea and Jurafsky (2002), which are directly applicable to nominal predicates. We then investigate new features which are designed to capture the novelties in nominal argument structure and show a significant performance improvement using these new features. We also investigate the parsing performance of nominalizations in Chinese and compare the salience of the features for the two languages.

Estimating semantic confidence for spoken dialogue systems

In order for a spoken dialogue system to carry on a fluent conversation, it must be able to estimate confidence in its interpretation of the input that it is receiving. It must realize when it doesnt understand the user and interact to correct the problem. To this end, most systems have some form of confidence assessment mechanism. These algorithms generally estimate confidence on a word-by-word basis and sometimes use these estimates to accept or reject an utterance as a whole. This paper presents the confidence assessment mechanism developed for the CU Communicator spoken dialogue system. The focus of this mechanism is to assess confidence in the semantic representation extracted from an utterance by the system rather than in the string of words produced by the recognizer. We use a decision tree classifier with features based on acoustic models, language models, word lattice density, parser output and dialogue context. The classification performance is evaluated on a test set from the CU Communicator data. CU Communicator is a telephone based spoken dialogue system for getting information on air travel, hotels and rental cars.