Charles F. Greenbacker

Access to multimodal articles for individuals with sight impairments

Although intelligent interactive systems have been the focus of many research efforts, very few have addressed systems for individuals with disabilities. This article presents our methodology for an intelligent interactive system that provides individuals with sight impairments with access to the content of information graphics (such as bar charts and line graphs) in popular media. The article describes the methodology underlying the systems intelligent behavior, its interface for interacting with users, examples processed by the implemented system, and evaluation studies both of the methodology and the effectiveness of the overall system. This research advances universal access to electronic documents.

UDel: Extending Reference Generation to Multiple Entities

We report on an attempt to extend a reference generation system, originally designed only for main subjects, to generate references for multiple entities in a single document. This endeavor yielded three separate systems: one utilizing the original classifier, another with a retrained classifier, and a third taking advantage of new data to improve the identification of interfering antecedents. Each subsequent system improved upon the results of the previous iteration.

Summarizing multimodal documents in popular media for people with visual impairments

People without visual impairments often skim a document and look at the graphics it contains before deciding whether to read it in detail. While this is easy for sighted people, it is far more challenging for people with visual impairments, especially since the graphical content is largely inaccessible. We propose to increase the universal accessibility of articles in popular media containing both text and graphical elements for people with visual impairments by automatically generating an abstractive summary of the entire multimodal document. Our goal is to enable users to quickly access the high-level content of a multimodal document, and to decide whether to commit to reading the full article.

UDel: Generating Referring Expressions Guided by Psycholinguistc Findings

We present an approach to generating referring expressions in context utilizing feature selection informed by psycholinguistic research. Features suggested by studies on pronoun interpretation were used to train a classifier system which determined the most appropriate selection from a list of possible references. This application demonstrates one way to help bridge the gap between computational and empirical means of reference generation.