Michael Fleischman

Fine grained classification of named entities

While Named Entity extraction is useful in many natural language applications, the coarse categories that most NE extractors work with prove insufficient for complex applications such as Question Answering and Ontology generation. We examine one coarse category of named entities, persons, and describe a method for automatically classifying person instances into eight finer-grained subcategories. We present a supervised learning method that considers the local context surrounding the entity as well as more global semantic information derived from topic signatures and WordNet. We reinforce this method with an algorithm that takes advantage of the presence of entities in multiple contexts.

The human speechome project

The Human Speechome Project is an effort to observe and computationally model the longitudinal course of language development for a single child at an unprecedented scale. We are collecting audio and video recordings for the first three years of one childs life, in its near entirety, as it unfolds in the childs home. A network of ceiling-mounted video cameras and microphones are generating approximately 300 gigabytes of observational data each day from the home. One of the worlds largest single-volume disk arrays is under construction to house approximately 400,000 hours of audio and video recordings that will accumulate over the three year study. To analyze the massive data set, we are developing new data mining technologies to help human analysts rapidly annotate and transcribe recordings using semi-automatic methods, and to detect and visualize salient patterns of behavior and interaction. To make sense of large-scale patterns that span across months or even years of observations, we are developing computational models of language acquisition that are able to learn from the childs experiential record. By creating and evaluating machine learning systems that step into the shoes of the child and sequentially process long stretches of perceptual experience, we will investigate possible language learning strategies used by children with an emphasis on early word learning.

Mining temporal patterns of movement for video content classification

Scalable approaches to video content classification are limited by an inability to automatically generate representations of events that encode abstract temporal structure. This paper presents a method in which temporal information is captured by representing events using a lexicon of hierarchical patterns of movement that are mined from large corpora of unannotated video data. These patterns are then used as features for a discriminative model of event classification that exploits tree kernels in a Support Vector Machine. Evaluations show the method learns informative patterns on a 1450-hour video corpus of natural human activities recorded in the home.

Situated Models of Meaning for Sports Video Retrieval

Situated models of meaning ground words in the non-linguistic context, or situation, to which they refer. Applying such models to sports video retrieval requires learning appropriate representations for complex events. We propose a method that uses data mining to discover temporal patterns in video, and pair these patterns with associated closed captioning text. This paired corpus is used to train a situated model of meaning that significantly improves video retrieval performance.

Temporal feature induction for baseball highlight classification

Most approaches to highlight classification in the sports domain exploit only limited temporal information. This paper presents a method, called temporal feature induction, which automatically mines complex temporal information from raw video for use in highlight classification. The method exploits techniques from temporal data mining to discover a codebook of temporal patterns that encode long distance dependencies and duration information. Preliminary experiments show that using such induced temporal features significantly improves performance of a baseball highlight classification system.

Recommendations without user preferences: a natural language processing approach

We examine the problems with automated recommendation systems when information about user preferences is limited. We equate the problem to one of content similarity measurement and apply techniques from Natural Language Processing to the domain of movie recommendation. We describe two algorithms, a naïve word-space approach and a more sophisticated approach using topic signatures, and evaluate their performance compared to baseline, gold standard, and commercial systems.

Unsupervised content-based indexing of sports video

This paper presents a methodology for automatically indexing a large corpus of broadcast baseball games using an unsupervised content-based approach. The method relies on the learning of a grounded language model which maps query terms to the non-linguistic context to which they refer. Grounded language models are learned from a large, unlabeled corpus of video events. Events are represented using a codebook of automatically discovered temporal patterns of low level features extracted from the raw video. These patterns are associated with words extracted from the closed captioning text using a generalization of Latent Dirichlet Allocation. We evaluate the benefit of the grounded language model by extending a traditional language model based approach to information retrieval. Experimental results indicate that using a grounded language model nearly doubles performance on a held out test set.

A maximum entropy approach to FrameNet tagging

The development of FrameNet, a large database of semantically annotated sentences, has primed research into statistical methods for semantic tagging. We advance previous work by adopting a Maximum Entropy approach and by using Viterbi search to find the highest probability tag sequence for a given sentence. Further we examine the use of syntactic pattern based re-ranking to further increase performance. We analyze our strategy using both extracted and human generated syntactic features. Experiments indicate 85.7% accuracy using human annotations on a held out test set.

FrameNet-based semantic parsing using maximum entropy models

As part of its description of lexico-semantic predicate frames or conceptual structures, the FrameNet project defines a set of semantic roles specific to the core predicate of a sentence. Recently, researchers have tried to automatically produce semantic interpretations of sentences using this information. Building on prior work, we describe a new method to perform such interpretations. We define sentence segmentation first and show how Maximum Entropy re-ranking helps achieve a level of 76.2% F-score (answer among topfive candidates) or 61.5% (correct answer).

Unsupervised content-based indexing for sports video retrieval

This demonstration presents an interface to a corpus of broadcast baseball games that have been indexed using an unsupervised content-based method introduced here. The method uses the concept of a grounded language model to motivate a framework in which video is searched using natural language with no reliance on predetermined concepts or hand labeled events. The interface demonstrates the effectiveness of the technique and the ease of use it affords the user.