David C. Gibbon

Generating hypermedia documents from transcriptions of television programs using parallel text alignment

This paper presents a method of automatically creating hypermedia documents from conventional transcriptions of television programs. Using parallel text alignment techniques, the temporal information derived from the closed caption signal is exploited to convert the transcription into a synchronized text stream. Given this text stream, we can create links between the transcription and the image and audio media streams. We describe a two-pass method for aligning parallel texts that first uses dynamic programming techniques to maximize the number of corresponding words (by minimizing the word edit distance). The second stage converts the word alignment into a sentence alignment, taking into account the cases of sentence split and merge. We present results of text alignment on a database of 610 programs (including three television news programs over a one-year period) for which we have closed caption, transcript, audio and image streams. The techniques presented can produce high quality hypermedia documents of video programs with little or no additional manual effort.

Automated generation of news content hierarchy by integrating audio, video, and text information

This paper addresses the problem of generating semantically meaningful content by integrating information from different media. The goal is to automatically construct a compact yet meaningful abstraction of the multimedia data that can serve as an effective index table, allowing users to browse through large amounts of data in a non-linear fashion with flexibility, efficiency, and confidence. We propose an integrated solution in the context of broadcast news that simultaneously utilizes cues from video, audio, and test to achieve the goal. Some experimental results are presented and discussed.

Geotracker: geospatial and temporal RSS navigation

The Web is rapidly moving towards a platform for mass collaboration in content production and consumption. Fresh content on a variety of topics, people, and places is being created and made available on the Web at breathtaking speed. Navigating the content effectively not only requires techniques such as aggregating various RSS-enabled feeds, but it also demands a new browsing paradigm. In this paper, we present novel geospatial and temporal browsing techniques that provide users with the capability of aggregating and navigating RSS-enabled content in a timely, personalized and automatic manner. In particular, we describe a system called GeoTracker that utilizes both a geospatial representation and a temporal (chronological) presentation to help users spot the most relevant updates quickly. Within the context of this work, we provide a middleware engine that supports intelligent aggregation and dissemination of RSS feeds with personalization to desktops and mobile devices. We study the navigation capabilities of this system on two kinds of data sets, namely, 2006 World Cup soccer data collected over two months and breaking news items that occur every day. We also demonstrate that the application of such technologies to the video search results returned by YouTube and Google greatly enhances a user.s ability in locating and browsing videos based on his or her geographical interests. Finally, we demonstrate that the location inference performance of GeoTracker compares well against machine learning techniques used in the natural language processing/information retrieval community. Despite its algorithm simplicity, it preserves high recall percentages.

Support vector machines: relevance feedback and information retrieval

We compare support vector machines (SVMs) to Rocchio, Ide regular and Ide dec-hi algorithms in information retrieval (IR) of text documents using relevancy feedback. It is assumed a preliminary search finds a set of documents that the user marks as relevant or not and then feedback iterations commence. Particular attention is paid to IR searches where the number of relevant documents in the database is low and the preliminary set of documents used to start the search has few relevant documents. Experiments show that if inverse document frequency (IDF) weighting is not used because one is unwilling to pay the time penalty needed to obtain these features, then SVMs are better whether using term-frequency (TF) or binary weighting. SVM performance is marginally better than Ide dec-hi if TF-IDF weighting is used and there is a reasonable number of relevant documents found in the preliminary search. If the preliminary search is so poor that one has to search through many documents to find at least one relevant document, then SVM is preferred.

Pictorial transcripts: multimedia processing applied to digital library creation

This paper describes a working system for the automated archiving and selective retrieval of textual, pictorial and auditory information contained in video programs. Video processing performs the task of representing the visual information using a small subset of the video frames. Linguistic processing refines the closed caption text, generates table of contents, and creates links to relevant multimedia documents. Audio and video information are compressed and indexed based on their temporal association with the selected video frames and processed text. The derived information is used to automatically generate a hypermedia rendition of the program contents. This provides a compact representation of the information contained in the video program. It also serves as a textual and pictorial index for selective retrieval of the full-motion video program. This fully automatic system generates HyperText Markup Language (HTML) renditions of television programs, and makes them available for access over the Internet within seconds of their broadcast. This digital library currently contains over 2200 hours of television programs.

A Fast, Comprehensive Shot Boundary Determination System

The proposed shot boundary determination (SBD) algorithm contains a set of finite state machine (FSM) based detectors for pure cut, fast dissolve, fade in, fade out, dissolve, and wipe. Support vector machines (SVM) are applied to the cut and dissolve detectors to further boost performance. Our SBD system was highly effective when evaluated in TRECVID 2006 (TREC video retrieval evaluation) and its performance was ranked highest overall.

Effective and scalable video copy detection

Video copy detection techniques are essential for a number of applications including discovering copyright infringement of multimedia content, monitoring commercial air time, and querying videos by example. Over the last decade, video copy detection has received rapidly growing attention from the multimedia research community. To encourage more innovative technology and benchmark the state of the art approaches in this field, the TRECVID conference series, sponsored by the NIST, initiated an evaluation task on content based copy detection in 2008. In this paper, we describe the content-based video copy detection framework developed at AT&T Labs - Research. We employed local visual features to match the video content, and adopted locality sensitve hashing and random sample consensus techniques to maintain the scalability and the robustness of our approach. Experimental results on TRECVID 2008 data show that our approach is effective and efficient.

GeoTV: navigating geocoded rss to create an iptv experience

The Web is rapidly moving towards a platform for mass collaboration in content production and consumption from three screens: computers, mobile phones, and TVs. While there has been a surge of interests in making Web content accessible from mobile devices, there is a significant lack of progress when it comes to making the web experience suitable for viewing on a television. Towards this end, we describe a novel concept, namely GeoTV, where we explore a framework by which web content can be presented or pushed in a meaningful manner to create an entertainment experience for the TV audience. Fresh content on a variety of topics, people, and places is being created and made available on the Web at breathtaking speed. Navigating fresh content effectively on TV demands a new browsing paradigm that requires fewer mouse clicks or user interactions from the remote control. Novel geospatial and temporal browsing techniques are provided in GeoTV that allow users the capability of aggregating and navigating RSS-enabled content in a timely, personalized and automatic manner for viewing in an IPTV environment. This poster is an extension of our previous work on GeoTracker that utilizes both a geospatial representation and a temporal (chronological) presentation to help users spot the most relevant updates quickly within the context of a Web-enabled environment. We demonstrate 1) the usability of such a tool that greatly enhances a user.s ability in locating and browsing videos based on his or her geographical interests and 2) various innovative interface designs for showing RSS-enabled information in an IPTV environment.

Clicker - An IPTV Remote Control in Your Cell Phone

This paper investigates a novel concept of providing seamless control and portability of an IPTV viewing session. A solution employing a middleware system, a secure hardware token and a cell phone are used to demonstrate how an IPTV session can be securely controlled remotely and moved between multiple viewing stations. We have built a prototype of the system and demonstrated its flexible features. Depending on the users protocol of choice, most remote control operations from a mobile device took less than 5 seconds to execute. An interesting capability of previewing content of other channels via the users device while still continuing to watch the program on the viewing station differentiates it from todays IPTV offers. Finally for mobile content delivery, we address the problem of dynamic device profile selection and content adaptation using a classification algorithm to match the best content alternative destined for a mobile.

The MIRACLE video search engine

This paper presents some of the searching and browsing features of the MIRACLE Video Search Engine. The rapid increase in the generation and dissemination of information and entertainment in video form has created a need for video search engines that facilitate finding and browsing relevant information. MIRACLE is an ongoing project at AT&T Labs aimed at addressing this need. This video search engine combines existing metadata with content-based information that is automatically extracted from the media components, or is obtained from other sources, to achieve this goal. A Web-based user interface provides browsing mechanisms that take advantage of the user’s perceptual abilities to refine the search results. The MIRACLE search engine currently operates on an archive of more than 32,000 hours of video that have been collected and automatically indexed over a ten year period.