Di Zhong

A fully automated content-based video search engine supporting spatiotemporal queries

The rapidity with which digital information, particularly video, is being generated has necessitated the development of tools for efficient search of these media. Content-based visual queries have been primarily focused on still image retrieval. In this paper, we propose a novel, interactive system on the Web, based on the visual paradigm, with spatiotemporal attributes playing a key role in video retrieval. We have developed innovative algorithms for automated video object segmentation and tracking, and use real-time video editing techniques while responding to user queries. The resulting system, called VideoQ , is the first on-line video search engine supporting automatic object-based indexing and spatiotemporal queries. The system performs well, with the user being able to retrieve complex video clips such as those of skiers and baseball players with ease.

VideoQ: an automated content based video search system using visual cues

The rapidity with which digitat information, particularly video, is being generated, has necessitated the development of tools for efficient search of these media. Content based visual queries have been primarily focussed on still image retrieval. In this papel; we propose a novel, real-time, interactive system on the Web, based on the visual paradigm, with spatio-temporal attributesplaying a key role in video retrieval. We have developed algorithms for automated video object segmentation and tracking and use real-time video editing techniques while responding to user queries. The resulting system pe

Clustering methods for video browsing and annotation

orms well, with the user being able to retrieve complex video clips such as those of skiers, baseball players, with ease.

Structure analysis of sports video using domain models

The large amount of video data makes it a tedious and hard job to browse and annotate them by just fast forward and rewind. Recent works in video parsing provide a foundation for building interactive and content based video browsing systems. In this paper, a generalized top-down hierarchial clustering process, which adopts partition clustering recursively at each level of the hierarchy, is studied and used to build hierarchical views of video shots. With the clustering processes, when a list of video programs or clips is provided, a browsing system can use either key-frame and/or shot features to cluster shots into classes, each of which consists of shots of similar content. After such clustering, each class of shots can be represented by an icon, which can then be displayed at the high levels of a hierarchical browser. As a result, users can know roughly the content of video shots even without moving down to a lower level of the hierarchy.

Scheme for visual feature-based image indexing

In this paper, we present an effective framework for scene detection and structure analysis for sports videos, using tennis and baseball as examples. Sports video can be characterized by its predictable temporal syntax, recurrent events with consistent features, and a fixed number of views. Our approach combines domain-specific knowledge, supervised machine learning techniques, and automatic feature analysis at multiple levels. Real time processing performance is achieved by utilizing compressed-domain processing techniques. High accuracy in view recognition is achieved by using compressed-domain global features as prefilters and object-level refined analysis in the latter verification stage. Applications include high-level structure browsing/navigation, highlight generation, and mobile media filtering.

An integrated approach for content-based video object segmentation and retrieval

As digital images are progressing into the mainstream of information systems, managing and manipulating them as images becomes an important issue to be resolved before we can take full advantage of their information content. To achieve content-based image indexing and retrieval, there are active research efforts in developing techniques to utilize visual features. On the other hand, without an effective indexing scheme, any visual content based image retrieval approach will lose its effectiveness as the number of features increases. This paper presents our initial work in developing an efficient indexing scheme using artificial neural network, which focuses on eliminating unlikely candidates rather than pin-pointing the targets directly. Experiment results in retrieving images using this scheme from a prototype visual database system are given.

Video object model and segmentation for content-based video indexing

Object-based video data representations enable unprecedented functionalities of content access and manipulation. We present an integrated approach using region-based analysis for semantic video object segmentation and retrieval. We first present an active system that combines low-level region segmentation with user inputs for defining and tracking semantic video objects. The proposed technique is novel in using an integrated feature fusion framework for tracking and segmentation at both region and object levels. Experimental results and extensive performance evaluation show excellent results compared to existing systems. Building upon the segmentation framework, we then present a unique region-based query system for semantic video object. The model facilitates powerful object search, such as spatio-temporal similarity searching at multiple levels.

Real-time content-based adaptive streaming of sports videos

Object segmentation and tracking is a key component for new generation of digital video representation, transmission and manipulations. Example applications include content based video database and video editing. We present a general schema for video object modeling, which incorporates low level visual features and hierarchical grouping. The schema provides a general framework for video object extraction, indexing, and classification. In addition, we present new video segmentation and tracking algorithms based on salient color and affine motion features. Color feature is used for intra frame segmentation; affine motion is used for tracking image segments over time. Experimental evaluation results using several test video streams are included.

Spatio-temporal video search using the object based video representation

We present a real-time software system for filtering live video content and then adoptively streaming the video over resource limited networks or devices. The rate adaptation is content-based and dynamically varied according to the event structure and the video content. The system includes a content analysis module for detecting important segments in the video, a variable-rate encoding module, and a buffer management module for streaming the variable rate video over low bandwidth networks. We have implemented a prototype system, which performs real-time content adaptation for tennis and baseball with promising results.

Real-time view recognition and event detection for sports video

Object-based video representation provides great promises for new search and editing functionalities. Feature regions in video sequences are automatically segmented, tracked, and grouped to form the basis for content-based video search and higher levels of abstraction. We present a new system for video object segmentation and tracking using feature fusion and region grouping. We also present efficient techniques for spatio-temporal video query based on the automatically segmented video objects.