Seungyup Paek

Integration of Visual and Text-Based Approaches for the Content Labeling and Classification of Photographs

Annotating photographs automatically with content descriptions facilitates organization, storage, and search over visual information. We present an integrated approach for scene classi cation that combines image-based and text-based approaches. On the text side, we use the text accompanying an image in a novel TF*IDF vector-based approach to classi cation. On the image side, we present a novel OF*IIF (object frequency) vector-based approach to classi cation. Objects are de ned by clustering of segmented regions of training images. The image based OF*IIF approach is synergistic with the text based TF*IDF approach. By integrating the TF*IDF approach and the OF*IIF approach, we achieved a classi cation accuracy of 86%. This is an improvement of approximately 12% over existing image classi ers, an improvement of approximately 3% over the TF*IDF image classi er based on textual information, and an improvement of approximately 4% over the OF*IIF image classi er based on visual information.

Object-based multimedia content description schemes and applications for MPEG-7

Abstract In this paper, we describe description schemes (DSs) for image, video, multimedia, home media, and archive content proposed to the MPEG-7 standard. MPEG-7 aims to create a multimedia content description standard in order to facilitate various multimedia searching and filtering applications. During the design process, special care was taken to provide simple but powerful structures that represent generic multimedia data. We use the extensible markup language (XML) to illustrate and exemplify the proposed DSs because of its interoperability and flexibility advantages. The main components of the image, video, and multimedia description schemes are object, feature classification, object hierarchy, entity-relation graph, code downloading, multi-abstraction levels, and modality transcoding. The home media description instantiates the former DSs proposing the 6-W semantic features for objects, and 1-P physical and 6-W semantic object hierarchies. The archive description scheme aims to describe collections of multimedia documents, whereas the former DSs only aim at individual multimedia documents. In the archive description scheme, the content of an archive is represented using multiple hierarchies of clusters, which may be related by entity-relation graphs. The hierarchy is a specific case of entity-relation graph using a containment relation. We explicitly include the hierarchy structure in our DSs because it is a natural way of defining composite objects, a more efficient structure for retrieval, and the representation structure used in MPEG-4. We demonstrate the feasibility and the efficiency of our description schemes by presenting applications that already use the proposed structures or will greatly benefit from their use. These applications are the visual apprentice, the AMOS-search system, a multimedia broadcast news browser, a storytelling system, and an image meta-search engine, MetaSEEk.

Detecting image purpose in World Wide Web documents

The number of WWW documents available to users of the Internet is growing at an incredible rate. Therefore, it is becoming increasingly important to develop systems that aid users in searching, filtering, and retrieving information from the Internet. Currently, only a few prototype systems catalog and index images in Web documents. To greatly improve the cataloging and indexing of images on the Web, we have developed a prototype rule-based systems that detects the content images in Web documents. Content images are images that are associated with the main content of Web documents, as opposed to a multitude of other images that exist in Web documents for different purposes, such as decorative, advertisement and logo images. We present a system that uses decision tree learning for automated rule induction for the content images detection system. The system uses visual features, text-related features and the document context of images in concert for fast and effective content image detection in Web documents. We have evaluated the system by collecting more than 1200 images from 4 different Web sites and we have achieved an overall classification accuracy of 84 percent.

Scalable MPEG2 Video Servers with Heterogeneous QoS on Parallel Disk Arrays

In this paper we focus on the video storage unit of a video server. We present a new, flexible data placement strategy for independent parallel disk arrays. The trade-off between utilization efficiency and interactive delay is investigated for this data placement strategy. Based on this trade-off, we show the advantage of video servers supporting a range of interactivity QoS. For our data placement strategy, we show that using scalable video improves the utilization and interactivity performance of a video server. We use three-layer, scalable MPEG2 digital video to support resolution QoS at a video server. Finally, we show that the data placement strategy reduces the complexity of admission control at the video server to that of a single disk system.

A knowledge engineering approach for image classification based on probabilistic reasoning systems

We present an image classification system based on probabilistic reasoning that was developed to meet two research objectives. The first objective was to exploit interactions between multiple classifiers to improve classification performance. The second objective was to exploit existing descriptions of an image to improve classification performance. A case study of applying this approach for the consumer photograph domain is presented, along with performance evaluation results.

Self-describing schemes for interoperable MPEG-7 multimedia content descriptions

In this paper, we present the self-describing schemes for interoperable image/video content descriptions, which are being developed as part of our proposal to the MPEG-7 standard. MPEG-7 aims to standardize content descriptions for multimedia data. The objective of this standard is to facilitate content-focused applications like multimedia searching, filtering, browsing, and summarization. To ensure maximum interoperability and flexibility, our descriptions are defined using the eXtensible Markup Language (XML), developed by the World Wide Web Consortium. We demonstrate the feasibility and efficiency of our self-describing schemes in our MPEG-7 testbed. First, we show how our scheme can accommodate image and video descriptions that are generated by a wide variety of systems. Then, we present two systems being developed that are enabled and enhanced by the proposed approach for multimedia content descriptions. The first system is an intelligent search engine with an associated expressive query interface. The second system is a new version of MetaSEEk, a metasearch system for mediation among multiple search engines for audio-visual information.

Video server retrieval scheduling for variable bit rate scalable video

In advanced multimedia networks, video servers with real time operating systems and high performance storage architectures will deliver multiple concurrent video streams to clients. The video server has a limited disk I/O bandwidth. This research presents a new framework for optimally utilizing the on board memory to alleviate the disk bandwidth bottleneck. The optimal utilization of the memory resource is achieved by using an optimal retrieval schedule for video data across the disk memory interface. The optimal retrieval schedule is directly applicable to video servers that are based on general storage architectures (e.g. RAID). Performance evaluation using real MPEG2 trace data verifies that this approach can provide large increases in the number of supported video streams.

Video-server retrieval scheduling and resource reservation for variable bit rate scalable video

State-of-the-art digital video compression produces bursty, variable bit rate video. The bursty nature of compressed video raises challenges in the design of video servers. In this paper, we first present a method for the efficient retrieval of bursty video data from the disk system to the memory of a digital video server. For a single video stream, the proposed retrieval schedule minimizes the buffer requirement for continuous retrieval, given that a fixed disk bandwidth is reserved for the entire duration of retrieval. Secondly, we present an optimal resource-reservation algorithm for multiple video streams based on the proposed retrieval schedule. The resource-reservation algorithm maximizes the number of bursty video streams that ran be supported by a video server given any disk bandwidth and memory resource. Thirdly, we present a progressive display scheme for scalable video that is based on the retrieval schedule and resource-reservation algorithm. Performance evaluations based on simulations using MPEG-2 trace data are presented. For a personal computer with four disks and a memory resource of 120 MB, our approach can support 50%-275% more video streams than previously proposed approaches, depending on the pre-fetch delay that users are willing to tolerate in interactive viewing of videos.

High Performance Digital Video Servers: Storage and Retrieval of Compressed Scalable Video

Abstract Developments in the fields of computing, computer networks and digital video compression technology have led to the emergence of unprecedented forms of video communications. In the future, it is envisioned that users will be able to connect to a massive number of distributed video servers, from which users will be able to select and receive high quality video and audio. High performance video servers will store a large number of compressed digital videos and allow multiple concurrent clients to connect and retrieve a video from the collection of videos. High performance video servers must provide guaranteed quality of service for each video stream that is being supported. The video server has to retrieve multiple video streams from the storage system and transmit the video data into the computer network. There has been a great deal of research and devel- opment in the past few years on the various issues related to high performance video servers. In this chapter, we provide an overview and snapshot of some of the main research areas that have been worked on recently. We critically evaluate and summarize the main research results. In particular, we focus on the interrelated issues of digital video compression, storage and retrieval for video servers.