Mehmet Emin Dönderler

BilVideo: Design and Implementation of a Video Database Management System

With the advances in information technology, the amount of multimedia data captured, produced, and stored is increasing rapidly. As a consequence, multimedia content is widely used for many applications in today’s world, and hence, a need for organizing this data, and accessing it from repositories with vast amount of information has been a driving stimulus both commercially and academically. In compliance with this inevitable trend, first image and especially later video database management systems have attracted a great deal of attention, since traditional database systems are designed to deal with alphanumeric information only, thereby not being suitable for multimedia data.In this paper, a prototype video database management system, which we call BilVideo, is introduced. The system architecture of BilVideo is original in that it provides full support for spatio-temporal queries that contain any combination of spatial, temporal, object-appearance, external-predicate, trajectory-projection, and similarity-based object-trajectory conditions by a rule-based system built on a knowledge-base, while utilizing an object-relational database to respond to semantic (keyword, event/activity, and category-based), color, shape, and texture queries. The parts of BilVideo (Fact-Extractor, Video-Annotator, its Web-based visual query interface, and its SQL-like textual query language) are presented, as well. Moreover, our query processing strategy is also briefly explained.

BilVideo: a video database management system

The BilVideo video database management system provides integrated support for spatiotemporal and semantic queries for video. A knowledge base, consisting of a fact base and a comprehensive rule set implemented in Prolog, handles spatio-temporal queries. These queries contain any combination of conditions related to direction, topology, 3D relationships, object appearance, trajectory projection, and similarity-based object trajectories. The rules in the knowledge base significantly reduce the number of facts representing the spatio-temporal relations that the system needs to store. A feature database stored in an object-relational database management system handles semantic queries. To respond to user queries containing both spatio-temporal and semantic conditions, a query processor interacts with the knowledge base and object-relational database and integrates the results returned from these two system components. Because of space limitations, we only discuss the Web-based visual query interface and its fact-extractor and video-annotator tools. These tools populate the systems fact base and feature database to support both query types.

Topic segmentation of message hierarchies for indexing and navigation support

Message hierarchies in web discussion boards grow with new postings. Threads of messages evolve as new postings focus within or diverge from the original themes of the threads. Thus, just by investigating the subject headings or contents of earlier postings in a message thread, one may not be able to guess the contents of the later postings. The resulting navigation problem is further compounded for blind users who need the help of a screen reader program that can provide only a linear representation of the content. We see that, in order to overcome the navigation obstacle for blind as well as sighted users, it is essential to develop techniques that help identify how the content of a discussion board grows through generalizations and specializations of topics. This knowledge can be used in segmenting the content in coherent units and guiding the users through segments relevant to their navigational goals. Our experimental results showed that the segmentation algorithm described in this paper provides up to 80-85% success rate in labeling messages. The algorithm is being deployed in a software system to reduce the navigational load of blind students in accessing web-based electronic course materials; however, we note that the techniques are equally applicable for developing web indexing and summarization tools for users with sight.

A rule-based video database system architecture

We propose a novel architecture for a video database system incorporating both spatio-temporal and semantic (keyword, event/activity and category-based) query facilities. The originality of our approach stems from the fact that we intend to provide full support for spatio-temporal, relative object-motion and similarity-based objecttrajectory queries by a rule-based system utilizing a knowledge-base while using an object-relational database to answer semantic-based queries. Our method of extracting and modeling spatio-temporal relations is also a unique one such that we segment video clips into shots using spatial relationships between objects in video frames rather than applying a traditional scene detection algorithm. The technique we use is simple, yet novel and powerful in terms of effectiveness and user query satisfaction: video clips are segmented into shots whenever the current set of relations between objects changes and the video frames, where these changes occur, are chosen as keyframes. The directional, topological and third-dimension relations used for shots are those of the keyframes selected to represent the shots and this information is kept, along with frame numbers of the keyframes, in a knowledge-base as Prolog facts. The system has a comprehensive set of inference rules to reduce the number of facts stored in the knowledge-base because a considerable number of facts, which otherwise would have to be stored explicitly, can be derived by rules with some extra effort.

Rule-based spatiotemporal query processing for video databases

Abstract.In our earlier work, we proposed an architecture for a Web-based video database management system (VDBMS) providing an integrated support for spatiotemporal and semantic queries. In this paper, we focus on the task of spatiotemporal query processing and also propose an SQL-like video query language that has the capability to handle a broad range of spatiotemporal queries. The language is rule-based in that it allows users to express spatial conditions in terms of Prolog-type predicates. Spatiotemporal query processing is carried out in three main stages: query recognition, query decomposition, and query execution.

A Rule-Based Approach to Represent Spatio-Temporal Relations in Video Data

In this paper, we propose a new approach for high level segmentation of a video clip into shots using spatio-temporal relationships between objects in video frames. The technique we use is simple, yet novel and powerful in terms of effectiveness and user query satisfaction. Video clips are segmented into shots whenever the current set of relations between objects changes and the video frames where these changes have occurred are chosen as key frames. The topological and directional relations used for shots are those of the key frames that have been selected to represent shots and this information is kept, along with key frame intervals, in a knowledge-base as Prolog facts. We also have a comprehensive set of inference rules in order to reduce the number of facts stored in our knowledge-base because a considerable number of facts, which otherwise would have to be stored explicitly, can be derived by these rules with some extra effort.

SEA: Segment-enrich-annotate paradigm for adapting dialog-based content for improved accessibility

While navigation within complex information spaces is a problem for all users, the problem is most evident with individuals who are blind who cannot simply locate, point, and click on a link in hypertext documents with a mouse. Users who are blind have to listen searching for the link in the document using only the keyboard and a screen reader program, which may be particularly inefficient in large documents with many links or deep hierarchies that are hard to navigate. Consequently, they are especially penalized when the information being searched is hidden under multiple layers of indirections. In this article, we introduce a segment-enrich-annotate (SEA) paradigm for adapting digital content with deep structures for improved accessibility. In particular, we instantiate and evaluate this paradigm through the iCare-Assistant, an assistive system for helping students who are blind in accessing Web and electronic course materials. Our evaluations, involving the participation of students who are blind, showed that the iCare-Assistant system, built based on the SEA paradigm, reduces the navigational overhead significantly and enables user who are blind access complex online course servers effectively.

BilVideo video database management system

A prototype video database management system, which we call BilVideo, is presented. BilVideo provides an integrated support for queries on spatio-temporal, semantic and low-level features (color, shape, and texture) on video data. BilVideo does not target a specific application, and thus, it can be used to support any application with video data. An example application, news archives search system, is presented with some sample queries.

An efficient query optimization strategy for spatio-temporal queries in video databases

The interest for multimedia database management systems has grown rapidly due to the need for the storage of huge volumes of multimedia data in computer systems. An important building block of a multimedia database system is the query processor, and a query optimizer embedded to the query processor is needed to answer user queries efficiently. Query optimization problem has been widely studied for conventional database systems however it is a new research area for multimedia database systems. Due to the differences in query processing strategies, query optimization techniques used in multimedia database systems are different from those used in traditional databases. In this paper, a query optimization strategy is proposed for processing spatio-temporal queries in video database systems. The proposed strategy includes reordering algorithms to be applied on query execution tree. The performance results obtained by testing the reordering algorithms on different query sets are also presented.

FMware: middleware for efficient filtering and matching of XML messages with local data

XML message filtering systems are used for sifting through real-time messages to support business data mining and reporting. An XML message filtering system needs to (a) process registered filter predicates on multiple distributed real-time streams and (b) match and validate the filter results with local data to identify the relevant data that can be used for higher-level processing. Although efficient real-time filtering schemes exists, the matching phase of the operation where filter results have to be matched against local data to select those matches that are relevant to the particular task remains to be expensive as it requires expensive join operations. In this paper, we present an efficient middleware (FMware) for filtering and matching XML messages against locally available data. The proposed operator relies on a novel cluster-domain matching scheme to reduce the cost of the process. We analytically study the cost of the proposed middleware and experimentally show that it adaptively reduces the number of local data accesses and provides large savings in matching time with respect to cluster-unaware matching.