Asha Vellaikal

Rule-based video classification system for basketball video indexing

Current information and communication technologies provide the infrastructure to send bits anywhere, but do not presume to handle information at the semantic level. This paper investigates the use of video content analysis and feature extraction and clustering techniques for further video semantic classifications and a supervised rule based video classification system is proposed. This system can be applied to the applications such as on-line video indexing, filtering and video summaries, etc. As an experiment, basketball video structure will be examined and categorized into different classes according to distinct visual and motional characteristics features by rule-based classifier. The semantics classes, the visual/motional feature descriptors and their statistical relationship are then studied in detail and experiment results based on basketball video will be provided and analyzed.

Semantic multicast: intelligently sharing collaborative sessions

We introduce the concept of semantic multicast to implement a large-scale shared interaction infrastructure providing mechanisms for collecting, indexing, and disseminating the information produced in collaborative sessions. This infrastructure captures the interactions between users (as video, text, audio and other data streams) and promotes a philosophy of filtering, archiving, and correlating collaborative sessions in user and context sensitive groupings. The semantic multicast service efficiently disseminates relevant information to every user engaged in the collaborative session, making the aggregated streams of the collaborative session available to the correct users at the right amount of detail. This contextual focus is accomplished by introducing proxy servers to gather, annotate, and filter the streams appropriate for specific interest groups. Users are subscribed to appropriate proxies, based on their profiles, and the collaborative session becomes a multi-level multicast of data from sources through proxies and to user interest groups.

Real-time content-based processing of multicast video

The access of high bandwidth multimedia data is generally limited to the network due to the limited network resources, inefficient indexing mechanism and less of semantic interpretations. Currently, most content-based video content representation involves the segmentation and indexing of video based on scene change and camera/object motion, and such research generally performs off-line video processing. Little research has been done on on-line video processing, which is crucial in video communication applications such as video conferencing, video multicasting and on-line video browsing and retrieval. This research investigates real-time content-based processing of multicast video over the Internet. New on-line video feature extraction schemes, such as scene change detection, on-line key frame classification, are considered to meet the requirement of real-time video multicasting filtering based on the users profile over the Internet. The annotation and features extracted from a multicast videoconference bitstream by the on-line video content analysis proxies, which are using the proposed video processing algorithms, are output to a separate metadata channel for the further assistance of semantic multicasting of video content. The performance of the proposed algorithms is also demonstrated.

Content-based image retrieval using multiresolution histogram representation

We present two new approaches based on color histogram indexing for content-based retrieval of image databases. Since the high computational complexity has been one of the main barriers towards the use of similarity measures such as histogram intersection in large databases, we propose a hierarchical indexing scheme where computationally efficient features are used to subset the image before more sophisticated techniques are applied for precise retrieval. The use of histograms at different color resolutions as filtering and matching features in a hierarchical scheme is studied. In the second approach, a multiresolution representation of the histogram using the indices and signs of its largest wavelet coefficients is examined. Excellent results have been observed using the latter method.

Hierarchical clustering techniques for image database organization and summarization

This paper investigates clustering techniques as a method of organizing image databases to support popular visual management functions such as searching, browsing and navigation. Different types of hierarchical agglomerative clustering techniques are studied as a method of organizing features space as well as summarizing image groups by the selection of a few appropriate representatives. Retrieval performance using both single and multiple level hierarchies are experimented with and the algorithms show an interesting relationship between the top k correct retrievals and the number of comparisons required. Some arguments are given to support the use of such cluster-based techniques for managing distributed image databases.

Online scene change detection of multicast (MBone) video

Many multimedia applications, such as multimedia data management systems and communication systems, require efficient representation of multimedia content. Thus semantic interpretation of video content has been a popular research area. Currently, most content-based video representation involves the segmentation of video based on key frames which are generated using scene change detection techniques as well as camera/object motion. Then, video features can be extracted from key frames. However most of such research performs off-line video processing in which the whole video scope is known as a priori which allows multiple scans of the stored video files during video processing. In comparison, relatively not much research has been done in the area of on-line video processing, which is crucial in video communication applications such as on-line collaboration, news broadcasts and so on. Our research investigates on-line real-time scene change detection of multicast video over the Internet. Our on-line processing system are designed to meet the requirements of real-time video multicasting over the Internet and to utilize the successful video parsing techniques available today. The proposed algorithms extract key frames from video bitstreams sent through the MBone network, and the extracted key frames are multicasted as annotations or metadata over a separate channel to assist in content filtering such as those anticipated to be in use by on-line filtering proxies in the Internet. The performance of the proposed algorithms are demonstrated and discussed in this paper.

Content-based retrieval of color and multispectral images using joint spatial-spectral indexing

A method based on the joint indexing of spatial and spectral information is presented for the purposes of content-based color and multispectral image retrieval. The image representation consists of hierarchical spatial structuring along with feature extraction in localized regions of the image. The algorithm comprises of a quadtree based image splitting method and a clustering technique to extract spectral information. It can support localized queries regarding either the colors or classes present in color and multispectral images. Experimental results suggest that the joint spatial and spectral indexing approach is a very flexible and efficient method for content-based queries in image database management.

On-line Scene Change Detection of Multicast Video

Network-based computing is becoming an increasingly important area of research, whereby computational elements within a distributed infrastructure process/enhance the data that traverses through its path. We refer to these computations as online processing and this paper investigates scene change detection in the context of MBone-based proxies in the network. On-line processing varies from traditional offline processing schemes, where for example, the whole video scope is known as a priori, which allows multiple scans of the stored video files during video processing. The on-line processing system is designed to meet the requirements of real-time video multicasting over the Internet and to utilize the successful video parsing techniques available today. The proposed algorithms do scene change detection and extract key frames from video bitstreams sent through the MBone network. We study several algorithms based on histogram differences and evaluate them with respect to precision, recall, and processing latency.

MB+-Tree: An Index Structure for Content-Based Retrieval

Though standard database management systems(DBMSs) dealing mainly with alphanumeric or spatial data have reached a high level of maturity, the techniques employed there cannot be effectively applied to the management of other multimedia entities such as images and video, primarily because of the differing nature of the data and the varying types of the queries posed to the system. Unlike traditional DBMSs, which normally retrieve a few records through the specification of exact queries based on the notion of “equality,” the types of queries expected in an image/video DBMS are relatively vague or fuzzy and are based on the notion of “similarity”. The result is that the similarity measure used can vary depending on the query posed to the system. Thus the indexing structure should be able to satisfy similarity-based queries for a wide range of similarity measures. Also, a realistic expectation of an image/video DBMS would be for it to reduce the search space by eliminating records which are completely irrelevant to the query. This “browsing” or “filtering” approach to query processing is suited to an image/video DBMS since the human visual system is quite capable of rapidly browsing through hundreds of images. In addition, the querying process in image/video DBMSs is expected to be iterative with progressively more refined queries being issued during the later stages. Thus the indexing structure should be able to efficiently support both vague queries(retrieving a large number of approximate matches) and “non-vague” queries(retrieving a small number of close matches).

Cooperative content analysis agents for online multimedia indexing and filtering

New integrated services are emerging from the rapid technological advances in networking, multi-agents, media and broadcasting technologies. This advancement allows for large amounts of multimedia information to be distributed and shared on the Internet. To fulfill the goal of the efficient searching and effective redistribution of online multimedia, automatic segmentation, efficient content analysis and effective semantic concept interpretation of visual data are required. In this research, we propose a system with cooperative content analysis agents for automatic video analysis to support fast online multimedia indexing and filtering. An independent metadata channel is also proposed as the communication mechanism among the content agents. The proposed system is evaluated and applied to CNN news distribution and sharing over the Mbone Multicast.