Rajiv Mehrotra

Semantics of multimedia in MPEG-7

In this paper, we present the tools standardized by MPEG-7 for describing the semantics of multimedia. In particular, we focus on the abstraction model, entities, attributes and relations of MPEG-7 semantic descriptions. MPEG-7 tools can describe the semantics of specific instances of multimedia such as one image or one video segment but can also generalize these descriptions either to multiple instances of multimedia or to a set of semantic descriptions. The key components of MPEG-7 semantic descriptions are semantic entities such as objects and events, attributes of these entities such as labels and properties, and, finally, relations of these entities such as an object being the patient of an event. The descriptive power and usability of these tools has been demonstrated in numerous experiments and applications, these make them key candidates to enable intelligent applications that deal with multimedia at human levels.

Establishing motion-based feature point correspondence

Abstract Given a sequence of frames corresponding to a dynamic scene, multiframe correspondence of a set of selected image points, called feature or interest points, determine the motion trajectories of those points. This paper describes a new polynomial time algorithm for finding feature point trajectories from a large sequence of frames. The proposed approach utilizes a constraint, called motion uniformity constraint. According to the motion uniformity constraint, the changes in the motion characteristics of a point are small over a short period of time and are temporally continuous. This implies that the trajectory of a point over a given time interval is as “smooth” and “short” as possible. The proposed approach permits point trajectories to terminate and start at intermediate frames of the sequence. For a given sequence, the trajectory set is obtained by considering every set of three consecutive frames and finding the locally best local trajectories (in terms of motion uniformity) that extend some of the existing point trajectories and/or define new point trajectories. Experimental results are presented to demonstrate the effectiveness of the proposed approach.

Object-based motion description: from low-level features to semantics

We present a generic model to describe image and video content by a combination of semantic entities and low level features for semantically meaningful and fast retrieval. The proposed model includes semantic entities such as Object, Event and Actors to express relations between the first two. The use of Actors entity increases the efficiency of certain types of search, while the use of semantic and linguistic roles increases the expression capability of the model. The model also contains links to high-level media segments such as actions and interactions, and low level media segments such as elementary motion and reaction units, as well as low-level features such as motion parameters and trajectories. Based on this model, we propose image and video retrieval combining semantic and low-level information. The retrieval performance of our system is tested by using query-by-annotation, query-by-example, query-by-sketch, and a combination of them.

Integrated semantic-syntactic video event modeling for search and retrieval

This paper proposes to integrate text-based database models (commonly used in the database and information retrieval literature) and low-level video features, such as object-based motion features (commonly used in image processing and computer vision), within a single framework to describe video events for fast and effective browsing and retrieval using a mixture of textual and low-level descriptors. The instantiations of the model are expressed as graphs. Users can formulate flexible queries that can be translated into such graphs. Alternatively, users can input query graphs by editing an abstract model (model template) derived either from a model database or from an example scene in the media database. Search and retrieval is accomplished by matching the query graph with those instantiated models in the database. The proposed approach allows for integrated textual and low-level, descriptor-based video search, where the integration is enabled by the model. Examples and experimental results are provided to demonstrate the effectiveness of the proposed integrated modeling and querying framework.

Robust face recognition based on motion pursuit

Robust face recognition is an important research area that has a great impact in the field of surveillance and human-computer interactions in general. This paper addresses how the concept of motion pursuit can bridge the alignment information with imagery similarities to form a joint decision for the purpose of robust face recognition. More explicitly, we perform multi-scale correspondence matching in a wavelet domain and utilize a simple motion measure in conjunction with wavelet response matching to form a discriminative face similarity measure. We demonstrate the efficacy of the proposed approach with a large consumer image database of diverse image capture conditions.

Book Rvw: Multimedia tools and applications. By Borko Furht

This PDF file contains the editorial “Book Rvw: Multimedia tools and applications, by Borko Furht for JEI Vol. 6 Issue 03