Chrisa Tsinaraki

Integration of OWL Ontologies in MPEG-7 and TV-Anytime Compliant Semantic Indexing

We describe a systematic methodology for extending the audiovisual content description standards (MPEG-7 and TV-Anytime) with domain-specific knowledge descriptions expressed in OWL. The domain-specific descriptions of the audiovisual content metadata are completely transparent to applications and tools that use MPEG-7 and TV-Anytime, allowing them to use the domain- specific ontologies without any software changes. We also present an interoperability mechanism between OWL and the audiovisual content description standards, which allows MPEG-7 and TV-Anytime descriptions and their domain-specific extensions to be described in OWL and vice versa. Thus, the methodology and the mechanisms presented here open up opportunities for reusing ontology tools and ontologies across a large number of applications (with or without audiovisual content) and across different professional and user communities. We present the details of the methodology and the implementation of the tools supporting it as well as its integration in a large framework for domain-specific indexing and retrieval of audiovisual content.

Interoperability Support for Ontology-Based Video Retrieval Applications

Domain ontologies are very useful for indexing, query specification, retrieval and filtering, user interfaces, even information extraction from audiovisual material. The dominant emerging language standard for the description of domain ontologies is OWL. We describe here a methodology and software that we have developed for the interoperability of OWL with the complete MPEG-7 MDS so that domain ontologies described in OWL can be transparently integrated with the MPEG-7 MDS metadata. This allows applications that recognize and use the MPEG-7 MDS constructs to make use of domain ontologies for applications like indexing, retrieval, filtering etc. resulting in more effective user retrieval and interaction with audiovisual material.

Ontology-Based Semantic Indexing for MPEG-7 and TV-Anytime Audiovisual Content

In this paper, we describe a framework that we have developed for the support of ontology-based semantic indexing and retrieval of audiovisual content following the MPEG-7 and TV-Anytime standard specifications for metadata descriptions. Our work aims to provide a methodology to enhance the retrieval effectiveness of audiovisual content, while maintaining compatibility with the international multimedia standards.In our framework, domain-specific ontologies guide the definition of both the application-specific metadata and the instance-description metadata that describe the contents of audiovisual programs and/or their segments. The ontologies are also used in order to provide compatible descriptions in both audiovisual content standards (MPEG-7 and TV-Anytime) for the same content. This approach allows indexing compatibility and interoperability of TV-Anytime and digital library applications.We describe the design and implementation of a system supporting this framework. The components of the system include a segmentation tool for segmenting audiovisual information, which also provides ontology-based semantic indexing capabilities, and an appropriate API for semantic query support. An application testbed for the domain of soccer games has been developed on top of this system. An ontology for soccer games has been defined and used for indexing and retrieval of soccer games that have been stored in the system database.The methodology we developed opens up a wide opportunity for the creation of MPEG-7 and TV-Anytime services offering structured domain-specific ontologies that can be integrated to these standards for enhancing audiovisual content retrieval performance.

Interoperability Support between MPEG-7/21 and OWL in DS-MIRF

In this paper, we focus on interoperable semantic multimedia services that are offered in open environments such as the Internet. The use of well-accepted standards is of paramount importance for interoperability support in open environments. In addition, the semantic description of multimedia content utilizing domain ontologies is very useful for indexing, query specification, retrieval, filtering, user Interfaces, and knowledge extraction from audiovisual material. With the MPEG-7 and MPEG-21 standards dominating the multimedia content and service description domain and OWL dominating the ontology description languages, it is important to establish a framework that allows these standards to interoperate. We describe here the DS-MIRF framework, a software engineering framework that facilitates the development of knowledge-based multimedia applications such as multimedia information retrieval, filtering, browsing, interaction, knowledge extraction, segmentation, and content description. DS-MIRF supports interoperability of OWL with the MPEG-7/21 so that domain and application ontologies expressed in OWL can be transparently integrated with MPEG-7/21 metadata. This allows applications that recognize and use the constructs provided by MPEG-7/21 to make use of domain and application ontologies, resulting in more effective retrieval and user interaction with the audiovisual material. We also present a retrieval evaluation methodology and comparative retrieval results

An ontology-driven framework for the management of semantic metadata describing audiovisual information

In this paper, we describe a framework for the management of semantic metadata that describe audiovisual content. The main objectives of this work are to provide a framework allowing enhanced retrieval performance and better user interfaces for audiovisual data, based on extensions of the well accepted international standards for the digital media, digital broadcasting and TV-Anytime domains like TV-Anytime and MPEG-7, as well as to provide an interoperability framework between TV-Anytime and MPEG-7 for the rapidly expanding set of interoperating digital library and interactive TV applications. The framework was developed in the context of the UP-TV project, which investigates TV-Anytime architectures and services. Our approach is ontology-driven, in the sense that we provide ontology based domain-specific extensions of the standards for describing multimedia content. Several aspects of the framework are based on the existence of ontologies. Our framework allows the creation of more efficient user interfaces for accessing content, better retrieval performance and better support in the indexing phase. It also provides compatibility in indexing between the two standards and interoperability for TV-Anytime and digital library applications supported by TV-Anytime servers. We have developed a domain-specific ontology (football games) to test our framework ideas and implementation. The developed ontology uses as primitive modelling elements those provided by the MPEG-7 metadata model. The mapping of the ontology to TV-Anytime metadata primitives is straightforward. We show in detail the concrete implementation of the framework on top of relational databases and its interfaces with the other system components.

A multimedia user preference model that supports semantics and its application to MPEG 7/21

Semantic interoperability is usually provided in open environments through standards and domain ontology. The dominant standards for multimedia content and service descriptions are MPEG-7 and MPEG-21. The MPEG-7 semantic DS has powerful semantic description capabilities and supports using semantic entities specified in domain ontology in multimedia content descriptions. However, the MPEG-7/21 usage environment allows neither the specification of semantic user preferences nor the exploitation of domain knowledge and MPEG-7 semantic metadata descriptions. In addition, the users cannot explicitly specify, in the hierarchical MPEG-7/21 filtering and search preferences, the Boolean operators that should be used during content filtering to combine the hierarchy components. We think these as serious limitations and we propose a hierarchical semantic user preference model that allows for the explicit specification of Boolean operators. Then, we present the application of the model in MPEG-7/21 and the model implementation within the DS-MIRF framework

Interoperability of XML schema applications with OWL domain knowledge and semantic web tools

Several standards are expressed using XML Schema syntax, since the XML is the default standard for data exchange in the Internet. However, several applications need semantic support offered by domain ontologies and semantic Web tools like logic-based reasoners. Thus, there is a strong need for interoperability between XML Schema and OWL. This can be achieved if the XML schema constructs are expressed in OWL, where the enrichment with OWL domain ontologies and further semantic processing are possible. After semantic processing, the derived OWL constructs should be converted back to instances of the original schema. We present in this paper XS2OWL, a model and a system that allow the transformation of XML Schemas to OWL-DL constructs. These constructs can be used to drive the automatic creation of OWL domain ontologies and individuals. The XS2OWL transformation model allows the correct conversion of the derived knowledge from OWL-DL back to XML constructs valid according to the original XML Schemas, in order to be used transparently by the applications that follow XML Schema syntax of the standards.

The MPEG-7 query format: a new standard in progress for multimedia query by content

In recent years, the amount of Internet accessible digital audiovisual media files has vastly increased. Therefore the need to describe the media (by way of metadata) has also increased significantly. MPEG-7 (finalized in 2001) provides a comprehensive and rich metadata standard for the description of multimedia content. Unfortunately, a standardized query format does not exist for MPEG-7, or other multimedia metadata. Such a standard would provide for communications between querying clients and databases, supporting cross-modal and cross-media retrieval. The lSO/lEC SC29WG11 committee decided therefore to contribute to this application space by adding such functionality as a new part of the MPEG-7 series of standards. In response to a Call for Proposals, six proposals were submitted. This paper describes the strengths of each proposal as well as the resulting draft standard for the MPEG-7 query format.

An MPEG-7 query language and a user preference model that allow semantic retrieval and filtering of multimedia content

We present in this paper the MPEG-7 Query Language (MP7QL), a powerful query language that we have developed for querying MPEG-7 descriptions, as well as its compatible Filtering and Search Preferences (FASP) model. The MP7QL has the MPEG-7 as data model and allows for querying every aspect of an MPEG-7 multimedia content description. It allows the users to express the conditions that should hold for the multimedia content returned to them regarding semantics, low-level visual and audio features and media-related aspects. The MP7QL queries may utilize the users’ FASP and Usage History as context, thus allowing for personalized multimedia content retrieval. The FASP model supported is compatible with the MP7QL and has the model of the standard MPEG-7 FASPs as a special case. The proposed FASPs essentially are MP7QL queries. Both the MP7QL and its compatible FASP model allow for the exploitation of domain knowledge encoded using pure MPEG-7 constructs. In addition, they allow the explicit specification of boolean operators and/or preference values in order to allow both the combination of the query conditions according to the user intentions and the expression of the importance of the individual conditions for the users. The MP7QL query results are represented as MPEG-7 documents, guaranteeing the closure of the results within the MPEG-7 space. The MP7QL and the FASP model have been expressed using both XML Schema and OWL syntax. An implementation of the MP7QL, on top of an XML Native Database is currently in progress. A real world-world evaluation study on the expressive power of the MP7QL shows that it covers both general purpose and domain specific requirements in multimedia content retrieval.

A User Preference Model and a Query Language that allow Semantic Retrieval and Filtering of Multimedia Content

We present in this paper MP7QL, a powerful query language that we are developing for querying MPEG-7 descriptions, and a user preference model that allows for expressing preferences about every aspect of an MPEG-7 multimedia content description. The MP7QL queries may utilize the user preferences as context, thus allowing for personalized multimedia content retrieval. The user preference model supports multimedia Filtering and Search Preferences (FASP) that allow for expressing semantic preferences, preferences regarding the low-level visual features and preferences regarding media-related aspects (creation, classification, source etc). The proposed FASPs essentially are MP7QL queries, allow for the systematic integration of domain knowledge in the semantic preferences of the users and have the standard MPEG-7 FASPs as a special case. The MP7QL and the FASP model have been expressed using both XML Schema and OWL syntax. An implementation on top of an XML Repository is currently under way.