Vassilis Tzouvaras

Semantic annotation of images and videos for multimedia analysis

Annotations of multimedia documents typically have been pursued in two different directions. Either previous approaches have focused on low level descriptors, such as dominant color, or they have focused on the content dimension and corresponding annotations, such as person or vehicle. In this paper, we present a software environment to bridge between the two directions. M-OntoMat-Annotizer allows for linking low level MPEG-7 visual descriptions to conventional Semantic Web ontologies and annotations. We use M-OntoMat-Annotizer in order to construct ontologies that include prototypical instances of high-level domain concepts together with a formal specification of corresponding visual descriptors. Thus, we formalize the interrelationship of high- and low-level multimedia concept descriptions allowing for new kinds of multimedia content analysis and reasoning.

Reasoning with very expressive fuzzy description logics

It is widely recognized today that the management of imprecision and vagueness will yield more intelligent and realistic knowledge-based applications. Description Logics (DLs) are a family of knowledge representation languages that have gained considerable attention the last decade, mainly due to their decidability and the existence of empirically high performance of reasoning algorithms. In this paper, we extend the well known fuzzy ALC DL to the fuzzy SHIN DL, which extends the fuzzy ALC DL with transitive role axioms (S), inverse roles (I), role hierarchies (H) and number restrictions (N). We illustrate why transitive role axioms are difficult to handle in the presence of fuzzy interpretations and how to handle them properly. Then we extend these results by adding role hierarchies and finally number restrictions. The main contributions of the paper are the decidability proof of the fuzzy DL languages fuzzy-SI and fuzzy-SHIN, as well as decision procedures for the knowledge base satisfiability problem of the fuzzy-SI and fuzzy-SHIN.

Enquiring MPEG-7 based multimedia ontologies

Machine understandable metadata forms the main prerequisite for the intelligent services envisaged in a Web, which going beyond mere data exchange and provides for effective content access, sharing and reuse. MPEG-7, despite providing a comprehensive set of tools for the standardised description of audiovisual content, is largely compromised by the use of XML that leaves the largest part of the intended semantics implicit. Aspiring to formalise MPEG-7 descriptions and enhance multimedia metadata interoperability, a number of multimedia ontologies have been proposed. Though sharing a common vision, the developed ontologies are characterised by substantial conceptual differences, reflected both in the modelling of MPEG-7 description tools as well as in the linking with domain ontologies. Delving into the principles underlying their engineering, we present a systematic survey of the state of the art MPEG-7 based multimedia ontologies, and highlight issues that hinder interoperability as well as possible directions towards their harmonisation.

An ontology infrastructure for multimedia reasoning

In this paper, an ontology infrastucture for multimedia reasoning is presented, making it possible to combine low-level visual descriptors with domain specific knowledge and subsequently analyze multimedia content with a generic algorithm that makes use of this knowledge. More specifically, the ontology infrastructure consists of a domain-specific ontology, a visual descriptor ontology (VDO) and an upper ontology. In order to interpret a scene, a set of atom regions is generated by an initial segmentation and their descriptors are extracted. Considering all descriptors in association with the related prototype instances and relations, a genetic algorithm labels the atom regions. Finally, a constraint reasoning engine enables the final region merging and labelling into meaningful objects.

Capturing MPEG-7 Semantics

The ambiguities due to the purely syntactical nature of MPEG7 have hindered its widespread application as they lead to serious interoperability issues in sharing and managing multimedia metadata. Acknowledging these limitations, a number of initiatives have been reported towards attaching formal semantics to the MPEG-7 specifications. In this paper we examine the rationale on which the relevant approaches build, and building on the experiences gained we present the approach followed in the BOEMIE project.

Multimedia Reasoning with f-SHIN

Effective management and exploitation of multimedia documents requires extraction of the underlying semantics. Multimedia analysis algorithms can produce fairly rich but imprecise information about a multimedia document. In this paper, a multimedia reasoning architecture is presented using the fuzzy extension of expressive SHIN, f-SHIN. First a segmentation algorithm generates a set of over-segmented regions and a classification process is employed to assign those regions with semantic labels. A semantic-based refinement of the segmentation is follows and this information initializes the ABox of a fuzzy-knowledge that is used for multimedia reasoning. The proposed approach was tested on outdoor domain and shows promising results.

Image analysis of soil micromorphology: feature extraction, segmentation, and quality inference

We present an automated system that we have developed for estimation of the bioecological quality of soils using various image analysis methodologies. Its goal is to analyze soilsection images, extract features related to their micromorphology, and relate the visual features to various degrees of soil fertility inferred from biochemical characteristics of the soil. The image methodologies used range from low-level image processing tasks, such as nonlinear enhancement, multiscale analysis, geometric feature detection, and size distributions, to object-oriented analysis, such as segmentation, region texture, and shape analysis.

Integrating Image Segmentation and Classification for Fuzzy Knowledge-Based Multimedia Indexing

In this paper we propose a methodology for semantic indexing of images, based on techniques of image segmentation, classification and fuzzy reasoning. The proposed knowledge-assisted analysis architecture integrates algorithms applied on three overlapping levels of semantic information: i) no semantics, i.e. segmentation based on low-level features such as color and shape, ii) mid-level semantics, such as concurrent image segmentation and object detection, region-based classification and, iii) rich semantics, i.e. fuzzy reasoning for extraction of implicit knowledge. In that way, we extract semantic description of raw multimedia content and use it for indexing and retrieval purposes, backed up by a fuzzy knowledge repository. We conducted several experiments to evaluate each technique, as well as the whole methodology in overall and, results show the potential of our approach.

A systemic approach for effective semantic access to cultural content

A large on-going activity for digitization, dissemination and preservation of cultural heritage is taking place in Europe and the United States, which involves all types of cultural institutions, i.e., galleries, libraries, museums, archives and all types of cultural content. The development of Europeana, as a single point of access to European Cultural Heritage, has probably been the most important result of the activities in the field till now. Semantic interoperability is a key issue in these developments. This paper presents a system that provides content providers and users with the ability to map, in an effective way, their own metadata schemas to common domain standards and the Europeana (ESE, EDM) data models. Based on these mappings, semantic enrichment and query answering techniques are proposed as a means tbr providing effective access of users to digital cultural heritage. An experimental study is presented involving content from national and thematic content aggregators in Europeana, which illustrates the proposed system capabilities.

Synthesis and applications of lattice image operators based on fuzzy norms

We use concepts from the lattice-based theory of morphological operators and fuzzy sets to develop generalized lattice image operators that can be expressed as nonlinear convolutions that are suprema or infima of fuzzy intersection or union norms. Our emphasis (different from previous works) is the construction of pairs of fuzzy dilation and erosion operators that form lattice adjunctions. This guarantees that their composition will be a valid algebraic opening or closing. The power, but also the difficulty, in applying these fuzzy operators to image analysis is the large variety of fuzzy norms and the absence of systematic ways in selecting them. Towards this goal, we have performed extensive experiments in applying these fuzzy operators to various nonlinear filtering and image analysis tasks, attempting first to understand the effect that the type of fuzzy norm and the shape/size of the structuring function has on the resulting new image operators. Further, we have developed some new fuzzy edge gradients and optimized their usage for edge detection on test problems via a parametric fuzzy norm.