Lynda Hardman

/facet: a browser for heterogeneous semantic web repositories

Facet browsing has become popular as a user friendly interface to data repositories. The Semantic Web raises new challenges due to the heterogeneous character of the data. First, users should be able to select and navigate through facets of resources of any type and to make selections based on properties of other, semantically related, types. Second, where traditional facet browsers require manual configuration of the software, a semantic web browser should be able to handle any RDFS dataset without any additional configuration. Third, hierarchical data on the semantic web is not designed for browsing: complementary techniques, such as search, should be available to overcome this problem. We address these requirements in our browser, /facet. Additionally, the interface allows the inclusion of facet-specific display options that go beyond the hierarchical navigation that characterizes current facet browsing. /facet is a tool for Semantic Web developers as an instant interface to their complete dataset. The automatic facet configuration generated by the system can then be further refined to configure it as a tool for end users. The implementation is based on current Web standards and open source software. The new functionality is motivated using a scenario from the cultural heritage domain.

Structured multimedia authoring

Authoring context sensitive, interactive multimedia presentations is much more complex than authoring either purely audiovisual applications or text. Interactions among media objects need to be described as a set of spatio-temporal relationships that account for synchronous and asynchronous interactions, as well as on-demand linking behavior. This article considers the issues that need to be addressed by an authoring environment. We begin with a partitioning of concerns based on seven classes of authoring problems. We then describe a selection of multimedia authoring environments within four different authoring paradigms: structured, timeline, graph and scripting. We next provide observations and insights into the authoring process and argue that the structured paradigm provides the most useful framework for presentation authoring. We close with an example application of the structured multimedia authoring paradigm in the context of our own structure-based system GRiNS.

COMM: designing a well-founded multimedia ontology for the web

Semantic descriptions of non-textual media available on the web can be used to facilitate retrieval and presentation of media assets and documents containing them. While technologies for multimedia semantic descriptions already exist, there is as yet no formal description of a high quality multimedia ontology that is compatible with existing (semantic) web technologies. We explain the complexity of the problem using an annotation scenario. We then derive a number of requirements for specifying a formal multimedia ontology before we present the developed ontology, COMM, and evaluate it with respect to our requirements. We provide an API for generating multimedia annotations that conform to COMM.

Towards second and third generation web-based multimedia

First generation Web-content encodes information in handwritten (HTML) Web pages. Second generation Web content generates HTML pages on demand, e.g. by filling in templates with content retrieved dynamically from a database or transformation of structured documents using style sheets (e.g. XSLT). Third generation Web pages will make use of rich markup (e.g. XML) along with metadata (e.g. RDF) schemes to make the content not only machine readable but also machine processable - a necessary pre-requisite to the emphSemantic Web. While text-based content on the Web is already rapidly approaching the third generation, multimedia content is still trying to catch up with second generation techniques. Multimedia document processing has a number of fundamentally different requirements from text which make it more difficult to incorporate within the document processing chain. In particular, multimedia transformation uses different document and presentation abstractions, its formatting rules cannot be based on text-flow, it requires feedback from the formatting back-end and is hard to describe in the functional style of current style languages. We state the requirements for second generation processing of multimedia and describe how these have been incorporated in our prototype multimedia document transformation environment, emphCuypers. The system overcomes a number of the restrictions of the text-flow based tool sets by integrating a number of conceptually distinct processing steps in a single runtime execution environment. We describe the need for these different processing steps and describe them in turn (semantic structure, communicative device, qualitative constraints, quantitative constraints, final form presentation), and illustrate our approach by means of an example. We conclude by discussing the models and techniques required for the creation of third generation multimedia content.

Semantic annotation and search of cultural-heritage collections: The MultimediaN E-Culture demonstrator

In this article we describe a Semantic Web application for semantic annotation and search in large virtual collections of cultural-heritage objects, indexed with multiple vocabularies. During the annotation phase we harvest, enrich and align collection metadata and vocabularies. The semantic-search facilities support keyword-based queries of the graph (currently 20M triples), resulting in semantically grouped result clusters, all representing potential semantic matches of the original query. We show two sample search scenarios. The annotation and search software is open source and is already being used by third parties. All software is based on established Web standards, in particular HTML/XML, CSS, RDF/OWL, SPARQL and JavaScript.

Semantic Multimedia

Multimedia constitutes an interesting field of application for Semantic Web and Semantic Web reasoning, as the access and management of multimedia content and context depends strongly on the semantic descriptions of both. At the same time, multimedia resources constitute complex objects, the descriptions of which are involved and require the foundation on sound modeling practice in order to represent findings of low- and high level multimedia analysis and to make them accessible via Semantic Web querying of resources. This tutorial aims to provide a red thread through these different issues and to give an outline of where Semantic Web modeling and reasoning needs to further contribute to the area of semantic multimedia for the fruitful interaction between these two fields of computer science.

MultimediaN e-culture demonstrator

The main objective of the MultimediaN E-Culture project is to demonstrate how novel semantic-web and presentation technologies can be deployed to provide better indexing and search support within large virtual collections of cultural-heritage resources. The architecture is fully based on open web standards, in particular XML, SVG, RDF/OWL and SPARQL. One basic hypothesis underlying this work is that the use of explicit background knowledge in the form of ontologies/vocabularies/thesauri is in particular useful in information retrieval in knowledge-rich domains.

That obscure object of desire: multimedia metadata on the Web, Part-1

In this article we discuss the advances in, and remaining problems of, making use of audio-visual media in a semantic-based environment, such as the Semantic Web, facilitated through media-aware and ontologybased metadata. Our discussion is predominantly motivated by the two most widely known approaches towards machine-processable and semantic-based content description, namely the Semantic Web activity of the W3C [2, 4] and ISO’s efforts in the direction of complex media content modeling, in particular the the Multimedia Content Description Interface (MPEG-7) [30, 31, 32, 33, 34]. We chose these two approaches as they provide the potential techniques to establish a media-aware Semantic Web, even though at the time of writing the approaches seem to be diverging rather than converging. The Semantic Web should bring machine-processable content to Web pages, thus being an extension of the current Web. The aim is to add ontology-based metadata to Web resources to improve Internet search and provide means for machine-based reasoning about the content. A major drawback of the current semantic Web developments, however, is its media-agnostic view on Web resources. The specific needs of dynamic audio-visual media with its variety of data representations is not recognized. That is, however, precisely what the intention of MPEG is, partially addressed in MPEG-4 [27] and MPEG-21 [35] and fully in MPEG-7. In this paper, we explain that the conceptual ideas and technologies discussed in both approaches are essential for the next step in Web-based multimedia development. Unfortunately, there are still many practical obstacles that block their widespread use for providing multimedia metadata on the Web. We show that a media-ware Semantic Web will blur the boundaries between traditional categories like preproduction, production, and postproduction, with far-reaching effects on concepts such as data, metadata, consumer and producer. The paper is structured as follows. We first provide a scenario to explain our vision of a media-aware Semantic Web and derive from it a number of problems regarding the semantic content description of media units. We then discuss the multimedia production chain, in particular emphasizing the role of progressive metadata production. As a result we distill a set of media-based metadata production requirements and show how current media production environments fail to address these. We then introduce those parts of the W3C and ISO standardization works that are relevant to our discussion. We analyze their abilities to define structures for describing media semantics, discuss syntactic and semantic problems, ontological problems for media semantics, and the problems of applying the theoretical concepts to real world problems.

Towards ontology-driven discourse: from semantic graphs to multimedia presentations

Traditionally, research in applying Semantic Web technology to multimedia information systems has focused on using annotations and ontologies to improve the retrieval process. This paper concentrates on improving the presentation of the retrieval results. First, our approach uses ontological domain knowledge to select and organize the content relevant to the topic the user is interested in. Domain ontologies are valuable in the presentation generation process, because effective presentations are those that succeed in conveying the relevant domain semantics to the user. Explicit discourse and narrative knowledge allows selection of appropriate presentation genres and creation of narrative structures, which are used for conveying these domain relations. In addition, knowledge of graphic design and media characteristics is essential to transform abstract presentation structures in real multimedia presentations. Design knowledge determines how the semantics and presentation structure are expressed in the multimedia presentation. In traditional Web environments, this type of design knowledge remains implicit, hidden in style sheets and other document transformation code. Our second use of Semantic Web technology is to model design knowledge explicitly, and to let it drive the transformations needed to turn annotated media items into structured presentations.

Links in hypermedia: the requirement for context

Taking the concept of a link from hypertext and adding to it the rich collection of information formats found in multimedia systems provides an extension to hypertext that is often called hypermedia. Unfortunately, the implicit assumptions under which hypertext links work do not extend well to time-based presentations that consist of a number of simultaneously active media items. It is not obvious where links should lead and there are no standard rules that indicate what should happen to other parts of the presentation that are active. This paper addresses the problems associated with links in hypermedia. In order to provide a solution, we introduce the notion of context for the source and the destination of a link. A context makes explicit which part of a presentation is affected when a link is followed from an anchor in the presentation. Given explicit source and destination contexts for a link, an author is able to state the desired presentation characteristics for following a link, including whether the presentation currently playing should continue playing or be replaced. We first give an intuitive description of contexts for links, then present a structure-based approach. We go on to describe the implementation of contexts in our hypermedia authoring system CMIFed.