Simon Goodall

SCULPTEUR: towards a new paradigm for multimedia museum information handling

This paper describes the design and prototype implementation of a novel architecture for integrated concept, metadata and content based browsing and retrieval of museum information. The work is part of a European project involving several major galleries and the aim is to provide more versatile access to digital collections of museum artefacts, including 2-D images, 3-D models and other multimedia representations. An ontology for the museum domain, based on the CIDOC Conceptual Reference Model, is being developed as a semantic layer with references to the digital collection as instance information. A graphical concept browser is an integral component in the user interface, allowing navigation through the semantic layer, display of thumbnails, or full representations of artefacts and textual information in appropriate viewers and the invocation of conventional content based searching or combined querying. Semantic Web technologies are used in system integration to describe how tools for analysis and visualisation can be applied to different data types and sources. This supports flexible and managed formulation, execution and interpretation of the results of distributed multimedia queries. Combined searches using concepts, content and metadata can be initiated from a single user interface.

SCULPTEUR: Multimedia Retrieval for Museums

The paper describes the prototype design and development of a multimedia system for museums and galleries. Key elements in the system are the introduction of 3-D models of museum artefacts together with 3-D as well as 2-D content based retrieval and navigation facilities and the development of a semantic layer, centred on an ontology for museums, which aims to expose the richness of knowledge associated with the museum collections and facilitate concept based retrieval and navigation integrated with that based on content and metadata. Interoperability protocols are designed to allow external applications to access the collection and an example is given of an e-Learning facility which uses models extracted to a virtual museum.

Integrated image content and metadata search and retrieval across multiple databases

This paper presents an updated technical overview of an integrated content and metadata-based image retrieval system used by several major art galleries in Europe including the Louvre in Paris, the Victoria and Albert Museum in London, the Uffizi Gallery in Florence and the National Gallery in London. In our approach, the subjects of a query (e.g. images, textual metadata attributes), the operators used in a query (e.g. SimilarTo, Contains, Equals) and the rules that constrain the query (e.g. SimilarTo can only be applied to Images) are all explicitly defined and published for each gallery collection. In this way, cross-collection queries are dynamically constructed and executed in a way that is automatically constrained to the capabilities of the particular image collections being searched. The application of existing, standards based, technology to integrate metadata and content based queries underpins an open standards approach to extending interoperability across multiple image databases.

3-D Shape Descriptors and Distance Metrics for Content-Based Artefact Retrieval

The growing number of large multimedia collections has led to an increased interest in content-based retrieval research. Applications of content-based techniques to image retrieval is an active research area but much less work has been reported on content-based retrieval of 3-D objects in a multimedia database context. Increasingly such objects are being captured and added to multimedia collections and the European project, SCULPTEUR, is developing a museum information system which includes the introduction of facilities for content-based retrieval of the 3-D representations. This paper provides a comparison and evaluation of a range of 3-D shape descriptors and distance metrics which have been introduced into the SCULPTEUR project to demonstrate their use for content-based retrieval applications. Results show that while particular descriptors and distance metrics provide good overall performance, it can be more appropriate to choose different descriptors for different search tasks.

Question Answering Towards Automatic Augmentations of Ontology Instances

Ontology instances are typically stored as triples which associate two named entities with a pre-defined relational description. Sometimes such triples can be incomplete in that one entity is known but the other entity is missing. The automatic discovery of the missing values is closely related to relation extraction systems that extract binary relations between two identified entities. Relation extraction systems rely on the availability of accurately named entities in that mislabelled entities can decrease the number of relations correctly identified. Although recent results demonstrate over 80% accuracy for recognising named entities, when input texts have less consistent patterns, the performance decreases rapidly. This paper presents OntotripleQA which is the application of question-answering techniques to relation extraction in order to reduce the reliance on the named entities and take into account other assessments when evaluating potential relations. Not only does this increase the number of relations extracted, but it also improves the accuracy of extracting relations by considering features which are not extractable with only comparisons of the named entities. A small dataset was collected to test the proposed approach and the experiment demonstrates that it is effective on sentences from Web documents with an accuracy of 68% on average.

Towards automatic classification of 3-d museum artifacts using ontological concepts

The development and use of content-based retrieval techniques for 3-D models is a relatively new departure in multimedia retrieval. We have extended our existing multimedia museum information system to support content-, metadata- and concept-based retrieval of 3-D models of museum artifacts and in this paper we describe a “classifier agent” to automatically assign associations between 3-D artifacts and concepts and metadata stored in a domain ontology. The context of the classifier agent is described, together with an overview of its architecture. Selecting appropriate parameters for the agent is an important activity and a comparison is made between manually selected parameters and the results of an automatic technique to determine “optimal” settings.

Colour cluster analysis for pigment identification

This paper presents image processing algorithms designed to analyse the colour CIE Lab histogram of high resolution images of paintings. Three algorithms are illustrated which attempt to identify colour clusters, cluster shapes due to shading and finally to identify pigments. Using the image collection and pigment list of the National Gallery London large numbers of images within a restricted period have been classified with a variety of algorithms. The image descriptors produced were also used with suitable comparison metrics to obtain content-based retrieval of the images.