Thomas Käster

Combining speech and haptics for intuitive and efficient navigation through image databases

Given the size of todays professional image databases, the stan-dard approach to object- or theme-related image retrieval is to in-teractively navigate through the content. But as most users of such databases are designers or artists who do not have a technical back-ground, navigation interfaces must be intuitive to use and easy to learn. This paper reports on efforts towards this goal. We present a system for intuitive image retrieval that features different moda-lities for interaction. Apart from conventional input devices like mouse or keyboard it is also possible to use speech or haptic gesture to indicate what kind of images one is looking for. Seeing a selection of images on the screen, the user provides relevance feedback to narrow the choice of motifs presented next. This is done either by scoring whole images or by choosing cer-tain image regions. In order to derive consistent reactions from multimodal user input, asynchronous integration of modalities and probabilistic reasoning based on Bayesian networks are applied. After addressing technical details, we will discuss a series of usability experiments, which we conducted to examine the impact of multimodal input facilities on interactive image retrieval. The results indicate that users appreciate multimodality. While we ob-served little decrease in task performance, measures of contentment exceeded those for conventional input devices.

Comparing Clustering Methods for Database Categorization in Image Retrieval

Applying image retrieval techniques to large image databases requires the restriction of search space to provide adequate response time. This restriction can be done by means of clustering techniques to partition the image data set into subspaces of similar elements. In this article several clustering methods and validity indices are examined with regard to image categorization. A subset of the COIL-100 image collection is clustered by different agglomerative hierarchical methods as well as the k-Means, PAM and CLARA clustering algorithms. The validity of the resulting clusters is determined by computing the Davies-Bouldin-Index and Calinski-Harabasz-Index. To evaluate the performance of the different combinations of clustering methods and validity indices with regard to semantically meaningful clusters, the results are compared with a given reference grouping by measuring the Rand-Index.

INDI - intelligent database navigation by interactive and intuitive content-based image retrieval

We present a content-based image retrieval system, INDI (techniques for Intelligent Navigation in Digital Image databases), that combines the use of low-level pattern recognition techniques, machine learning and an intuitive human-computer interface in order to support intelligent and user-friendly semantic navigation in large image databases. To keep independence from specific image domains and to encompass different search tasks, the system is highly modular and contains a hierarchical mechanism for the adaptive reweighting of similarity measures implemented by dynamically reloadable modules at different semantic levels.

Vision systems with the human in the loop

The emerging cognitive vision paradigm deals with vision systems that apply machine learning and automatic reasoning in order to learn from what they perceive. Cognitive vision systems can rate the relevance and consistency of newly acquired knowledge, they can adapt to their environment and thus will exhibit high robustness. This contribution presents vision systems that aim at flexibility and robustness. One is tailored for content-based image retrieval, the others are cognitive vision systems that constitute prototypes of visual active memories which evaluate, gather, and integrate contextual knowledge for visual analysis. All three systems are designed to interact with human users. After we will have discussed adaptive content-based image retrieval and object and action recognition in an office environment, the issue of assessing cognitive systems will be raised. Experiences from psychologically evaluated human-machine interactions will be reported and the promising potential of psychologically-based usability experiments will be stressed.

Content-based image retrieval by multimodal interaction

Due to the size of todays professional image databases, the standard approach to content-based image retrieval is to interactively navigate through the content. However, most people whose job necessitates working with such databases do not have a technical background. Commercial practice thus requires efficient retrieval techniques as well as navigation interfaces that are intuitive to use and easy to learn. In this paper we introduce a system for interactive image retrieval that combines different approaches to feature based queries. Furthermore, it allows multimodal interaction because apart from conventional input devices like mouse and keyboard, it is possible to operate the system using a touch screen or even natural language. Besides technical details and results on retrieval accuracy, we also present results of usability experiments which underline that users well appreciate multimodal interfaces for image retrieval.

Usability Evaluation for Image Retrieval Beyond Desktop Applications

Interactivity is a key concept in modern content-based retrieval. Therefore, in addition to the ability to learn from user generated data, easy and intuitive to use interfaces are an important area of research in (multi)media retrieval. In this contribution, we focus on the latter aspect and present how different modalities like speech and gestures on super sized touch screen facilities may be integrated to accomplish the goal of intuitive interaction. In order to evaluate our approach, we conducted a series of usability experiments. Their results demonstrate that our multimodal user interface allows for both, comfortable and successful interactive image retrieval

Fast, Illumination Insensitive Face Detection Based on Multilinear Techniques and Curvature Features

This paper brings together two recent developments in image analysis. We consider a new mathematical framework that provides illumination invariant descriptors for face detection. Towards fast learning and processing, we understand images and the corresponding feature maps as multilinear entities and apply higher order classifiers for image analysis and object detection. Experimental results underline that this approach indeed provides quick training, fast runtime and robust performance across a variety of illumination conditions