Rainer Lienhart

On the optimal placement of multiple visual sensors

Many novel multimedia systems and applications use visual sensor arrays. An important issue in designing sensor arrays is the appropriate placement of the visual sensors such that they achieve a predefined goal. In this paper we focus on the placement with respect to maximizing coverage or achieving coverage at a certain resolution. We identify and consider four different problems: maximizing coverage subject to a given number of cameras (a) or a maximum total price of the sensor array (b), optimizing camera poses given fixed locations (c), and minimizing the cost of a sensor array given a minimally required percentage of coverage (d). To solve these problems, we propose different algorithms. Our approaches can be subdivided into algorithms which give a global optimum solution and heuristics which solve the problem within reasonable time and memory consumption at the cost of not necessarily determining the global optimum. We also present a user-interface to enter and edit the spaces under analysis, the optimization problems as well as the other setup parameters. The different algorithms are experimentally evaluated and results are presented. The results show that the algorithms work well and are suited for different practical applications. For the final paper it is planned to have the user interface running as a web service.

Scalable logo recognition in real-world images

In this paper we propose a highly effective and scalable framework for recognizing logos in images. At the core of our approach lays a method for encoding and indexing the relative spatial layout of local features detected in the logo images. Based on the analysis of the local features and the composition of basic spatial structures, such as edges and triangles, we can derive a quantized representation of the regions in the logos and minimize the false positive detections. Furthermore, we propose a cascaded index for scalable multi-class recognition of logos.n For the evaluation of our system, we have constructed and released a logo recognition benchmark which consists of manually labeled logo images, complemented with non-logo images, all posted on Flickr. The dataset consists of a training, validation, and test set with 32 logo-classes. We thoroughly evaluate our system with this benchmark and show that our approach effectively recognizes different logo classes with high precision.

Image retrieval on large-scale image databases

Online image repositories such as Flickr contain hundreds of millions of images and are growing quickly. Along with that the needs for supporting indexing, searching and browsing is becoming more and more pressing. In this work we will employ the image content as a source of information to retrieve images. We study the representation of images by Latent Dirichlet Allocation (LDA) models for content-based image retrieval. Image representations are learned in an unsupervised fashion, and each image is modeled as the mixture of topics/object parts depicted in the image. This allows us to put images into subspaces for higher-level reasoning which in turn can be used to find similar images. Different similarity measures based on the described image representation are studied. The presented approach is evaluated on a real world image database consisting of more than 246,000 images and compared to image models based on probabilistic Latent Semantic Analysis (pLSA). Results show the suitability of the approach for large-scale databases. Finally we incorporate active learning with user relevance feedback in our framework, which further boosts the retrieval performance.

Multilayer pLSA for multimodal image retrieval

It is current state of knowledge that our neocortex consists of six layers [10]. We take this knowledge from neuroscience as an inspiration to extend the standard single-layer probabilistic Latent Semantic Analysis (pLSA) [13] to multiple layers. As multiple layers should naturally handle multiple modalities and a hierarchy of abstractions, we denote this new approach multilayer multimodal probabilistic Latent Semantic Analysis (mm-pLSA). We derive the training and inference rules for the smallest possible non-degenerated mm-pLSA model: a model with two leaf-pLSAs (here from two different data modalities: image tags and visual image features) and a single top-level pLSA node merging the two leaf-pLSAs. From this derivation it is obvious how to extend the learning and inference rules to more modalities and more layers. We also propose a fast and strictly stepwise forward procedure to initialize bottom-up the mm-pLSA model, which in turn can then be post-optimized by the general mm-pLSA learning algorithm. We evaluate the proposed approach experimentally in a query-by-example retrieval task using 50-dimensional topic vectors as image models. We compare various variants of our mm-pLSA system to systems relying solely on visual features or tag features and analyze possible pitfalls of the mm-pLSA training. It is shown that the best variant of the the proposed mm-pLSA system outperforms the unimodal systems by approximately 19% in our query-by-example task.

PLSA on Large Scale Image Databases

The Web and image repositories such as Fickrtrade are the largest image databases in the world. There are billions of images on the web, and hundreds of million high-quality images in image repositories. Currently, these images are indexed based on manually-entered tags and individual and group usage patterns. In this work we a exploring a third information dimension: image features. We are exploring probabilistic latent semantic analysis in order to infer which visual patterns describe each object. We wish to build models that connect words and image features, and use content features and tags to better find similar images.

The Holy Grail of Multimedia Information Retrieval: So Close or Yet So Far Away?

The papers in this issue cover the main aspects of multimedia information retrieval research, assess the applicability of the obtained results in real-life scenarios, and address the many future challenges in this field.

Approximating Optimal Visual Sensor Placement

Many novel multimedia applications use visual sensor arrays. In this paper we address the problem of optimally placing multiple visual sensors in a given space. Our linear programming approach determines the minimum number of cameras needed to cover the space completely at a given sampling frequency. Simultaneously it determines the optimal positions and poses of the visual sensors. We also show how to account for visual sensors with different properties and costs if more than one kind is available, and report performance results.

Bundle min-hashing for logo recognition

We present a scalable logo recognition technique based on feature bundling. Individual local features are aggregated with features from their spatial neighborhood into bundles. These bundles carry more information about the image content than single visual words. The recognition of logos in novel images is then performed by querying a database of reference images.n We further propose a novel WGC-constrained RANSAC and a technique that boosts recall for object retrieval by synthesizing images from original query or reference images. We demonstrate the benefits of these techniques for both small object retrieval and logo recognition. Our logo recognition system clearly outperforms the current state-of-the-art with a recall of 83% at a precision of 99%.

Deep networks for image retrieval on large-scale databases

Currently there are hundreds of millions (high-quality) images in online image repositories such as Flickr. This makes is necessary to develop new algorithms that allow for searching and browsing in those large-scale databases. In this work we explore deep networks for deriving a low-dimensional image representation appropriate for image retrieval. A deep network consisting of multiple layers of features aims to capture higher order correlations between basic image features. We will evaluate our approach on a real world large-scale image database and compare it to image representations based on topic models. Our results show the suitability of the approach for very large databases.

A survey on visual adult image recognition

We provide an overview of state-of-the-art approaches to visual adult image recognition which is a special case of one-class image classification. We present a representative selection of methods which we coarsely divide into three main groups. First we discuss color-based approaches which rely on the intuitive assumption that adult images usually feature skin-colored regions. Different ways of defining skin colors are described and example classification frameworks built on skin color models are presented. Another main group of approaches to adult image recognition is based on shape information which usually also exploit color information to find skin-colored regions of interest. Color and texture features are often used to augment such shape features. Finally we introduce approaches based on local feature descriptors.