Thomas Riegel

AIR: Architecture for interoperable retrieval on distributed and heterogeneous multimedia repositories

Nowadays multimedia data is produced and consumed at an ever increasing rate. Similarly to this trend, diverse storage approaches for multimedia data have been introduced. These observations lead to the fact that distributed and heterogeneous multimedia repositories exist whereas an unified and easy access to the stored multimedia data is not given. This paper presents an architecture, named AIR, that offers the aformentioned retrieval possibilites. To ensure interoperability, AIR makes use of recently issued standards, namely the MPEG Query Format (MPQF) (multimedia query language) and the JPSearch transformation rules (metadata interoperability).

Core Technologies for the Internet of Services

Information and knowledge are growing permanently and represent valuable resources for many enterprises. The efficient access to knowledge of an enterprise like expertise, contact persons, project and milestone plans, etc. may simplify business processes and lead to time and cost savings. Semantic technologies offer numerous possibilities to enrich data with background information about their meaning. Such semantic relations do not only lead to more efficient search in larger information repositories but they also assist the user in diverse processes like editing, annotation and processing of information. In addition they offer new means of access and transfer of knowledge. Each information unit is linked to other units in the same domain, which allows faster search and offers a way of information access that is close to the habits of humans, i.e. the creation of new knowledge and its association to already existing knowledge. This approach of the whole THESEUS research program was also the basis of the Core Technology Cluster of THESEUS.

Role-based identity recognition for telecasts

Semantic queries involving image understanding aspects require the exploitation of multiple clues, namely the (inter-)relations between objects and events across multiple images, the situational context, and the application context. A prominent example for such queries is the identification of individuals in video sequences. Straightforward face recognition approaches require a model of the persons in question and tend to fail in ill conditioned environments. Therefore, an alternative approach is to involve contextual conditions of observations in order to determine the role a person plays in the current context. Due to the strong relation between roles, persons and their identities, knowing either often allows inferring about the other. This paper presents a system that implements this approach: First, robust face detection localizes the actors in the video. By clustering similar face instances the relative frequency of their appearance within a sequence is determined. In combination with a coarse textual annotation manually created by the broadcast stations archivist the roles and consequently the identities can be assigned and labeled in the video. Starting with unambiguous assignments and cascading appropriately most of the persons can be identified and labeled successfully. The feasibility and performance of the role-based person identification is demonstrated on basis of several programs of a popular German TV show, which consists of various elements like interview scenes, games and musical show acts.

Role-based identity recognition for TV broadcasts

Semantic queries involving image understanding aspects require the exploitation of multiple clues, namely the (inter-) relations between objects and events across multiple images, the situational context, and the application context. A prominent example for such queries is the identification of individuals in video sequences. Straightforward face recognition approaches require a model of the persons in question and tend to fail in ill-conditioned environments. Therefore, an alternative approach is to involve contextual conditions of observations in order to determine the role a person plays in the current context. Due to the strong relation between roles, persons and their identities, knowing either often allows inferring about the other. This paper presents a system that implements this approach: First, robust face detection localizes the actors in the video. By clustering similar face instances the relative frequency of their appearance within a sequence is determined. In combination with a coarse textual annotation manually created by the broadcast station’s archivist the roles and consequently the identities can be assigned and labeled in the video. Starting with unambiguous assignments and cascading, most of the persons can be identified and labeled successfully. The feasibility and performance of the role-based person identification is demonstrated on the basis of several programs of a popular German TV show, which consists of various elements like interview scenes, games and musical show acts.

Moving object contour detection based on S-T characteristics in surveillance

We present a method for moving object contours detection based on spatial-temporal characteristics. Using S-T features, the contour of moving object can be well distinguished from background; therefore the moving objects are detected without the need of establishing and updating background models. The detection method can handle situations where the background of the scene suffers from the noises due to the various facts, including the weather condition such as snow or fog and flicker of leafs on trees, and bushes. The algorithm estimates the probability of observing pixel as a contour pixel based on a sample of intensity values for each pixel during a period of time and its local gradient in current frame. The experiments show that this method is sensitive to changes caused by moving objects and is able to avoid the affection of complex background. The paper also shows how to separate multi-person based on the contour detection results using template matching. The approach runs in realtime and achieves sensitive detection.

Basistechnologien für das Internet der Dienste

Wissen und Informationen wachsen nicht nur stetig in ihrer Menge, sie stellen heute vielmehr eine bedeutende Ressource vieler Unternehmen dar. Der effiziente Zugriff auf Unternehmenswissen, wie etwa Expertisen, Ansprechpartner, Projekt- und Meilensteinplane etc. kann Unternehmensprozesse vereinfachen und somit zur Zeit- und Kostenreduktion beitragen.

3D shape initialization of objects in multiview image sequences

The ultimate goal for future telecommunication is highly effective inter-personal information exchange. The effectiveness of telecommunication is greatly enhanced by 3-D telepresence. This requires that visual information is presented in such a way that the viewer is under the impression of actually being physically close to the party with whom the communication takes place. One way to achieve a natural 3-D impression is to encode image sequences using 3-D model objects and animate them again by computer graphic means regarding the observers eye positions. This concept will use a parametric 3-D scene description in order to model a scene. The parameters of the model objects will be estimated from trinocular input image sequences by means of image analysis. This paper starts with an overview on the European ACTS project PANORAMA, in which the above mentioned concept will be realized and evaluated. In the main part the shape initialization of physical objects from a multiview image sequence will be discussed. For this the range information given by three disparity maps from different stereo views is backprojected into 3-D space. The resulting cloud of 3-D points is then approximated by a flexible triangular net by using a technique named discrete smooth interpolation. The discrete smooth interpolation is a particular surface interpolation technique, which is solved by an iterative approach. It allows to generate a surface, defined as a wireframe mesh, that fits (or interpolates) a given set of 3D points by observing, at the same time, some given constraints about the surface characteristics, like roughness, behavior at the boundaries, etc. The finally presented results show the capabilities of this approach in video communication.

Automatic Face Modeling and Synthesis Based on Image Pairs

Unlike traditional 3D model based or image based animation methods, in this paper a novel approach is presented to generate both facial actions and head rotations for photo-realistic facial animation based on one frontal and one half-profile facial image taken with an uncalibrated camera. We represent faces with 2D wire-frame models and use MPEG4 FAPs to encode basic facial actions. Hierarchical Direct Appearance Model is employed for facial feature localization. 3D deformable model is applied for pose estimation. By affine projection 3D deformable model and facial actions are mapped to 2D facial models and actions at various head poses. Coarse 2D models are refined with extracted facial features by RBF interpolation. Pose-variable facial animation is generated by synthesizing facial actions on 2D models and morphing facial textures between frontal and halfprofile views. Experimental results demonstrate the effectiveness of our approach.

Coding of partially computer-rendered image sequences

A new coding scheme for partially computer-rendered image sequences will be presented. It is specifically suited for heterogeneous data sets containing symbolic and pixel-based image descriptions, which are used by an electronic set system at the receiver site for the synthesis and mixture of transmitted image sequences. The different types of data sets and their particular properties regarding data compression are explained. Finally, results are given comparing the new coding scheme with traditional MPEG2 coding based on typical test sequences.

The Usage of Turntable Sequences for Disparity/Depth Estimation

For the modelling of natural objects various approaches are known. Buschmann/Niem [1] developed an algorithm for the automatic modelling of isolated objects with convex shapes using silhouettes from multiple views of the object. One major drawback of this approach is its limitation on modelling objects with concavities. One way to overcome this restriction is to integrate depth information into the convex volume model. This additional information can be obtained by disparity/depth estimation using the same images. The aim of this contribution is to describe the individual steps which are necessary to calculate a depth value for each pixel of the object in each perspective view.