Olivier Steiger

Tracking video objects in cluttered background

We present an algorithm for tracking video objects which is based on a hybrid strategy. This strategy uses both object and region information to solve the correspondence problem. Low-level descriptors are exploited to track objects regions and to cope with track management issues. Appearance and disappearance of objects, splitting and partial occlusions are resolved through interactions between regions and objects. Experimental results demonstrate that this approach has the ability to deal with multiple deformable objects, whose shape varies over time. Furthermore, it is very simple, because the tracking is based on the descriptors, which represent a very compact piece of information about regions, and they are easy to define and track automatically. Finally, this procedure implicitly provides one with a description of the objects and their track, thus enabling indexing and manipulation of the video content.

Semantic video analysis for adaptive content delivery and automatic description

We present an encoding framework which exploits semantics for video content delivery. The video content is organized based on the idea of main content message. In the work reported in this paper, the main content message is extracted from the video data through semantic video analysis, an application-dependent process that separates relevant information from non relevant information. We use here semantic analysis and the corresponding content annotation under a new perspective: the results of the analysis are exploited for object-based encoders, such as MPEG-4, as well as for frame-based encoders, such as MPEG-1. Moreover, the use of MPEG-7 content descriptors in conjunction with the video is used for improving content visualization for narrow channels and devices with limited capabilities. Finally, we analyze and evaluate the impact of semantic video analysis in video encoding and show that the use of semantic video analysis prior to encoding sensibly reduces the bandwidth requirements compared to traditional encoders not only for an object-based encoder but also for a frame-based encoder.

MPEG-based personalized content delivery

In this paper, we present a personalized multimedia content delivery system dealing with both user preferences and terminal/network capabilities. In order to ease interoperability with third-party applications, content annotation, user preferences handling and terminal/network capabilities description are managed by MPEG-7 and MPEG-21 standards. Offline content adaptation and annotation tools are proposed for content preparation. Content delivery is handled by client-server architecture. This architecture is built around custom personalization and adaptation tools, and a commercial content streamer. The proposed system has been used to provide universal multimedia access within the EC-funded R&D project PERSEO.

Multiple video object tracking in complex scenes

We present an automatic video object tracking algorithm capable of dealing with multiple simultaneous objects. The tracking is based on interactions between high-level and low-level image analysis results. The high-level result is a partition defining video objects, and the low-level result is a partition formed by homogeneous regions. For each region, a set of characteristic descriptors is produced. These region descriptors, and not regions themselves, are used to track the regions (and thus the objects) along time. Track management issues such as appearance and disappearance of objects, splitting and partial occlusions are resolved through interactions between regions and objects. Defining the tracking based on the parts of objects, identified by region segmentation, has led to a flexible technique that exploits the nature of the video object tracking problem. Experimental results show that the proposed method is able to track multiple rigid and deformable objects in indoor and outdoor scenes.

Semantic segmentation and description for video transcoding

We present an automatic content-based video transcoding algorithm, which is based on how humans perceive visual information. The transcoder support multiple video objects and their description. First the video is decomposed into meaningful objects through semantic segmentation. Then the transcoder adapts its behavior to code relevant (foreground) and non-relevant objects differently. Both objects-based and frame-based encoders are combined with semantic segmentation. Experimental results show that the use of semantics and description prior to transcoding reduces the bandwidth requirements and makes it possible to adapt the video representation to limited network and terminal device capabilities still retaining the essential information.

Calibration of time-of-flight range imaging cameras

In this paper, a procedure for the systematic, generic and reproducible calibration of 3-D time-of-flight range imaging cameras is proposed. First, the effect of a number of internal and environmental influence factors is studied. Internal factors include the illumination radiation pattern, inner temperature and pixel integration period. Environmental factors are the ambient temperature, light, and target properties such as position, reflectance and color. Then, the influence of these factors is compensated. The procedure is illustrated with the SR-3000 camera, and validation results show that systematic improvements of the measurement accuracy have been obtained.

Perceptual prefiltering for video coding

Semantic segmentation is generally associated with second generation video coders, or object-based coders. Object-based coders encode different video objects separately in order to achieve lower bitrates and to enable object-based functionalities. In this paper, we present an encoding framework that uses semantic segmentation to improve the performance of first generation video coders, or frame-based coders. Semantic segmentation is exploited in a prefiltering step prior to encoding. This prefiltering step mimics the way humans treat visual information by separating relevant information from contextual information. Contextual information is then simplified, thus reducing the information to be coded. Experimental results on indoor as well as on outdoor test sequences, when the semantics is defined by motion, show that the proposed prefiltering improves the perceived quality with respect to traditional video coders.

Surveillance video for mobile devices

In this paper, we present a video encoding scheme that uses object-based adaptation to deliver surveillance video to mobile devices. The method relies on a set of complementary video adaptation strategies and generates content that matches various appliance and network resources. Prior to encoding, some of the adaptation strategies exploit video object segmentation and selective filtering in order to improve the perceived quality. Moreover, object segmentation enables the generation of automatic summaries and of simplified versions of the monitored scene. The performance of individual adaptation strategies is assessed using an objective video quality metric, which is also used to select the strategy that provides maximum value for the user under a given set of constraints. We demonstrate the effectiveness of the scheme on standard surveillance test sequences and realistic mobile client resource profiles.

MPEG-7 Description of Generic Video Objects for Scene Reconstruction

We present an MPEG--7 compliant description of generic video sequences aiming at their scalable transmission and reconstruction. The proposed method allows efficient and flexible video coding while keeping the advantages of textual descriptions in database applications. Visual objects are described in terms of their shape, color, texture and motion; these features can be extracted automatically and are sufficient in a wide range of applications. To permit partial sequence reconstruction, at least one simple qualitative as well as a quantitative descriptor is provided for each feature. In addition, we propose a structure for the organization of the descriptors into objects and scenes and some possible applications for our method. Experimental results obtained with news and video surveillance sequences validate our method and highlight its main features.

Evaluating Perceptually Prefiltered Video

Perceptual prefiltering is the process of enhancing relevant portions of an image or of a video, and of simplifying contextual information in order to improve the perceived quality or the compression ratio. In this paper, we discuss the results of subjective quality evaluation experiments performed to assess the impact of perceptual prefiltering on video coding and we propose an objective quality metric that mimics the behavior of human observers. The predicted performance of the proposed metric is consistent with the subjective evaluation scores. Experimental results demonstrate that perceptual prefiltering leads to quality improvements by up to 10% at low bit rates