Xavier Marichal

Blur determination in the compressed domain using DCT information

The paper presents a simple yet robust measure of image quality in terms of global (camera) blur. It is based on histogram computation of non-zero DCT coefficients. The technique is directly applicable to images and video frames in compressed (MPEG or JPEG) domain and to all types of MPEG frames (I-, P- or B-frames). The resulting quality measure is proved to be in concordance with subjective testing and is therefore suitable for quick qualitative characterization of images and video frames.

Perceptually-weighted evaluation criteria for segmentation masks in video sequences

In order to complement subjective evaluation of the quality of segmentation masks, this paper introduces a procedure for automatically assessing this quality. Algorithmically computed figures of merit are proposed. Assuming the existence of a perfect reference mask (ground truth), generated manually or with a reliable procedure over a test set, these figures of merit take into account visually desirable properties of a segmentation mask in order to provide the user with metrics that best quantify the spatial and temporal accuracy of the segmentation masks. For the sake of easy interpretation, results are presented on a peaked signal-to-noise ratio-like logarithmic scale.

Automatic detection of interest areas of an image or of a sequence of images

The tool introduced in this paper allows to automatically decide in an image or in a video sequence which regions are important and which ones are not. For this purpose, fuzzy logic has been used to modelize human subjective knowledge about the way to allocate priorities to regions. The resulting classification can be used in a wide range of applications going from image coding to image understanding.

Real-time segmentation of video objects for mixed-reality interactive applications

The present paper introduces a very specific and pragmatic approach to segmentation. It is driven by a particular application context: in the framework of mixed-reality, Tranfiction (“transportation into fictional spaces”) is designed to mix synthetic and natural images in real time while allowing users to interact in these input/output screens. Segmentation is therefore used to provide both the immersion and interaction capabilities. The former aspect is achieved by composing the image of the user within the projected virtual scenes, while the later is achieved thanks to basic body/gesture analysis on the segmented silhouettes. According to indoor or outdoor usages, two real-time techniques are developed. Results are analyzed with respect to the overall application, not only in terms of absolute quality but also in terms of perception by the users.

Interacting with Virtual Agents in Mixed Reality Interactive Storytelling

User interaction with virtual agents generally takes place in virtual environments in which there is clear separation between the virtual actors and the user, due to the fact that in most cases, the user is in some way external to the virtual world. In Mixed-Reality Interactive Storytelling, the user’s video image is captured in real time and inserted into a virtual world populated by autonomous synthetic actors with which the user interacts. The user in turn watches the composite world projected on a large screen, following a ”magic mirror” metaphor. This context leads to re-investigating the techniques by which the user interacts with virtual actors, as well as exploring specific research problems. In this paper, we discuss some specificities of user interaction with virtual actors in Mixed Reality Interactive Storytelling. After a brief introduction to our system’s architecture and the example scenario supporting our experiments, we describe various techniques supporting multi-modal interaction with virtual actors.

Users acting in mixed reality interactive storytelling

Entertainment systems promise to be a significant application for Mixed Reality. Recently, a growing number of Mixed Reality applications have included interaction with synthetic characters and storytelling. However, AI-based Interactive Storytelling techniques have not yet been explored in the context of Mixed Reality. In this paper, we describe a first experiment in the adaptation of an Interactive Storytelling technique to a Mixed Reality system. After a description of the real time image processing techniques that support the creation of a hybrid environment, we introduce the storytelling technique and the specificities of user interaction in the Mixed Reality context. We illustrate these experiments by discussing examples obtained from the system.

3D posture estimation using geodesic distance maps

This paper presents a novel technique for three-dimensional (3D) human motion capture using a set of two non-calibrated cameras. The user’s five extremities (head, hands and feet) are extracted, labeled and tracked after silhouette segmentation. As they are the minimal number of points that can be used in order to enable whole body gestural interaction, we will henceforth refer to these features as crucial points. Features are subsequently labelled using 3D triangulation and inter-image tracking. The crucial point candidates are defined as the local maxima of the geodesic distance with respect to the center of gravity of the actor region that lie on the silhouette boundary. Due to its low computational complexity, the system can run at real-time paces on standard personal computers, with an average error rate range between 4% and 9% in realistic situations, depending on the context and segmentation quality.

Silhouette-based probabilistic 2D human motion estimation for real-time applications

This paper presents a novel technique for 2D human motion estimation using a single non calibrated camera. The users five crucial human features (head, hands and feet) are extracted, labeled and tracked, after silhouette segmentation. The crucial points candidates are defined as the local maxima of the geodesic distance with respect to the center of gravity of the actor region (silhouette) following the silhouette boundary. Selected crucial points are then classified as head, hands or feet using a probabilistic approach weighted by a prior human model. The system can run at 50 Hz paces on standard personal computers.

Senses of spaces through transfiction

Transfiction is a system allowing the exploration and the realisation of concepts involving multiple places, both physical and virtual, and multiple media channels (visual, audiochrw...) for enhancing the human-to-human communication. The human variable is back in the loop of complex Mixed Reality systems design and is positioned at the core of the narrative through the participation of users as “interactors”.

A fuzzy logic system for content-based bit-rate allocation

When video coders communicate at very-low bit-rate, it is often difficult for them to preserve an acceptable global quality of the images. A selection of the key regions according to the semantic relevance may therefore be useful for improving the quality, as it would permit an adaptive bit allocation, which is important for good subjective quality at (very-)low bit-rate: the essential features could be extracted and coded with a good quality, while the remaining portions of the image would be coarsely transmitted. This paper describes tools to perform classification of regions according to the subjective priority, and a generic algorithm to optimally share out the bit-rate. Among others, an important contribution of this paper is the use of Fuzzy Logic in order to model human subjective knowledge about the attribution of priorities to picture regions. An original bit-rate allocation method is also proposed as a solution to the optimization problem posed by the rate-distortion theory. The paper concludes presenting some preliminary results