Olivier Nocent

Toward an immersion platform for the world wide web using autostereoscopic displays and tracking devices

A few years ago, the introduction of the WebGL API has allowed displaying 3D content in web browsers very efficiently by using the power of 3D accelerators. Nowadays, 3D web applications, from games to medical imaging, tend to compete with their desktop counterparts. Among the main factors that can improve the feeling of presence in terms of impressiveness, immersion and natural interaction play a prominent role in enhancing the quality of the user experience. Both immersion and natural interaction rely on dedicated hardware like 3D displays and tracking devices. Unfortunately, browser makers do not supply JavaScript mechanisms for accessing hardware for security reasons. In this paper, we propose a plugin-free solution using the new features of HTML5 (WebGL and WebSockets) in order to handle autostereoscopic displays and widespread tracking devices like IR depth sensors for providing immersion and natural interaction within the web browser.

3D volume matching for mesh animation of moving actors

4D multiview reconstruction of moving actors has many applications in the entertainment industry and although studios providing such services become more accessible, efforts have to be done in order to improve the underlying technology to produce high-quality 4D contents. In this paper, we enable surface matching for an animated mesh sequence in order to introduce coherence in the data. The context is provided by an indoor multi-camera system which performs synchronized video captures from multiple viewpoints in a chroma key studio. Our input is given by a volumetric silhouette-based reconstruction algorithm that generates a visual hull at each frame of the video sequence. These 3D volumetric models differ from one frame to another, in terms of structure and topology, which makes them very difficult to use in post-production and 3D animation software solutions. Our goal is to transform this input sequence of independent 3D volumes into a single dynamic volumetric structure, directly usable in post-production. These volumes are then transformed into an animated mesh. Our approach is based on a motion estimation procedure. An unsigned distance function on the volumes is used as the main shape descriptor and a 3D surface matching algorithm minimizes the interference between unrelated surface regions. Experimental results, tested on our multiview datasets, show that our method outperforms approaches based on optical flow when considering robustness over several frames.

RECOVER3D: A Hybrid Multi-View System for 4D Reconstruction of Moving Actors

4D multi-view reconstruction of moving actors has many applications in the entertainment industry and although studios providing such services become more accessible, efforts have to be done in order to improve the underlying technology and to produce high-quality 3D contents. The RECOVER3D project aim is to elaborate an integrated virtual video system for the broadcast and motion pictures markets. In particular, we present a hybrid acquisition system coupling mono and multiscopic video cameras where actor’s performance is captured as 4D data set: a sequence of 3D volumes over time. The visual improvement of the software solutions being implemented relies on “silhouette-based” techniques and (multi-)stereovision, following several hybridization scenarios integrating GPU-based processing. Afterwards, we transform this sequence of independent 3D volumes in a unique dynamic mesh. Our approach is based on a motion estimation procedure. An adaptive signed volume distance function is used as the principal shape descriptor and an optical flow algorithm is adapted to the surface setting with a modification that minimizes the interference between unrelated surface regions.

Abdominal morphometric data acquisition using depth sensors

The treatment of eating disorders is now part of the priority actions of the public health policy. For several years, nutritionists have been using tools relying on new digital technologies, able to provide new diagnostic elements. In this paper, we propose a complete methodology to acquire, analyse morphological data and establish typologies with low cost consumer electronics devices. We use a Microsoft® Kinect™ like peripheral to capture abdominal measurements. For each individual, we calculate two profiles in the sagittal and transverse planes respectively. The extracted quantitative information is then analyzed to build typologies of abdominal morphology.

LeBonGeste: basketball training by entertaining

This paper describes LeBonGeste, a device consisting of a basketball basket equipped with sensors, actuators and computers to create a so called augmented basket. It is used to train beginner players to the free throw by helping them to achieve the right stance. Thanks to the Kinect depth-sensor, the system delivers realtime feedback to the player. LeBonGeste has been showcased two times indoor and outdoor using different communication channels for feedback: visual (screen or LED strips) and audio. Developed as an interactive installation for training by entertaining, the system will be tested soon on basketball courts with experienced players.

SPLINE AND IDEAL: FROM REAL TO VIRTUAL SCULPTURES, AND BACK

We present an original approach of virtual sculpting incarnated in a unified space-time interactive modeler named Splimo (for Spline Modeler), which is specifically designed for the creation of tubular spline works. Oriented towards artistic creation, it allows the realization of virtual transcriptions of real art pieces, as well as the creation of animated virtual sculptures, presented in an exhibition mode. Drawing curves in the virtual world is a way to be free of the constraints of the real sculptures, that’s why we call it “ideal”. Splimo may however also be used as a virtual prototyping tool to experiment new shapes to be later designed in the real world.