Alexandre Chapiro

Optimizing stereo-to-multiview conversion for autostereoscopic displays

We present a novel stereo‐to‐multiview video conversion method for glasses‐free multiview displays. Different from previous stereo‐to‐multiview approaches, our mapping algorithm utilizes the limited depth range of autostereoscopic displays optimally and strives to preserve the scenes artistic composition and perceived depth even under strong depth compression. We first present an investigation of how perceived image quality relates to spatial frequency and disparity. The outcome of this study is utilized in a two‐step mapping algorithm, where we (i) compress the scene depth using a non‐linear global function to the depth range of an autostereoscopic display and (ii) enhance the depth gradients of salient objects to restore the perceived depth and salient scene structure. Finally, an adapted image domain warping algorithm is proposed to generate the multiview output, which enables overall disparity range extension.

Towards Mobile HDR Video

We present a novel method for High Dynamic Range video where the critical phases of the pipeline are based on histograms. It is possible to achieve high framerates, since the algorithm generates one HDR frame per captured frame. Also, the method is of low computational cost, making it particularly suited for devices with less powerful processors. An implementation of the capture process for the Nokia N900 smartphone, using the recent FCam API, is detailed.

Filter based deghosting for exposure fusion video

This work deals with a well known problem - the fact that consumer cameras are unable to capture the whole range of color luminance variations the human eye can perceive. Many techniques deal with this, the most widespread probably being High Dynamic Range Imaging. These works focus mainly on still images. Our focus in this work, however, is to improve videos taken from mobile devices by extending one such technique, Exposure Fusion, introduced in [Mertens et al. ].

Stereo from Shading

We present a new method for creating and enhancing the stereoscopic 3D (S3D) sensation without using the parallax disparity between an image pair. S3D relies on a combination of cues to generate a feeling of depth, but only a few of these cues can easily be modified within a rendering pipeline without significantly changing the content. We explore one such cue—shading stereopsis—which to date has not been exploited for 3D rendering. By changing only the shading of objects between the left and right eye renders, we generate a noticeable increase in perceived depth. This effect can be used to create depth when applied to flat images, and to enhance depth when applied to shallow depth S3D images. Our method modifies the shading normals of objects or materials, such that it can be flexibly and selectively applied in complex scenes with arbitrary numbers and types of lights and indirect illumination. Our results show examples of rendered stills and video, as well as live action footage.

The influence of visual salience on video consumption behavior: A survival analysis approach

In an increasingly competitive media environment, producers of online content need analytics that can predict the success of a video. In recent years the field of visual computation has produced a variety of mathematical models that quantify an images salience, that is, its potential to capture attention. To test how a videos content might predict its success, we applied the standard saliency model of Itti, Koch, and Niebur [1] to more than 1000 video clips that were broadcast on a large video streaming website. We also obtained fine-grained data on the viewership of these clips. Based on a survival analysis, we find that people prefer more salient videos. The results were robust towards the inclusion of other predictors such as the genre of the video, but not to video length, which remains correlated with salience even after comparing videos only within show and genre. Our analyses suggest that visual salience provides an objective and easy-to-compute supplement to previously suggested predictors of video consumption behavior.

Video content and structure description based on keyframes, clusters and storyboards

In this paper we present a novel system to extract keyframes, shot clusters and structural storyboards for video content description, which can be used for a variety of summarization, visualization, classification, indexing and retrieval applications. The system automatically selects an appealing set of keyframes and creates meaningful clusters of shots. It further identifies sections that appear recurrently, which are called anchors, and typically divide television shows into different parts. This information about anchors can then be used to browse video content in a new fashion. Finally, our system creates a new type of interactive storyboard suitable to visualize and analyze the structure of the video in a novel way.

Unfolding the 8-bit era

We propose a hardware and software system that transforms 8-bit side-scrolling console video games into immersive multiplayer experiences. We enhance a classic video game console with custom hardware that time-multiplexes eight gamepad inputs to automatically hand off control from one gamepad to the next. Because control transfers quickly, people at a large event can frequently step in and out of a game and naturally call to their peers to join any time a gamepad is vacant. Video from the game console is captured and processed by a vision algorithm that stitches it into a continuous, expanding panoramic texture, which is displayed in real time on a 360 degree projection system at a large event space. With this system, side-scrolling games unfold across the walls of the room to encircle a large party, giving the feeling that the entire party is taking place inside of the games world. When such a display system is not available, we also provide a virtual reality recreation of the experience. We show results of our system for a number of classic console games tested at a large live event. Results indicate that our work provides a successful recipe to create immersive, multiplayer, interactive experiences that leverage the nostalgic appeal of 8-bit games.

Art-directable Continuous Dynamic Range video

We present a novel, end-to-end workflow for content creation and distribution to a multitude of displays that have different dynamic ranges. The emergence of new, consumer level HDR displays with various peak luminances expected in 2015 gives rise to two new research questions: (i) how can the raw source content be graded for a diverse set of displays both efficiently and without restricting artistic freedom, and (ii) how can an arbitrary number of graded video streams be represented and encoded in an efficient way. In this work we propose a new editing paradigm which we call dynamic range mapping to obtain a novel Continuous Dynamic Range (CDR) video representation, where the luminance of the video content, instead of being a scalar value, is defined as a continuous function of the display dynamic range. We present an interactive interface where CDR videos can be efficiently created while providing full artistic control. In addition, we discuss the efficient approximation of CDR video using a polynomial series approximation, and its encoding and distribution to an arbitrary set of target displays. We validate our workflow in a subjective study, which suggests that a visually lossless CDR video representation can be achieved with little bandwidth overhead. Our solution can be implemented easily in the current distribution infrastructure and consists of transmitting two gradings and an additional meta-data stream, which occupies less than 13% current standard video distribution bandwidth. Graphical abstractDisplay Omitted HighlightsA pipeline for creation and distribution of HDR content for a variety of displays.We allow the creation of Continuous Dynamic Range video with full artistic control.An efficient representation of CDR video using a series approximation.We present a demonstration of efficient encoding of Continuous Dynamic Range video.