Péter Tamás Kovács

Holovizio 3D Display System

We present the HoloVizio system design and give an overview of Holografikas approach to the 3D displaying. The patented HoloVizio technology uses a specially arranged array of optical modules and a holographic screen. Each point of the holographic screen emits light beams of different color and intensity to various directions. With proper software control, light beams leaving the pixels propagate in multiple directions, as if they were emitted from the points of 3D objects at fixed spatial locations. We show that the direction selective light emission is a general requirement for every 3D systems and the advantages of light field reconstruction over the multiview approach. We describe the 10 Mpixel desktop display and the 50 Mpixel large-scale system. We cover the real-time control issues at high pixel-count systems with the HoloVizio software environment and describe concrete 3D applications developed in the frame of European projects.

Subjective evaluation of Super Multi-View compressed contents on high-end light-field 3D displays

Super Multi-View (SMV) video content is composed of tens or hundreds of views that provide a light-field representation of a scene. This representation allows a glass-free visualization and eliminates many causes of discomfort existing in current available 3D video technologies. Efficient video compression of SMV content is a key factor for enabling future 3D video services. This paper first compares several coding configurations for SMV content and several inter-view prediction structures are also tested and compared. The experiments mainly suggest that large differences in coding efficiency can be observed from one configuration to another. Several ratios for the number of coded and synthesized views are compared, both objectively and subjectively. It is reported that view synthesis significantly affects the coding scheme. The amount of views to skip highly depends on the sequence and on the quality of the associated depth maps. Reported ranges of bitrates required to obtain a good quality for the tested SMV content are realistic and coherent with future 4K/8K needs. The reliability of the PSNR metric for SMV content is also studied. Objective and subjective results show that PSNR is able to reflect increase or decrease in subjective quality even in the presence of synthesized views. However, depending on the ratio of coded and synthesized views, the order of magnitude of the effective quality variation is biased by PSNR. Results indicate that PSNR is less tolerant to view synthesis artifacts than human viewers. Finally, preliminary observations are initiated. First, the light-field conversion step does not seem to alter the objective results for compression. Secondly, the motion parallax does not seem to be impacted by specific compression artifacts. The perception of the motion parallax is only altered by variations of the typical compression artifacts along the viewing angle, in cases where the subjective image quality is already low. To the best of our knowledge, this paper is the first to carry out subjective experiments and to report results of SMV compression for light-field 3D displays. It provides first results showing that improvement of compression efficiency is required, as well as depth estimation and view synthesis algorithms improvement, but that the use of SMV appears realistic according to next generation compression technology requirements. HighlightsStudy of the impact of compression on subjective quality for lightfield SMV content.To the best of our knowledge, this paper is the first to report results of this kind.Several SMV coding configurations are compared both objectively and subjectively.Compression efficiency, depth estimation and view synthesis require improvements.SMV appears realistic according to next generation compression technology requirements.

Freehand interaction with large-scale 3D map data

In this paper, we present our method and apparatus to visualize and interact with large-scale 3D map data on a 3D light-field display in real time. 3D map data are streamed over Internet to the display in real-time based on request sent by the application. On the user side, data is processed and visualized on a large-scale 3D light field display. We present a method to efficiently interact with the visualized 3D map using freehand gestures. We use a Leap Motion sensor that supports sufficient refresh rate for our current application and is able to accurately track and acquire information on the user hand position. This information is further processed to support natural and fast interaction. We have also developed a method to automatically adjust the maps plane to the screen plane of the display. The visualization and interaction method is scalable and allows the exploration of large-scale 3D maps down to the street level.

Measurement of perceived spatial resolution in 3D light-field displays

Effective spatial resolution of projection-based 3D light-field (LF) displays is an important quantity, which is informative about the capabilities of the display to recreate views in space and is important for content creation. We propose a subjective experiment to measure the spatial resolution of LF displays and compare it to our objective measurement technique. The subjective experiment determines the limit of visibility on the screen as perceived by viewers. The test involves subjects determining the direction of patterns that resemble tumbling E eye test charts. These results are checked against the LF display resolution determined by objective means. The objective measurement models the display as a signal-processing channel. It characterizes the display throughput in terms of passband, quantified by spatial resolution measurements in multiple directions. We also explore the effect of viewing angle and motion parallax on the spatial resolution.

Overview of the applicability of H.264/MVC for real-time light-field applications

Several methods for compressing light-fields (LF) and multiview 3D video content have been proposed in the literature. The most widely accepted and standardized method is the Multi View Coding (MVC) extension of H.264, which is considered appropriate for use with stereoscopic and multiview 3D displays. In this paper we will focus on light-field 3D displays, outline typical use cases for such displays, analyze processing requirements for display-specific and display-independent light-fields, and see how these map to MVC as the underlying 3D video compression method. We also provide an overview of available MVC implementations, and the support these provide for multiview 3D video. Directions for future research and additional features supporting LF video compression are presented.

New visual coding exploration in MPEG: Super-MultiView and Free Navigation in Free viewpoint TV

ISO/IEC MPEG and ITU-T VCEG have recently jointly issued a new multiview video compression standard, called 3D-HEVC, which reaches unpreceded compression performances for linear,dense camera arrangements. In view of supporting future highquality,auto-stereoscopic 3D displays and Free Navigation virtual/augmented reality applications with sparse, arbitrarily arranged camera setups, innovative depth estimation and virtual view synthesis techniques with global optimizations over all camera views should be developed. Preliminary studies in response to the MPEG-FTV (Free viewpoint TV) Call for Evidence suggest these targets are within reach, with at least 6% bitrate gains over 3DHEVC technology.

Light-field video coding using geometry-based disparity compensation

One of the main challenges in 3D light-field imaging approaches lies in the massive amount of visual information involved in providing 3D content with sufficient resolution. Consequently, adequate coding tools are essential for efficient transmission and storage of this type of content. In this context, this paper presents and evaluates two coding solutions based on the High Efficiency Video Coding (HEVC) scheme and for efficient compression of the 3D light-field content. These two coding schemes aim to exploit the 3D geometry-based disparity information in the 3D light field content and replace the block-based disparity estimation. In the first scheme, the disparity map of each view is used to directly derive the vectors for compensation, and in the second scheme these disparity vectors (for all views) are calculated (for non-occluded areas) from the disparity map of the base view. A comparative study of these proposed coding schemes is performed and future research directions are also discussed.

Optimization of light field display-camera configuration based on display properties in spectral domain.

The visualization capability of a light field display is uniquely determined by its angular and spatial resolution referred to as display passband. In this paper we use a multidimensional sampling model for describing the display-camera channel. Based on the model, for a given display passband, we propose a methodology for determining the optimal distribution of ray generators in a projection-based light field display. We also discuss the required camera setup that can provide data with the necessary amount of details for such display that maximizes the visual quality and minimizes the amount of data.

Viva la resolution: the perceivable differences between image resolutions for light field displays

As 3D display technologies are becoming more and more common in commercial, everyday usage, a special type of 3D known as projection-based light field displays is emerging as well. While holding many key characteristics such as field of view or angular resolution, traditional image resolution also plays a major role in the overall determination of user experience, similarly to 2D displays. The paper investigates the perceivable differences between display resolutions, and presents the acceptability of resolution degradation should it be visible. A total of 20 test participants provided subjective assessment in a series of pair comparisons between 5 resolutions for various stimuli. Beyond mean scores, the results are presented in terms of score distribution in this analysis, separately for each and every stimulus.

Natural 3D content on glasses-free light-field 3D cinema

This paper presents a complete framework for capturing, processing and displaying the free viewpoint video on a large scale immersive light-field display. We present a combined hardware-software solution to visualize free viewpoint 3D video on a cinema-sized screen. The new glasses-free 3D projection technology can support larger audience than the existing autostereoscopic displays. We introduce and describe our new display system including optical and mechanical design considerations, the capturing system and render cluster for producing the 3D content, and the various software modules driving the system. The indigenous display is first of its kind, equipped with front-projection light-field HoloVizio technology, controlling up to 63 MP. It has all the advantages of previous light-field displays and in addition, allows a more flexible arrangement with a larger screen size, matching cinema or meeting room geometries, yet simpler to set-up. The software system makes it possible to show 3D applications in real-time, besides the natural content captured from dense camera arrangements as well as from sparse cameras covering a wider baseline. Our software system on the GPU accelerated render cluster, can also visualize pre-recorded Multi-view Video plus Depth (MVD4) videos on this light-field glasses-free cinema system, interpolating and extrapolating missing views.