Ronan Boitard

Temporal coherency for video tone mapping

Tone Mapping Operators (TMOs) aim at converting real world high dynamic range (HDR) images captured with HDR cameras, into low dynamic range (LDR) images that can be displayed on LDR displays. Several TMOs have been proposed over the last decade, from the simple global mapping to the more complex one simulating the human vision system. While these solutions work generally well for still pictures, they are usually less e_cient for video sequences as they are source of visual artifacts. Only few of them can be adapted to cope with a sequence of images. In this paper we present a major problem that a static TMO usually encounters while dealing with video sequences, namely the temporal coherency. Indeed, as each tone mapper deals with each frame separately, no temporal coherency is taken into account and hence the results can be quite disturbing for high varying dynamics in a video. We propose a temporal coherency algorithm that is designed to analyze a video as a whole, and from its characteristics adapts each tone mapped frame of a sequence in order to preserve the temporal coherency. This temporal coherency algorithm has been tested on a set of real as well as Computer Graphics Image (CGI) content and put in competition with several algorithms that are designed to be time-dependent. Results show that temporal coherency preserves the overall contrast in a sequence of images. Furthermore, this technique is applicable to any TMO as it is a post-processing that only depends on the used TMO.

Evaluation of color encodings for high dynamic range pixels

Traditional Low Dynamic Range (LDR) color spaces encode a small fraction of the visible color gamut, which does not encompass the range of colors produced on upcoming High Dynamic Range (HDR) displays. Future imaging systems will require encoding much wider color gamut and luminance range. Such wide color gamut can be represented using floating point HDR pixel values but those are inefficient to encode. They also lack perceptual uniformity of the luminance and color distribution, which is provided (in approximation) by most LDR color spaces. Therefore, there is a need to devise an efficient, perceptually uniform and integer valued representation for high dynamic range pixel values. In this paper we evaluate several methods for encoding colour HDR pixel values, in particular for use in image and video compression. Unlike other studies we test both luminance and color difference encoding in a rigorous 4AFC threshold experiments to determine the minimum bit-depth required. Results show that the Perceptual Quantizer (PQ) encoding provides the best perceptual uniformity in the considered luminance range, however the gain in bit-depth is rather modest. More significant difference can be observed between color difference encoding schemes, from which YDuDv encoding seems to be the most efficient.

Demystifying High-Dynamic-Range Technology: A new evolution in digital media.

High-dynamic-range (HDR) technology has attracted a lot of attention recently, especially in commercial trade shows such as the Consumer Electronics Show, the National Association of Broadcasters Show, the International Broadcasting Convention, and Internationale Funkausstellung Berlin. However, a great deal of mystery still surrounds this new evolution in digital media. In a nutshell, HDR technology aims at capturing, distributing, and displaying a range of luminance and color values that better correspond to what the human eye can perceive. Here, the term luminance stands for the photometric quantity of light arriving at the human eye measured in candela per square meter or nits. The color refers to all the weighted combinations of spectral wavelengths, expressed in nanometers (nm), emitted by the sun that are visible by the human eye (see Figure 1). The human eye can perceive a dynamic range of over 14 orders of magnitude (i.e., the difference in powers of ten between highest and lowest luminance value) in the real world through adaptation. However, at a single adaptation time, the human eye can only resolve up to five orders of magnitude, as illustrated in Figure 2. Dynamic range denotes the ratio between the highest and lowest luminance value. As reported in Table 1, there are different interpretations for dynamic range, depending on the application. For instance, in photography, dynamic range is measured in terms of f-stops, which correspond to the number of times that the light intensity can be doubled.

Zonal brightness coherency for video tone mapping

Tone Mapping Operators (TMOs) compress High Dynamic Range (HDR) contents to address Low Dynamic Range (LDR) displays. While many solutions have been designed over the last decade, only few of them can cope with video sequences. Indeed, these TMOs tone map each frame of a video sequence separately, which results in temporal incoherency. Two main types of temporal incoherency are usually considered: flickering artifacts and temporal brightness incoherency. While the reduction of flickering artifacts has been well studied, less work has been performed on brightness incoherency. In this paper, we propose a method that aims at preserving spatio-temporal brightness coherency when tone mapping video sequences. Our technique computes HDR video zones which are constant throughout a sequence, based on the luminance of each pixel. Our method aims at preserving the brightness coherency between the brightest zone of the video and each other zone. This technique adapts to any TMO and results show that it preserves well spatio-temporal brightness coherency. We validate our method using a subjective evaluation. In addition, unlike local TMOs, our method, when applied to still images, is capable of ensuring spatial brightness coherency. Finally, it also preserves video fade effects commonly used in post-production.

Compression efficiency of HDR/LDR content

High Dynamic Range (HDR) imaging is capable of delivering a wider range of luminance and color gamut compared to Standard Dynamic Range (SDR), offering to viewers a visual quality of experience close to that of real-life. In this study, we evaluate the quality of coded original HDR streams and HDR streams reconstructed from SDR videos and metadata, both compressed by the HEVC standard. Our evaluations have shown that the single HDR approach is largely preferred over the SDR counterpart.

Motion-guided quantization for video tone mapping

Tone Mapping Operators (TMOs) transform High Dynamic Range (HDR) contents to address Low Dynamic Range (LDR) displays. However, before reaching the end-user, these contents are usually compressed using a codec (coder-decoder) for broadcasting or storage purposes. Achieving the best trade-off between rendering and compression efficiency is of prime importance. Any TMO includes a rounding quantization to convert floating point values to integer ones. In this work, we propose to modify this quantization to increase the compression efficiency of the tone mapped content. By using a motion compensation, our technique preserves the rendering intent of the TMO while maximizing the correlations between successive frames. Experimental results show that we can save up to 12% of the total bit-rate as well as an average bit-rate reduction of 8.5% for all the test sequences. We show that our technique can be applied to other applications such as denoising.

prediction-guided quantization for video tone mapping

Tone Mapping Operators (TMOs) compress High Dynamic Range (HDR) content to address Low Dynamic Range (LDR) displays. However, before reaching the end-user, this tone mapped content is usually compressed for broadcasting or storage purposes. Any TMO includes a quantization step to convert floating point values to integer ones. In this work, we propose to adapt this quantization, in the loop of an encoder, to reduce the entropy of the tone mapped video content. Our technique provides an appropriate quantization for each mode of both the Intra and Inter-prediction that is performed in the loop of a block-based encoder. The mode that minimizes a rate-distortion criterion uses its associated quantization to provide integer values for the rest of the encoding process. The method has been implemented in HEVC and was tested over two different scenarios: the compression of tone mapped LDR video content (using the HM10.0) and the compression of perceptually encoded HDR content (HM14.0). Results show an average bit-rate reduction under the same PSNR for all the sequences and TMO considered of 20.3% and 27.3% for tone mapped content and 2.4% and 2.7% for HDR content.

Evaluation of color mapping algorithms in different color spaces

The color gamut supported by current commercial displays is only a subset of the full spectrum of colors visible by the human eye. In High-Definition (HD) television technology, the scope of the supported colors covers 35.9% of the full visible gamut. For comparison, Ultra High-Definition (UHD) television, which is currently being deployed on the market, extends this range to 75.8%. However, when reproducing content with a wider color gamut than that of a television, typically UHD content on HD television, some original color information may lie outside the reproduction capabilities of the television. Efficient gamut mapping techniques are required in order to fit the colors of any source content into the gamut of a given display. The goal of gamut mapping is to minimize the distortion, in terms of perceptual quality, when converting video from one color gamut to another. It is assumed that the efficiency of gamut mapping depends on the color space in which it is computed. In this article, we evaluate 14 gamut mapping techniques, 12 combinations of two projection methods across six color spaces as well as R’G’B’ Clipping and wrong gamut interpretation. Objective results, using the CIEDE2000 metric, show that the R’G’B’ Clipping is slightly outperformed by only one combination of color space and projection method. However, analysis of images shows that R’G’B’ Clipping can result in loss of contrast in highly saturated images, greatly impairing the quality of the mapped image.

A hybrid approach for efficient color gamut mapping

Ultra High Definition (UHD) and High Definition (HD) Television standard recommendations support different color gamuts with the HD gamut much smaller than that of the UHD one. To adapt UHD content to the restricted gamut of HD televisions, a process known as gamut mapping is required. This gamut mapping projects out-of-gamut colors inside the targeted color gamut. Gamut mapping can be performed in any color space and using different projection methods. In this paper, we present a hybrid gamut mapping approach which selects one combination of color space and projection method for each UHD color representable. The selection is based on the CIE ΔE2000 metric. Results show improvements in terms of CIE ΔE2000 when comparing original and projected colors over existing methods.

Evaluation of chroma subsampling for high dynamic range video compression

Subsampling chroma channels of video content is a processing performed in most video distribution pipelines. In such pipelines, chroma subsampled content also undergoes video coding, using a codec, before being distributed. Presently, it is common practice to optimize compression efficiency and visual quality of content with respect to its subsampled version rather than the original one. Although this may suffice for traditional imagery, it is unclear if emerging technologies such as Wide Color Gamut and High Dynamic Range are more sensitive to quality loss due to chroma subsampling. In this article, we assess the efficiency of chroma subsampling when compressing HDR content. We compare the performance of two different downsampling filters against the full chroma sampling. Objective results show that distributing 4:4:4 is more efficient than its 4:2:0 counterpart at medium to high bit-rates. For low bit-rates, this increase in efficiency is reduced and in some cases even reversed.