Louis Joseph Kerofsky

Display adaptive tone mapping

We propose a tone mapping operator that can minimize visible contrast distortions for a range of output devices, ranging from e-paper to HDR displays. The operator weights contrast distortions according to their visibility predicted by the model of the human visual system. The distortions are minimized given a display model that enforces constraints on the solution. We show that the problem can be solved very efficiently by employing higher order image statistics and quadratic programming. Our tone mapping technique can adjust image or video content for optimum contrast visibility taking into account ambient illumination and display characteristics. We discuss the differences between our method and previous approaches to the tone mapping problem.

Brightness preservation for LCD backlight dimming

— Backlight dimming is a dominant method for power reduction in LCDs. Image processing with a simple boost and clip can compensate for such dimming-restoring image brightness. We propose a low complexity approach which replaces hard clipping with smooth roll-off to reduce clipping artifacts. An additional high-quality approach uses a two-channel spatial-frequency decomposition to preserve highlight detail attenuated by the roll-off. Image quality is improved and power savings can be increased by more-aggressive backlight dimming.

Low-complexity transform and quantization with 16-bit arithmetic for H.26L

This paper presents an overview of the latest transform and quantization designs for H.26L. Unlike the popular discrete cosine transform (DCT) used in previous standards, the transforms in H.26L can be computed exactly in integer arithmetic, thus avoiding inverse transform mismatch problems. The new transforms can also be computed without multiplications, just additions and shifts, in 16-bit arithmetic, thus minimizing computational complexity, especially for low-end processors. By using short tables, the new quantization formulas use multiplications but avoid divisions.

All-phase motion compensated prediction in the wavelet domain for high performance video coding

This paper presents a novel framework of motion compensated prediction (MCP) techniques in the wavelet domain for high performance video coding. Our analysis reveals fundamental limitations with previous ad-hoc wavelet-based video coders from the motion accuracy point of view. We demonstrate that the phase associated with any wavelet transform carries critical information of the motion accuracy and we propose to restore the motion accuracy by considering the wavelet coefficients of the previous frame with all different phases. Our all-phase MCP approach can be viewed as predicting the wavelet coefficients from an over-complete expansion of the previous frame. Experimental results have shown that restoration of motion accuracy in the wavelet domain can dramatically improve the efficiency of MCP. The video coder (MCP-WT) built upon the MCP of wavelet coefficients has achieved 2 to 3 dB gain over existing the MPEG-2 coder at the bit rate of 1 to 9 Mbps. Moreover, MCP techniques in the wavelet domain offer a promising new ground for developing efficient scalable video coders.

26.2: Distinguished Paper: Brightness Preservation for LCD Backlight Reduction

Backlight dimming is the predominant method for power reduction in LDC displays and image processing can compensate for such dimming. We propose a low complexity approach which replaces hard clipping with smooth roll-off. A high quality approach preserves highlight detail. Image quality is improved and power savings can be increases by more aggressive BL dimming capable with these methods.

Color Gamut Scalable Video Coding

This paper describes a scalable extension of the High Efficiency Video Coding (HEVC) standard that supports different color gamuts in an enhancement and base layer. Here, the emphasis is on scenarios with BT.2020 color gamut in an enhancement layer and BT.709 color gamut in the base layer. This is motivated by a need to provide content for both high definition and ultra-high definition devices in the near future. The paper describes a method for predicting the enhancement layer samples from a decoded base layer using a series of multiplies and adds to account for both color gamut and bit-depth changes. Results show an improvement in coding efficiency between 65% and 84% for luma (57% and 85% for chroma) compared to simulcast in quality (SNR) scalable coding.

Subpixel rendering on nonstriped colour matrix displays

Luminance resolution of colour matrix displays can be enhanced when subpixel resampling is used to render images, but at the cost of colour aliasing if appropriate filtering is not done. The problem has been effectively solved for the case of one-dimensional RGB striped matrix displays by using a high-pass chrominance filter, based on a human perceptual model, following resampling. In this paper we motivate the case of 2D subpixel geometries and suggest an extension to the ID solution.

Compression Efficiency Improvement over HEVC Main 10 Profile for HDR and WCG Content

The paper presents the joint proposal by Arris, Dolby and InterDigital as a response to the Call-for-Evidence of the High Dynamic Range and Wide Color Gamut (HDR/WCG) video compression in MPEG. The joint proposal introduces a set of new HDR coding technologies, including the IPT-PQ color space, the adaptive reshaping process, the color enhancement filters, and the adaptive transfer function. These new coding technologies are applied to the decoded output of an HEVC decoder. Hence, no changes to the lower level logics of the HEVC decoder are required to implement the proposal. Formal subjective tests conducted by MPEG confirmed that the proposal could achieve significant subjective quality improvements over the HEVC Main10 anchors at similar bit rates for HDR/WCG video content.

Improving content visibility for high‐ambient‐illumination viewable display and energy‐saving display

— Nowadays, low-contrast viewing of LC displays (LCDs) occurs very often, which includes the viewing of mobile LCDs at high ambient illumination and the viewing of LCDs at low-power mode. These cases result in low-content visibility and low contrast, leading to an unpleasant viewing experience. In this paper, a technique to improve the perceived contrast and visibility of images at low-contrast viewing conditions is proposed. The proposed approach enhances image brightness with content and ambient adaptive image brightening and highlights visual parts and boundaries with non-photorealistic rendering. The proposed technique enables longer battery life for mobile LC devices and makes mobile LC devices viewable at high ambient illumination. It also enables TVs with extreme low-power consumption and smart-grid responsive TVs.

67.4: Use of a Spatial Multiresolution Visual Model with Display Characterization and Ambient Information to Adaptively Shape the Tonescales of Mobile Displays

An image-dependent, display-dependent, and ambient illumination-dependent tone-mapping operator is developed that is driven by a human vision system model. It aims to maximize visible contrast information for the full range of display devices, from e-paper to HDR displays. The operator weights contrast distortions according to their visibility predicted by the model of the human visual system. The distortions are minimized given a display model that incorporates ambient information to enforce constraints on the solution. A scene cut algorithm is used for video content to allow rapid tonescale changes across scenes while a temporal LPF is used to avoid visible temporal fluctuations within scenes. We discuss the differences between our method and previous approaches to the tone-mapping problem, as well as newer tone mapping approaches that do not seek to preserve image information. This paper will especially highlight the issues important for current display technology for mobile devices.