Erik Reinhard

Colour Mapping: A Review of Recent Methods, Extensions and Applications

The objective of colour mapping or colour transfer methods is to recolour a given image or video by deriving a mapping between that image and another image serving as a reference. These methods have received considerable attention in recent years, both in academic literature and industrial applications. Methods for recolouring images have often appeared under the labels of colour correction, colour transfer or colour balancing, to name a few, but their goal is always the same: mapping the colours of one image to another. In this paper, we present a comprehensive overview of these methods and offer a classification of current solutions depending not only on their algorithmic formulation but also their range of applications. We also provide a new dataset and a novel evaluation technique called ‘evaluation by colour mapping roundtrip’. We discuss the relative merit of each class of techniques through examples and show how colour mapping solutions can have been applied to a diverse range of problems.

On Visual Realism of Synthesized Imagery

Traditionally, computer graphics has been concerned with producing imagery that is as physically accurate as possible. But accurate physical simulation of geometry, lighting, and material properties of a visual scene can be cumbersome and time consuming. At the same time, human vision is far from accurate, which offers an enormous opportunity to create imagery at a reduced computational cost as well as with less reliance on human modelers. As a result, a recent trend is toward accepting perceptual plausibility instead of physical accuracy as a guiding principle in the design of modeling and rendering systems. This requires us to understand visual realism, which involves both learning statistical regularities of the world, for instance, by employing huge amounts of data, as well as humans visual perception of it. This paper addresses issues related to understanding realism, presents several applications, and discusses what this interesting approach may lead to in the future.

Motion aware exposure bracketing for HDR video

Mobile phones and tablets are rapidly gaining significance as omnipresent image and video capture devices. In this context we present an algorithm that allows such devices to capture high dynamic range (HDR) video. The design of the algorithm was informed by a perceptual study that assesses the relative importance of motion and dynamic range. We found that ghosting artefacts are more visually disturbing than a reduction in dynamic range, even if a comparable number of pixels is affected by each. We incorporated these findings into a real‐time, adaptive metering algorithm that seamlessly adjusts its settings to take exposures that will lead to minimal visual artefacts after recombination into an HDR sequence. It is uniquely suitable for real‐time selection of exposure settings. Finally, we present an off‐line HDR reconstruction algorithm that is matched to the adaptive nature of our real‐time metering approach.

Sky Based Light Metering for High Dynamic Range Images

Image calibration requires both linearization of pixel values and scaling so that values in the image correspond to real‐world luminances. In this paper we focus on the latter and rather than rely on camera characterization, we calibrate images by analysing their content and metadata, obviating the need for expensive measuring devices or modeling of lens and camera combinations. Our analysis correlates sky pixel values to luminances that would be expected based on geographical metadata. Combined with high dynamic range (HDR) imaging, which gives us linear pixel data, our algorithm allows us to find absolute luminance values for each pixel—effectively turning digital cameras into absolute light meters. To validate our algorithm we have collected and annotated a calibrated set of HDR images and compared our estimation with several other approaches, showing that our approach is able to more accurately recover absolute luminance. We discuss various applications and demonstrate the utility of our method in the context of calibrated color appearance reproduction and lighting design.

A Gamut-Mapping Framework for Color-Accurate Reproduction of HDR Images.

An integrated gamut- and tone-management framework for color-accurate reproduction of high dynamic range images can prevent hue and luminance shifts while taking gamut boundaries into consideration. The proposed approach is conceptually and computationally simple, parameter-free, and compatible with existing tone-mapping operators.

Perceptual Lightness Modeling for High-Dynamic-Range Imaging

The human visual system (HVS) non-linearly processes light from the real world, allowing us to perceive detail over a wide range of illumination. Although models that describe this non-linearity are constructed based on psycho-visual experiments, they generally apply to a limited range of illumination and therefore may not fully explain the behavior of the HVS under more extreme illumination conditions. We propose a modified experimental protocol for measuring visual responses to emissive stimuli that do not require participant training, nor requiring the exclusion of non-expert participants. Furthermore, the protocol can be applied to stimuli covering an extended luminance range. Based on the outcome of our experiment, we propose a new model describing lightness response over an extended luminance range. The model can be integrated with existing color appearance models or perceptual color spaces. To demonstrate the effectiveness of our model in high dynamic range applications, we evaluate its suitability for dynamic range expansion relative to existing solutions.

Optical center estimation for lenslet-based plenoptic cameras

Plenoptic cameras enable a variety of novel post-processing applications, including refocusing and single-shot 3D imaging. To achieve high accuracy, such applications typically require knowledge of intrinsic camera parameters. One such parameter is the location of the main lens optical center relative to the sensor, which is required for modeling radially symmetric optical effects. We show that estimating this parameter can be achieved to an accuracy of less than half a pixel by utilising the symmetry inherent in each micro-image. Further, we show that estimating this parameter separately allows all other intrinsic camera parameters to be estimated with higher accuracy than can be achieved using a single optimization scheme, and leads to better vignetting correction than with an inaccurate optical center.

Cornsweet surfaces for selective contrast enhancement

Abstract A typical goal when enhancing the contrast of images is to increase the perceived contrast without altering the original feel of the image. Such contrast enhancement can be achieved by modelling Cornsweet profiles into the image. We demonstrate that previous methods aiming to model Cornsweet profiles for contrast enhancement, often employing the unsharp mask operator, are not robust to image content. To achieve robustness, we propose a fundamentally different vector-centric approach with Cornsweet surfaces. Cornsweet surfaces are parametrised 3D surfaces (2D in space, 1D in luminance enhancement) that are extruded or depressed in the luminance dimension to create countershading that respects image structure. In contrast to previous methods, our method is robust against the topology of the edges to be enhanced and the relative luminance across those edges. In user trials, our solution was significantly preferred over the most related contrast enhancement method.

Over- and Under-Exposure Reconstruction of a Single Plenoptic Capture

Light field images, for example, taken with plenoptic cameras, offer interesting post-processing opportunities, including depth-of-field management, depth estimation, viewpoint selection, and 3D image synthesis. Like most capture devices, however, plenoptic cameras have a limited dynamic range, so that over- and under-exposed areas in plenoptic images are commonplace. We therefore present a straightforward and robust plenoptic reconstruction technique based on the observation that vignetting causes peripheral views to receive less light than central views. Thus, corresponding pixels in different views can be used to reconstruct illumination, especially in areas where information missing in one view is present in another. Our algorithm accurately reconstructs under- and over-exposed regions (known as declipping), additionally affording an increase in peak luminance by up to two f-stops, and a comparable lowering of the noise floor. The key advantages of this approach are that no hardware modifications are necessary to improve the dynamic range, that no multiple exposure techniques are required, and therefore that no ghosting or other artifacts are introduced.

Color Management in HDR Imaging

Abstract A large volume of research exists that addresses the mapping, compression, expansion, and other types of manipulation of the extended luminance range afforded by high dynamic range technologies. However, most of this research focuses exclusively on the luminance dimension, ignoring the chromatic information in images. This is particularly problematic given the parallel push toward extending not only the luminance range but also the chromatic range. In this chapter, we look at solutions that combine the ideas of high dynamic range imaging with color management approaches in different contexts.