Praveen Cyriac

A tone mapping operator based on neural and psychophysical models of visual perception

High dynamic range imaging techniques involve capturing and storing real world radiance values that span many orders of magnitude. However, common display devices can usually reproduce intensity ranges only up to two to three orders of magnitude. Therefore, in order to display a high dynamic range image on a low dynamic range screen, the dynamic range of the image needs to be compressed without losing details or introducing artefacts, and this process is called tone mapping. A good tone mapping operator must be able to produce a low dynamic range image that matches as much as possible the perception of the real world scene. We propose a two stage tone mapping approach, in which the first stage is a global method for range compression based on a gamma curve that equalizes the lightness histogram the best, and the second stage performs local contrast enhancement and color induction using neural activity models for the visual cortex.

Optimized tone curve for in-camera image processing

Comunicacio presentada al IS&T International Symposium on Electronic Imaging, celebrat del 14 al 18 de febrer de 2016 a San Francisco (CA, USA) i organitzat per la Society for Imaging Science and Technology.

Perceptual dynamic range for in-camera image processing.

A patent application based on the research in this article has been filed at the European patent/noffice, Application no 15154172.9-1906. This work was supported by the European Research/nCouncil, Starting Grant ref. 306337, by the Spanish government, grant ref. TIN2012-/n38112, and by the Icrea Academia Award.

Derivatives and inverse of cascaded linear+nonlinear neural models

In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences. However, the conventional literature is usually too focused on only describing the forward input-output transform. Instead, in this work we present the mathematics of such cascades beyond the forward transform, namely the Jacobian matrices and the inverse. The fundamental reason for this analytical treatment is that it offers useful analytical insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (volume of the discrimination regions) and the adaptation of the receptive fields can be identified in the expression of the Jacobian w.r.t. the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian w.r.t. the parameters shows which aspects of the model have bigger impact in the response, and hence their relative relevance. The analytic inverse implies conditions for the response and model parameters to ensure appropriate decoding. From the experimental and applied perspective, (a) the Jacobian w.r.t. the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometrical properties, (b) the Jacobian matrices w.r.t. the parameters are convenient to learn the model from classical experiments or alternative goal optimization, and (c) the inverse is a promising model-based alternative to blind machine-learning methods for neural decoding that do not include meaningful biological information. The theory is checked by building and testing a vision model that actually follows a modular Linear+Nonlinear program. Our illustrative derivable and invertible model consists of a cascade of modules that account for brightness, contrast, energy masking, and wavelet masking. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization modules are substituted by equivalent modules such as the Wilson-Cowan interaction model (at the V1 cortex) or a tone-mapping model (at the retina).

The Wilson-Cowan model describes Contrast Response and Subjective Distortion

The Wilson-Cowan equations were originally proposed to describe the low-level dynamics of neural populations (Wilson&Cowan 1972). These equations have been extensively used in modelling the oscillations of cortical activity (Cowan et al. 2016). However, due to their low-level nature, very few works have attempted connections to higher level psychophysics (Herzog et al. 2003, Hermens et al. 2005) and, to the best of our knowledge, they have not been used to predict contrast response curves or subjective image quality. Interestingly (Bertalmío&Cowan 2009) showed that Wilson-Cowan models may lead to (high level) color constancy. Moreover, these models may have positive statistical effects similarly to Divisive Normalization, which is the canonical choice to understand contrast response (Watson&Solomon 1997, Carandini&Heeger 2012): while Divisive Normalization reduces redundancy due to predictive coding (Malo&Laparra 2010), Wilson-Cowan leads to local histogram equalization (Bertalmío 2014), another route to increase channel capacity.

A Nonlocal Variational Formulation for the Improvement of Tone Mapped Images

Due to technical limitations, common display devices can only reproduce images having a low range of intensity values (dynamic range). As a consequence, the dynamic range of images encoding real world scenes, which is large, has to be compressed in order for them to be reproduced on a common display, and this technique is called tone mapping. Because there is no ground truth to compare with, evaluation of a tone mapped image has to be done by comparing with the original high dynamic range image. As standard metrics based on pixelwise comparisons are not suitable for comparing images of different dynamic range, nonlocal perceptual based metrics are commonly used. We propose a general method for optimizing tone mapped images with respect to a given nonlocal metric. In particular, if the metric is perceptual, i.e., it involves perceptual concepts, we provide an adequate minimization strategy. Experiments on a particular perceptual metric tested with different tone mapped images provided by several tone mappi...

A Variational Method for the Optimization of Tone Mapping Operators

Given any metric that compares images of different dynamic range, we propose a method to reduce their distance with respect to this metric. The key idea is to consider the metric as a non local operator. Then, we transform the problem of distance reduction into a non local variational problem. In this context, the low dynamic range image having the smallest distance with a given high dynamic range is the minimum of a suitable energy, and can be reached through a gradient descent algorithm. Dealing with an appropriate metric, we present an application to Tone Mapping Operator (TMO) optimization. We apply our gradient descent algorithm, where the initial conditions are Tone Mapped (TM) images. Experiments show that our algorithm does reduce the distance of the TM images with the high dynamic range source images, meaning that our method improves the corresponding TMOs.

A fast image dehazing method that does not introduce color artifacts

We propose a method for color dehazing with four main characteristics: it does not introduce color artifacts, it does not depend on inverting any physical equation, it is based on models of visual perception, and it is fast, potentially real time. Our method converts the original input image to the HSV color space and works in the saturation and value domains by: (1) reducing the value component via a global constrained histogram flattening; (2) modifying the saturation component in consistency with the previous reduced value; and (3) performing a local contrast enhancement in the value component. Results show that our method competes with the state-of-the-art when dealing with standard hazy images, and outperforms it when dealing with challenging haze cases. Furthermore, our method is able to dehaze a FullHD image on a GPU in 90 ms.

Automatic, Viewing-Condition Dependent Contrast Grading based on Perceptual Models

Cameras automatically apply non-linear transformations to the sensor data, allowing for perceptually-uniform quantization suited to standard dynamic range displays in dim conditions. In the cinema industry, data is recorded in raw (linear) format and non-linearly corrected in post-production by a skilled technician who optimizes image appearance for cinema (dark) conditions. We propose a method that automatically performs this non-linear transformation taking into account the intended viewing conditions. It is based on visual perception models and produces results that look natural, without any spatio-temporal artifacts. User preference tests show that our method outperforms state of the art approaches. The technique is fast and could be implemented on camera hardware. It can be used for on-set monitoring on regular displays, as a substitute for gamma-correction, and as a way of providing the colorist with content that is both natural looking and has a crisp, clear image.