David Alleysson

Linear demosaicing inspired by the human visual system

There is an analogy between single-chip color cameras and the human visual system in that these two systems acquire only one limited wavelength sensitivity band per spatial location. We have exploited this analogy, defining a model that characterizes a one-color per spatial position image as a coding into luminance and chrominance of the corresponding three colors per spatial position image. Luminance is defined with full spatial resolution while chrominance contains subsampled opponent colors. Moreover, luminance and chrominance follow a particular arrangement in the Fourier domain, allowing for demosaicing by spatial frequency filtering. This model shows that visual artifacts after demosaicing are due to aliasing between luminance and chrominance and could be solved using a preprocessing filter. This approach also gives new insights for the representation of single-color per spatial location images and enables formal and controllable procedures to design demosaicing algorithms that perform well compared to concurrent approaches, as demonstrated by experiments.

Coarse scales are sufficient for efficient categorization of emotional facial expressions: Evidence from neural computation

The human perceptual system performs rapid processing within the early visual system: low spatial frequency information is processed rapidly through magnocellular layers, whereas the parvocellular layers process all the spatial frequencies more slowly. The purpose of the present paper is to test the usefulness of low spatial frequency (LSF) information compared to high spatial frequency (HSF) and broad spatial frequency (BSF) visual stimuli in a classification task of emotional facial expressions (EFE) by artificial neural networks. The connectionist modeling results show that an LSF information provided by the frequency domain is sufficient for a distributed neural network to correctly classify EFE, even when all the spatial information relating to these images is discarded. These results suggest that the HSF signal, which is also present in BSF faces, acts as a source of noisy information for classification tasks in an artificial neural system.

The importance of low spatial frequency information for recognising fearful facial expressions

A recent brain imaging study (Vuilleumier, Armony, Driver and Dolan 2003, Nature Neuroscience, 6, 624–631) has shown that amygdala responses to fearful expressions are preferentially driven by intact or low spatial frequency (LSF) images of faces, rather than by high spatial frequency (HSF) images. These results suggest that LSF components processed rapidly via magnocellular pathways within the visual system might be very efficiently conveyed to the amygdala for the rapid recognition of fearful expressions, perhaps via a subcortical pathway that activates the pulvinar and superior colliculus, but which bypasses any finer visual analysis of HSF cues in the striate and temporal extrastriate cortex. The purpose of this paper is to analyse the statistical properties of LSF compared with HSF and intact faces. The statistical analysis shows that the LSF components in faces, which are typically extracted rapidly by the visual system, provide a better source of information than HSF components for the correct categorisation of fearful expressions in faces. These results support the idea that visual pathways from the magnocellular visual neurons might be optimal, at a computational level, for the rapid classification of fearful emotional expressions in human faces.

Spatio-temporal Tone Mapping Operator Based on a Retina Model

From moonlight to bright sun shine, real world visual scenes contain a very wide range of luminance; they are said to be High Dynamic Range (HDR). Our visual system is well adapted to explore and analyze such a variable visual content. It is now possible to acquire such HDR contents with digital cameras; however it is not possible to render them all on standard displays, which have only Low Dynamic Range (LDR) capabilities. This rendering usually generates bad exposure or loss of information. It is necessary to develop locally adaptive Tone Mapping Operators (TMO) to compress a HDR content to a LDR one and keep as much information as possible. The human retina is known to perform such a task to overcome the limited range of values which can be coded by neurons. The purpose of this paper is to present a TMO inspired from the retina properties. The presented biological model allows reliable dynamic range compression with natural color constancy properties. Moreover, its non-separable spatio-temporal filter enhances HDR video content processing with an added temporal constancy.

Digital Camera Workflow for High Dynamic Range Images Using a Model of Retinal Processing

We propose a complete digital camera workflow to capture and render high dynamic range (HDR) static scenes, from RAW sensor data to an output-referred encoded image. In traditional digital camera processing, demosaicing is one of the first operations done after scene analysis. It is followed by rendering operations, such as color correction and tone mapping. In our workflow, which is based on a model of retinal processing, most of the rendering steps are performed before demosaicing. This reduces the complexity of the computation, as only one third of the pixels are processed. This is especially important as our tone mapping operator applies local and global tone corrections, which is usually needed to well render high dynamic scenes. Our algorithms efficiently process HDR images with different keys and different content.

Hue discrimination, unique hues and naming

The hue discrimination curve (HDC) that characterizes performances over the entire hue circle was determined by using sinusoidally modulated spectral power distributions of 1.5 c/300 nm with fixed amplitude and twelve reference phases. To investigate relationship between hue discrimination and appearance, observers further performed a free color naming and unique hue tasks. The HDC consistently displayed two minima and two maxima; discrimination is optimal at the yellow/orange and blue/magenta boundaries and pessimal in green and in the extra-spectral magenta colors. A linear model based on Müller zone theory correctly predicts a periodical profile but with a phase-opponency (minima/maxima at 180° apart) which is inconsistent with the empirical HDCs profile.

Spatial bias induced by a non-conflictual task reveals the nature of space perception

The aim of the present study was to show that space perception depends on sensori-motor experience. We induced spatial biases by a non-conflictual lateralized sensori-motor task on twenty seven right-handed healthy volunteers (left-to-right readers). After a pre-test and before a post-test, which assessed visuo-motor and perceptual subjective midpoint in line bisection, participants performed a short lateralized pointing task (towards the left or right hemispace). Results indicated that this lateralized pointing task induced deviations towards the stimulated hemispace in both the visuo-motor and the perceptual estimations of the subjective line centre. These spatial biases varied as a function of pointing direction (left or right pointing), spatial location and line lengths. These findings suggest that a preceding non-conflictual lateralized sensori-motor experience influences subsequent space perception. Accordingly, ecological sensori-motor experience could be involved in asymmetric perception exhibited by normal individuals and neglect patients.

Joint Demosaicing and Super-Resolution Imaging from a Set of Unregistered Aliased Images

We present a new algorithm that performs demosaicing and super-resolution jointly from a set of raw images sampled with a color filter array. Such a combined approach allows us to compute the alignment parameters between the images on the raw camera data before interpolation artifacts are introduced. After image registration, a high resolution color image is reconstructed at once using the full set of images. For this, we use normalized convolution, an image interpolation method from a nonuniform set of samples. Our algorithm is tested and compared to other approaches in simulations and practical experiments.

Frequency selection demosaicking: a review and a look ahead

In this paper, we restate the model of spatio-chromatic sampling in single-chip digital cameras covered by Color Filter Array (CFA)1. The model shows that a periodic arrangement of chromatic samples in the CFA gives luminance and chromatic information that is localized in the Fourier domain. This representation allows defining a space invariant uniform demosaicking method which is based on the frequency selection of the luminance and chrominance information. We then show two extended methods which used the frequency representation of the Bayer CFA2,3 to derive adaptive demosaicking. Finally, we will show the application of the model for CFA with random arrangement of chromatic samples, either using a linear method based on Wiener estimation4 or with an adaptive method5.

Neurogeometry of color vision

In neurogeometry, principles of differential geometry and neuron dynamics are used to model the representation of forms in the primary visual cortex, V1. This approach is well-suited for explaining the perception of illusory contours such as Kanizsas figure (see Petitot (2008) for a review). In its current version, neurogeometry uses achromatic inputs to the visual system as the starting-point for form estimation. Here we ask how neurogeometry operates when the input is chromatic as in color vision. We propose that even when considering only the perception of form, the random nature of the cone mosaic must be taken into account. The main challenge for neurogeometry is to explain how achromatic information could be estimated from the sparse chromatic sampling provided by the cone mosaic. This article also discusses the non-linearity involved in a neural geometry for chromatic processing. We present empirical results on color discrimination to illustrate the geometric complexity for the discrimination contour when the adaptation state of the observer is not conditioned. The underlying non-linear geometry must conciliate both mosaic sampling and regulation of visual information in the visual system.