Frédéric Guichard

Axioms and fundamental equations of image processing

Image-processing transforms must satisfy a list of formal requirements. We discuss these requirements and classify them into three categories: “architectural requirements” like locality, recursivity and causality in the scale space, “stability requirements” like the comparison principle and “morphological requirements”, which correspond to shape-preserving properties (rotation invariance, scale invariance, etc.). A complete classification is given of all image multiscale transforms satisfying these requirements. This classification yields a characterization of all classical models and includes new ones, which all are partial differential equations. The new models we introduce have more invariance properties than all the previously known models and in particular have a projection invariance essential for shape recognition. Numerical experiments are presented and compared. The same method is applied to the multiscale analysis of movies. By introducing a property of Galilean invariance, we find a single multiscale morphological model for movie analysis.

Extended depth-of-field (EDoF) using sharpness transport across colour channels

In this paper we present an approach to obtain an extended Depth-of-Field (DoF) for cell phone miniature camera by jointly optimizing optical system and post-capture digital processing techniques. Using a computational imaging approach, we demonstrate how to increase, to a useful operating range, the effective DoF of a specifically designed fixed focus lens operating e.g. at f/2.8. This is achieved with a lens design where the longitudinal chromatic aberration has been increased. This increase is controlled so as to have, for any distance within the extended DoF, at least one colour channel of a RGB image which contains the in-focus scene information (e.g. high frequencies). By determining the sharpest colour (for each region in the digital image) and reflecting its sharpness on the others, we show that it is possible to get a sharp image for all colours through the merged DoF of the three of them. We compare our technique with other approaches that also aimed to increase the DoF such as Wavefront coding.

Does resolution really increase image quality

A general trend in the CMOS image sensor market is for increasing resolution (by having a larger number of pixels) while keeping a small form factor by shrinking photosite size. This article discusses the impact of this trend on some of the main attributes of image quality. The first example is image sharpness. A smaller pitch theoretically allows a larger limiting resolution which is derived from the Modulation Transfer Function (MTF). But recent sensor technologies (1.75μm, and soon 1.45μm) with typical aperture f/2.8 are clearly reaching the size of the diffraction blur spot. A second example is the impact on pixel light sensitivity and image sensor noise. For photonic noise, the Signal-to-Noise-Ratio (SNR) is typically a decreasing function of the resolution. To evaluate whether shrinking pixel size could be beneficial to the image quality, the tradeoff between spatial resolution and light sensitivity is examined by comparing the image information capacity of sensors with varying pixel size. A theoretical analysis that takes into consideration measured and predictive models of pixel performance degradation and improvement associated with CMOS imager technology scaling, is presented. This analysis is completed by a benchmarking of recent commercial sensors with different pixel technologies.

Information capacity: a measure of potential image quality of a digital camera

The aim of the paper is to define an objective measurement for evaluating the performance of a digital camera. The challenge is to mix different flaws involving geometry (as distortion or lateral chromatic aberrations), light (as luminance and color shading), or statistical phenomena (as noise). We introduce the concept of information capacity that accounts for all the main defects than can be observed in digital images, and that can be due either to the optics or to the sensor. The information capacity describes the potential of the camera to produce good images. In particular, digital processing can correct some flaws (like distortion). Our definition of information takes possible correction into account and the fact that processing can neither retrieve lost information nor create some. This paper extends some of our previous work where the information capacity was only defined for RAW sensors. The concept is extended for cameras with optical defects as distortion, lateral and longitudinal chromatic aberration or lens shading.

Measuring texture sharpness of a digital camera

A method for evaluating texture quality as shot by a camera is presented. It is shown that usual sharpness measurements are not completely satisfying for this task. A new target based on random geometry is proposed. It uses the so-called dead leaves model. It contains objects of any size at any orientation and follows some common statistics with natural images. Some experiments show that the correlation between objectives measurements derived from this target and subjective measurements conducted in the Camera Phone Image Quality initiative are excellent.

From spectral sensitivities to noise characteristics

For a given noise at the photosite level and a given output color space, the spectral sensitivities of a sensor constrain the color processing and therefore impact the level of noise in the output. In particular, this noise may be very different from the usually documented photosite noise. A key phenomenon is the appearance of strong correlations between channels which makes individual channel measures (including the classical signal-to-noise ratio, SNR) misleading. We evaluate existing chains and isolated sensors by several indicators including the previously developed color sensitivity. We finally apply this approach to the understanding of good spectral sensitivities by considering hypothetical spectral sensitivities and simulating their performances.

Sensor information capacity and spectral sensitivities

In this paper, we numerically quantify the information capacity of a sensor, by examining the different factors than can limit this capacity, namely sensor spectral response, noise, and sensor blur (due to fill factor, cross talk and diffraction, for given aperture). In particular, we compare the effectiveness of raw color space for different kinds of sensors. We also define an intrinsic notion of color sensitivity that generalizes some of our previous works. We also attempt to discuss how metamerism can be represented for a sensor.

Nonlinear frame fusion by minimizing divergence magnitude of optical flow

We define a variational method to perform frame-fusion. The process is in three steps: we first estimate the velocities and occlusions using optical flow and spatial constraint on the velocities based on the L1 norm of the divergence. We then collect non-occluded points from the sequence, and estimate their locations at a chosen time, on which we perform the fusion. From this list of points, we reconstruct the super-frame by minimizing a total variation energy which forces the super-frame to look like each frame of the sequence (after shifting) and select among the least oscillatory solutions. We display some examples.

Electronic trigger for capacitive touchscreen and extension of ISO 15781 standard time lag measurements to smartphones

We present in this paper a novel capacitive device that stimulates the touchscreen interface of a smartphone (or of any imaging device equipped with a capacitive touchscreen) and synchronizes triggering with the DxO LED Universal Timer to measure shooting time lag and shutter lag according to ISO 15781:2013. The device and protocol extend the time lag measurement beyond the standard by including negative shutter lag, a phenomenon that is more and more commonly found in smartphones. The device is computer-controlled, and this feature, combined with measurement algorithms, makes it possible to automatize a large series of captures so as to provide more refined statistical analyses when, for example, the shutter lag of “zero shutter lag” devices is limited by the frame time as our measurements confirm.

Measurement and protocol for evaluating video and still stabilization systems

This article presents a system and a protocol to characterize image stabilization systems both for still images and videos. It uses a six axes platform, three being used for camera rotation and three for camera positioning. The platform is programmable and can reproduce complex motions that have been typically recorded by a gyroscope mounted on different types of cameras in different use cases. The measurement uses a single chart for still image and videos, the texture dead leaves chart. Although the proposed implementation of the protocol uses a motion platform, the measurement itself does not rely on any specific hardware. For still images, a modulation transfer function is measured in different directions and is weighted by a contrast sensitivity function (simulating the human visual system accuracy) to obtain an acutance. The sharpness improvement due to the image stabilization system is a good measurement of performance as recommended by a CIPA standard draft. For video, four markers on the chart are detected with sub-pixel accuracy to determine a homographic deformation between the current frame and a reference position. This model describes well the apparent global motion as translations, but also rotations along the optical axis and distortion due to the electronic rolling shutter equipping most CMOS sensors. The protocol is applied to all types of cameras such as DSC, DSLR and smartphones.