John J. McCann

Lightness and Retinex Theory

Sensations of color show a strong correlation with reflectance, even though the amount of visible light reaching the eye depends on the product of reflectance and illumination. The visual system must achieve this remarkable result by a scheme that does not measure flux. Such a scheme is described as the basis of retinex theory. This theory assumes that there are three independent cone systems, each starting with a set of receptors peaking, respectively, in the long-, middle-, and short-wavelength regions of the visible spectrum. Each system forms a separate image of the world in terms of lightness that shows a strong correlation with reflectance within its particular band of wavelengths. These images are not mixed, but rather are compared to generate color sensations. The problem then becomes how the lightness of areas in these separate images can be independent of flux. This article describes the mathematics of a lightness scheme that generates lightness numbers, the biologic correlate of reflectance, independent of the flux from objects

Retinex in MATLAB

Many different descriptions of Retinex methods of light- ness computation exist. We provide concise MATLAB™ implemen- tations of two of the spatial techniques of making pixel comparisons. The code is presented, along with test results on several images and a discussion of the results. We also discuss the calibration of input images and the postRetinex processing required to display the output images.

Quantitative studies in retinex theory a comparison between theoretical predictions and observer responses to the “color mondrian” experiments

Abstract Lands Color Mondrian experiments showed that a single wavelength-radiance distribution falling on a point on the retina can generate nearly any color sensation. In Part I we repeated that experiment, quantifying the color sensations for each of the many Mondrian areas. In Part II we show that each areas color sensation correlates with a triplet of reflectances measured with photodetectors having the same spectral sensitivities as the cone pigments in the eye. This result provides a description of what the visual system does, but it does not provide a mechanism for how the visual system can do it because the reflectance measurements required the use of a reflectance standard and unchanging illumination. In Part III we describe a model for color sensations that computes three reflectances from the wavelength-radiance distribution without reflectance or illumination standards: hence, it is able to predict the color sensations seen by the observer. The model is able to predict gray, red. yellow, green and blue sensations associated with areas that send identical wavelength-radiance distributions to the eye.

Visibility of low-spatial-frequency sine-wave targets: Dependence on number of cycles

The number of cycles in a low-frequency sinusoidal display is a crucial variable in determining the visibility of the display. In particular, the threshold contrast is essentially independent of spatial frequency for these displays. We have extended the above experiments, using more cycles and a variety of targets and observer tasks. The results confirm previous findings; they also show that the type of target or task has little influence. For low-frequency sinusoids that contain up to about 3 cycles, the threshold contrast is determined by the number of cycles. For high-number-of-cycles targets with spatial frequencies above 6-10 cycles per degree, visibility is predominantly dependent on the spatial frequency. The results suggest that the low-frequency decrease in reported MTFs is due to the decrease of the number of cycles used in determining them.

Interaction of the long-wave cones and the rods to produce color sensations.

The duplicity theory states that cones produce photopic or color vision, whereas the rods produce scotopic or colorless night vision. This paper reports experimental findings which demonstrate the capacity of the rods to interact with the long-wave cones to produce color sensations. Radiances of 546 and 450 nm that excited only the rods, and radiances of 656 nm that excited only the long-wave cones were determined. When the rods and long-wave cones were selectively excited with the minimum radiance necessary to see form, the observers reported seeing a large variety of color sensations. These observers also reported the same variety of color sensations at greater radiances when the rods and long-wave cones were selectively excited. Color sensations produced by the excitation of rods and long-wave cones were independent of the wavelength used to excite the rods. Color sensations produced by rods and long-wave cones were identical, except for slight differences of brightness and sharpness, to the color sensations produced by 656 and 495±5 nm light when both were above cone threshold. Therefore, under the described conditions, the rods can be as much a part of the human color-producing system as the cones. All of the above results can be explained by Land’s retinex theory of color vision.

Visibility of low-frequency sine-wave targets: Dependence on number of cycles and surround parameters

Several papers have described experiments which have shown that human visual sensitivity to sinusoidal gratings is dependent on the number of cycles of sinusoid for low-spatial-frequency targets (McCann, Savoy and Hall 1973; Hoekstra, van der Goat, van den Brink and Bilsen, 1974; McCann, Savoy, Hall and Scarpetti 1974; Savoy and McCann, 1975). A recent article by Estevez and Cavonius (1976) presented the argument that dependence on the number of cycles occurred only in the presence of a dark surround They reported a dependence on number of cycles with sinusoids that have a black surround, but they also reported a lack of dependence on number of cycles with targets that have averageluminance areas adjacent to the sinusoidal area. They pointed out that the majority of previous experiments had been performed with a dark surround.’ The exception was one of the experiments by Savoy and McCann (1975) in which the luminance of the area surrounding the sinusoidal grating was equal to the average luminance of the grating. In this case, Estevez and Cavonius (1976) argued that a hairline edge of a mirror, which produced a faint visible line around the sinusoidal portion of the display, was having an effect very similar to that of a black surround. This Letter presents experimental results which replicate the Savoy and McCann (1975) data using targets without any visible line. This shows that the conclusions reached in the 1975 paper reporting a dependence on number of cycles are correct. Furthermore, the experiments show that the experimental data of Estevez and Cavonius are due to substantial differences in non-sinusoidal parameters of their displays compared to those used by Savoy and McCann. We began by repeating the Savoy and McCann experiments without the hairline edge created by the mirror. William Wray. of our laboratory, in collaboration with John Hall designed a display system that provides stimuli without visible lines around the sinusoidal portion. The unusual property of this electronic display system is that it allows us to vary the luminance and the size of the average-luminance surround on all four sides of the sinusoidal portion of the display. Estevez and Cavonius displayed areas of average luminance on the left and the right, but not on the top and the bottom of the sinusoidal portion

Visibility of continuous luminance gradients.

Abstract A plateau of illumination was modulated with various patterns of gradual change: linear slopes and small numbers of low spatial frequency sinusoidal oscillations. Over the range of parameters tested, the threshold contrast necessary for the detection of these modulations was found to be largely independent of the steepness of the gradient, the frequency of the sinusoids, and the size of the target on the retina. Visibility was found to be a function of the fractional change in luminance across the target (contrast) and the pattern of the modulation (characterized by the number of cycles of sinusoid).

Capturing a black cat in shade: past and present of Retinex color appearance models

This work recounts the research on capturing real-life scenes, calculating appearances, and rendering sensations on film and other limited dynamic-range media. It describes the first pat- ents, a hardware display used in Lands Ives Medal Address in 1968, the first computer simulations using 20 324 pixel arrays, psy- chophysical experiments and computational models of color con- stancy and dynamic range compression, and the Frankle-McCann computationally efficient retinex image processing of 512 3512 im- ages. It includes several modifications of the original approach, in- cluding recent models of human vision and gamut-mapping applica- tions. This work emphasizes the need for parallel studies of psychophysical measurements of human vision and computational algorithms used in commercial imaging systems.

Effects of average-luminance surrounds on the visibility of sine-wave gratings

In the low-spatial-frequency region (below 2 c/deg) contrast sensitivity to sinusoids does not depend on spatial frequency, but does depend on the number of cycles of sinusoid. Contrast sensitivity to sinusoids can vary from 14 to 60 depending on the amount of average-luminance area or flank adjacent to the sinusoid. The influence of average-luminance flanks does not depend on the width of the flank, but does depend on the equivalent number of cycles of flank.

Retinex at 40

Edwin Land first described the Retinex idea in the 1963 RESA William Proctor Prize Address, Cleveland, Ohio, on December 30, 1963. He said that it was fruitful to suggest that receptors exist in sets. A Retinex is all mechanisms from retina to cortex necessary to form images in terms of lightness. This was a distinct departure from the point-bypoint thinking that dominated physics and colorimetry. It required that models of color appearance evaluate all the pixels in the field of view as input. It is difficult in today’s world, dominated by digital images, to imagine just how novel this idea was in the 1960s. Nevertheless, many experiments in the 60s were fundamental to our understanding of human vision. Hubel and Wiesel’s studies of cat and monkey cortex, Land’s Mondrians, and Campbell and Robson’s work on human spatial frequency responses all made a strong