Kinjiro Amano

Frequency of Metamerism in Natural Scenes

Estimates of the frequency of metameric surfaces, which appear the same to the eye under one illuminant but different under another, were obtained from 50 hyperspectral images of natural scenes. The degree of metamerism was specified with respect to a color-difference measure after allowing for full chromatic adaptation. The relative frequency of metameric pairs of surfaces, expressed as a proportion of all pairs of surfaces in a scene, was very low. Depending on the criterion degree of metamerism, it ranged from about 10−6 to 10−4 for the largest illuminant change tested, which was from a daylight of correlated color temperature 25,000 K to one of 4000 K. But, given pairs of surfaces that were indistinguishable under one of these illuminants, the conditional relative frequency of metamerism was much higher, from about 10−2 to 10−1, sufficiently large to affect visual inferences about material identity.

Information limits on neural identification of colored surfaces in natural scenes.

If surfaces in a scene are to be distinguished by their color, their neural representation at some level should ideally vary little with the color of the illumination. Four possible neural codes were considered: von-Kries-scaled cone responses from single points in a scene, spatial ratios of cone responses produced by light reflected from pairs of points, and these quantities obtained with sharpened (opponent-cone) responses. The effectiveness of these codes in identifying surfaces was quantified by information-theoretic measures. Data were drawn from a sample of 25 rural and urban scenes imaged with a hyperspectral camera, which provided estimates of surface reflectance at 10-nm intervals at each of 1344 x 1024 pixels for each scene. In computer simulations, scenes were illuminated separately by daylights of correlated color temperatures 4000 K, 6500 K, and 25,000 K. Points were sampled randomly in each scene and identified according to each of the codes. It was found that the maximum information preserved under illuminant changes varied with the code, but for a particular code it was remarkably stable across the different scenes. The standard deviation over the 25 scenes was, on average, approximately 1 bit, suggesting that the neural coding of surface color can be optimized independent of location for any particular range of illuminants.

Colour constancy from temporal cues: better matches with less variability under fast illuminant changes

To test whether temporal transient cues could improve colour-constancy estimates, surface-colour matches were made across two Mondrian patterns illuminated by different daylights: the patterns were presented either in the same position in an alternating sequence or, as a control, simultaneously side-by-side. The degree of colour constancy was significantly higher with sequential stimulus presentation than with simultaneous presentation, in the best condition reaching 0.87 on a scale of 0 to 1 for matches averaged over 20 observers. The variance between observers was also markedly reduced with sequential stimulus presentation. The visual system appears to have mechanisms not requiring adaptation that can provide almost unbiased information about surface colour under changing illuminants.

Parallel detection of violations of color constancy

The perceived colors of reflecting surfaces generally remain stable despite changes in the spectrum of the illuminating light. This color constancy can be measured operationally by asking observers to distinguish illuminant changes on a scene from changes in the reflecting properties of the surfaces comprising it. It is shown here that during fast illuminant changes, simultaneous changes in spectral reflectance of one or more surfaces in an array of other surfaces can be readily detected almost independent of the numbers of surfaces, suggesting a preattentive, spatially parallel process. This process, which is perfect over a spatial window delimited by the anatomical fovea, may form an early input to a multistage analysis of surface color, providing the visual system with information about a rapidly changing world in advance of the generation of a more elaborate and stable perceptual representation.

Color Constancy: Phenomenal or Projective?

Naive observers viewed a sequence of colored Mondrian patterns, simulated on a color monitor. Each pattern was presented twice in succession, first under one daylight illuminant with a correlated color temperature of either 16,000 or 4,000 K and then under the other, to test for color constancy. The observers compared the central square of the pattern across illuminants, either rating it for sameness of material appearance or sameness of hue and saturation or judging an objective property—that is, whether its change of color originated from a change in material or only from a change in illumination. Average color constancy indices were high for material appearance ratings and binary judgments of origin and low for hue-saturation ratings. Individuals’ performance varied, but judgments of material and of hue and saturation remained demarcated. Observers seem able to separate phenomenal percepts from their ontological projections of mental appearance onto physical phenomena; thus, even when a chromatic change alters perceived hue and saturation, observers can reliably infer the cause, the constancy of the underlying surface spectral reflectance.

Colour constancy under simultaneous changes in surface position and illuminant

Two kinds of constancy underlie the everyday perception of surface colour: constancy under changes in illuminant and constancy under changes in surface position. Classically, these two constancies seem to place conflicting demands on the visual system: to both take into account the region surrounding a surface and also discount it. It is shown here, however, that the ability of observers to make surface–colour matches across simultaneous changes in test–surface position and illuminant in computer–generated ‘Mondrian’ patterns is almost as good as across changes in illuminant alone. Performance was no poorer when the surfaces surrounding the test surface were permuted, or when information from a potential comparison surface, the one with the highest luminance, was suppressed. Computer simulations of cone–photoreceptor activity showed that a reliable cue for making surface–colour matches in all experimental conditions was provided by the ratios of cone excitations between the test surfaces and a spatial average over the whole pattern.

Visual sensitivity to color errors in images of natural scenes

Simple color-difference formulae and pictorial images have traditionally been used to estimate the visual impact of color errors introduced by image-reproduction processes. But the limited gamut of RGB cameras constrains such analyses, particularly of natural scenes. The purpose of this work was to estimate visual sensitivity to color errors introduced deliberately into pictures synthesized from hyperspectral images of natural scenes without gamut constraints and to compare discrimination thresholds expressed in CIELAB and S-CIELAB color spaces. From each original image, a set of approximate images with variable color errors were generated and displayed on a calibrated RGB color monitor. The threshold for perceptibility of the errors was determined in a paired-comparison experiment. In agreement with previous studies, it was found that discrimination between original and approximate images needed on average a CIELAB color difference DeltaEab* of about 2.2. Although a large variation of performance across the nine images tested was found when errors were expressed in CIELAB units, little variation was obtained when they were expressed in S-CIELAB units.

Minimalist surface-colour matching

Some theories of surface-colour perception assume that observers estimate the illuminant on a scene so that its effects can be discounted. A critical test of this interpretation of colour constancy is whether surface-colour matching is worse when the number of surfaces in a scene is so small that any illuminant estimate is unreliable. In the experiment reported here, observers made asymmetric colour matches between pairs of simultaneously presented Mondrian-like patterns under different daylights. The patterns had either 49 surfaces or a minimal 2 surfaces. No significant effect of number was found, suggesting that illuminant estimates are unnecessary for surface-colour matching.

Approaching ideal observer efficiency in using color to retrieve information from natural scenes

Variations in illumination on a scene and trichromatic sampling by the eye limit inferences about scene content. The aim of this work was to elucidate these limits in relation to an ideal observer using color signals alone. Simulations were based on 50 hyperspectral images of natural scenes and daylight illuminants with correlated color temperatures 4000 K, 6500 K, and 25,000 K. Estimates were made of the (Shannon) information available from each scene, the redundancies in receptoral and postreceptoral coding, and the information retrieved by an observer identifying corresponding points across image pairs. For the largest illuminant difference, between 25,000 K and 4000 K, a postreceptoral transformation providing minimum redundancy yielded an efficiency of about 80% in the information retrieved. This increased to about 89% when the transformation was optimized directly for information retrieved, corresponding to an equivalent Gaussian noise amplitude of 3.0% or to a mean of 3.6 x 10(4) distinct identifiable points per scene. Using color signals to retrieve information from natural scenes can approach ideal observer efficiency levels.

Spatial distributions of local illumination color in natural scenes.

In natural complex environments, the elevation of the sun and the presence of occluding objects and mutual reflections cause variations in the spectral composition of the local illumination across time and location. Unlike the changes in time and their consequences for color appearance and constancy, the spatial variations of local illumination color in natural scenes have received relatively little attention. The aim of the present work was to characterize these spatial variations by spectral imaging. Hyperspectral radiance images were obtained from 30 rural and urban scenes in which neutral probe spheres were embedded. The spectra of the local illumination at 17 sample points on each sphere in each scene were extracted and a total of 1904 chromaticity coordinates and correlated color temperatures (CCTs) derived. Maximum differences in chromaticities over spheres and over scenes were similar. When data were pooled over scenes, CCTs ranged from 3000 K to 20,000 K, a variation of the same order of magnitude as that occurring over the day. Any mechanisms that underlie stable surface color perception in natural scenes need to accommodate these large spatial variations in local illumination color.