Sarah R. Allred

Stimulus-specific variability in color working memory with delayed estimation

Working memory for color has been the central focus in an ongoing debate concerning the structure and limits of visual working memory. Within this area, the delayed estimation task has played a key role. An implicit assumption in color working memory research generally, and delayed estimation in particular, is that the fidelity of memory does not depend on color value (and, relatedly, that experimental colors have been sampled homogeneously with respect to discriminability). This assumption is reflected in the common practice of collapsing across trials with different target colors when estimating memory precision and other model parameters. Here we investigated whether or not this assumption is secure. To do so, we conducted delayed estimation experiments following standard practice with a memory load of one. We discovered that different target colors evoked response distributions that differed widely in dispersion and that these stimulus-specific response properties were correlated across observers. Subsequent experiments demonstrated that stimulus-specific responses persist under higher memory loads and that at least part of the specificity arises in perception and is eventually propagated to working memory. Posthoc stimulus measurement revealed that rendered stimuli differed from nominal stimuli in both chromaticity and luminance. We discuss the implications of these deviations for both our results and those from other working memory studies.

The Dynamic Range of Human Lightness Perception

Natural viewing challenges the visual system with images that have a dynamic range of light intensity (luminance) that can approach 1,000,000:1 and that often exceeds 10,000:1 [1, 2]. The range of perceived surface reflectance (lightness), however, can be well approximated by the Munsell matte neutral scale (N 2.0/ to N 9.5/), consisting of surfaces whose reflectance varies by about 30:1. Thus, the visual system must map a large range of surface luminance onto a much smaller range of surface lightness. We measured this mapping in images with a dynamic range close to that of natural images. We studied simple images that lacked segmentation cues that would indicate multiple regions of illumination. We found a remarkable degree of compression: at a single image location, a stimulus luminance range of 5,905:1 can be mapped onto an extended lightness scale that has a reflectance range of 100:1. We characterized how the luminance-to-lightness mapping changes with stimulus context. Our data rule out theories that predict perceived lightness from luminance ratios or Weber contrast. A mechanistic model connects our data to theories of adaptation and provides insight about how the underlying visual response varies with context.

Lightness perception in high dynamic range images: local and remote luminance effects

We measured the perceived lightness of target patches embedded in high dynamic range checkerboards. We independently varied the luminance of checks immediately surrounding the test and those remote from it. The data establish context transfer functions (CTFs) that characterize perceptual matches across checkerboard contexts. Several features of the CTFs are broadly consistent with previous research: Matched luminance decreases when overall context luminance decreases; matched luminance increases when overall context luminance increases; manipulating context locations near the target has a greater effect than manipulating locations far from the target patch. The measured CTFs are not well described, however, by changes with context in multiplicative gain alone or by changes in both multiplicative and subtractive adaptation parameters. We were able to fit the data with a three-parameter model of adaptation. This allowed us to characterize the CTFs by specifying the luminances that appeared white, black, and gray (white point, black point, and gray point, respectively). The white and black points depended additively on the local and remote contrasts, but accounting for the gray point required an interaction term. Analysis of this effect suggests that the target patch itself must be included in a description of the visual context.

Short-Term Memory Affects Color Perception in Context

Color-based object selection — for instance, looking for ripe tomatoes in the market — places demands on both perceptual and memory processes: it is necessary to form a stable perceptual estimate of surface color from a variable visual signal, as well as to retain multiple perceptual estimates in memory while comparing objects. Nevertheless, perceptual and memory processes in the color domain are generally studied in separate research programs with the assumption that they are independent. Here, we demonstrate a strong failure of independence between color perception and memory: the effect of context on color appearance is substantially weakened by a short retention interval between a reference and test stimulus. This somewhat counterintuitive result is consistent with Bayesian estimation: as the precision of the representation of the reference surface and its context decays in memory, prior information gains more weight, causing the retained percepts to be drawn toward prior information about surface and context color. This interaction implies that to fully understand information processing in real-world color tasks, perception and memory need to be considered jointly.

The central tendency bias in color perception: Effects of internal and external noise

Perceptual estimates can be biased by previously seen stimuli in delayed estimation tasks. These biases are often toward the mean of the whole stimulus set. Recently, we demonstrated such a central tendency bias in delayed color estimation. In the Bayesian framework of perceptual inference, perceptual biases arise when noisy sensory measurements are combined with prior information about the world. Here, we investigate this idea in color perception by manipulating stimulus range and stimulus noise while characterizing delayed color estimates. First, we manipulated the experimental prior for stimulus color by embedding stimuli in collections with different hue ranges. Stimulus range affected hue bias: Hue estimates were always biased toward the mean of the current set. Next, we studied the effect of internal and external noise on the amount of hue bias. Internal noise was manipulated by increasing the delay between the reference and test from 0.4 to 4 s. External noise was manipulated by increasing the amount of chromatic noise in the reference stimulus, while keeping the delay between the reference and test constant at 2 s. Both noise manipulations had a reliable effect on the strength of the central tendency bias. Furthermore, there was a tendency for a positive relationship between variability of the estimates and bias in both noise conditions. In conclusion, observers are able to learn an experimental hue prior, and the weight on the prior can be manipulated by introducing noise in the estimation process.

Cognitive Load and Strategic Sophistication

We study the relationship between the cognitive load manipulation and strategic sophistication. The cognitive load manipulation is designed to reduce the subjects cognitive resources that are available for deliberation on a choice. In our experiment, subjects are placed under a large cognitive load (given a difficult number to remember) or a low cognitive load (given a number which is not difficult to remember). Subsequently, the subjects play a one-shot game then they are asked to recall the number. This procedure is repeated for various games, where a new number is given for each game. We find a nuanced and nonmonotonic relationship between cognitive load and strategic sophistication. This relationship is consistent with two effects. First, subjects under a high cognitive load tend to exhibit behavior consistent with the reduced ability to compute the optimal decision. Second, the cognitive load tends to affect the subjects perception of their relative standing in the distribution of the available cognitive resources. The net result of these two effects depends on the strategic setting. Our experiment provides evidence on the literature which examines the relationship between measures of cognitive ability and strategic sophistication.

A Bayesian model of lightness perception that incorporates spatial variation in the illumination

The lightness of a test stimulus depends in a complex manner on the context in which it is viewed. To predict lightness, it is necessary to leverage measurements of a feasible number of contextual configurations into predictions for a wider range of configurations. Here we pursue this goal, using the idea that lightness results from the visual systems attempt to provide stable information about object surface reflectance. We develop a Bayesian algorithm that estimates both illumination and reflectance from image luminance, and link perceived lightness to the algorithms estimates of surface reflectance. The algorithm resolves ambiguity in the image through the application of priors that specify what illumination and surface reflectances are likely to occur in viewed scenes. The prior distributions were chosen to allow spatial variation in both illumination and surface reflectance. To evaluate our model, we compared its predictions to a data set of judgments of perceived lightness of test patches embedded in achromatic checkerboards (Allred, Radonjić, Gilchrist, & Brainard, 2012). The checkerboard stimuli incorporated the large variation in luminance that is a pervasive feature of natural scenes. In addition, the luminance profile of the checks both near to and remote from the central test patches was systematically manipulated. The manipulations provided a simplified version of spatial variation in illumination. The model can account for effects of overall changes in image luminance and the dependence of such changes on spatial location as well as some but not all of the more detailed features of the data.

Relating color working memory and color perception

Color is the most frequently studied feature in visual working memory (VWM). Oddly, much of this work de-emphasizes perception, instead making simplifying assumptions about the inputs served to memory. We question these assumptions in light of perception research, and we identify important points of contact between perception and working memory in the case of color. Better characterization of its perceptual inputs will be crucial for elucidating the structure and function of VWM.

The effect of memory and context changes on color matches to real objects

Real-world color identification tasks often require matching the color of objects between contexts and after a temporal delay, thus placing demands on both perceptual and memory processes. Although the mechanisms of matching colors between different contexts have been widely studied under the rubric of color constancy, little research has investigated the role of long-term memory in such tasks or how memory interacts with color constancy. To investigate this relationship, observers made color matches to real study objects that spanned color space, and we independently manipulated the illumination impinging on the objects, the surfaces in which objects were embedded, and the delay between seeing the study object and selecting its color match. Adding a 10-min delay increased both the bias and variability of color matches compared to a baseline condition. These memory errors were well accounted for by modeling memory as a noisy but unbiased version of perception constrained by the matching methods. Surprisingly, we did not observe significant increases in errors when illumination and surround changes were added to the 10-minute delay, although the context changes alone did elicit significant errors.

The effect of background and illumination on color identification of real, 3D objects

For the surface reflectance of an object to be a useful cue to object identity, judgments of its color should remain stable across changes in the objects environment. In 2D scenes, there is general consensus that color judgments are much more stable across illumination changes than background changes. Here we investigate whether these findings generalize to real 3D objects. Observers made color matches to cubes as we independently varied both the illumination impinging on the cube and the 3D background of the cube. As in 2D scenes, we found relatively high but imperfect stability of color judgments under an illuminant shift. In contrast to 2D scenes, we found that background had little effect on average color judgments. In addition, variability of color judgments was increased by an illuminant shift and decreased by embedding the cube within a background. Taken together, these results suggest that in real 3D scenes with ample cues to object segregation, the addition of a background may improve stability of color identification.