Katherine R. Storrs

Are high-level aftereffects perceptual?

Imagineanexperimentinwhichyoushowsomeonepicturesofacomputer-generatedface displaying random expressions, rang-ing from happy, through neutral, to sad.You tell the participant that she must clas-sify each picture as being either “happy”or “sad,” and you note the point onthe expression continuum at which sheswitches from mostly-“happy” classifica-tions to mostly-“sad.” Then, you ask herto repeat the task, but between each pic-ture you have her feel, with her hands butout of sight, the contours of a smiling facemask. You find that her category bound-ary has shifted: she now classifies more ofthe pictures as “sad.” This result (reportedby Matsumiya, 2013)isanexampleofan aftereffect, in which adaptation to oneinput (the mask) has altered responsesto subsequent inputs (the images). It is“high-level” in the sense that the adaptingand test stimuli have little overlap in theirinitial sensory encoding (they are pre-sentedinseparatemodalities).Thereareatleast two ways to interpret this finding.

Not all face aftereffects are equal

After prolonged exposure to a female face, faces that had previously seemed androgynous are more likely to be judged as male. Similarly, after prolonged exposure to a face with expanded features, faces that had previously seemed normal are more likely to be judged as having contracted features. These facial aftereffects have both been attributed to the impact of adaptation upon a norm-based opponent code, akin to low-level analyses of colour. While a good deal of evidence is consistent with this, some recent data is contradictory, motivating a more rigorous test. In behaviourally matched tasks we compared the characteristics of aftereffects generated by adapting to colour, to expanded or contracted faces, and to male or female faces. In our experiments opponent coding predicted that the appearance of the adapting image should change and that adaptation should induce symmetrical shifts of two category boundaries. This combination of predictions was firmly supported for colour adaptation, somewhat supported for facial distortion aftereffects, but not supported for facial gender aftereffects. Interestingly, the two face aftereffects we tested generated discrepant patterns of response shifts. Our data suggest that superficially similar aftereffects can ensue from mechanisms that differ qualitatively, and therefore that not all high-level categorical face aftereffects can be attributed to a common coding strategy.

Face aftereffects involve local repulsion, not renormalization

After looking at a photograph of someone for a protracted period (adaptation), a previously neutral-looking face can take on an opposite appearance in terms of gender, identity, and other attributes-but what happens to the appearance of other faces? Face aftereffects have repeatedly been ascribed to perceptual renormalization. Renormalization predicts that the adapting face and more extreme versions of it should appear more neutral after adaptation (e.g., if the adaptor was male, it and hyper-masculine faces should look more feminine). Other aftereffects, such as tilt and spatial frequency, are locally repulsive, exaggerating differences between adapting and test stimuli. This predicts that the adapting face should be little changed in appearance after adaptation, while more extreme versions of it should look even more extreme (e.g., if the adaptor was male, it should look unchanged, while hyper-masculine faces should look even more masculine). Existing reports do not provide clear evidence for either pattern. We overcame this by using a spatial comparison task to measure the appearance of stimuli presented in differently adapted retinal locations. In behaviorally matched experiments we compared aftereffect patterns after adapting to tilt, facial identity, and facial gender. In all three experiments data matched the predictions of a locally repulsive, but not a renormalizing, aftereffect. These data are consistent with the existence of similar encoding strategies for tilt, facial identity, and facial gender.

Sociality of facial expressions in immersive virtual environments: a facial EMG study.

Immersive virtual environment technology is increasingly used by psychologists as a tool for researching social influence in realistic, yet experimentally controllable, settings. The present study demonstrates the validity and reliability of facial electromyography as a marker of affect in immersive virtual environments and further shows that the mere presence of virtual humans is enough to elicit sociality effects on facial expressiveness. Participants viewed pleasant and unpleasant images in a virtual room either alone or with two virtual humans present. The patterns of smiling and frowning activity elicited by positive and negative stimuli in the virtual environment were the same as those found in laboratory settings. Moreover, when viewing positive stimuli, smiling activity was greater when two agents were present than in the alone condition. The results provide new psychophysiological evidence for the potency of social agents in immersive virtual environments.

Shape Aftereffects Reflect Shape Constancy Operations: Appearance Matters

One of the oldest known visual aftereffects is the shape aftereffect, wherein looking at a particular shape can make subsequent shapes seem distorted in the opposite direction. After viewing a narrow ellipse, for example, a perfect circle can look like a broad ellipse. It is thought that shape aftereffects are determined by the dimensions of successive retinal images. However, perceived shape is invariant for large retinal image changes resulting from different viewing angles; current understanding suggests that shape aftereffects should not be impacted by the operations responsible for this viewpoint invariance. By viewing adaptors from an angle, with subsequent frontoparallel tests, we establish that shape aftereffects are not solely determined by the dimensions of successive retinal images. Moreover, by comparing performance with and without stereo surface slant cues, we show that shape aftereffects reflect a weighted function of retinal image shape and surface slant information, a hallmark of shape constancy operations. Thus our data establish that shape aftereffects can be influenced by perceived shape, as determined by constancy operations, and must therefore involve higher-level neural substrates than previously thought.

Shape adaptation exaggerates shape differences

Adaptation to different visual properties can produce distinct patterns of perceptual aftereffect. Some, such as those following adaptation to color, seem to arise from recalibrative processes. These are associated with a reappraisal of which physical input constitutes a normative value in the environment—in this case, what appears “colorless,” and what “colorful.” Recalibrative aftereffects can arise from coding schemes in which inputs are referenced against malleable norm values. Other aftereffects seem to arise from contrastive processes. These exaggerate differences between the adaptor and other inputs without changing the adaptor’s appearance. There has been conjecture over which process best describes adaptation-induced distortions of spatial vision, such as of apparent shape or facial identity. In 3 experiments, we determined whether recalibrative or contrastive processes underlie the shape aspect ratio aftereffect. We found that adapting to a moderately elongated shape compressed the appearance of narrower shapes and further elongated the appearance of more-elongated shapes (Experiment 1). Adaptation did not change the perceived aspect ratio of the adaptor itself (Experiment 2), and adapting to a circle induced similar bidirectional aftereffects on shapes narrower or wider than circular (Experiment 3). Results could not be explained by adaptation to retinotopically local edge orientation or single linear dimensions of shapes. We conclude that aspect ratio aftereffects are determined by contrastive processes that can exaggerate differences between successive inputs, inconsistent with a norm-referenced representation of aspect ratio. Adaptation might enhance the salience of novel stimuli rather than recalibrate one’s sense of what constitutes a “normal” shape.