Elise A. Piazza

The perceptual basis of common photographic practice

Photographers, cinematographers, and computer-graphics engineers use certain techniques to create striking pictorial effects. By using lenses of different focal lengths, they can make a scene look compressed or expanded in depth, make a familiar object look natural or distorted, or make a person look smarter, more attractive, or more neurotic. We asked why pictures taken with a certain focal length look natural, while those taken with other focal lengths look distorted. We found that peoples preferred viewing distance when looking at pictures leads them to view long-focal-length pictures from too near and short-focal-length pictures from too far. Perceptual distortions occur because people do not take their incorrect viewing distances into account. By following the rule of thumb of using a 50-mm lens, photographers greatly increase the odds of a viewer looking at a photograph from the correct distance, where the percept will be undistorted. Our theory leads to new guidelines for creating pictorial effects that are more effective than conventional guidelines.

Humans Use Summary Statistics to Perceive Auditory Sequences

In vision, humans use summary statistics (e.g., the average facial expression of a crowd) to efficiently perceive the gist of groups of features. Here, we present direct evidence that ensemble coding is also important for auditory processing. We found that listeners could accurately estimate the mean frequency of a set of logarithmically spaced pure tones presented in a temporal sequence (Experiment 1). Their performance was severely reduced when only a subset of tones from a given sequence was presented (Experiment 2), which demonstrates that ensemble coding is based on a substantial number of the tones in a sequence. This precise ensemble coding occurred despite very limited representation of individual tones from the sequence: Listeners were poor at identifying specific individual member tones (Experiment 3) and at determining their positions in the sequence (Experiment 4). Together, these results indicate that summary statistical coding is not limited to visual processing and is an important auditory mechanism for extracting ensemble frequency information from sequences of sounds.

Measuring speaker–listener neural coupling with functional near infrared spectroscopy

The present study investigates brain-to-brain coupling, defined as inter-subject correlations in the hemodynamic response, during natural verbal communication. We used functional near-infrared spectroscopy (fNIRS) to record brain activity of 3 speakers telling stories and 15 listeners comprehending audio recordings of these stories. Listeners’ brain activity was significantly correlated with speakers’ with a delay. This between-brain correlation disappeared when verbal communication failed. We further compared the fNIRS and functional Magnetic Resonance Imaging (fMRI) recordings of listeners comprehending the same story and found a significant relationship between the fNIRS oxygenated-hemoglobin concentration changes and the fMRI BOLD in brain areas associated with speech comprehension. This correlation between fNIRS and fMRI was only present when data from the same story were compared between the two modalities and vanished when data from different stories were compared; this cross-modality consistency further highlights the reliability of the spatiotemporal brain activation pattern as a measure of story comprehension. Our findings suggest that fNIRS can be used for investigating brain-to-brain coupling during verbal communication in natural settings.

Predictive context influences perceptual selection during binocular rivalry

Prediction may be a fundamental principle of sensory processing: it has been proposed that the brain continuously generates predictions about forthcoming sensory information. However, little is known about how prediction contributes to the selection of a conscious percept from among competing alternatives. Here, we used binocular rivalry to investigate the effects of prediction on perceptual selection. In binocular rivalry, incompatible images presented to the two eyes result in a perceptual alternation between the images, even though the visual stimuli remain constant. If predictive signals influence the competition between neural representations of rivalrous images, this influence should generate a bias in perceptual selection that depends on predictive context. To manipulate predictive context, we developed a novel binocular rivalry paradigm in which rivalrous test images were immediately preceded by a sequence of context images presented identically to the two eyes. One of the test images was consistent with the preceding image sequence (it was the expected next image in the series), and the other was inconsistent (non-predicted). We found that human observers were more likely to perceive the consistent image at the onset of rivalry, suggesting that predictive context biased selection in favor of the predicted percept. This prediction effect was distinct from the effects of adaptation to stimuli presented before the binocular rivalry test. In addition, perceptual reports were speeded for predicted percepts relative to non-predicted percepts. These results suggest that predictive signals related to visual stimulus history exist at neural sites that can bias conscious perception during binocular rivalry. Our paradigm provides a new way to study how prior information and incoming sensory information combine to generate visual percepts.

Persistent hemispheric differences in the perceptual selection of spatial frequencies

Previous research has shown that the right hemisphere processes low spatial frequencies more efficiently than the left hemisphere, which preferentially processes high spatial frequencies. These studies have typically measured RTs to single, briefly flashed gratings and/or have directed observers to attend to a particular spatial frequency immediately before making a judgment about a subsequently presented stimulus. Thus, it is unclear whether the hemispheres differ in perceptual selection from multiple spatial frequencies that are simultaneously present in the environment, without bias from selective attention. Moreover, the time course of hemispheric asymmetry in spatial frequency processing is unknown. We addressed both of these questions with binocular rivalry, a measure of perceptual selection from competing alternatives over time. Participants viewed a pair of rivalrous orthogonal gratings with different spatial frequencies, presented either to the left or right of central fixation, and continuously reported which grating they perceived. At the beginning of a trial, the low spatial frequency grating was perceptually selected more often when presented in the left hemifield (right hemisphere) than in the right hemifield (left hemisphere), whereas the high spatial frequency grating showed the opposite pattern of results. This hemispheric asymmetry in perceptual selection persisted for the entire 30-sec stimulus presentation, continuing long after stimulus onset. These results indicate stable differences in the resolution of ambiguity across spatial locations and demonstrate the importance of considering sustained differences in perceptual selection across space when characterizing conscious representations of complex scenes.

Camera Focal Length and the Perception of Pictures

Photographers, cinematographers, and computer-graphics engineers use certain techniques to create striking pictorial effects. By using lenses of different focal lengths, they can make a scene look compressed or expanded in depth, make a familiar object look natural or distorted, or make a person look smarter, more attractive, or more neurotic. Photographers have a rule of thumb that a 50 mm lens produces natural-looking pictures. We asked why pictures taken with a 50 mm lens look natural, while those taken with other focal lengths look distorted. We found that peoples preferred viewing distance when looking at pictures leads them to view long-focal-length pictures from too near and short-focal-length pictures from too far. Perceptual distortions occur because people do not take their incorrect viewing distances into account. By following the rule of thumb of using a 50 mm lens, photographers greatly increase the odds of a viewer looking at a photograph from the correct distance, where the percept will be undistorted. Our theory leads to new guidelines for creating pictorial effects that are more effective than conventional guidelines.

Infant and adult brains are coupled to the dynamics of natural communication

Infancy is the foundational period for learning from adults, and the dynamics of the social environment have long been proposed as central to children’s development. Here we reveal a novel, highly naturalistic approach for studying live interactions between infants and adults. Using functional near-infrared spectroscopy (fNIRS), we simultaneously and continuously measured the brains of infants (9-15 months) and an adult while they communicated and played with each other in real time. We found that time-locked neural coupling within dyads was significantly greater when they interacted with each other than with control individuals. In addition, we found that both infant and adult brains continuously tracked the moment-to-moment fluctuations of mutual gaze, infant emotion, and adult speech prosody with high temporal precision. This investigation advances what is currently known about how the brains and behaviors of infants both shape and reflect those of adults during real-life communication.

Rapid Adaptation to the Timbre of Natural Sounds

Timbre, the unique quality of a sound that points to its source, allows us to quickly identify a loved one’s voice in a crowd and distinguish a buzzy, bright trumpet from a warm cello. Despite its importance for perceiving the richness of auditory objects, timbre is a relatively poorly understood feature of sounds. Here we demonstrate for the first time that listeners adapt to the timbre of a wide variety of natural sounds. For each of several sound classes, participants were repeatedly exposed to two sounds (e.g., clarinet and oboe, male and female voice) that formed the endpoints of a morphed continuum. Adaptation to timbre resulted in consistent perceptual aftereffects, such that hearing sound A significantly altered perception of a neutral morph between A and B, making it sound more like B. Furthermore, these aftereffects were robust to moderate pitch changes, suggesting that adaptation to timbral features used for object identification drives these effects, analogous to face adaptation in vision.

Relative spatial frequency processing drives hemispheric asymmetry in conscious awareness

Visual stimuli with different spatial frequencies (SFs) are processed asymmetrically in the two cerebral hemispheres. Specifically, low SFs are processed relatively more efficiently in the right hemisphere than the left hemisphere, whereas high SFs show the opposite pattern. In this study, we ask whether these differences between the two hemispheres reflect a low-level division that is based on absolute SF values or a flexible comparison of the SFs in the visual environment at any given time. In a recent study, we showed that conscious awareness of SF information (i.e., visual perceptual selection from multiple SFs simultaneously present in the environment) differs between the two hemispheres. Building upon that result, here we employed binocular rivalry to test whether this hemispheric asymmetry is due to absolute or relative SF processing. In each trial, participants viewed a pair of rivalrous orthogonal gratings of different SFs, presented either to the left or right of central fixation, and continuously reported which grating they perceived. We found that the hemispheric asymmetry in perception is significantly influenced by relative processing of the SFs of the simultaneously presented stimuli. For example, when a medium SF grating and a higher SF grating were presented as a rivalry pair, subjects were more likely to report that they initially perceived the medium SF grating when the rivalry pair was presented in the left visual hemifield (right hemisphere), compared to the right hemifield. However, this same medium SF grating, when it was paired in rivalry with a lower SF grating, was more likely to be perceptually selected when it was in the right visual hemifield (left hemisphere). Thus, the visual system’s classification of a given SF as “low” or “high” (and therefore, which hemisphere preferentially processes that SF) depends on the other SFs that are present, demonstrating that relative SF processing contributes to hemispheric differences in visual perceptual selection.

Implicit multisensory statistical learning influences visual perceptual selection

Subjects selected from all 6 images (simultaneously presented and randomly ordered on the screen). Subjects demonstrate above-chance recognition of the auditory-visual pairings immediately after the end of the exposure period, but it varies across sounds and images. Rivalrous images were more likely to be initially selected when preceded by their matching sound than when preceded by non-matching sounds. Relationship Between Recognition Accuracy and Rivalry E ect