Wilma A. Bainbridge

The intrinsic memorability of face photographs.

The faces we encounter throughout our lives make different impressions on us: Some are remembered at first glance, while others are forgotten. Previous work has found that the distinctiveness of a face influences its memorability--the degree to which face images are remembered or forgotten. Here, we generalize the concept of face memorability in a large-scale memory study. First, we find that memorability is an intrinsic feature of a face photograph--across observers some faces are consistently more remembered or forgotten than others--indicating that memorability can be used for measuring, predicting, and manipulating subsequent memories. Second, we determine the role that 20 personality, social, and memory-related traits play in face memorability. Whereas we find that certain traits (such as kindness, atypicality, and trustworthiness) contribute to face memorability, they do not suffice to explain the variance in memorability scores, even when accounting for noise and differences in subjective experience. This suggests that memorability itself is a consistent, singular measure of a face that cannot be reduced to a simple combination of personality and social facial attributes. We outline modern neuroscience questions that can be explored through the lens of memorability.

Interaction envelope: Local spatial representations of objects at all scales in scene-selective regions

While several cortical regions have been highlighted for their category selectivity (e.g., scene-selective regions like the parahippocampal place area, object selective regions like the lateral occipital complex), a growing trend in cognitive neuroscience has been to investigate what particular perceptual properties these regions calculate. Classical scene-selective regions have been particularly targeted in recent work as being sensitive to object size or other related properties. Here we test to which extent these regions are sensitive to spatial information of stimuli at any size. We introduce the spatial object property of “interaction envelope,” defined as the space through which a user trans-verses to interact with an object. In two functional magnetic resonance imaging experiments, we examined activity in a comprehensive set of perceptual regions of interest for when human participants viewed object images varying along the dimensions of interaction envelope and physical size. Importantly, we controlled for confounding perceptual and semantic object properties. We find that scene-selective regions are in fact sensitive to object interaction envelope for small, manipulable objects regardless of real-world size and task. Meanwhile, small-scale entity regions maintain selectivity to stimulus physical size. These results indicate that regions traditionally associated with scene processing may not be solely sensitive to larger object and scene information, but instead are calculating local spatial information of objects and scenes of all sizes.

A toolbox and sample object perception data for equalization of natural images

For psychologists and neuroscientists, careful selection of their stimuli is essential, so that low-level visual features such as color or spatial frequency do not serve as confounds between conditions of interest. Here, we detail the Natural Image Statistical Toolbox, which allows scientists to measure, visualize, and control stimulus sets along a set of low-level visual properties. Additionally, we provide a set of object images varying along several perceptual object properties, including physical size and interaction envelope size (i.e., the space around an object transversed during an interaction), serving as a test-bed for the Natural Image Statistical Toolbox. This stimulus set is also a highly characterized set useful to psychology and neuroscience studies on object perception.

Memorability: A stimulus-driven perceptual neural signature distinctive from memory

ABSTRACT A long‐standing question in neuroscience is how perceptual processes select stimuli for encoding and later retrieval by memory processes. Using a functional magnetic resonance imaging study with human participants, we report the discovery of a global, stimulus‐driven processing stream that we call memorability. Memorability automatically tags the statistical distinctiveness of stimuli for later encoding, and shows separate neural signatures from both low‐level perception (memorability shows no signal in early visual cortex) and classical subsequent memory based on individual memory. Memorability and individual subsequent memory show dissociable neural substrates: first, memorability effects consistently emerge in the medial temporal lobe (MTL), whereas individual subsequent memory effects emerge in the prefrontal cortex (PFC). Second, memorability effects remain consistent even in the absence of memory (i.e., for forgotten images). Third, the MTL shows higher correlations with memorability‐based patterns, while the PFC shows higher correlations with individual memory voxels patterns. Taken together, these results support a reformulated framework of the interplay between perception and memory, with the MTL determining stimulus statistics and distinctiveness to support later memory encoding, and the PFC comparing stimuli to specific individual memories. As stimulus memorability is a confound present in many previous memory studies, these findings should stimulate a revisitation of the neural streams dedicated to perception and memory. HighlightsMemorability is an intrinsic perceptual property with dedicated neural signals.Ventral visual stream, medial temporal lobe sensitive to face, scene memorability.Early visual cortex and attention regions show no memorability sensitivity.Memorability effects exist even in the absence of memory (i.e., forgotten images).Dissociation of neural patterns and loci between memorability and memory.

From what we perceive to what we remember: Characterizing representational dynamics of visual memorability

Not all visual memories are equal—some endure in our minds, while others quickly disappear. Recent behavioral work shows we can reliably predict which images will be remembered. This image property is called memorability. Memorability is intrinsic to an image, robust across observers, and unexplainable by low-level visual features. However, its neural bases and relation with perception and memory remain unknown. Here we characterize the representational dynamics of memorability using magnetoencephalography (MEG). We find memorability is indexed by brain responses starting at 218ms for faces and 371ms for scenes—later than classical early face/scene discrimination perceptual signals, yet earlier than the late memory encoding signal observed at ~700ms. The results show memorability is a high-level image property whose spatio-temporal neural dynamics are different from those of memory encoding. Together, this work brings new insights into the underlying neural processes of the transformation from what we perceive to what we remember.

Neural Signatures of Visual Memorability: Memory in the First Perception of an Image

Whereas some places or people leave a memorable first impression, others are immediately forgotten. Recent work has shown that memorability of scene and face pictures is highly consistent across people, providing a basis to predict later memory behavior (Bainbridge, 2013; Isola, 2011). Here, we investigate the neural signatures of memorability during the first perception of an image. In two fMRI experiments, participants were shown blocks of novel images grouped by stimulus type (face or scene) and memorability level (high or low memorability). Stimuli were carefully controlled for attributes including gender, race, attractiveness and emotional content for faces, and indoor/outdoor, natural/manmade and category type for scenes, as well as a range of low-level image statistics for both. To validate the robustness of the findings, different sets of participants performed a 1-back task in Experiment 1 (N=24), and a perceptual task in Experiment 2 (N=13). None were told about the memory-related nature of the study. For each participant, independent functional localizers were used to localize perceptual regions, and regions in their medial-temporal lobe (MTL) were segmented using anatomical landmarks. Wholebrain analyses, multivariate analyses, and region of interest analyses pinpointed areas of responsiveness to memorable versus forgettable images. In both experiments, signatures of memorability were not found in low-level visual areas, but were consistently found in several perceptual regions specific to faces and scenes. Different regions in the MTL (e.g., the perirhinal cortex) also show preferential activity for memorable images, regardless of the stimulus type. Multivoxel pattern analyses reveal pattern encoding of more memorable items, after a single exposure, in the hippocampus. These results show that signatures of memorability of an image can be found both in ventral neocortical and medial temporal lobe regions, questioning to which extent perception and memory representations are separated in the brain.

What makes a picture memorable

‘Mental sports’ have become a new trend in self-improvement, with video games designed to improve mental fitness. At the World Memory Championships, athletes compete to recall massive amounts of information; contestants must memorize and recall sequences of abstract images and the names of people whose faces are shown in photographs. While these tasks might seem challenging, our research suggests that images that possess certain properties are memorable. Our findings can explain why we have all had some images stuck in our minds, but ignored or quickly forgotten others. Although image memorability seems subjective and hard to quantify, our recent work1–5 shows that it is not an inexplicable phenomenon. We found that visual memorability is largely intrinsic to the image and reproducible across a diverse population. This means that despite varied experiences, individuals tend to remember and forget the same images. Using experimental data detailing the types of images people remember or quickly forget, we developed an algorithm that automatically predicts whether an image will be memorable. To determine the intrinsic features that make an image memorable, we first asked 665 individuals to participate in a computer memory game. During each level of the game, for up to 30 levels, participants viewed a stream of images and then pressed the space bar whenever they saw one of those images repeated in a subsequent sequence. In total, the image database contained 2222 repeated images and 8220 unrepeated images that included faces, interior-design photos, nature scenes, streetscapes, and others. We found that photographs with people or central objects were memorable, whereas landscapes—that one might expect to be memorable—were among the most forgettable (see Figure 1).1 Next, we assigned a ‘memorability score’ to each image, which was defined as the percent of correct detections by participants in the study. On average, 78 participants scored each Figure 1. A sample of the photos used to develop an algorithm that predicts image memorability.1 The photos are scaled according to their experimentally measured memorability, with the most memorable photos appearing larger and in the center.

Dissociating neural markers of stimulus memorability and subjective recognition during episodic retrieval

While much of memory research takes an observer-centric focus looking at participant performance, recent work has pinpointed important item-centric effects on memory, or how intrinsically memorable a given stimulus is. However, little is known about the neural correlates of memorability during memory retrieval, or how such correlates relate to subjective memory behavior. Here, stimuli and blood-oxygen-level dependent data from a prior functional magnetic resonance imaging (fMRI) study were reanalyzed using a memorability-based framework. In that study, sixteen participants studied 200 novel face images and were scanned while making recognition memory judgments on those faces, interspersed with 200 unstudied faces. In the current investigation, memorability scores for those stimuli were obtained through an online crowd-sourced (N = 740) continuous recognition test that measured each image’s corrected recognition rate. Representational similarity analyses were conducted across the brain to identify regions wherein neural pattern similarity tracked item-specific effects (stimulus memorability) versus observer-specific effects (individual memory performance). We find two non-overlapping sets of regions, with memorability-related information predominantly represented within ventral and medial temporal regions and memory retrieval outcome-related information within fronto-parietal regions. These memorability-based effects persist regardless of image history, implying that coding of stimulus memorability may be a continuous and automatic perceptual process.

The Intrinsic Memorability of Face Identities

Some people in our lives stick in our memories while others are instantly forgotten. Memorability — whether a stimulus is likely to be later remembered—is found to be highly consistent for face images; people tend to remember and forget the same images. However, is memorability also intrinsic to a facial identity, generalizable across different images of a single person? 5,210 participants completed an online memory experiment testing face identity recognition over five different emotional and viewpoint transformations (neutral, happy, angry, 3/4 view, and pro-file view). Participants saw a stream of novel face images and indicated when they saw a repeated person, regardless of if the image was different or not . Memorability was found to be highly consistent within each image, as well as when training and testing on different images of the same person . Memorability ranking was also consistent across transformations—if a face was remembered in one image, it was also likely to be remembered in another. These results held true over both emotion expression transformations as well as viewpoint transformations. In addition, an asymmetry was found where encoding neutral (forward-facing, neutral emotion) images did not diminish memory performance, while encoding transformations did, hinting towards a prototype-based face memory representation. In whole, these results provide first evidence for an intrinsic, core memorability to a person or entity, beyond specific images.

Temporal dynamics of memorability: an intrinsic brain signal distinct from memory

Can we predict what people will remember, as they are perceiving an image? Recent work has identified that images carry the attribute of memorability, a predictive value of whether a novel image will be later remembered or forgotten (Isola et al . 2011, 2014; Bainbridge et al . 2013) . Despite the separate subjective experiences people have, certain faces and scenes are consistently remembered and others forgotten, independent of observer . Whereas many studies have concentrated on an observer-centric predictor of memory (e .g . Kuhl et al . 2012), memorability is a complementary, stimulus-centric predictor, generalizable across observers and context. How is memorability manifested in the brain, and how does it differ from pure memory encoding? In this study we characterized temporal dynamics of memorability, and showed that magnetoencephalography (MEG) brain signals are predictive of memorability . We further showed that the neural signature of memorability exists for both faces and scenes; however each of them has its own specific temporal dynamics. Faces showed a persistent memorability signal whereas scenes had more transient characteristics . We also found that neural signatures of memorability across time are different from that of memory encoding, as measured by a post-MEG memory recognition task. This work is the first to measure memorability, as an innate property of images, from electrophysiological brain signals and characterize its temporal dynamics .