Michèle Fabre-Thorpe

A Limit to the Speed of Processing in Ultra-Rapid Visual Categorization of Novel Natural Scenes

The processing required to decide whether a briefly flashed natural scene contains an animal can be achieved in 150 msec (Thorpe, Fize, & Marlot, 1996). Here we report that extensive training with a subset of photographs over a 3-week period failed to increase the speed of the processing underlying such Rapid Visual Categorizations: Completely novel scenes could be categorized just as fast as highly familiar ones. Such data imply that the visual system processes new stimuli at a speed and with a number of stages that cannot be compressed. This rapid processing mode was seen with a wide range of visual complex images, challenging the idea that short reaction times can only be seen with simple visual stimuli and implying that highly automatic feed-forward mechanisms underlie a far greater proportion of the sophisticated image analysis needed for everyday vision than is generally assumed.

Parallel processing in high-level categorization of natural images.

Models of visual processing often include an initial parallel stage that is restricted to relatively low-level features, whereas activation of higher-level object descriptions is generally assumed to require attention. Here we report that even high-level object representations can be accessed in parallel: in a rapid animal versus non-animal categorization task, both behavioral and electrophysiological data show that human subjects were as fast at responding to two simultaneously presented natural images as they were to a single one. The implication is that even complex natural images can be processed in parallel without the need for sequential focal attention.

Rapid categorization of natural images by rhesus monkeys

TWO rhesus macaques were tested on a categorization task in which they had to classify previously unseen photographs flashed for only 80 ms. One monkey was trained to respond to the presence of an animal, the second to the presence of food. Although the monkeys were not quite as accurate as humans tested on the same material, they nevertheless performed this very challenging visual task remarkably well. Furthermore, their reaction times were considerably shorter than even the fastest human subject. Such data, combined with the detailed knowledge of the monkeys visual system, provide a severe challenge to current theories of visual processing. They also argue that this form of rapid visual categorization is fundamentally similar in both monkeys and humans.

The time course of visual processing: Backward masking and natural scene categorisation

Human observers are very good at deciding whether briefly flashed novel images contain an animal and previous work has shown that the underlying visual processing can be performed in under 150 ms. Here we used a masking paradigm to determine how information accumulates over time during such high-level categorisation tasks. As the delay between test image and mask is increased, both behavioural accuracy and differential ERP amplitude rapidly increase to reach asymptotic levels around 40-60 ms. Such results imply that processing at each stage in the visual system is remarkably rapid, with information accumulating almost continuously following the onset of activation.

Is it an animal? Is it a human face? Fast processing in upright and inverted natural scenes

Object categorization can be extremely fast. But among all objects, human faces might hold a special status that could depend on a specialized module. Visual processing could thus be faster for faces than for any other kind of object. Moreover, because face processing might rely on facial configuration, it could be more disrupted by stimulus inversion. Here we report two experiments that compared the rapid categorization of human faces and animals or animal faces in the context of upright and inverted natural scenes. In Experiment 1, the natural scenes contained human faces and animals in a full range of scales from close-up to far views. In Experiment 2, targets were restricted to close-ups of human faces and animal faces. Both experiments revealed the remarkable object processing efficiency of our visual system and further showed (1) virtually no advantage for faces over animals; (2) very little performance impairment with inversion; and (3) greater sensitivity of faces to inversion. These results are interpreted within the framework of a unique system for object processing in the ventral pathway. In this system, evidence would accumulate very quickly and efficiently to categorize visual objects, without involving a face module or a mental rotation mechanism. It is further suggested that rapid object categorization in natural scenes might not rely on high-level features but rather on features of intermediate complexity.

Detection of animals in natural images using far peripheral vision

It is generally believed that the acuity of the peripheral visual field is too poor to allow accurate object recognition and, that to be identified, most objects need to be brought into foveal vision by using saccadic eye movements. However, most measures of form vision in the periphery have been done at eccentricities below 10° and have used relatively artificial stimuli such as letters, digits and compound Gabor patterns. Little is known about how such data would apply in the case of more naturalistic stimuli. Here humans were required to categorize briefly flashed (28 ms) unmasked photographs of natural scenes (39° high, and 26° across) on the basis of whether or not they contained an animal. The photographs appeared randomly in nine locations across virtually the entire extent of the horizontal visual field. Accuracy was 93.3% for central vision and decreased almost linearly with increasing eccentricity (89.8% at 13°, 76.1% at 44.5° and 71.2% at 57.5°). Even at the most extreme eccentricity, where the images were centred at 70.5°, subjects scored 60.5% correct. No evidence was found for hemispheric specialization. This level of performance was achieved despite the fact that the position of the image was unpredictable, ruling out the use of precued attention to target locations. The results demonstrate that even high‐level visual tasks involving object vision can be performed using the relatively coarse information provided by the peripheral retina.

How parallel is visual processing in the ventral pathway

Visual object perception is usually studied by presenting one object at a time at the fovea. However, the world around us is composed of multiple objects. The way our visual system deals with this complexity has remained controversial in the literature. Some models claim that the ventral pathway, a set of visual cortical areas responsible for object recognition, can process only one or very few objects at a time without ambiguity. Other models argue in favor of a massively parallel processing of objects in a scene. Recent experiments in monkeys have provided important data about this issue. The ventral pathway seems to be able to perform complex analyses on several objects simultaneously, but only during a short time period. Subsequently only one or very few objects are explicitly selected and consciously perceived. Here, we survey the implications of these new findings for our understanding of object processing.

Ultra-rapid categorisation of natural scenes does not rely on colour cues: a study in monkeys and humans.

In a rapid categorisation task, monkeys and humans had to detect a target (animal or food) in briefly flashed (32 ms) and previously unseen natural images. Removing colour cues had very little effect on average performance. Impairments were restricted to a mild accuracy drop (in some human subjects) and a small reaction time mean increase (10-15 ms) observed both in monkeys and humans but only in the detection of food targets. In both tasks, accuracy and latency of the fastest behavioural responses were unaffected, suggesting that such ultra-rapid categorizations could depend on feed-forward processing of early coarse achromatic magnocellular information.

How long to get to the gist of real-world natural scenes?

This study aimed at assessing the processing time of a natural scene in a fast categorization task of its context or “gist”. In Experiment 1, human subjects performed 4 go/no-go categorization tasks in succession with colour pictures of real-world scenes belonging to 2 natural categories: “Sea” and “mountain”, and 2 artificial categories: “Indoor” and “urban”. Experiment 2 used colour and grey-level scenes in the same tasks to assess the role of colour cues on performance. Pictures were flashed for 26 ms. Both experiments showed that the gist of real-world scenes can be extracted with high accuracy (>90%), short median RT (400-460 ms) and early responses triggered with latencies as short as 260-300 ms. Natural scenes were processed faster than artificial scenes. Categories for which colour could have a diagnostic value were processed faster in colour than in grey. Finally, processing speed is compared for scene and object categorization tasks.

Interaction of the Amygdala with the Frontal Lobe in Reward Memory

Five cynomolgus monkeys (Macaca fascicularis) were assessed for their ability to associate visual stimuli with food reward. They learned a series of new two‐choice visual discriminations between coloured patterns displayed on a touch‐sensitive monitor screen; the feedback for correct choice was delivery of food. Normal learning in this task is known to be dependent on the amygdala. The monkeys received brain lesions which were designed to disconnect the amygdala from interaction with other brain structures thought to be involved in this memory task. All the monkeys received an amygdalectomy in one hemisphere and lesions in the other hemisphere of some of the projection targets of the amygdala, namely the ventral striatum, the mediodorsal thalamus and the ventromedial prefrontal cortex. The rate of learning new problems was assessed before and after each operation. Disconnection of the amygdala from the ventral striatum was without effect on learning rate. An earlier study had shown that disconnection of the amygdala from either the mediodorsal thalamus or the ventromedial prefrontal cortex produced only a mild impairment, significantly less severe than that produced by bilateral lesions of any of these three structures. The present results show, however, that disconnection of the amygdala from both the mediodorsal thalamus and the ventromedial prefrontal cortex in the same animal, by crossed unilateral lesions of the amygdala in one hemisphere and of both the mediodorsal thalamus and the ventromedial prefrontal cortex in the other hemisphere, produces an impairment as severe as that which follows bilateral lesions of any of these three structures. These results show that, in stimulus – reward associative memory, the role of the amygdala is entirely dependent on its interaction with the frontal lobe, either by direct projections or by indirect subcortical pathways including the mediodorsal nucleus of the thalamus; and that there are at least two partially independent pathways by which the amygdala can influence the frontal lobe.