Rüdiger von der Heydt

A Century of Gestalt Psychology in Visual Perception I. Perceptual Grouping and Figure-Ground Organization

In 1912, Max Wertheimer published his paper on phi motion, widely recognized as the start of Gestalt psychology. Because of its continued relevance in modern psychology, this centennial anniversary is an excellent opportunity to take stock of what Gestalt psychology has offered and how it has changed since its inception. We first introduce the key findings and ideas in the Berlin school of Gestalt psychology, and then briefly sketch its development, rise, and fall. Next, we discuss its empirical and conceptual problems, and indicate how they are addressed in contemporary research on perceptual grouping and figure-ground organization. In particular, we review the principles of grouping, both classical (e.g., proximity, similarity, common fate, good continuation, closure, symmetry, parallelism) and new (e.g., synchrony, common region, element and uniform connectedness), and their role in contour integration and completion. We then review classic and new image-based principles of figure-ground organization, how it is influenced by past experience and attention, and how it relates to shape and depth perception. After an integrated review of the neural mechanisms involved in contour grouping, border ownership, and figure-ground perception, we conclude by evaluating what modern vision science has offered compared to traditional Gestalt psychology, whether we can speak of a Gestalt revival, and where the remaining limitations and challenges lie. A better integration of this research tradition with the rest of vision science requires further progress regarding the conceptual and theoretical foundations of the Gestalt approach, which is the focus of a second review article.

Subjective contours - bridging the gap between psychophysics and physiology

Much is known about the initial stages of visual processing up to the striate cortex, but how is visual information represented and handled at subsequent stages? Phenomena of contour, color and movement perception have been used to identify functions of neurons and to reveal functional differences between cortical areas that application of classical receptive-field concepts has not suggested. These differences can be related to theoretical stages of visual processing that provide stability of perception under changing conditions of stimulation.

Representation of stereoscopic edges in monkey visual cortex

Form perception in random-dot stereograms is based on information that resides in the correlation between the two images, but is not present in either image alone. We have studied the coding of stereoscopic figures in the neural activity of areas V1 and V2 of alert behaving monkeys. While cells in V1 generally responded according to the disparity of the surface at the receptive field, we found cells in area V2 that responded selectively to the figure edges. These cells signaled the location and orientation of contrast borders as well as stereoscopic edges, and were often selective for the direction of the step in depth. We concluded that stereoscopic edges are explicitly represented in area V2.

Figure-ground mechanisms provide structure for selective attention

Attention depends on figure-ground organization: figures draw attention, whereas shapes of the ground tend to be ignored. Recent research has revealed mechanisms for figure-ground organization in the visual cortex, but how these mechanisms relate to the attention process remains unclear. Here we show that the influences of figure-ground organization and volitional (top-down) attention converge in single neurons of area V2 in Macaca mulatta. Although we found assignment of border ownership for attended and for ignored figures, attentional modulation was stronger when the attended figure was located on the neurons preferred side of border ownership. When the border between two overlapping figures was placed in the receptive field, responses depended on the side of attention, and enhancement was generally found on the neurons preferred side of border ownership. This correlation suggests that the neural network that creates figure-ground organization also provides the interface for the top-down selection process.

The coding of uniform colour figures in monkey visual cortex.

Psychophysical studies indicate that perception of the colour and brightness of a surface depends on neural signals evoked by the borders of the surface rather than its interior. The visual cortex emphasizes contrast borders, but it is unclear whether colour surface signals also exist, whether colour border signals are orientation selective or mainly non‐oriented, and whether cortical processing tends to separate colour and form information. To address these questions we examined the representation of uniform colour figures by recording single neuron activity from areas V1 and V2 in alert macaque monkeys during behaviourally induced fixation. Three aspects of coding were quantified: colour, orientation and edge selectivity. The occurrence of colour selectivity was not correlated with orientation or edge selectivity. The fraction of colour‐selective cells was the same (64 % in layers 2 and 3 of V1, 45 % in V2) for oriented and non‐oriented cells, and for edge‐selective and surface‐responsive cells. Oriented cells were often highly selective in colour space, and about 40 % of them were selective for edge polarity or border ownership. Thus, contrary to the idea of feature maps, colour, orientation and edge polarity are multiplexed in cortical signals. The results from V2 were similar to those from upper‐layer V1, indicating that cortical processing does not strive to separate form and colour information. Oriented cells were five times more frequent than non‐oriented cells. Thus, the vast majority of colour‐coded cells are orientation tuned. Based on response profiles across a 4 deg square figure, and the relative frequency of oriented and non‐oriented cells, we estimate that the cortical colour signal is 5–6 times stronger for the edges than for the surface of the figure. The frequency of oriented colour cells and their ability to code edge polarity indicate that these cells play a major role in the representation of surface colour.

Simulation of neural contour mechanisms : representing anomalous contours

Abstract We present a computational model of a contour mechanism first identified by neurophysiological methods in monkey visual cortex. The scope is the definition of occluding contours in static monocular images. The model employs convolutions and non-linear operations, but does not require feedback loops. Contours are defined by the local response maxima of a contour operator applied in six orientations. The operator sums the activities of a ‘C-operator’, sensitive to contrast borders and a ‘grouping operator’ that integrates collinear aggregations of termination features, such as line-ends and corners. The grouping process is selective for termination features which are consistent with the interpretation of occlusion. Contrast edges are represented by C-operators simulating the function of cortical complex cells, termination features by ES-operators simulating the function of cortical end-stopped cells. The concepts of ortho and para curvilinear grouping are introduced. Ortho grouping applies to terminations of the background, which tend to be orthogonal to the occluding contour. Para grouping applies to discontinuities of the foreground and is used to interpolate the contour in the direction of termination. Both grouping modes also identify the direction of figure and ground at such contours. The simulation reproduces well-known illusory figures, including curved Kanizsa triangles and the circular disk of the four-armed Ehrenstein figure. Further, it improves the definition of occluding contours in natural, gray value images.

Analysis of the context integration mechanisms underlying figure-ground organization in the visual cortex.

Most neurons in visual cortex respond to contrast borders and are orientation selective, and some are also selective for which side of a border is figure and which side is ground (“border ownership coding”). These neurons are influenced by the image context far beyond the classical receptive field (CRF) and as early as 25 ms after the onset of activity in the cortex. The nature of the fast context integration mechanism is not well understood. What parts of a figure contribute to the context effect? What is the structure of the “extraclassical surround”? Is the context information propagated through horizontal fibers within cortex or through reciprocal connections via higher-level areas? To address these questions, we studied border ownership modulation with fragmented figures. Neurons were recorded in areas V1 and V2 of Macaca mulatta under behaviorally induced fixation. Test figures were fragmented rectangles. While one edge was centered on the CRF, the presence of the fragments outside the CRF was varied. The surround fragments produced facilitation on the preferred border ownership side as well as suppression on the nonpreferred side, with ∼80% of the locations contributing on average. Fragments far from the CRF influenced the responses even in the absence of fragments closer to the CRF, and without the extra delay that would incur from propagation through horizontal fibers. Three principally different models are discussed. The results support a model in which the antagonistic surround influences are produced by reentrant signals from a higher-level area.

Detection of General Edges and Keypoints

A computational framework for extracting (1) edges with an arbitrary profile function and (2) keypoints such as corners, vertices and terminations is presented. Using oriented filters with even and odd symmetry we combine their convolution outputs to oriented energy resulting in a unified representation of edges, lines and combinations thereof. We derive an ” edge quality” measure which allows to test the validity of a general edge model. A detection scheme for keypoints is proposed based on an analysis of oriented energy channels using differential geometry.

Mechanisms of perceptual organization provide auto-zoom and auto-localization for attention to objects.

Visual attention is often understood as a modulatory field acting at early stages of processing, but the mechanisms that direct and fit the field to the attended object are not known. We show that a purely spatial attention field propagating downward in the neuronal network responsible for perceptual organization will be reshaped, repositioned, and sharpened to match the objects shape and scale. Key features of the model are grouping neurons integrating local features into coherent tentative objects, excitatory feedback to the same local feature neurons that caused grouping neuron activation, and inhibition between incompatible interpretations both at the local feature level and at the object representation level.

Synchrony and the binding problem in macaque visual cortex

We tested the binding-by-synchrony hypothesis which proposes that object representations are formed by synchronizing spike activity between neurons that code features of the same object. We studied responses of 32 pairs of neurons recorded with microelectrodes 3 mm apart in the visual cortex of macaques performing a fixation task. Upon mapping the receptive fields of the neurons, a quadrilateral was generated so that two of its sides were centered in the receptive fields at the optimal orientations. This one-figure condition was compared with a two-figure condition in which the neurons were stimulated by two separate figures, keeping the local edges in the receptive fields identical. For each neuron, we also determined its border ownership selectivity (H. Zhou, H. S. Friedman, & R. von der Heydt, 2000). We examined both synchronization and correlation at nonzero time lag. After correcting for effects of the firing rate, we found that synchrony did not depend on the binding condition. However, finding synchrony in a pair of neurons was correlated with finding border-ownership selectivity in both members of the pair. This suggests that the synchrony reflected the connectivity in the network that generates border ownership assignment. Thus, we have not found evidence to support the binding-by-synchrony hypothesis.