Charles E. Connor

Underlying principles of visual shape selectivity in posterior inferotemporal cortex.

Object perception depends on shape processing in the ventral visual pathway, which in monkeys culminates in inferotemporal cortex (IT). Here we provide a description of fundamental quantitative principles governing neural selectivity for complex shape in IT. By measuring responses to large, parametric sets of two-dimensional (2D) silhouette shapes, we found that neurons in posterior IT (Brodmanns areas TEO and posterior TE) integrate information about multiple contour elements (straight and curved edge fragments of the type represented in lower-level areas) using both linear and nonlinear mechanisms. This results in complex, distributed response patterns that cannot be characterized solely in terms of example stimuli. We explained these response patterns with tuning functions in multidimensional shape space and accurately predicted neural responses to the widely varying shapes in our stimulus set. Integration of contour element information in earlier stages of IT represents an important step in the transformation from low-level shape signals to complex object representation.

A neural code for three-dimensional object shape in macaque inferotemporal cortex

Previous investigations of the neural code for complex object shape have focused on two-dimensional pattern representation. This may be the primary mode for object vision given its simplicity and direct relation to the retinal image. In contrast, three-dimensional shape representation requires higher-dimensional coding derived from extensive computation. We found evidence for an explicit neural code for complex three-dimensional object shape. We used an evolutionary stimulus strategy and linear/nonlinear response models to characterize three-dimensional shape responses in macaque monkey inferotemporal cortex (IT). We found widespread tuning for three-dimensional spatial configurations of surface fragments characterized by their three-dimensional orientations and joint principal curvatures. Configural representation of three-dimensional shape could provide specific knowledge of object structure to support guidance of complex physical interactions and evaluation of object functionality and utility.

Transformation of shape information in the ventral pathway

Object perception seems effortless to us, but it depends on intensive neural processing across multiple stages in ventral pathway visual cortex. Shape information at the retinal level is hopelessly complex, variable and implicit. The ventral pathway must somehow transform retinal signals into much more compact, stable and explicit representations of object shape. Recent findings highlight key aspects of this transformation: higher-order contour derivatives, structural representation in object-based coordinates, composite shape tuning dimensions, and long-term storage of object knowledge. These coding principles could help to explain our remarkable ability to perceive, distinguish, remember and understand a virtual infinity of objects.

Toward a Unified Theory of Visual Area V4

Visual area V4 is a midtier cortical area in the ventral visual pathway. It is crucial for visual object recognition and has been a focus of many studies on visual attention. However, there is no unifying view of V4s role in visual processing. Neither is there an understanding of how its role in feature processing interfaces with its role in visual attention. This review captures our current knowledge of V4, largely derived from electrophysiological and imaging studies in the macaque monkey. Based on recent discovery of functionally specific domains in V4, we propose that the unifying function of V4 circuitry is to enable selective extraction of specific functional domain-based networks, whether it be by bottom-up specification of object features or by top-down attentionally driven selection.

Neural activity in areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing

UNDER natural viewing conditions primates make frequent exploratory eye movements across complex scenes. We recorded neural activity of 62 cells in visual areas V1, V2 and V4 in an awake behaving monkey that freely viewed natural images. About half of the cells studied showed a modulation in firing rate following some of the eye movements made during free viewing, though the proportions showing a discernible modulation varied across areas. These cells were also examined under controlled viewing conditions in which gratings or natural image patches were flashed in and around the classical receptive field while the animal performed a fixation task. Activity rates were generally highest with flashed gratings and lowest during free viewing. Flashed natural image patches evoked responses between these two extremes, and the responses were higher when the patches were confined to the classical receptive field than when they extended into the non-classical surround. Thus the reduction of activity during free viewing relative to that obtained with flashed gratings is partly attributable to natural images being less effective stimuli and partly to suppressive spatio-temporal neural mechanisms that are important during natural vision.

Three-dimensional orientation tuning in macaque area V4

Tuning for the orientation of elongated, linear image elements (edges, bars, gratings), first discovered by Hubel and Wiesel, is considered a key feature of visual processing in the brain. It has been studied extensively in two dimensions (2D) using frontoparallel stimuli, but in real life most lines, edges and contours are slanted with respect to the viewer. Here we report that neurons in macaque area V4, an intermediate stage in the ventral (object-related) pathway of visual cortex, were tuned for 3D orientation—that is, for specific slants as well as for 2D orientation. The tuning for 3D orientation was consistent across depth position (binocular disparity) and position within the 2D classical receptive field. The existence of 3D orientation signals in the ventral pathway suggests that the brain may use such information to interpret 3D shape.

Neural representations for object perception: Structure, category, and adaptive coding

Object perception is one of the most remarkable capacities of the primate brain. Owing to the large and indeterminate dimensionality of object space, the neural basis of object perception has been difficult to study and remains controversial. Recent work has provided a more precise picture of how 2D and 3D object structure is encoded in intermediate and higher-level visual cortices. Yet, other studies suggest that higher-level visual cortex represents categorical identity rather than structure. Furthermore, object responses are surprisingly adaptive to changes in environmental statistics, implying that learning through evolution, development, and also shorter-term experience during adulthood may optimize the object code. Future progress in reconciling these findings will depend on more effective sampling of the object domain and direct comparison of these competing hypotheses.

Medial Axis Shape Coding in Macaque Inferotemporal Cortex

The basic, still unanswered question about visual object representation is this: what specific information is encoded by neural signals? Theorists have long predicted that neurons would encode medial axis or skeletal object shape, yet recent studies reveal instead neural coding of boundary or surface shape. Here, we addressed this theoretical/experimental disconnect, using adaptive shape sampling to demonstrate explicit coding of medial axis shape in high-level object cortex (macaque monkey inferotemporal cortex or IT). Our metric shape analyses revealed a coding continuum, along which most neurons represent a configuration of both medial axis and surface components. Thus, IT response functions embody a rich basis set for simultaneously representing skeletal and external shape of complex objects. This would be especially useful for representing biological shapes, which are often characterized by both complex, articulated skeletal structure and specific surface features.

A sparse object coding scheme in area V4

Sparse coding has long been recognized as a primary goal of image transformation in the visual system. Sparse coding in early visual cortex is achieved by abstracting local oriented spatial frequencies and by excitatory/inhibitory surround modulation. Object responses are thought to be sparse at subsequent processing stages, but neural mechanisms for higher-level sparsification are not known. Here, convergent results from macaque area V4 neural recording and simulated V4 populations trained on natural object contours suggest that sparse coding is achieved in midlevel visual cortex by emphasizing representation of acute convex and concave curvature. We studied 165 V4 neurons with a random, adaptive stimulus strategy to minimize bias and explore an unlimited range of contour shapes. V4 responses were strongly weighted toward contours containing acute convex or concave curvature. In contrast, the tuning distribution in nonsparse simulated V4 populations was strongly weighted toward low curvature. But as sparseness constraints increased, the simulated tuning distribution shifted progressively toward more acute convex and concave curvature, matching the neural recording results. These findings indicate a sparse object coding scheme in midlevel visual cortex based on uncommon but diagnostic regions of acute contour curvature.

Disparity tuning in macaque area V4

Neural processing of stereoscopic depth is conventionally associated with the dorsal (spatial) pathway in primate visual cortex. The role of depth information in the ventral (object) pathway has been less certain. We found prominent tuning for stereoscopic disparity in area V4, an intermediate stage in the ventral pathway. Eighty percent of the cells in our sample exhibited significant disparity tuning over the −1.0° to 1.0° range, and the majority showed > 2:1 response differences. Tuning function shapes were similar to those reported previously in other visual areas. We observed a significant tuning bias towards crossed (near) disparities. This could reflect an emphasis in the ventral pathway on foreground objects or parts of objects projecting towards the viewer.