Margaret E. Sereno

Shape Selectivity in Primate Frontal Eye Field

Previous neurophysiological studies of the frontal eye field (FEF) in monkeys have focused on its role in saccade target selection and gaze shift control. It has been argued that FEF neurons indicate the locations of behaviorally significant visual stimuli and are not inherently sensitive to specific features of the visual stimuli per se. Here, for the first time, we directly examined single cell responses to simple, two-dimensional shapes and found that shape selectivity exists in a substantial number of FEF cells during a passive fixation task or during the sample, delay (memory), and eye movement periods in a delayed match to sample (DMTS) task. Our data demonstrate that FEF neurons show sensory and mnemonic selectivity for stimulus shape features whether or not they are behaviorally significant for the task at hand. We also investigated the extent and localization of activation in the FEF using a variety of shape stimuli defined by static or dynamic cues employing functional magentic resonance imaging (fMRI) in anesthetized and paralyzed monkeys. Our fMRI results support the electrophysiological findings by showing significant FEF activation for a variety of shape stimuli and cues in the absence of attentional and motor processing. This shape selectivity in FEF is comparable to previous reports in the ventral pathway, inviting a reconsideration of the functional organization of the visual system.

Size matters: Bigger is faster

A largely unexplored aspect of lexical access in visual word recognition is “semantic size”—namely, the real-world size of an object to which a word refers. A total of 42 participants performed a lexical decision task on concrete nouns denoting either big or small objects (e.g., bookcase or teaspoon). Items were matched pairwise on relevant lexical dimensions. Participants’ reaction times were reliably faster to semantically “big” versus “small” words. The results are discussed in terms of possible mechanisms, including more active representations for “big” words, due to the ecological importance attributed to large objects in the environment and the relative speed of neural responses to large objects.

2-D center-surround effects on 3-D structure-from-motion.

This study investigates how mechanisms for amplifying 2-D motion contrast influence the assignment of 3-D depth values. The authors found that the direction of movement of a random-dot conveyor belt strongly inclined observers to report that the front surface of a superimposed, transparent, rotating, random-dot sphere moved in a direction opposite to the belt. This motion-contrast effect was direction selective and demonstrated substantial spatial integration. Varying the stereo depth of the belt did not compromise the main effect, precluding a mechanical interpretation (sphere rolling on belt). Varying the speed of the surfaces of the sphere also did not greatly affect the interpretation of rotation direction. These results suggest that 2-D center-surround interactions influence 3-D depth assignment by differentially modulating the strength of response to the moving surfaces of an object (their prominence) without affecting featural specificity.

Aesthetic Responses to Exact Fractals Driven by Physical Complexity

Fractals are physically complex due to their repetition of patterns at multiple size scales. Whereas the statistical characteristics of the patterns repeat for fractals found in natural objects, computers can generate patterns that repeat exactly. Are these exact fractals processed differently, visually and aesthetically, than their statistical counterparts? We investigated the human aesthetic response to the complexity of exact fractals by manipulating fractal dimensionality, symmetry, recursion, and the number of segments in the generator. Across two studies, a variety of fractal patterns were visually presented to human participants to determine the typical response to exact fractals. In the first study, we found that preference ratings for exact midpoint displacement fractals can be described by a linear trend with preference increasing as fractal dimension increases. For the majority of individuals, preference increased with dimension. We replicated these results for other exact fractal patterns in a second study. In the second study, we also tested the effects of symmetry and recursion by presenting asymmetric dragon fractals, symmetric dragon fractals, and Sierpinski carpets and Koch snowflakes, which have radial and mirror symmetry. We found a strong interaction among recursion, symmetry and fractal dimension. Specifically, at low levels of recursion, the presence of symmetry was enough to drive high preference ratings for patterns with moderate to high levels of fractal dimension. Most individuals required a much higher level of recursion to recover this level of preference in a pattern that lacked mirror or radial symmetry, while others were less discriminating. This suggests that exact fractals are processed differently than their statistical counterparts. We propose a set of four factors that influence complexity and preference judgments in fractals that may extend to other patterns: fractal dimension, recursion, symmetry and the number of segments in a pattern. Conceptualizations such as Berlyne’s and Redies’ theories of aesthetics also provide a suitable framework for interpretation of our data with respect to the individual differences that we detect. Future studies that incorporate physiological methods to measure the human aesthetic response to exact fractal patterns would further elucidate our responses to such timeless patterns.

Recovering stimulus locations using populations of eye-position modulated neurons in dorsal and ventral visual streams of non-human primates.

We recorded visual responses while monkeys fixated the same target at different gaze angles, both dorsally (lateral intraparietal cortex, LIP) and ventrally (anterior inferotemporal cortex, AIT). While eye-position modulations occurred in both areas, they were both more frequent and stronger in LIP neurons. We used an intrinsic population decoding technique, multidimensional scaling (MDS), to recover eye positions, equivalent to recovering fixated target locations. We report that eye-position based visual space in LIP was more accurate (i.e., metric). Nevertheless, the AIT spatial representation remained largely topologically correct, perhaps indicative of a categorical spatial representation (i.e., a qualitative description such as “left of” or “above” as opposed to a quantitative, metrically precise description). Additionally, we developed a simple neural model of eye position signals and illustrate that differences in single cell characteristics can influence the ability to recover target position in a population of cells. We demonstrate for the first time that the ventral stream contains sufficient information for constructing an eye-position based spatial representation. Furthermore we demonstrate, in dorsal and ventral streams as well as modeling, that target locations can be extracted directly from eye position signals in cortical visual responses without computing coordinate transforms of visual space.

Population coding and the labeling problem: Extrinsic versus intrinsic representations

Current population coding methods, including weighted averaging and Bayesian estimation, are based on extrinsic representations. These require that neurons be labeled with response parameters, such as tuning curve peaks or noise distributions, which are tied to some external, world-based metric scale. Firing rates alone, without this external labeling, are insufficient to represent a variable. However, the extrinsic approach does not explain how such neural labeling is implemented. A radically different and perhaps more physiological approach is based on intrinsic representations, which have access only to firing rates. Because neurons are unlabeled, intrinsic coding represents relative, rather than absolute, values of a variable. We show that intrinsic coding has representational advantages, including invariance, categorization, and discrimination, and in certain situations it may also recover absolute stimulus values.

Semantic Size of Abstract Concepts: It Gets Emotional When You Can't See It

Size is an important visuo-spatial characteristic of the physical world. In language processing, previous research has demonstrated a processing advantage for words denoting semantically “big” (e.g., jungle) versus “small” (e.g., needle) concrete objects. We investigated whether semantic size plays a role in the recognition of words expressing abstract concepts (e.g., truth). Semantically “big” and “small” concrete and abstract words were presented in a lexical decision task. Responses to “big” words, regardless of their concreteness, were faster than those to “small” words. Critically, we explored the relationship between semantic size and affective characteristics of words as well as their influence on lexical access. Although a word’s semantic size was correlated with its emotional arousal, the temporal locus of arousal effects may depend on the level of concreteness. That is, arousal seemed to have an earlier (lexical) effect on abstract words, but a later (post-lexical) effect on concrete words. Our findings provide novel insights into the semantic representations of size in abstract concepts and highlight that affective attributes of words may not always index lexical access.

Relationship between Fractal Dimension and Spectral Scaling Decay Rate in Computer-Generated Fractals

Two measures are commonly used to describe scale-invariant complexity in images: fractal dimension (D) and power spectrum decay rate (β). Although a relationship between these measures has been derived mathematically, empirical validation across measurements is lacking. Here, we determine the relationship between D and β for 1- and 2-dimensional fractals. We find that for 1-dimensional fractals, measurements of D and β obey the derived relationship. Similarly, in 2-dimensional fractals, measurements along any straight-line path across the fractal’s surface obey the mathematically derived relationship. However, the standard approach of vision researchers is to measure β of the surface after 2-dimensional Fourier decomposition rather than along a straight-line path. This surface technique provides measurements of β that do not obey the mathematically derived relationship with D. Instead, this method produces values of β that imply that the fractal’s surface is much smoother than the measurements along the straight lines indicate. To facilitate communication across disciplines, we provide empirically derived equations for relating each measure of β to D. Finally, we discuss implications for future research on topics including stress reduction and the perception of motion in the context of a generalized equation relating β to D.

Characteristics of Eye-Position Gain Field Populations Determine Geometry of Visual Space

We have previously demonstrated differences in eye-position spatial maps for anterior inferotemporal cortex (AIT) in the ventral stream and lateral intraparietal cortex (LIP) in the dorsal stream, based on population decoding of gaze angle modulations of neural visual responses (i.e., eye-position gain fields). Here we explore the basis of such spatial encoding differences through modeling of gain field characteristics. We created a population of model neurons, each having a different eye-position gain field. This population was used to reconstruct eye-position visual space using multidimensional scaling. As gain field shapes have never been well-established experimentally, we examined different functions, including planar, sigmoidal, elliptical, hyperbolic, and mixtures of those functions. All functions successfully recovered positions, indicating weak constraints on allowable gain field shapes. We then used a genetic algorithm to modify the characteristics of model gain field populations until the recovered spatial maps closely matched those derived from monkey neurophysiological data in AIT and LIP. The primary differences found between model AIT and LIP gain fields were that AIT gain fields were more foveally dominated. That is, gain fields in AIT operated on smaller spatial scales and smaller dispersions than in LIP. Thus, we show that the geometry of eye-position visual space depends on the population characteristics of gain fields, and that differences in gain field characteristics for different cortical areas may underlie differences in the representation of space.

Depth reversals in stereoscopic displays driven by apparent size

In visual scenes, depth information is derived from a variety of monocular and binocular cues. When in conflict, a monocular cue is sometimes able to override the binocular information. We examined the accuracy of relative depth judgments in orthographic, stereoscopic displays and found that perceived relative size can override binocular disparity as a depth cue in a situation where the relative size information is itself generated from disparity information, not from retinal size difference. A size discrimination task confirmed the assumption that disparity information was perceived and used to generate apparent size differences. The tendency for the apparent size cue to override disparity information can be modulated by varying the strength of the apparent size cue. In addition, an analysis of reaction times provides supporting evidence for this novel depth reversal effect. We believe that human perception must be regarded as an important component of stereoscopic applications. Hence, if applications are to be effective and accurate, it is necessary to take into account the richness and complexity of the human visual perceptual system that interacts with them. We discuss implications of this and similar research for human performance in virtual environments, the design of visual presentations for virtual worlds, and the design of visualization tools.