Fabrizio Santini

Miniature eye movements enhance fine spatial detail

Our eyes are constantly in motion. Even during visual fixation, small eye movements continually jitter the location of gaze. It is known that visual percepts tend to fade when retinal image motion is eliminated in the laboratory. However, it has long been debated whether, during natural viewing, fixational eye movements have functions in addition to preventing the visual scene from fading. In this study, we analysed the influence in humans of fixational eye movements on the discrimination of gratings masked by noise that has a power spectrum similar to that of natural images. Using a new method of retinal image stabilization, we selectively eliminated the motion of the retinal image that normally occurs during the intersaccadic intervals of visual fixation. Here we show that fixational eye movements improve discrimination of high spatial frequency stimuli, but not of low spatial frequency stimuli. This improvement originates from the temporal modulations introduced by fixational eye movements in the visual input to the retina, which emphasize the high spatial frequency harmonics of the stimulus. In a natural visual world dominated by low spatial frequencies, fixational eye movements appear to constitute an effective sampling strategy by which the visual system enhances the processing of spatial detail.

EyeRIS: a general-purpose system for eye-movement-contingent display control.

In experimental studies of visual performance, the need often emerges to modify the stimulus according to the eye movements performed by the subject. The eye-movement-contingent display (EMCD) methodology enables accurate control of the position and motion of the stimulus on the retina. EMCD procedures have been used successfully in many areas of vision science, including studies of visual attention or eye movements and physiological characterization of neuronal response properties. Unfortunately, the difficulty of real-time programming and the unavailability of flexible and economical systems that can be easily adapted to the diversity of experimental needs and laboratory setups have prevented the widespread use of EMCD control. This article describes EyeRIS, a general-purpose system for performing EMCD experiments on a Windows computer. Based on a digital signal processor with analog and digital interfaces, this integrated hardware and software system is responsible for sampling and processing oculomotor signals and subject responses and for modifying the stimulus displayed on a CRT according to a gaze-contingent procedure specified by the experimenter. EyeRIS is designed to update the stimulus with a delay of only 10 msec. To thoroughly evaluate EyeRIS’s performance, this study was designed to (1) examine the response of the system in a number of EMCD procedures and computational benchmarking tests; (2) compare the accuracy of implementation of one particular EMCD procedure, retinal stabilization, with that produced by a standard tool used for this task; and (3) examine EyeRIS’s performance in one of the many EMCD procedures that cannot be executed by means of any other currently available device.

Integrating robotics and neuroscience: brains for robots, bodies for brains

Researchers in robotics and artificial intelligence have often looked at biology as a source of inspiration for solving their problems. From the opposite perspective, neuroscientists have recently turned their attention to the use of robotic systems as a way to quantitatively test and analyze theories that would otherwise remain at a speculative stage. Computational models of neurons and networks of neurons are often activated with simplified artificial patterns that bear little resemblance to natural stimuli. The use of robotic systems has the advantage of introducing phenotypic and environmental constraints similar to those that brains of animals have to face during development and in everyday life. Consideration of these constraints is particularly important in light of modern brain theories, which emphasize the importance of closing the perception/action loop between the agent and the environment. To provide concrete examples of the use of robotic systems in neuroscience, this paper reviews our work in the areas of sensory perception and motor learning. The interdisciplinary approach followed by this research establishes a direct link between natural sciences and engineering. This research can lead to the understanding of basic biological problems while producing robust and flexible systems that operate in the real world.

Active estimation of distance in a robotic system that replicates human eye movement

In a moving agent, the different apparent motion of objects located at various distances provides an important source of depth information. While motion parallax is evident for large translations of the agent, a small parallax also occurs in most head/eye systems during rotations of the cameras. A similar parallax is also present in the human eye, so that a redirection of gaze shifts the projection of an object on the retina by an amount that depends not only on the amplitude of the rotation, but also on the distance of the object with respect to the observer. This study examines the accuracy of distance estimation on the basis of the parallax produced by camera rotations. Sequences of human eye movements were used to control the motion of a pan/tilt system specifically designed to reproduce the oculomotor parallax present in the human eye. We show that the oculomotor strategies by which humans scan visual scenes produce parallaxes that provide accurate estimation of distance. This information simplifies challenging visual tasks such as image segmentation and figure/ground segregation.

ACTIVE 3D VISION THROUGH GAZE RELOCATION IN A HUMANOID ROBOT

Motion parallax, the relative motion of 3D space at different distances experienced by a moving agent, is one of the most informative visual cues of depth and distance. While motion parallax is typically investigated during navigation, it also occurs in most robotic head/eye systems during rotations of the cameras. In these systems, as in the eyes of many species, the optical nodal points do not lie on the axes of rotation. Thus, a camera rotation shifts an objects projection on the sensor by an amount that depends not only on the rotation amplitude, but also on the distance of the object with respect to the camera. Several species rely on this cue to estimate distance. An oculomotor parallax is present also in the human eye, and during normal eye movements, displaces the stimulus on the retina by an amount that is well within the range of sensitivity of the visual system. We developed an anthropomorphic robot equipped with an oculomotor system specifically designed to reproduce the images impinging on t...

Depth perception in an anthropomorphic robot that replicates human eye movements

In the eyes of many species, the optical nodal points of the cornea and lens do not lie on the axes of rotation of the eye. During eye movements, this lack of alignment produces depth information in the form of an oculomotor parallax. That is, a redirection of gaze shifts the projection of an object on the retina by an amount that depends not only on the amplitude of the rotation of the eye, but also on the distance of the object with respect to the observer. Species as diverse as the chameleon and the sandlance critically rely on this depth cue to estimate distance. An oculomotor parallax is present also in the human eye and, during natural eye movements, it produces retinal shifts that are well within the range of sensitivity of the human visual system. We have developed an anthropomorphic robot equipped with a pan/tilt head specifically designed to reproduce the oculomotor parallax present in the human eye. We show that replication of sequences of human eye movements with this robot produces accurate estimation of distance. In robotic vision it is often debated whether the dynamic analysis of a visual scene by means of a mobile camera presents advantages with respect to the static analysis provided by a stationary camera with a wide field of view. This study shows that by generating depth information, replication of the dynamic strategy by which humans scan a visual scene greatly facilitates the processes of figure/ground segregation and image segmentation, two of the hardest tasks of machine vision