Hanspeter A. Mallot

Integration of depth modules: stereo and shading

We studied the integration of image disparities, edge information, and shading in the three-dimensional perception of complex yet well-controlled images generated with a computer-graphics system. The images showed end-on views of flat- and smooth-shaded ellipsoids, i.e., images with and without intensity discontinuities (edges). A map of perceived depth was measured by adjusting a small stereo depth probe interactively to the perceived surface. Our data show that disparate shading (even in the absence of disparate edges) yields a vivid stereoscopic depth perception. The perceived depth is significantly reduced if the disparities are completely removed (shape-from-shading). If edge information is available, it overrides both shape-from-shading and disparate shading. Degradations of depth perception corresponded to a reduced depth rather than to an increased scatter in the depth measurement. The results are compared with computer-vision algorithms for both single cues and their integration for three-dimensional vision.

Biomimetic robot navigation

In the past decade, a large number of robots has been built that explicitly implement biological navigation behaviours. We review these biomimetic approaches using a framework that allows for a common description of biological and technical navigation behaviour. The review shows that biomimetic systems make significant contributions to two fields of research: First, they provide a real world test of models of biological navigation behaviour; second, they make new navigation mechanisms available for technical applications, most notably in the field of indoor robot navigation. While simpler insect navigation behaviours have been implemented quite successfully, the more complicated way-finding capabilities of vertebrates still pose a challenge to current systems. ©2000 Elsevier Science B.V. All rights reserved.

Navigation and Acquisition of Spatial Knowledge in a Virtual Maze

Spatial behavior in humans and animals includes a wide variety of behavioral competences and makes use of a large number of sensory cues. Here we studied the ability of human subjects to search locations, to find shortcuts and novel paths, to estimate distances between remembered places, and to draw sketch maps of the explored environment; these competences are related to goal-independent memory of space, or cognitive maps. Information on spatial relations was restricted to two types: a visual motion sequence generated by simulated movements in a virtual maze and the subjects own movement decisions defining the path through the maze. Visual information was local (i.e., no global landmarks or compass information was provided). Other position and movement information (vestibular or proprioceptive) was excluded. The amount of visual information provided was varied over four experimental conditions. The results indicate that human subjects are able to learn a virtual maze from sequences of local views and movements. The information acquired is local, consisting of recognized positions and movement decisions associated to them. Although simple associations of this type can be shown to be present in some subjects, more complete configurational knowledge is acquired as well. The results are discussed in a view-based framework of navigation and the representation of spatial knowledge by means of a view graph.

Inverse perspective mapping simplifies optical flow computation and obstacle detection

We present a scheme for obstacle detection from optical flow which is based on strategies of biological information processing. Optical flow is established by a local “voting” (non-maximum suppression) over the outputs of correlation-type motion detectors similar to those found in the fly visual system. The computational theory of obstacle detection is discussed in terms of space-variances of the motion field. An efficient mechanism for the detection of disturbances in the expected motion field is based on “inverse perspective mapping”, i.e., a coordinate transform or retinotopic mapping applied to the image. It turns out that besides obstacle detection, inverse perspective mapping has additional advantages for regularizing optical flow algorithms. Psychophysical evidence for body-scaled obstacle detection and related neurophysiological results are discussed.

Where did I take that snapshot? Scene-based homing by image matching

Abstract. In homing tasks, the goal is often not marked by visible objects but must be inferred from the spatial relation to the visual cues in the surrounding scene. The exact computation of the goal direction would require knowledge about the distances to visible landmarks, information, which is not directly available to passive vision systems. However, if prior assumptions about typical distance distributions are used, a snapshot taken at the goal suffices to compute the goal direction from the current view. We show that most existing approaches to scene-based homing implicitly assume an isotropic landmark distribution. As an alternative, we propose a homing scheme that uses parameterized displacement fields. These are obtained from an approximation that incorporates prior knowledge about perspective distortions of the visual environment. A mathematical analysis proves that both approximations do not prevent the schemes from approaching the goal with arbitrary accuracy, but lead to different errors in the computed goal direction. Mobile robot experiments are used to test the theoretical predictions and to demonstrate the practical feasibility of the new approach.

Learning view graphs for robot navigation

We present a purely vision-based scheme for learning a topological representation of an open environment. The system represents selected places by local views of the surrounding scene, and finds traversable paths between them. The set of recorded views and their connections are combined into a graph model of the environment. To navigate between views connected in the graph, we employ a homing strategy inspired by findings of insect ethology. In robot experiments, we demonstrate that complex visual exploration and navigation tasks can thus be performed without using metric information.

Differential Recruitment of the Hippocampus, Medial Prefrontal Cortex, and the Human Motion Complex during Path Integration in Humans

Path integration, the ability to sense self-motion for keeping track of changes in orientation and position, constitutes a fundamental mechanism of spatial navigation and a keystone for the development of cognitive maps. Whereas animal path integration is predominantly supported by the head-direction, grid, and place cell systems, the neural foundations are not well understood in humans. Here we used functional magnetic resonance imaging and a virtual rendition of a triangle completion paradigm to test whether human path integration recruits a cortical system similar to that of rodents and nonhuman primates. Participants traveled along two legs of a triangle before pointing toward the starting location. In accordance with animal models, stronger right hippocampal activation predicted more accurate updating of the starting location on a trial-by-trial basis. Moreover, between-subjects fluctuations in response consistency were negatively correlated with bilateral hippocampal and medial prefrontal activation, and bilateral recruitment of the human motion complex (hMT+) covaried with individual path integration capability. Given that these effects were absent in a perceptual control task, the present study provides the first evidence that visual path integration is related to the dynamic interplay of self-motion processing in hMT+, higher-level spatial processes in the hippocampus, and spatial working memory in medial prefrontal cortex.

View-Based Cognitive Mapping and Path Planning

This article presents a scheme for learning a cognitive map of a maze from a sequence of views and movement decisions. The scheme is based on an intermediate representation called the view graph, whose nodes correspond to the views whereas the labeled edges represent the movements leading from one view to another. By means of a graph theoretical reconstruction method, the view graph is shown to carry complete information on the topological and directional structure of the maze. Path planning can be carried out directly in the view graph without actually performing this reconstruction. A neural network is presented that learns the view graph during a random exploration of the maze. lt is based on an unsupervised competitive learning rule translating temporal sequence (rather than similarity) of views into connectedness in the network. The network uses its knowledge of the topological and directional structure of the maze to generate expectations about which views are likely to be encountered next, improving the view-recognition performance. Numerical simulations illustrate the networks ability for path planning and the recognition of views degraded by random noise. The results are compared to findings of behavioral neuroscience.

Miniature curved artificial compound eyes.

In most animal species, vision is mediated by compound eyes, which offer lower resolution than vertebrate single-lens eyes, but significantly larger fields of view with negligible distortion and spherical aberration, as well as high temporal resolution in a tiny package. Compound eyes are ideally suited for fast panoramic motion perception. Engineering a miniature artificial compound eye is challenging because it requires accurate alignment of photoreceptive and optical components on a curved surface. Here, we describe a unique design method for biomimetic compound eyes featuring a panoramic, undistorted field of view in a very thin package. The design consists of three planar layers of separately produced arrays, namely, a microlens array, a neuromorphic photodetector array, and a flexible printed circuit board that are stacked, cut, and curved to produce a mechanically flexible imager. Following this method, we have prototyped and characterized an artificial compound eye bearing a hemispherical field of view with embedded and programmable low-power signal processing, high temporal resolution, and local adaptation to illumination. The prototyped artificial compound eye possesses several characteristics similar to the eye of the fruit fly Drosophila and other arthropod species. This design method opens up additional vistas for a broad range of applications in which wide field motion detection is at a premium, such as collision-free navigation of terrestrial and aerospace vehicles, and for the experimental testing of insect vision theories.

'Fine-to-Coarse' Route Planning and Navigation in Regionalized Environments

Environments that are divided into regions lead to hierarchical encoding of space. Such memory structures are known to systematically distort estimates of distance and direction and affect spatial priming and memory recall. Here we present two navigation experiments in virtual environments that reveal an influence of environmental regions on human route planning and navigation behaviour. Following the hierarchical theories of spatial representations, it is argued that environmental regions are explicitly represented in spatial memory and that human route planning takes into account region-connectivity and is not based on place-connectivity alone. We also propose a fine-to-coarse planning heuristic that could account for the empirical data by planning in a representation that uses fine-space information for close locations and coarse-space information for distant locations simultaneously.