Don Ray Murray

Using Real-Time Stereo Vision for Mobile Robot Navigation

This paper describes a working vision-based mobile robot that navigates and autonomously explores its environment while building occupancy grid maps of the environment. We present a method for reducing stereo vision disparity images to two-dimensional map information. Stereo vision has several attributes that set it apart from other sensors more commonly used for occupancy grid mapping. We discuss these attributes, the errors that some of them create, and how to overcome them. We reduce errors by segmenting disparity images based on continuous disparity surfaces to reject “spikes” caused by stereo mismatches. Stereo vision processing and map updates are done at 5 Hz and the robot moves at speeds of 300 cm/s.

Stereo vision based mapping and navigation for mobile robots

This paper describes a visually guided robot that can plan paths, construct maps and explore an indoor environment. The robot uses a trinocular stereo vision system to produce highly accurate depth images at 2 Hz allowing it to safely travel through the environment at 0.5 m/s. The algorithm integrates stereo vision, occupancy grid mapping, and potential field path planning techniques to form a robust and cohesive robotic system for mapping and navigation. Stereo vision is shown to be a viable alternative to active sensing devices such as sonar and laser range finders.

Patchlets: Representing Stereo Vision Data with Surface Elements

This paper describes a class of augmented surface elements which we call patchlets. Patchlets are planar surface elements generated from dense stereo vision 3D range images. Patchlets have a position, surface normal and size. In addition they have confidence measures on the position and normal direction that are based on the sensor accuracy. These confidence measures facilitate their use with probabilistic methods such as clustering for range image segmentation. Patchlets are formed by the projection of a pixel within the stereo image onto a sensed surface. They are surface elements that are constructed directly from the sensor data and can be used as a fundamental sensed-data primitive. We describe patchlet formation from the stereo disparity image, the propagation of errors from the stereo sensor model, and confirm experimentally the patchlet model representation. We provide surface segmentation as a sample patchlet application.

Spinoza: a stereoscopic visually guided mobile robot

Our mobile robot, Spinoza, embodies a sophisticated real-time vision system for the control of a mobile robot in a dynamic environment. The complexity of our robot architecture arises from the wide variety of tasks that need to be performed and the resulting challenge of coordinating multiple distributed, concurrent processes on a diverse range of processor architectures, including transputers, digital signal processors and a workstation host. The system handles the sensing, reasoning and action components of a robot, distributed over these architectures, and responds to unpredictable events in an unknown dynamic environment. Spinoza relies heavily on its capability to perform real-time vision processing in order to perform task such as mapping, navigation, exploration, tracking and simple manipulation.

Environment modeling with stereo vision

We consider the problem of creating compact surface-based environment models from stereo vision images taken from a stereo-camera equipped mobile robot. The stereo images can be quite complex and correlation stereo suffers from considerable noise at ranges over a few metres. We construct the environment models by segmenting the scene viewed from a stereo camera into rectangular planar surfaces through the use of the patchlets surface element data structure. Patchlets are the projection of the stereo pixels onto detected surfaces in the scene. They have position, orientation, size and sensor-based confidence measures. The confidence measures allow proper weighting of patchlet parameters when aggregating patchlets into larger surfaces.

Segmenting correlation stereo range images using surface elements

This work describes methods for segmenting planar surfaces from noisy 3D data obtained from correlation stereo vision. We make use of local planar surface elements called patchlets. Patchlets have 3D position, orientation and size parameters. As well, they have positional confidence measures based on the stereo sensor model. Patchlet orientations (i.e., surface normals) provide important additional dimensionality that reduces the ambiguity of segmentation-by-clustering. Patchlet size allows the use of continuity or coverage constraints when segmenting bounded surfaces from depth images. We use a region-growing approach to identify the number of surfaces that exist in a stereo image and obtain an initial estimate of the surface parameters. We refine segmentation using a maximum likelihood clustering approach that is optimised with Expectation-Maximisation. Confidence measures on the patchlet parameters allow proper weighting of patchlet contributions to the solution. We provide experimental results of the segmentation on complex outdoor scenes.

Vision-based mapping with cooperative robots

Two stereo-vision-based mobile robots navigate and autonomously explore their environment safely while building occupancy grid maps of the environment. The robots maintain position estimates within a global coordinate frame using landmark recognition. This allows them to build a common map by sharing position information and stereo data. Stereo vision processing and map updates are done at 3 Hz and the robots move at speeds of 200 cm/s. Cooperative mapping is achieved through autonomous exploration of unstructured and dynamic environments. The map is constructed conservatively, so as to be useful for collision-free path planning. Each robot maintains a separate copy of a shared map, and then posts updates to the common map when it returns to observe a landmark at home base. Issues include synchronization, mutual localization, navigation, exploration, registration of maps, merging repeated views (fusion), centralized vs decentralized maps.