Nicholas Molton

Locally Planar Patch Features for Real-Time Structure from Motion

The performance of sequential structure from motion systems, where scene mapping is sparse to permit real-time operation, depends greatly on the ability to repeatedly measure the same visual features from a wide range of viewpoints. While previous systems have tracked features as 2D templates in image space, we show that long-term tracking is improved by treating salient feature patches as observations of locally planar regions on 3D world surfaces. Within a SLAM framework for motion and structure estimation, a gradient-based image alignment method is used to deduce estimates feature surface normal estimates, enabling pre-warping of templates for matching. As an added benet these normals provide a richer description of the scene.

A stereo vision-based aid for the visually impaired

This paper describes a portable vision-based obstacle detection system, intended for use by blind people. The system combines an obstacle detection system designed for AGVs with recalibration of ground position and a Kalman Filter based model of the persons walking movement. The system uses stereo vision. Obstacle detection is achieved through comparison of the disparity seen with that expected from the position of the ground. Recalibration of ground position is made by plane fitting in the ground region. Motion estimation using two visual methods and the use of an inclinometer is described. The results show satisfactory success in all parts of the system.

Practical Structure and Motion from Stereo When Motion is Unconstrained

This paper describes a system which robustly estimates motion, and the 3D structure of a rigid environment, as a stereo vision platform moves through it. The system can cope with any camera motion, and any scene structure and is successful even in the presence of large jumps in camera position between the capture of successive image pairs, and when point matching is ambiguous. The system was developed to provide robust obstacle avoidance for a partially sighted person.The process described attempts to maximise use of the abundant information present in a stereo sequence. Key features include the use of multiple stereo match hypotheses, efficient motion computation from three images, and the use of this motion to ensure reliable matching, and to eliminate multiple stereo matches. Points are reconstructed in 3D space and tracked in a static coordinate frame with a Kalman Filter.This results in good 3D scene reconstructions. Structure which is impossible to match with certainty is absent, rather than being incorrectly reconstructed. As a result, the system is appropriate for obstacle detection. The results of processing some indoor and outdoor scenes, are given in the paper, and practical issues are highlighted throughout.

Robotic sensing for the partially sighted

Partial sightedness is a sensory disability which can to some extent be alleviated by artificial aids. Many of the sensory methods used in robotics can be applied in attempts to recapture some of the sensory information a partially sighted person has lost. This paper describes a device which uses sonar and stereo vision sensors for this task. The device is portable, and is worn by the user, giving them freedom of movement over kerbs, stairs and rough ground. Sensor motion during walking is measured using visual egomotion recovery and odometry, and has been modelled to allow compensation in the sensor readings. A ground position estimate is continually updated by scene ground-plane fitting, or from the walk-motion model, and is used to classify scene features as obstacles or parts of the ground. Methods for the robust reconstruction of image points and lines into scene features are developed. The recognition of world objects of exceptional significance to a mobile person - kerbs and stairs - is given particular attention. A user interface, which has undergone limited real world testing, is also described. Experimental results are presented for the various parts of the system.

Interaction between hand and wearable camera in 2D and 3D environments

This paper is concerned with allowing the user of a wearable, portable, vision system to interact with the visual information using hand movements and gestures. Two example scenarios are explored. The first, in 2D, uses the wearer’s hand to both guide an active wearable camera and to highlight objects of interest using a grasping vector. The second is based in 3D, and builds on earlier work which recovers 3D scene structure at video-rate, allowing real-time purposive redirection of the camera to any scene point. Here, a range of hand gestures are used to highlight and select 3D points within the structure and in this instance used to insert 3D graphical objects into the scene. Structure recovery, gesture recognition, scene annotation and augmentation are achieved in parallel and at video-rate.

Robotic Sensing for the Guidance of the Visually Impaired

This paper describes ongoing work into a portable mobility aid, worn by the visually impaired. The system uses stereo vision and sonar sensors for obstacle avoidance and recognition of kerbs. Because the device is carried, the user is given freedom of movement over kerbs, stairs and rough ground, not traversable with a wheeled aid. Motion of the sensor due to the walking action is measured using a digital compass and inclinometer. This motion has been modelled and is tracked to allow compensation of sensor measurements. The vision obstacle detection method uses comparison of image feature disparity with a ground feature disparity function. The disparity function is continually updated by the walk-motion model and by scene ground-plane fitting. Kerb detection is achieved by identifying clusters of parallel lines using the Hough transform. Experimental results are presented from the vision and sonar parts of the system.

Visual enhancement of incised text

This paper describes ongoing research into the application of pattern recognition to incised documents, for which grooves and other structural surface markings are significant, rather than surface coloration. We deduce incised stroke information by imaging the document under a carefully selected set of lighting conditions that cause shadows to be cast, and observing the position and motion of shadow areas. The removal of wood grain noise, image registration, and the detection of image features using phase congruency techniques form the basis of our work. We also briefly describe our approach to interpolating broken strokes and recovering 3D surface structure.

Modelling the motion of a sensor attached to a walking person

Abstract This paper describes an implemented system to measure and track the motion of a sensor which is attached to a person or walking robot. The complex gait motion undergone by a structure-measuring sensor during walking introduces significant uncertainty into the measurements it takes. By measuring and tracking the sensor’s motion, more meaningful information can be obtained from the measurements, and predictions can be made about the future observed position of important objects in the world. The use of motion measuring devices is discussed and compared to the possible estimation of motion by the structural sensor itself. Analysis of walker motion is performed through the use of Gabor wavelets and the Extended Kalman Filter. The method tracks and predicts motions reliably while the walker walks in a straight path or turns corners. Particular attention is focussed on the case where the structural sensor is computer vision, and the benefit that a motion model can provide with it.