Michael Happold

Toward Reliable Off Road Autonomous Vehicles Operating in Challenging Environments

The DARPA PerceptOR program has implemented a rigorous evaluative test program which fosters the development of field relevant outdoor mobile robots. Autonomous ground vehicles were deployed on diverse test courses throughout the USA and quantitatively evaluated on such factors as autonomy level, waypoint acquisition, failure rate, speed, and communications bandwidth. Our efforts over the three year program have produced new approaches in planning, perception, localization, and control which have been driven by the quest for reliable operation in challenging environments. This paper focuses on some of the most unique aspects of the systems developed by the CMU PerceptOR team, the lessons learned during the effort, and the most immediate challenges that remain to be addressed.

The Demeter System for Automated Harvesting

Automation of agricultural harvesting equipment in the near term appears both economically viable and technically feasible. This paper describes the Demeter system for automated harvesting. Demeter is a computer-controlled speedrowing machine, equipped with a pair of video cameras and a global positioning sensor for navigation. Demeter is capable of planning harvesting operations for an entire field, and then executing its plan by cutting crop rows, turning to cut successive rows, repositioning itself in the field, and detecting unexpected obstacles. In August of 1997, the Demeter system autonomously harvested 40 hectares (100 acres) of crop in a continuous run (excluding stops for refueling). During 1998, the Demeter system harvested in excess of 48.5 hectares (120 acres) of crop, cutting in a variety of fields.

Enhancing Supervised Terrain Classification with Predictive Unsupervised Learning

This paper describes a method for classifying the traversability of terrain by combining unsupervised learning of color models that predict scene geometry with supervised learning of the relationship between geometric features and traversability. A neural network is trained offline on hand-labeled geometric features computed from stereo data. An online process learns the association between color and geometry, enabling the robot to assess the traversability of regions for which there is little range information by estimating the geometry from the color of the scene and passing this to the neural network. This online process is continuous and extremely rapid, which allows for quick adaptations to different lighting conditions and terrain changes. The sensitivity of the traversability judgment is further adjusted online by feedback from the robot’s bumper. Terrain assessments from the color classifier are merged with pure geometric classifications in an occupancy grid by computing the intersection of the ray associated with a pixel with a ground plane computed from the stereo range data. We present results from DARPA-conducted tests that demonstrate its effectiveness in a variety of outdoor environments.

A Bayesian approach to imitation learning for robot navigation

Driving in unknown natural outdoor terrain is a challenge for autonomous ground vehicles. It can be difficult for a robot to discern obstacles and other hazards in its environment, and characteristics of this high cost terrain may change from one environment to another, or even with different lighting conditions. One successful approach to this problem is for a robot to learn from a demonstration by a human operator. In this paper, we describe an approach to calculating terrain costs from Bayesian estimates using feature vectors measured during a short teleoperated training run in similar terrain and conditions. We describe the theory, its implementation on two different robotic systems, and results of several independently conducted field tests.

Autonomous Learning of Terrain Classification within Imagery for Robot Navigation

Stereo matching in unstructured, outdoor environments is often confounded by the complexity of the scenery and thus may yield only sparse disparity maps. Two-dimensional visual imagery, on the other hand, offers dense information about the environment of mobile robots, but is often difficult to exploit. Training a supervised classifier to identify traversable regions within images that generalizes well across a large variety of environments requires a vast corpus of labeled examples. Autonomous learning of the traversable/untraversable distinction indicated by scene appearance is therefore a highly desirable goal of robot vision. We describe here a system for learning this distinction online without the involvement of a human supervisor. The system takes in imagery and range data from a pair of stereo cameras mounted on a small mobile robot and autonomously learns to produce a labeling of scenery. Supervision of the learning process is entirely through information gathered from range data. Two types of boosted weak learners, Nearest Means and naive Bayes, are trained on this autonomously labeled corpus. The resulting classified images provide dense information about the environment which can be used to fill-in regions where stereo cannot find matches or in lieu of stereo to direct robot navigation. This method has been tested across a large array of environment types and can produce very accurate labelings of scene imagery as judged by human experts and compared against purely geometric-based labelings. Because it is online and rapid, it eliminates some of the problems related to color constancy and dynamic environments.

Using Learned Features from 3D Data for Robot Navigation

Summary. We describe a novel method for classifying terrain in unstructured, natural environments for the purpose of aiding mobile robot navigation. This method operates on range data provided by stereo without the traditional preliminary extraction of geometric features such as height and slope, replacing these measurements with 2D histograms representing the shape and permeability of objects within a local region. A convolutional neural network is trained to categorize the histogram samples according to the traversability of the terrain they represent for a small mobile robot. In live and offline testing in a wide variety of environments, it demonstrates state-of-the-art performance.