Korbinian Schmid

Toward a Fully Autonomous UAV: Research Platform for Indoor and Outdoor Urban Search and Rescue

Urban search and rescue missions raise special requirements on robotic systems. Small aerial systems provide essential support to human task forces in situation assessment and surveillance. As external infrastructure for navigation and communication is usually not available, robotic systems must be able to operate autonomously. A limited payload of small aerial systems poses a great challenge to the system design. The optimal tradeoff between flight performance, sensors, and computing resources has to be found. Communication to external computers cannot be guaranteed; therefore, all processing and decision making has to be done on board. In this article, we present an unmanned aircraft system design fulfilling these requirements. The components of our system are structured into groups to encapsulate their functionality and interfaces. We use both laser and stereo vision odometry to enable seamless indoor and outdoor navigation. The odometry is fused with an inertial measurement unit in an extended Kalman filter. Navigation is supported by a module that recognizes known objects in the environment. A distributed computation approach is adopted to address the computational requirements of the used algorithms. The capabilities of the system are validated in flight experiments, using a quadrotor.

Stereo vision based indoor/outdoor navigation for flying robots

We introduce our new quadrotor platform for realizing autonomous navigation in unknown indoor/outdoor environments. Autonomous waypoint navigation, obstacle avoidance and flight control is implemented on-board. The system does not require a special environment, artificial markers or an external reference system. We developed a monolithic, mechanically damped perception unit which is equipped with a stereo camera pair, an Inertial Measurement Unit (IMU), two processor-and an FPGA board. Stereo images are processed on the FPGA by the Semi-Global Matching algorithm. Keyframe-based stereo odometry is fused with IMU data compensating for time delays that are induced by the vision pipeline. The system state estimate is used for control and on-board 3D mapping. An operator can set waypoints in the map, while the quadrotor autonomously plans its path avoiding obstacles. We show experiments with the quadrotor flying from inside a building to the outside and vice versa, traversing a window and a door respectively. A video of the experiments is part of this work. To the best of our knowledge, this is the first autonomously flying system with complete on-board processing that performs waypoint navigation with obstacle avoidance in geometrically unconstrained, complex indoor/outdoor environments.

Autonomous Vision-based Micro Air Vehicle for Indoor and Outdoor Navigation

Micro air vehicles have become very popular in recent years. Autonomous navigation of such systems plays an important role in many industrial applications as well as in search-and-rescue scenarios. We present a quadrotor that performs autonomous navigation in complex indoor and outdoor environments. An operator selects target positions in the onboard map and the system autonomously plans an obstacle-free path and flies to these locations. An onboard stereo camera and inertial measurement unit are the only sensors. The system is independent of external navigation aids such as GPS. No assumptions are made about the structure of the unknown environment. All navigation tasks are implemented onboard the system. A wireless connection is only used for sending images and a three-dimensional 3D map to the operator and to receive target locations. We discuss the hardware and software setup of the system in detail. Highlights of the implementation are the field-programmable-gate-array-based dense stereo matching of 0.5 Mpixel images at a rate of 14.6 Hz using semiglobal matching, locally drift-free visual odometry with key frames, and sensor data fusion with compensation of measurement delays of 220i¾?ms. We show the robustness of the approach in simulations and experiments with ground truth. We present the results of a complex, autonomous indoor/outdoor flight and the exploration of a coal mine with obstacle avoidance and 3D mapping.

Stereo vision and IMU based real-time ego-motion and depth image computation on a handheld device

Real-time environmental depth perception and ego-motion estimation is essential for all mobile robotic systems. We present a system that computes high quality depth images with 0.5 MPixel resolution using Semi-Global Matching (SGM) and estimates the ego-motion by key frame based visual odometry fused with the data of an inertial measurement unit (IMU). The hardware includes a pair of cameras, a small Intel Core2Duo CPU board, a Spartan 6 FPGA board, an OMAP3530 ARM processor board as well as an IMU. The total weight of the experimental setup is 830 g and is, thus, also feasible for hand-held or flying platforms. Experiments show that the vision system runs at 14.6 Hz with a latency of around 250 ms and produces high quality depth images as well as reliable 6D ego-motion estimates. In the fusion algorithm of visual odometry and IMU data, time delays of the vision system are compensated and a system state estimate is available at the full data rate of the IMU which is important for system control. This paper presents the integration of different techniques into a fast, light weight, real-time system and validates its performance by experiments on real data.

State estimation for highly dynamic flying systems using key frame odometry with varying time delays

System state estimation is an essential part for robot navigation and control. A combination of Inertial Navigation Systems (INS) and further exteroceptive sensors such as cameras or laser scanners is widely used. On small robotic systems with limitations in payload, power consumption and computational resources the processing of exteroceptive sensor data often introduces time delays which have to be considered in the sensor data fusion process. These time delays are especially critical in the estimation of system velocity. In this paper we present a state estimation framework fusing an INS with time delayed, relative exteroceptive sensor measurements. We evaluate its performance for a highly dynamic flight system trajectory including a flip. The evolution of velocity and position errors for varying measurement frequencies from 15Hz to 1Hz and time delays up to 1s is shown in Monte Carlo simulations. The filter algorithm with key frame based odometry permits an optimal, local drift free navigation while still being computationally tractable on small onboard computers. Finally, we present the results of the algorithm applied to a real quadrotor by flying from inside a house out through the window.

View Planning for Multi-View Stereo 3D Reconstruction Using an Autonomous Multicopter

Multi-view stereo algorithms are an attractive technique for the digital reconstruction of outdoor sites. Concerning the data acquisition process a vertical take off and landing UAV carrying a digital camera is a suitable platform in terms of mobility and flexibility in viewpoint placement. We introduce an automated UAV based data acquisition and outdoor site reconstruction system. A special focus is set on the problem of model based view planning using a coarse digital surface model (DSM) with minimal data preprocessing. The developed view planning heuristic considers a coverage, a maximum view angle and an overlapping constraint imposed by multi-view stereo reconstruction techniques. The time complexity of the algorithm is linear with respect to the size of the area of interest. We demonstrate the efficiency of the entire system in two scenarios, a building and a hillside.

The flying anemometer: Unified estimation of wind velocity from aerodynamic power and wrenches

We consider the problem of estimating the wind velocity perceived by a flying multicopter, from data acquired by onboard sensors and knowledge of its aerodynamics model only. We employ two complementary methods. The first is based on the estimation of the external wrench (force and torque) due to aerodynamics acting on the robot in flight. Wind velocity is obtained by inverting an identified model of the aerodynamic forces. The second method is based on the estimation of the propeller aerodynamic power, and provides an estimate independent of other sensors. We show how to calculate components of the wind velocity using multiple aerodynamic power measurements, when the poses between them are known. The method uses the motor current and angular velocity as measured by the electronic speed controllers, essentially using the propellers as wind sensors. Verification of the methods and model identification were done using measurements acquired during autonomous flights in a 3D wind tunnel.

Distributed stereo vision-based 6D localization and mapping for multi-robot teams: SCHUSTER et al.

Joint simultaneous localization and mapping (SLAM) constitutes the basis for cooperative action in multi-robot teams. We designed a stereo vision-based 6D SLAM system combining local and global methods to benefit from their particular advantages: (1) Decoupled local reference filters on each robot for real-time, long-term stable state estimation required for stabilization, control and fast obstacle avoidance; (2) Online graph optimization with a novel graph topology and intra- as well as inter-robot loop closures through an improved submap matching method to provide global multi-robot pose and map estimates; (3) Distribution of the processing of high-frequency and high-bandwidth measurements enabling the exchange of aggregated and thus compacted map data. As a result, we gain robustness with respect to communication losses between robots. We evaluated our improved map matcher on simulated and real-world datasets and present our full system in five real-world multi-robot experiments in areas of up 3,000 m2 (bounding box), including visual robot detections and submap matches as loop-closure constraints. Further, we demonstrate its application to autonomous multi-robot exploration in a challenging rough-terrain environment at a Moon-analogue site located on a volcano.