Sammy Omari

The EuRoC micro aerial vehicle datasets

This paper presents visual-inertial datasets collected on-board a micro aerial vehicle. The datasets contain synchronized stereo images, IMU measurements and accurate ground truth. The first batch of datasets facilitates the design and evaluation of visual-inertial localization algorithms on real flight data. It was collected in an industrial environment and contains millimeter accurate position ground truth from a laser tracking system. The second batch of datasets is aimed at precise 3D environment reconstruction and was recorded in a room equipped with a motion capture system. The datasets contain 6D pose ground truth and a detailed 3D scan of the environment. Eleven datasets are provided in total, ranging from slow flights under good visual conditions to dynamic flights with motion blur and poor illumination, enabling researchers to thoroughly test and evaluate their algorithms. All datasets contain raw sensor measurements, spatio-temporally aligned sensor data and ground truth, extrinsic and intrinsic calibrations and datasets for custom calibrations.

Three-dimensional coverage path planning via viewpoint resampling and tour optimization for aerial robots

This paper presents a new algorithm for three-dimensional coverage path planning for autonomous structural inspection operations using aerial robots. The proposed approach is capable of computing short inspection paths via an alternating two-step optimization algorithm according to which at every iteration it attempts to find a new and improved set of viewpoints that together provide full coverage with decreased path cost. The algorithm supports the integration of multiple sensors with different fields of view, the limitations of which are respected. Both fixed-wing as well as rotorcraft aerial robot configurations are supported and their motion constraints are respected at all optimization steps, while the algorithm operates on both mesh- and occupancy map-based representations of the environment. To thoroughly evaluate this new path planning strategy, a set of large-scale simulation scenarios was considered, followed by multiple real-life experimental test-cases using both vehicle configurations.

Hardware and Software Architecture for Nonlinear Control of Multirotor Helicopters

This paper presents the design and implementation of a nonlinear control scheme for multirotor helicopters that takes first-order drag effects into account explicitly. A dynamic model including the blade flapping and induced drag forces is provided and a hierarchical nonlinear controller is presented. This controller is designed for both high-precision flights as well as robustness against model uncertainties and external disturbances. This is achieved by using saturated integrators with fast desaturation properties. The implementation of the controller on the flybox hexacopter platform is described. The hardware and software architecture of this UAV is discussed, and useful hints and insights gained during its design process are presented. Finally, experimental results and videos are reported to demonstrate the successful implementation and the performance of the overall system.

Nonlinear control of VTOL UAVs incorporating flapping dynamics

This paper presents the design and evaluation of a nonlinear control scheme for multirotor helicopters that takes first-order drag effects into account explicitly. A dynamic model including the blade flapping and induced drag forces is presented. Based on this model, a hierarchical nonlinear controller is designed to actively compensates for the nonlinear effects these drag forces. Reported simulation and experimental results indicate the significant performance improvement of the proposed drag-augmented control scheme with respect to a conventional nonlinear controller. For completeness, an offline procedure allowing for efficiently identifying the drag parameters is proposed.

Bilateral haptic teleoperation of VTOL UAVs

This paper presents an intuitive teleoperation scheme to safely operate a wide range of VTOL UAVs by an untrained user in a cluttered environment. This scheme includes a novel force-feedback algorithm that enables the user to feel the texture of the environment. In addition, a novel mapping function is introduced to teleoperate the UAV in an unlimited workspace in position control mode with a joystick which has a limited workspace. An obstacle avoidance strategy is designed to autonomously modify the position set point of the UAV independently of the pilots commands. The stability analysis of the whole teleoperation loop is provided. Experiments demonstrate the successful teleoperation of a UAV using an haptic joystick and a hexacopter UAV equipped with a 2D laser-range scanner.

Dense visual-inertial navigation system for mobile robots

Real-time dense mapping and pose estimation is essential for a wide range of navigation tasks in mobile robotic applications. We propose an odometry and mapping system that leverages the full photometric information from a stereo-vision system as well as inertial measurements in a probabilistic framework while running in real-time on a single low-power Intel CPU core. Instead of performing mapping and localization on a set of sparse image features, we use the complete dense image intensity information in our navigation system. By incorporating a probabilistic model of the stereo sensor and the IMU, we can robustly estimate the ego-motion as well as a dense 3D model of the environment in real-time. The probabilistic formulation of the joint odometry estimation and mapping process enables to efficiently reject temporal outliers in ego-motion estimation as well as spatial outliers in the mapping process. To underline the versatility of the proposed navigation system, we evaluate it in a set of experiments on a multi-rotor system as well as on a quadrupedal walking robot. We tightly integrate our framework into the stabilization-loop of the UAV and the mapping framework of the walking robot. It is shown that the dense framework exhibits good tracking and mapping performance in terms of accuracy as well as robustness in scenarios with highly dynamic motion patterns while retaining a relatively small computational footprint. This makes it an ideal candidate for control and navigation tasks in unstructured GPS-denied environments, for a wide range of robotic platforms with power and weight constraints. The proposed framework is released as an open-source ROS package.

Visual industrial inspection using aerial robots

The use of unmanned aerial vehicles (UAV) offers a unique possibility to capture visual information in areas which are hard to reach or dangerous for humans. For UAVs to become a standard tool in visual inspection, it is of utmost importance that the aerial robot can be operated efficiently by a non-expert UAV pilot and that the navigation system is robust enough to remain operational in rough, industrial conditions. To this end, we present a UAV navigation system setup that uses visual-inertial sensor cues to estimate the UAV pose as well as to create a dense 3D map of the environment in real-time onboard the UAV, completely independent of GPS. The proposed navigation system enables the operator to directly interface the UAV using high-level commands such as waypoints or velocity commands while the navigation system ensures a stable and collision-free flight.

Omnidirectional visual obstacle detection using embedded FPGA

For autonomous navigation of Micro Aerial Vehicles (MAVs) in cluttered environments, it is essential to detect potential obstacles not only in the direction of flight but in their entire local environment. While there exist systems that do vision based obstacle detection, most of them are limited to a single perception direction. Extending these systems to a multi-directional sensing approach would exhaust the payload limit in terms of weight and computational power. We present a novel light-weight sensor setup comprising of four stereo heads and an inertial measurement unit (IMU) to perform FPGA-based dense reconstruction for obstacle detection in all directions. As the data-rate scales up with the number of cameras we use an FPGA to perform streaming based tasks in real-time and show a light-weight polar-coordinate map to allow a companion computer to fully process the data of all the cameras and perform obstacle detection in real-time. The system is able to process up to 80 frames per second (fps) freely distributed on the four stereo heads while maintaining a low power budget. The perception system including FPGA, image sensors and stereo mounts is 235 g in weight.

An incremental sampling-based approach to inspection planning: the rapidly exploring random tree of trees

A new algorithm, called rapidly exploring random tree of trees (RRTOT) is proposed, that aims to address the challenge of planning for autonomous structural inspection. Given a representation of a structure, a visibility model of an onboard sensor, an initial robot configuration and constraints, RRTOT computes inspection paths that provide full coverage. Sampling based techniques and a meta-tree structure consisting of multiple RRT* trees are employed to find admissible paths with decreasing cost. Using this approach, RRTOT does not suffer from the limitations of strategies that separate the inspection path planning problem into that of finding the minimum set of observation points and only afterwards compute the best possible path among them. Analysis is provided on the capability of RRTOT to find admissible solutions that, in the limit case, approach the optimal one. The algorithm is evaluated in both simulation and experimental studies. An unmanned rotorcraft equipped with a vision sensor was utilized as the experimental platform and validation of the achieved inspection properties was performed using 3D reconstruction techniques.

Towards autonomous mine inspection

The purpose of this paper is to evaluate the use of a micro aerial vehicle (MAV) for autonomous inspection and 3D reconstruction of underground mines. The goal is to manually fly an MAV equipped with cameras and a laser range sensor into a vertical shaft to collect data. This data can be used to evaluate the performance of the localization system as well as post processed to reconstruct a 3D model of the shaft. Due to its novelty of flying an MAV in a deep mine, we report gained experience of the effect of the hot, wet and dusty environment on the system as well as the influence of turbulences from vertical winds on the flight performance. Further we evaluated the quality of the recorded data and there applicability for a fully autonomous mine inspection system.