Janosch Nikolic

The EuRoC micro aerial vehicle datasets

This paper presents visual-inertial datasets collected on-board a micro aerial vehicle. The datasets contain synchronized stereo images, IMU measurements and accurate ground truth. The first batch of datasets facilitates the design and evaluation of visual-inertial localization algorithms on real flight data. It was collected in an industrial environment and contains millimeter accurate position ground truth from a laser tracking system. The second batch of datasets is aimed at precise 3D environment reconstruction and was recorded in a room equipped with a motion capture system. The datasets contain 6D pose ground truth and a detailed 3D scan of the environment. Eleven datasets are provided in total, ranging from slow flights under good visual conditions to dynamic flights with motion blur and poor illumination, enabling researchers to thoroughly test and evaluate their algorithms. All datasets contain raw sensor measurements, spatio-temporally aligned sensor data and ground truth, extrinsic and intrinsic calibrations and datasets for custom calibrations.

Compressive power spectral density estimation

In this paper, we consider power spectral density estimation of bandlimited, wide-sense stationary signals from sub-Nyquist sampled data. This problem has recently received attention from within the emerging field of cognitive radio for example, and solutions have been proposed that use ideas from compressed sensing and the theory of digital alias-free signal processing. Here we develop a compressed sensing based technique that employs multi-coset sampling and produces multi-resolution power spectral estimates at arbitrarily low average sampling rates. The technique applies to spectrally sparse and nonsparse signals alike, but we show that when the wide-sense stationary signal is spectrally sparse, compressed sensing is able to enhance the estimator. The estimator does not require signal reconstruction and can be directly obtained from a straightforward application of nonnegative least squares.

A UAV system for inspection of industrial facilities

This work presents a small-scale Unmanned Aerial System (UAS) capable of performing inspection tasks in enclosed industrial environments. Vehicles with such capabilities have the potential to reduce human involvement in hazardous tasks and can minimize facility outage periods. The results presented generalize to UAS exploration tasks in almost any GPS-denied indoor environment. The contribution of this work is twofold. First, results from autonomous flights inside an industrial boiler of a power plant are presented. A lightweight, vision-aided inertial navigation system provides reliable state estimates under difficult environmental conditions typical for such sites. It relies solely on measurements from an on-board MEMS inertial measurement unit and a pair of cameras arranged in a classical stereo configuration. A model-predictive controller allows for efficient trajectory following and enables flight in close proximity to the boiler surface. As a second contribution, we highlight ongoing developments by displaying state estimation and structure recovery results acquired with an integrated visual/inertial sensor that will be employed on future aerial service robotic platforms. A tight integration in hardware facilitates spatial and temporal calibration of the different sensors and thus enables more accurate and robust ego-motion estimates. Comparison with ground truth obtained from a laser tracker shows that such a sensor can provide motion estimates with drift rates of only few centimeters over the period of a typical flight.

Aerial service robots for visual inspection of thermal power plant boiler systems

This work focuses on the use of MAVs for industrial inspection tasks. An efficient flight controller based on a model predictive control paradigm is developed. It allows for agile maneuvers in confined spaces while incorporating delays, saturations and inaccurate vehicle state estimates only available at low rate. The fast gradient method is used to solve the optimization problem and meet real-time constraints, given limited computational resources. The vehicle state is estimated from an on-board forward-looking camera system, tightly fused with inertial measurements. Experiments using a realistic industrial mock environment demonstrate the effectiveness, robustness and limitations of the proposed approach. The results show that egomotion estimation is robust under rapid motion, in poorly textured environments and under challenging lighting conditions. When coupled with the model predictive controller, the system requires only limited computational resources and sufficiently tracks an arbitrary trajectory.

Robust embedded egomotion estimation

This work presents a method for estimating the egomotion of an aerial vehicle in challenging industrial environments. It combines binocular visual and inertial cues in a tightly-coupled fashion and operates in real time on an embedded platform. An extended Kalman filter fuses measurements and makes motion estimation rely more on inertial data if visual feature constellation is degenerate. Errors in roll and pitch are bounded implicitly by the gravity vector. Inertial sensors are used for efficient outlier detection and enable operation in poorly and repetitively textured environments. We demonstrate robustness and accuracy in an industrial scenario as well as in general indoor environments. The former is accompanied by a detailed performance evaluation supported with ground truth measurements from an external tracking system.

Aerial service robotics: The AIRobots perspective

This paper presents the main vision and research activities of the ongoing European project AIRobots (Innovative Aerial Service Robot for Remote Inspection by Contact, www.airobots.eu). The goal of AIRobots is to develop a new generation of aerial service robots capable of supporting human beings in all those activities that require the ability to interact actively and safely with environments not constrained on ground but, indeed, airborne. Besides presenting the main ideas and the research activities within the three-year project, the paper shows the first technological outcomes obtained during the first year and a half of activity.

Real-time 6D stereo Visual Odometry with non-overlapping fields of view

In this paper, we present a framework for 6D absolute scale motion and structure estimation of a multi-camera system in challenging indoor environments. It operates in real-time and employs information from two cameras with non-overlapping fields of view. Monocular Visual Odometry supplying up-to-scale 6D motion information is carried out in each of the cameras, and the metric scale is recovered via a linear solution by imposing the known static transformation between both sensors. The redundancy in the motion estimates is finally exploited by a statistical fusion to an optimal 6D metric result. The proposed technique is robust to outliers and able to continuously deliver a reasonable measurement of the scale factor. The quality of the framework is demonstrated by a concise evaluation on indoor datasets, including a comparison to accurate ground truth data provided by an external motion tracking system.

Maximum Likelihood Identification of Inertial Sensor Noise Model Parameters

Accurate visual-inertial localization and mapping systems require accurate calibration and good sensor error models. To this end, we present a simple offline method to automatically determine the parameters of inertial sensor noise models. The proposed methodology identifies noise processes across a large range of strength and time-scales, for example, weak gyroscope bias fluctuations buried in broadband noise. This is accomplished with a classical maximum likelihood estimator, based on the integrated process (i.e., the angle, velocity, or position), rather than on the angular rate or acceleration as is standard in the literature. This trivial modification allows us to capture noise processes according to their effect on the integrated process, irrespective of their contribution to rate or acceleration noise. The cause of the noise is not discussed in this article. The method is tested on different classes of sensors by automatically identifying the parameters of a standard inertial sensor noise model. The results are analyzed qualitatively by comparing the models Allan variance to the Allan variance computed directly from sensor data. A simulation that resembles one of the devices under test facilitates a quantitative analysis of the proposed estimator. Comparison with a competing, state-of-the-art method shows the advantages of the algorithm.

Aerial Service Robots: An overview of the AIRobots activity

This video paper outlines some of the results achieved during the first two years of the ongoing European project AIRobots (Innovative Aerial Service Robots for Remote Inspection by Contact, www.airobots.eu). Goal of AIRobots is to develop a new generation of aerial service robots capable of supporting human beings in all those activities that require the ability to interact actively and safely with environments not constrained on ground but, indeed, airborne. The flight tests presented in the video show the capabilities of the prototypes developed in the project to accomplish tasks including 3D environmental reconstruction using stereo vision, obstacle avoidance and remote manipulation by means of an on-board robotic arm.

Extending kalibr: Calibrating the extrinsics of multiple IMUs and of individual axes

An increasing number of robotic systems feature multiple inertial measurement units (IMUs). Due to competing objectives-either desired vicinity to the center of gravity when used in controls, or an unobstructed field of view when integrated in a sensor setup with an exteroceptive sensor for ego-motion estimation-individual IMUs are often mounted at considerable distance. As a result, they sense different accelerations when the platform is subjected to rotational motions. In this work, we derive a method for spatially calibrating multiple IMUs in a single estimator based on the open-source camera/IMU calibration toolbox kalibr. We further extend the toolbox to determine IMU intrinsics, enabling accurate calibration of low-cost IMUs. The results suggest that the extended estimator is capable of precisely determining these intrinsics and even of localizing individual accelerometer axes inside a commercial grade IMU to millimeter precision.