Depth distillation: unsupervised metric depth estimation for UAVs by finding consensus between kinematics, optical flow and deep learning

Abstract

Estimating precise metric depth is an essential task for UAV navigation, which is very difficult to learn unsupervised without access to odometry. At the same time, depth recovery from kinematics and optical flow is mathematically precise, but less numerically stable and robust, especially in the focus of expansion areas. We propose a model that combines the analytical, vision-with-odometry approach, with deep unsupervised learning, into a single formulation for metric depth estimation, which is both fast and accurate. The two pathways – analytical and data-driven – form a robust ensemble, which provides supervisory signal to a single deep net that distills the consensus between scene geometry, pose, kinematics, camera intrinsics and the input RGB. The distilled net has low runtime and memory costs, being suitable for embedded devices. We validate our results against an off-the-shelf SfM-based solution. We also introduce a new real-world dataset of almost 20 minutes of continuous UAV flight, on which we demonstrate better accuracy and capabilities than the deep learning and analytical approaches.