Monocular Semantic Mapping Based on 3D Cuboids Tracking

Abstract

Semantic mapping based on information of objects has become a crucial component for the surrounding comprehension and the more robust navigation. In this paper, we propose a system for simultaneous localization and mapping (SLAM) that combines multiple objects tracking and factor graph optimization with semantically meaningful landmarks to achieve accurate monocular semantic mapping. Firstly, the process of object recognition uses a vanishing point sampling-based approach to efficiently infer the class and position of object landmarks from 2D bounding box object detection. Secondly, The semantic frontend utilizes local matching-based data association to track raw cuboid proposals. It can provide semantic constraints to reduce cuboid scale drift and improve its position estimation. Finally, we present a multi-view factor graph optimization which can use motion modal of the camera to optimize stable cuboids. The semantic mapping experiments on our own built virtual scene show better accuracy and robustness over existing approaches. We evaluate the effectiveness of our approach on public KITTI datasets and a real scene.