Steffen Urban

Distinctive 2D and 3D features for automated large-scale scene analysis in urban areas

We propose a new methodology for large-scale urban 3D scene analysis in terms of automatically assigning 3D points the respective semantic labels. The methodology focuses on simplicity and reproducibility of the involved components as well as performance in terms of accuracy and computational efficiency. Exploiting a variety of low-level 2D and 3D geometric features, we further improve their distinctiveness by involving individual neighborhoods of optimal size. Due to the use of individual neighborhoods, the methodology is not tailored to a specific dataset, but in principle designed to process point clouds with a few millions of 3D points. Consequently, an extension has to be introduced for analyzing huge 3D point clouds with possibly billions of points for a whole city. For this purpose, we propose an extension which is based on an appropriate partitioning of the scene and thus allows a successive processing in a reasonable time without affecting the quality of the classification results. We demonstrate the performance of our methodology on two labeled benchmark datasets with respect to robustness, efficiency, and scalability. Graphical abstractWe propose a new methodology for large-scale urban 3D scene analysis which is based on distinctive 2D and 3D features derived from optimal neighborhoods.Display Omitted HighlightsWe present a new methodology for large-scale urban 3D point cloud classification.We analyze a strategy for recovering individual 3D neighborhoods of optimal size.Our methodology involves efficient feature extraction and classification.Our methodology contains an extension towards data-intensive processing.We evaluate our methodology on two recent, publicly available point cloud datasets.

MultiCol Bundle Adjustment: A Generic Method for Pose Estimation, Simultaneous Self-Calibration and Reconstruction for Arbitrary Multi-Camera Systems

In this paper, we present a generic, modular bundle adjustment method for pose estimation, simultaneous self-calibration and reconstruction for multi-camera systems. In contrast to other approaches that use bearing vectors (camera rays) as observations, we extend the common collinearity equations with a general camera model and include the relative orientation of each camera w.r.t to the fixed multi-camera system frame yielding the extended collinearity equations that directly express all image observations as functions of all unknowns. Hence, we can either calibrate the camera system, the cameras, reconstruct the observed scene, and/or simply estimate the pose of the system by including the corresponding parameter block into the Jacobian matrix. Apart from evaluating the implementation with comprehensive simulations, we benchmark our method against recently published methods for pose estimation and bundle adjustment for multi-camera systems. Finally, all methods are evaluated using a 6 degree of freedom ground truth data set, that was recorded with a lasertracker.

mdBRIEF - a fast online-adaptable, distorted binary descriptor for real-time applications using calibrated wide-angle or fisheye cameras

Abstract Fast binary descriptors build the core for many vision based applications with real-time demands like object detection, visual odometry or SLAM. Commonly it is assumed, that the acquired images and thus the patches extracted around keypoints originate from a perspective projection ignoring image distortion or completely different types of projections such as omnidirectional or fisheye. Usually the deviations from a perfect perspective projection are corrected by using standard undistortion models. The latter, however, introduce artifacts if the camera’s field-of-view gets larger. In addition, many applications (e.g. monocular SLAM) require only undistorted points and holistic undistortion of every image for descriptor extraction could be eluded. In this paper, we propose a distorted and masked version of the BRIEF descriptor for calibrated cameras, called dBRIEF and mdBRIEF respectively. Instead of correcting the distortion holistically, we distort the binary tests and thus adapt the descriptor to different image regions. The implementation of the proposed method along with evaluation scripts can be found online at https://github.com/urbste/mdBRIEF .

LaFiDa—A Laserscanner Multi-Fisheye Camera Dataset

In this article, the Laserscanner Multi-Fisheye Camera Dataset (LaFiDa) for applying benchmarks is presented. A head-mounted multi-fisheye camera system combined with a mobile laserscanner was utilized to capture the benchmark datasets. Besides this, accurate six degrees of freedom (6 DoF) ground truth poses were obtained from a motion capture system with a sampling rate of 360 Hz. Multiple sequences were recorded in an indoor and outdoor environment, comprising different motion characteristics, lighting conditions, and scene dynamics. The provided sequences consist of images from three—by hardware trigger—fully synchronized fisheye cameras combined with a mobile laserscanner on the same platform. In total, six trajectories are provided. Each trajectory also comprises intrinsic and extrinsic calibration parameters and related measurements for all sensors. Furthermore, we generalize the most common toolbox for an extrinsic laserscanner to camera calibration to work with arbitrary central cameras, such as omnidirectional or fisheye projections. The benchmark dataset is available online released under the Creative Commons Attributions Licence (CC-BY 4.0), and it contains raw sensor data and specifications like timestamps, calibration, and evaluation scripts. The provided dataset can be used for multi-fisheye camera and/or laserscanner simultaneous localization and mapping (SLAM).

MLPnP - A real-time maximum likelihood solution to the perspective-n-point problem

Abstract. In this paper, a statistically optimal solution to the Perspective-n-Point (PnP) problem is presented. Many solutions to the PnP problem are geometrically optimal, but do not consider the uncertainties of the observations. In addition, it would be desirable to have an internal estimation of the accuracy of the estimated rotation and translation parameters of the camera pose. Thus, we propose a novel maximum likelihood solution to the PnP problem, that incorporates image observation uncertainties and remains real-time capable at the same time. Further, the presented method is general, as is works with 3D direction vectors instead of 2D image points and is thus able to cope with arbitrary central camera models. This is achieved by projecting (and thus reducing) the covariance matrices of the observations to the corresponding vector tangent space.