Franz Kurz

An Operational System for Estimating Road Traffic Information from Aerial Images

Given that ground stationary infrastructures for traffic monitoring are barely able to handle everyday traffic volumes, there is a risk that they could fail altogether in situations arising from mass events or disasters. In this work, we present an alternative approach for traffic monitoring during disaster and mass events, which is based on an airborne optical sensor system. With this system, optical image sequences are automatically examined on board an aircraft to estimate road traffic information, such as vehicle positions, velocities and driving directions. The traffic information, estimated in real time on board, is immediately downlinked to a ground station. The airborne sensor system consists of a three-head camera system, a real-time-capable GPS/INS unit, five industrial PCs and a downlink unit. The processing chain for automatic extraction of traffic information contains modules for the synchronization of image and navigation data streams, orthorectification and vehicle detection and tracking modules. The vehicle detector is based on a combination of AdaBoost and support vector machine classifiers. Vehicle tracking relies on shape-based matching operators. The processing chain is evaluated on a large number of image sequences recorded during several campaigns, and the data quality is compared to that obtained from induction loops. In summary, we can conclude that the achieved overall quality of the traffic data extracted by the airborne system is in the range of 68% and 81%. Thus, it is comparable to data obtained from stationary ground sensor networks.

Low-cost optical Camera System for real-time Mapping Applications

Die Beobachtung von Naturkatastrophen, Grosereignissen und Unfallen mit flugzeuggestutzten optischen Sensoren in Echtzeit ist ein derzeit wichtiges Thema in Forschung und Entwicklung. In diesem Zusammenhang wird die Leistungsfahigkeit von preisgunstigen Kamerasystemen fur Echtzeitanwendungen in Hinblick auf geometrische Genauigkeit, radiometrische Eigenschaften und Prozessierungszeiten evaluiert. Der Schwerpunkt liegt bei der Analyse der geometrischen Stabilitat von preisgunstigen Kameras im langjahrigen Betrieb und den Grenzen der direkten Georeferenzierung. Weiterhin wird eine echtzeitfahige Prozessierungskette mit einer GPU (Graphical Processing Unit) basierten Orthorektifizierungsmethode fur eine maximal mogliche Aufnahmerate von 5Hz vorgestellt.

Detection of Traffic Congestion in Optical Remote Sensing Imagery

A new approach for the traffic congestion detection in time series of optical digital camera images is proposed. It is well suited to derive various traffic parameters such as vehicle density, average vehicle velocity, beginning and end of congestion, length of congestion or for other traffic monitoring applications. The method is based on the vehicle detection on the road segment by change detection between two images with a short time lag, the usage of a priori information such as road data base, vehicle sizes and road parameters and a simple linear traffic model based on the spacing between vehicles. The estimated velocity profiles for experimental data acquired by airborne optical remote sensing sensor - 3 K camera system - coincide quite well with the reference measurements.

A new software/hardware architecture for real time image processing of wide area airborne camera images

This paper describes a new software/hardware architecture for processing wide area airborne camera images in real time. The images under consideration are acquired from the 3K-camera system developed at DLR (German Aerospace Center). It consists of three off-the-shelf cameras, each of it delivers 16 Mpixel three times a second. One camera is installed in nadir, whereas the other two cameras are looking in side direction. Main applications of our system are supposed to be automotive traffic monitoring, determining the workload of public road networks during mass events, or obtaining a survey of damages in disaster areas in real time. Altogether, this demands a fast image processing system on the aircraft, because the amount of original high resolution images can not be sent to ground by up-to-date transfer mode systems. The on-board image processing system is distributed over a local network. On each PC several modules are running concurrently. In order to synchronize several processes and to assure access to commonly used data, a new distributed middleware for real time image processing is introduced. Two sophisticated modules one for orthorectification of images and one for traffic monitoring are explained in more detail. The orthorectification and mosaicking is executed on the fast graphics processing unit on one PC, whereas the traffic monitoring module runs on another PC in the on-board network. The resulting image data and evaluated traffic parameters are sent to a ground station in near real time and are distributed to the involved users. Thus, with the here suggested software/hardware system it becomes possible to support rescue forces and security forces in disaster areas or during mass events in near real time.

Automatic UAV Image Geo-Registration by Matching UAV Images to Georeferenced Image Data

Recent years have witnessed the fast development of UAVs (unmanned aerial vehicles). As an alternative to traditional image acquisition methods, UAVs bridge the gap between terrestrial and airborne photogrammetry and enable flexible acquisition of high resolution images. However, the georeferencing accuracy of UAVs is still limited by the low-performance on-board GNSS and INS. This paper investigates automatic geo-registration of an individual UAV image or UAV image blocks by matching the UAV image(s) with a previously taken georeferenced image, such as an individual aerial or satellite image with a height map attached or an aerial orthophoto with a DSM (digital surface model) attached. As the biggest challenge for matching UAV and aerial images is in the large differences in scale and rotation, we propose a novel feature matching method for nadir or slightly tilted images. The method is comprised of a dense feature detection scheme, a one-to-many matching strategy and a global geometric verification scheme. The proposed method is able to find thousands of valid matches in cases where SIFT and ASIFT fail. Those matches can be used to geo-register the whole UAV image block towards the reference image data. When the reference images offer high georeferencing accuracy, the UAV images can also be geolocalized in a global coordinate system. A series of experiments involving different scenarios was conducted to validate the proposed method. The results demonstrate that our approach achieves not only decimeter-level registration accuracy, but also comparable global accuracy as the reference images.

The TUM-DLR Multimodal Earth Observation Evaluation Benchmark

We present a new dataset for development, benchmarking, and evaluation of remote sensing and earth observation approaches with special focus on converging perspectives. In order to provide data with different modalities, we observed the same scene using satellites, airplanes, unmanned aerial vehicles (UAV), and smartphones. The dataset is further complemented by ground-truth information and baseline results for different application scenarios. The provided data can be freely used by anybody interested in remote sensing and earth observation and will be continuously augmented and updated.

CHICAGO – An Airborne Observation System for Security Applications

The paper describes the layout of a new research aircraft for security applications. The typical applications for such an aircraft are outlined and the system requirements are derived. Furthermore, the performance of the system and a first mission is described.

Traffic Data Collection with TerraSAR-X and Performance Evaluation

As the amount of traffic has dramatically increased over the last years, traffic monitoring and traffic data collection have become more and more important. The acquisition of traffic data in almost real-time is essential to immediately react to current traffic situations. Stationary data collectors such as induction loops and video cameras mounted on bridges or traffic lights are matured methods. However, they only provide local data and are not able to observe the traffic situation in a large road network. Hence, traffic monitoring approaches relying on airborne and space-borne remote sensing come into play. Especially space-borne sensors do cover very large areas, even though image acquisition is strictly restricted to certain time slots predetermined by the respective orbit parameters. Space-borne systems thus contribute to the periodic collection of statistical traffic data in order to validate and improve traffic models. On the other hand, the concepts developed for space-borne imagery can be easily transferred to future HALE (High Altitude Long Endurance) systems, which show great potential to meet the demands of both temporal flexibility and spatial coverage.

Optimization of OpenStreetMap Building Footprints Based on Semantic Information of Oblique UAV Images

Building footprint information is vital for 3D building modeling. Traditionally, in remote sensing, building footprints are extracted and delineated from aerial imagery and/or LiDAR point cloud. Taking a different approach, this paper is dedicated to the optimization of OpenStreetMap (OSM) building footprints exploiting the contour information, which is derived from deep learning-based semantic segmentation of oblique images acquired by the Unmanned Aerial Vehicle (UAV). First, a simplified 3D building model of Level of Detail 1 (LoD 1) is initialized using the footprint information from OSM and the elevation information from Digital Surface Model (DSM). In parallel, a deep neural network for pixel-wise semantic image segmentation is trained in order to extract the building boundaries as contour evidence. Subsequently, an optimization integrating the contour evidence from multi-view images as a constraint results in a refined 3D building model with optimized footprints and height. Our method is leveraged to optimize OSM building footprints for four datasets with different building types, demonstrating robust performance for both individual buildings and multiple buildings regardless of image resolution. Finally, we compare our result with reference data from German Authority Topographic-Cartographic Information System (ATKIS). Quantitative and qualitative evaluations reveal that the original OSM building footprints have large offset, but can be significantly improved from meter level to decimeter level after optimization.

Fusion and classification of aerial images from MAVS and airplanes for local information enrichment

Despite the existence of various matching algorithms, matching of images from Micro Aerial Vehicles (MAVs) and airplanes is still a tough problem due to the substantial differences in scale and rotation. This paper investigates the fusion of MAV imagery and airplane imagery and proposes a new robust image matching method with self-adaption to differences in scale and viewing direction. This method is further applied to register a MAV image block with reference to the orthophoto and DSM of a previously-geolocalized aerial image dataset. After registration, a fused 3D point cloud is generated and then combined with images as inputs for land cover (here roofs) classification. Experiments show that the proposed matching method outperforms SIFT/ASIFT methods in both quantity and reliability of matching results, while the registration of MAV imagery achieves decimeter-level accuracy without using any onboard GPS/IMU data. Besides, the pixel-level classification that integrates information of point clouds and images achieves significantly higher accuracy than simply image-based classification.