Gerhard Schall

Handheld Augmented Reality for underground infrastructure visualization

In this paper, we present an Augmented Reality (AR) system for aiding field workers of utility companies in outdoor tasks such as maintenance, planning or surveying of underground infrastructure. Our work addresses these issues using spatial interaction and visualization techniques for mobile AR applications and as well as for a new mobile device design. We also present results from evaluations of the prototype application for underground infrastructure spanning various user groups. Our application has been driven by feedback from industrial collaborators in the utility sector, and includes a translation tool for automatically importing data from utility company databases of underground assets.

Global pose estimation using multi-sensor fusion for outdoor Augmented Reality

Outdoor Augmented Reality typically requires tracking in unprepared environments. For global registration, Global Positioning System (GPS) is currently the best sensing technology, but its precision and update rate are not sufficient for high quality tracking. We present a system that uses Kalman filtering for fusion of Differential GPS (DGPS) or Real-Time Kinematic (RTK) based GPS with barometric heights and also for an inertial measurement unit with gyroscopes, magnetometers and accelerometers to improve the transient oscillation. Typically, inertial sensors are subjected to drift and magnetometer measurements are distorted by electro-magnetic fields in the environment. For compensation, we additionally apply a visual orientation tracker which is drift-free through online mapping of the unknown environment. This tracker allows for correction of distortions of the 3-axis magnetic compass, which increases the robustness and accuracy of the pose estimates. We present results of applying this approach in an industrial application scenario.

Visual tracking for Augmented Reality

Localization of mobile devices is an essential task in Augmented Reality and has therefore been an active research topic for many years. Typically, indoor tracking approaches, such as methods based on infrared or ultra-wide-band, require preparations of the environment and special hardware sensors. Conversely, image feature tracking approaches can provide orientation estimates without special tracking hardware installations. With the advent of mobile devices equipped with sensors such as digital cameras, image based localization gains importance in Augmented Reality. Typically, fiducial marker tracking was considered as a standard image based localization method. We propose the use of natural image feature based tracking methods, which are a generalization of the same principals but do not require the presence of fiducial tracking targets.

Managing Complex Augmented Reality Models

Mobile augmented reality requires georeferenced data to present world-registered overlays. To cover a wide area and all artifacts and activities, a database containing this information must be created, stored, maintained, delivered, and finally used by the client application. We present a data model and a family of techniques to address these needs.

Mobile Augmented Reality: Robust detection and tracking of annotations for outdoor augmented reality browsing

A common goal of outdoor augmented reality (AR) is the presentation of annotations that are registered to anchor points in the real world. We present an enhanced approach for registering and tracking such anchor points, which is suitable for current generation mobile phones and can also successfully deal with the wide variety of viewing conditions encountered in real life outdoor use. The approach is based on on-the-fly generation of panoramic images by sweeping the camera over the scene. The panoramas are then used for stable orientation tracking, while the user is performing only rotational movements. This basic approach is improved by several new techniques for the re-detection and tracking of anchor points. For the re-detection, specifically after temporal variations, we first compute a panoramic image with extended dynamic range, which can better represent varying illumination conditions. The panorama is then searched for known anchor points, while orientation tracking continues uninterrupted. We then use information from an internal orientation sensor to prime an active search scheme for the anchor points, which improves matching results. Finally, global consistency is enhanced by statistical estimation of a global rotation that minimizes the overall position error of anchor points when transforming them from the source panorama in which they were created, to the current view represented by a new panorama. Once the anchor points are redetected, we track the users movement using a novel 3-degree-of-freedom orientation tracking approach that combines vision tracking with the absolute orientation from inertial and magnetic sensors. We tested our system using an AR campus guide as an example application and provide detailed results for our approach using an off-the-shelf smartphone. Results show that the re-detection rate is improved by a factor of 2 compared to previous work and reaches almost 90% for a wide variety of test cases while still keeping the ability to run at interactive frame rates.

Virtual redlining for civil engineering in real environments

Field workers of utility companies are regularly engaged in outdoor tasks such as network planning and inspection of underground infrastructure. Redlining is the term used for manually annotating either printed paper maps or a 2D geographic information system on a notebook computer taken to the field. Either of these approaches requires finding the physical location to be annotated on the physical or digital map. In this paper, we describe a mobile Augmented Reality (AR) system capable of supporting virtual redlining. The AR visualization delivered by the system is constructed from data directly extracted from a GIS used in day-to-day production by utility companies. We also report on encouraging trials and interviews performed with professional field workers from the utility sector.

Smart Vidente: advances in mobile augmented reality for interactive visualization of underground infrastructure

Many civil engineering tasks require to access geospatial data in the field and reference the stored information to the real-world situation. Augmented reality (AR), which interactively overlays 3D graphical content directly over a view of the world, can be a useful tool to visualize but also create, edit and update geospatial data representing real-world artifacts. We present research results on the next-generation field information system for companies relying on geospatial data, providing mobile workforces with capabilities for on-site inspection and planning, data capture and as-built surveying. To achieve this aim, we used mobile AR technology for on-site surveying of geometric and semantic attributes of geospatial 3D models on the user’s handheld device. The interactive 3D visualizations automatically generated from production databases provide immediate visual feedback for many tasks and lead to a round-trip workflow where planned data are used as a basis for as-built surveying through manipulation of the planned data. Classically, surveying of geospatial objects is a typical scenario performed from utility companies on a daily basis. We demonstrate a mobile AR system that is capable of these operations and present first field trials with expert end users from utility companies. Our initial results show that the workflows of planning and surveying of geospatial objects benefit from our AR approach.

North-centred orientation tracking on mobile phones

Magnetic compasses and accelerometers provide absolute orientation measurements within the earths reference frame. However, sensor output typically suffers from jitter and external disturbances. Conversely, visual tracking provides more stable orientation estimation relative to an unknown initial orientation rather than to true north. We propose a 3-degree-of-freedom orientation tracking approach combining the accuracy and stability of vision tracking with the absolute orientation from inertial and magnetic sensors by estimating the offset between the initial orientation of the vision tracker and true north. We demonstrate that the approach improves absolute orientation estimation on a mobile phone device.

3D tracking in unknown environments using on-line keypoint learning for mobile augmented reality

In this paper we present a natural feature tracking algorithm based on on-line boosting used for localizing a mobile computer. Mobile augmented reality requires highly accurate and fast six degrees of freedom tracking in order to provide registered graphical overlays to a mobile user. With advances in mobile computer hardware, vision-based tracking approaches have the potential to provide efficient solutions that are non-invasive in contrast to the currently dominating marker-based approaches. We propose to use a tracking approach which can use in an unknown environment, i.e. the target has not be known beforehand. The core of the tracker is an on-line learning algorithm, which updates the tracker as new data becomes available. This is suitable in many mobile augmented reality applications. We demonstrate the applicability of our approach on tasks where the target objects are not known beforehand, i.e. interactive planing.

Simultaneous Localization and Mapping for Augmented Reality

Recently, the methods of Simultaneous Localization and Mapping (SLAM) have received great interest in the field of Augmented Reality. Accurate tracking in unknown and new environments promises to reduce the initial costs of building AR systems which often require extensive and accurate models of the environments, interaction objects and virtual annotations. However, it is still an open question how interesting and useful annotations can be created, attached and stored for unknown and arbitrary locations. In this paper, we discuss possible uses of SLAM in the different components of typical AR systems to provide meaningful applications and go beyond current limitations.