Svenja Kahn

Towards precise real-time 3D difference detection for industrial applications

3D difference detection is the task to verify whether the 3D geometry of a real object exactly corresponds to a 3D model of this object. We present an approach for 3D difference detection with a hand-held depth camera. In contrast to previous approaches, with the presented approach geometric differences can be detected in real-time and from arbitrary viewpoints. The 3D difference detection accuracy is improved by two approaches: first, the precision of the depth cameras pose estimation is improved by coupling the depth camera with a high precision industrial measurement arm. Second, the influence of the depth measurement noise is reduced by integrating a 3D surface reconstruction algorithm. The effects of both enhancements are quantified by a ground-truth based quantitative evaluation, both for a time-of-flight (SwissRanger 4000) and a structured light depth camera (Kinect). With the proposed enhancements, differences of few millimeters can be detected from 1m measurement distance.

Enhancing realism of mixed reality applications through real-time depth-imaging devices in X3D

Until recently, depth sensing cameras have been used almost exclusively in research due to the high costs of such specialized equipment. With the introduction of the Microsoft Kinect device, realtime depth imaging is now available for the ordinary developer at low expenses and so far it has been received with great interest from both the research and hobby developer community. The underlying OpenNI framework not only allows to extract the depth image from the camera, but also provides tracking information of gestures or user skeletons. In this paper, we present a framework to include depth sensing devices into X3D in order to enhance visual fidelity of X3D Mixed Reality applications by introducing some extensions for advanced rendering techniques. We furthermore outline how to calibrate depth and image data in a meaningful way through calibration for devices that do not already come with precalibrated sensors, as well as a discussion of some of the OpenNI functionality that X3D can benefit from in the future.

Beyond 3D "As-Built" Information Using Mobile AR Enhancing the Building Lifecycle Management

The success of smart phone technologies changed the way information is processed as more and more geo-referenced structures are emerging linking information to specific locations within our environment. Thereby, Augmented Reality has become a key technology as it analyses the smart phone sensor data (camera, GPS, inertial) to derive the detailed pose of the smart phone, with the aim to correlate our real environment to the geo-referenced information space. In particular, this is relevant for application fields where 3Dmodels are used in planning and organization processes as e.g. facility management. In facility construction and management Building Information Model (BIM) was established as a standard that not only holds the 3D-building-geometry but encompasses pipe/electrical systems as well as semantic building information, for example properties and conditions of building components. With this motivation our work integrates BIM and Augmented Reality.

3D discrepancy check via Augmented Reality

For many tasks like markerless model-based camera tracking it is essential that the 3D model of a scene accurately represents the real geometry of the scene. It is therefore very important to detect deviations between a 3D model and a scene. We present an innovative approach which is based on the insight that camera tracking can not only be used for Augmented Reality visualization but also to solve the correspondence problem between 3D measurements of a real scene and their corresponding positions in the 3D model. We combine a time-of-flight camera (which acquires depth images in real time) with a custom 2D camera (used for the camera tracking) and developed an analysis-by-synthesis approach to detect deviations between a scene and a 3D model of the scene.

Fusing Real-Time Depth Imaging with High Precision Pose Estimation by a Measurement Arm

Recently, depth cameras have emerged which capture dense depth images in real-time. To benefit from their 3D imaging capabilities in interactive applications which support an arbitrary camera movement, the position and orientation of the depth camera needs to be robustly estimated in real time for each captured depth image. Therefore, this paper describes how to combine a depth camera with a mechanical measurement arm to fuse real-time depth imaging with real-time, high precision pose estimation. Estimating the pose of a depth camera with a measurement arm has three major advantages over 2D/3D image based optical pose estimation: The measurement arm has a very precise guaranteed accuracy better than 0.1mm, the pose estimation accuracy is not influenced by the captured scene and the computational load is much lower than for optical pose estimation, leaving more processing power for the applications themselves.

Towards an Affordable Markerless Acquisition of Intangible Contemporary Dance Choreographies at Large-Scaled Stages

While the documentation and preservation of rigid cultural heritage objects has become much easier with technologies such as 3D scanning or photogrammetry technologies, the digitalization of 3D intangible moving content is still a major issue. This concerns also the in situ creation of digital dance representations and the question of how to preserve and disseminate dance performances. In this paper, we present a generic and affordable approach for an automatized and markerless capturing of the movements of dancers, which was developed in the Motion Bank research project, as well as first application examples which analyse and visualize the captured dance data. The captured data is stored in a cloud based service and is thus made available for online and offline processing.

Accelerated real-time reconstruction of 3D deformable objects from multi-view video channels

In this paper we present a new framework for an accelerated 3D reconstruction of deformable objects within a multi-view setup. It is based on a new memory management and an enhanced algorithm pipeline of the well known Image-Based Visual Hull (IBVH) algorithm that enables efficient and fast reconstruction results and opens up new perspectives for the scalability of time consuming computations within larger camera environments. As a result, a significant increase of frame rates for the volumetric reconstruction of deformable objects can be achieved using an optimized CUDA-based implementation on NVIDIAs Fermi-GPUs.

Towards Symmetry Axis based Markerless Motion Capture

A natural interaction with virtual environments is one of the key issues for the usability of Virtual Reality applications. Device-free, intuitive interactions with the virtual world can be achieved by capturing the movements of the user with markerless motion capture. In this work we present a markerless motion capture approach which can be used to estimate the human body pose in real-time with a single depth camera. The presented approach requires neither a 3D shape model of the tracked person nor a training phase in which body shapes are learned a priori. Instead, it analyzes the curvature of the human body to estimate the symmetry axes of the body joints. These symmetry axes are then used to calculate the pose of the tracked human in real-time. The presented approach was evaluated qualitatively with a time-of-flight and a Kinect depth camera. Furthermore, quantitative simulation results show that the proposed approach is promising for depth cameras which can reliably capture the surface curvature (and thus the normals) of a person and which have a resolution of at least 320x240 pixel.

Capturing of contemporary dance for preservation and presentation of choreographies in online scores

In this paper, we present a generic and affordable approach for an automatized and markerless capturing of movements in dance, which was developed in the Motion Bank / The Forsythe Company project (www.motionbank.org). Thereby within Motion Bank we are considering the complete digitalization workflow starting with the setup of the camera array and ending with a web-based presentation of “Online Scores” visualizing different elements of choreography. Within our project, we have used our technology in two modern dance projects, one “Large Motion Space Performance” covering a large stage in solos and trios and one “Restricted Motion Space Performance” that is suited to be captured with range cameras. The project is realized in close cooperation with different choreographers and dance companies of modern ballet and with multi-media artists forming the visual representations of dance.