Tobias Salb

Using augmented reality to interact with an autonomous mobile platform

To allow users without special knowledge to interact with robots, it is desirable to make interaction methods as intuitive as possible. This goal is in many cases difficult to achieve since data flow from the robot to the human is limited, especially if free locomotion of both the human and the robot are required. Therefore, new communication channels need to be created. We propose the use of an augmented reality display together with a wearable, wirelessly networked computer to achieve this goal. This system makes it possible to overlay planning, world model and sensory data provided by the robot over the wearers field of view. We discuss the system architecture, interaction methods and experimental results. We demonstrate an example application for rapid prototyping of a warehouse transport system using the augmented reality system and a mobile platform. The user can create a topological map in an unknown environment on-the fly by setting and manipulating map nodes. This is done by pointing at the floor with a special interaction device, and issuing voice commands. The map is shown to the user as an augmentation of the real world view. Additionally, the robots path planning data is visualized.

A system for facial reconstruction using distraction and symmetry considerations

Abstract The goal of the system presented in this paper is to support several facial surgeries that are aiming to transform an unsymmetrical face to a symmetric one. There are two main techniques to achieve this goal: distraction of the lower jaw and adding or removing tissue or bone at certain facial regions. Both planning tasks are done based on segmented CT or MRI data. The distraction of the lower jaw is simulated by cutting the jaw, moving it in 3D along a distraction axis and choosing a distractor for this procedure after determining, which is the best result. Furthermore, we are working on consideration of soft tissue movement so the patient can see what he or she will look like after the surgery. For supporting implants in craniofacial surgery, we are proposing a tool based on symmetry considerations. Nowadays, if implants have to be fitted to the patients bone, they are handmade while the surgery is running. This is quite time consuming and it would be helpful to shorten this procedure by producing the implants preoperatively. Our system allows to determine regions where implants should be set and proposes an initial contour for the implants.

Interactive simulation of teeth cleaning

Abstract In this paper, an interactive simulation system for teeth cleaning is presented. This simulation system offers assistance for optimizing design and manufacturing of new toothbrushes. Data acquisition and pre-processing techniques for the model generation are shown, an editor for modelling of the elastic behaviour of the toothbrush parts is presented and the mathematical method for modelling of flexible parts of the toothbrush is explained. After the introduction of the PHANToM haptoid and the explanation of a new approach to collision detection, first results of the project are shown.

Technical experience from clinical studies with INPRES and a concept for a miniature augmented reality system

This paper is going to present a summary of our technical experience with the INPRES System -- an augmented reality system based upon a tracked see-through head-mounted display. With INPRES a complete augmented reality solution has been developed that has crucial advantages when compared with previous navigation systems. Using these techniques the surgeon does not need to turn his head from the patient to the computer monitor and vice versa. The systems purpose is to display virtual objects, e.g. cutting trajectories, tumours and risk-areas from computer-based surgical planning systems directly in the surgical site. The INPRES system was evaluated in several patient experiments in craniofacial surgery at the Department of Oral and Maxillofacial Surgery/University of Heidelberg. We will discuss the technical advantages as well as the limitations of INPRES and present two strategies as a result. On the one hand we will improve the existing and successful INPRES system with new hardware and a new calibration method to compensate for the stated disadvantage. On the other hand we will focus on miniaturized augmented reality systems and present a new concept based on fibre optics. This new system should be easily adaptable at surgical instruments and capable of projecting small structures. It consists of a source of light, a miniature TFT display, a fibre optic cable and a tool grip. Compared to established projection systems it has the capability of projecting into areas that are only accessible by a narrow path. No wide surgical exposure of the region is necessary for the use of augmented reality.

Evaluation of an augmented reality system for intraoperative data presentation — preliminary results

For intraoperative support of surgeons and reduction of clinical risks we developed an augmented reality system called INPRES — “INtraoperative PRESentation of surgical planning and simulation results”.

Symmetry considerations for volumetric implant-planning.

Symmetry Considerations can be used not only to plan the desired shape of reconstructured bone structures, but also to generate prototypes for soft tissue implants. The paper describes a system which allows to calculate a symmetry plane in the facial area automatically and computes proposals for implants or transplants. This is done by using a Simulated Annealing algorithm to determine the best reflection and morphologig operators to calculate the implant. The system presented has been used to calculate soft tissue implants in the facial area and a replacement for parts of the lower jaw.