Daniel Kade

Head-mounted mixed reality projection display for games production and entertainment

AbstractThis research presents a mixed reality (MR) application that is designed to be usable during a motion capture shoot and supports actors with their task to perform. Through our application, we allow seeing and exploring a digital environment without occluding an actor’s field of vision. A prototype was built by combining a retroreflective screen covering surrounding walls and a headband consisting of a laser scanning projector with a smartphone. Built-in sensors of a smartphone provide navigation capabilities in the digital world. The presented system was demonstrated in an initially published paper. Here, we extend these research results with our advances and discuss the potential use of our prototype in gaming and entertainment applications. To explore this potential use case, we built a gaming application using our MR prototype and tested it with 45 participants. In these tests, we use head movements as rather unconventional game controls. According to the performed user tests and their feedback, our prototype shows a potential to be used for gaming applications as well. Therefore, our MR prototype could become of special interest because the prototype is lightweight, allows for freedom of movement and is a low-cost, stand-alone mobile system. Moreover, the prototype also allows for 3D vision by mounting additional hardware.

Head-worn mixed reality projection display application

The main goal of this research is to develop a mixed reality (MR) application to support motion capture actors. This application allows seeing and exploring a digital environment without occluding the actors visual field. A prototype is built by combining a retro-reflective screen covering surrounding walls and a headband consisting of a laser scanning projector with a smartphone. Built-in sensors of a smartphone provide navigation capabilities in the digital world. The integrated system has some unique advantages, which are collectively demonstrated for the first time: (i) providing fixed field-of-view (50° in diagonal), fixed retinal images at full-resolution, and distortion-free images that are independent of the screen distance and shape; (ii) presenting different perspectives to the users as they move around or tilt their heads, (iii) allowing a focus-free and calibration-free display even on non-flat surfaces using laser scanning technology, (iv) enabling multiple users to share the same screen without crosstalk due to the use of retro-reflectors, and (v) producing high brightness pictures with a projector of only 15 lm; due to a high-gain retro-reflective screen. We demonstrated a lightweight, comfortable to wear and low cost head-mounted projection display (HMPD) which acts as a stand-a-lone mobile system. Initial informal functionality tests have been successfully performed. The prototype can also be used as a 3D stereo system using the same hardware by additionally mounting polarized glasses and an active polarization rotator, while maintaining all of the advantages listed above.

Low-cost mixed reality simulator for industrial vehicle environments

High-end industrial vehicle simulators are generally expensive and aim at providing a high level of realism. The access to such simulators is often a limited resource to researchers and developers who find themselves using a PC-based simulator instead. We challenge this approach by introducing a low-cost mixed reality simulator for industrial vehicles that allows to test new vehicle control concepts and design ideas in a rapid prototyping manner. Our simulator prototype consists of a head-mounted projection display, a CAVE-like room covered with a retro-reflective cloth and a rotatable chair with controls to steer an industrial vehicle. The created digital environment represents an obstacle course for an excavator and can be controlled by a joystick, a keyboard and can be explored by natural head movements.

Perception of Delay in Computer Input Devices Establishing a Baseline for Signal Processing of Motion Sensor Systems

New computer input devices in healthcare applications using small embedded sensors need firmware filters to run smoothly and to provide a better user experience. Therefore, it has to be investigated how much delay can be tolerated for signal processing before the users perceive a delay when using a computer input device. This paper is aimed to find out a threshold of unperceived delay by performing user tests with 25 participants. A communication retarder was used to create delays from 0 to 100 ms between a receiving computer and three different USB-connected computer input devices. A wired mouse, a wifi mouse and a head-mounted mouse were used as input devices. The results of the user tests show that delays up to 50 ms could be tolerated and are not perceived as delay, or depending on the used device still perceived as acceptable.

360 Degree Mixed Reality Environment to Evaluate Interaction Design for Industrial Vehicles Including Head-Up and Head-Down Displays

Designing and testing new information and safety features for industrial vehicles do not need to involve the realization of high-fidelity and expensive simulators. We propose a low-cost mixed reality environment which allows for rapid development and rearrangement of a virtual and physical setup of a simulator for industrial vehicles.

Evaluation of a Mixed Reality Head-Mounted Projection Display to Support Motion Capture Acting

Motion capture acting is a challenging task, it requires trained and experienced actors who can highly rely on their acting and imagination skills to deliver believable performances. This is especially the case when preparation times are short and scenery needs to be imagined, as it is commonly the case for shoots in the gaming industry. To support actors in such cases, we developed a mixed reality application that allows showing digital scenery and triggering emotions while performing.

Acting 2.0: when entertainment technology helps actors to perform

Motion capture shoots involve a wide range of technology and entertainment production systems such as motion capture cameras, tracking software and digital environments to create entertainment applications. However, acting in this high-tech environment is still traditional and brings its own challenges to the actors. Good acting and imagination skills are highly needed for many motion capture shoots to deliver satisfying results. In our research, we are exploring how to support the actors and use a head-mounted projection display to create a mixed reality application helping actors to perform during motion capture shoots. This paper presents the latest enhancements of our head-mounted projection display application and discusses the use of this technology for motion capture acting as well as the potential use for entertainment purposes.