Johnny Lee

Optical gesture sensing and depth mapping technologies for head-mounted displays: an overview

Head Mounted Displays (HMDs), and especially see-through HMDs have gained renewed interest in recent time, and for the first time outside the traditional military and defense realm, due to several high profile consumer electronics companies presenting their products to hit market. Consumer electronics HMDs have quite different requirements and constrains as their military counterparts. Voice comments are the de-facto interface for such devices, but when the voice recognition does not work (not connection to the cloud for example), trackpad and gesture sensing technologies have to be used to communicate information to the device. We review in this paper the various technologies developed today integrating optical gesture sensing in a small footprint, as well as the various related 3d depth mapping sensors.

User sensitivity to speed- and height-mismatch in VR

Facebooks purchase of Oculus VR in 2014 ushered in a new era of consumer virtual reality head-mounted displays (HMDs). Converging technological advancements in small, high-resolution displays and motion-detection devices propelled VR beyond the purview of high-tech research laboratories and into the mainstream. However, technological hurdles still remain. As more consumer grade products develop, user comfort and experience will be of the utmost importance. One of the biggest issues for HMDs that lack external tracking is drift in the user position and rotation sensors. Drift can cause motion sickness and make stationary items in the virtual environment to appear to shift in position. For developers who seek to design VR experiences that are rooted in real environments, drift can create large errors in positional tracking if left uncorrected over time. Although much of the current VR hardware makes use of external tracking devices to mitigate positional and rotational drift, the creation of head-mounted displays that can operate without the use of extremal tracking devices would make VR hardware more portable and flexible, and may therefore be a goal for future development. Until technology advances sufficiently to completely overcome the hardware problems that cause drift, software solutions are a viable option to correct for it. It may be possible to speed up and slow down users as they move though the virtual world in order to bring their tracked position back into alignment with their position in the real world. If speed changes can be implemented without users noticing the alteration, it may offer a seamless solution that does not interfere with the VR experience. In Experiments 1 and 2, we artificially introduced speed changes that made users move through the VR environment either faster than or slower than their actual real-world speed. Users were tasked with correctly identifying when they were moving at the correct true-to-life speed when compared to an altered virtual movement speed. Fore and aft movement and movement from side to side initiated by seated users bending at the waist were tested separately in two experiments. In Experiment 3, we presented alternating views of the virtual scene from different user heights. In this study, users had to correctly distinguish the view of the virtual scene presented at the correct height from incorrect shorter and taller heights. In Experiments 1 and 2, we found that on average speed increases and decreases up to approximately 25% went unnoticed by users, suggesting that there is flexibility for programs to add speed changes imperceptible to users to correct for drift. In contrast, Experiment 3 demonstrates that on average users were aware of height changes after virtual heights were altered by just 5 cm. These thresholds can be used by VR developers to compensate for tracking mismatches between real and virtual positions of users of virtual environments, and also by engineers to benchmark new virtual reality hardware against human perceptual abilities.