Ismo Rakkolainen

Head-mounted display with mid-air tactile feedback

Virtual and physical worlds are merging. Currently users of head-mounted displays cannot have unobtrusive tactile feedback while touching virtual objects. We present a mid-air tactile feedback system for head-mounted displays. Our prototype uses the focus of a modulated ultrasonic phased array for unobtrusive mid-air tactile feedback generation. The array and the hand position sensor are mounted on the front surface of a head-mounted virtual reality display. The presented system can enhance 3D user interfaces and virtual reality in a new way. To evaluate the tactile feedback together with visuals on an Oculus Rift VR headset, we had 13 participants do a simple virtual keypad tapping task with and without tactile feedback. The results indicate that while the measured speed and accuracy differed only a little, the subjects were nearly unanimous in that they preferred to use the tactile feedback. The raw NASA TLX questionnaires conducted after use revealed that the participants felt slightly less mental, physical and temporal demand with the tactile feedback. The participants self-assessment of their performance was also higher with the tactile feedback.

A hand-held immaterial volumetric display

We have created an ultralight, movable, “immaterial” fogscreen. It is based on the fogscreen mid-air imaging technology. The hand-held unit is roughly the size and weight of an ordinary toaster. If the screen is tracked, it can be swept in the air to create mid-air slices of volumetric objects, or to show augmented reality (AR) content on top of real objects. Interfacing devices and methodologies, such as hand and gesture trackers, camera-based trackers and object recognition, can make the screen interactive. The user can easily interact with any physical object or virtual information, as the screen is permeable. Any real objects can be seen through the screen, instead of e.g., through a video-based augmented reality screen. It creates a mixed reality setup where both the real world object and the augmented reality content can be viewed and interacted with simultaneously. The hand-held mid-air screen can be used e.g., as a novel collaborating or classroom tool for individual students or small groups.

Interaction with WebVR 360° video player: Comparing three interaction paradigms

Immersive 360° video needs new ways of interaction. We compared three different interaction methods to find out which one of them is the most applicable for controlling 360° video playback. The compared methods were: remote control, pointing with head orientation, and hand gestures. A WebVR-based 360° video player was built for the experiment.

A compact, wide-FOV optical design for head-mounted displays

We present a new optical design for head-mounted displays (HMD) which has an exceptionally wide field of view (FOV). It can cover even the full human FOV. It is based on seamless lenses and screens curved around the eyes. The proof-of-concept prototypes are promising, and one of them far exceeds the human FOV, although the effective FOV is limited by the anatomy of the human head. The presented optical design has advantages such as compactness, light weight, low cost and super-wide FOV with high resolution. Even though this is still work-in-progress and display functionality is not yet implemented, it suggests a feasible way to significantly expand the FOV of HMDs.

Casual immersive viewing with smartphones

In this paper, we explore how to better integrate virtual reality viewing to a smartphone. We present novel designs for casual (short-term) immersive viewing of spatial and 3D content, such as augmented and virtual reality, with smartphones. Our goal is to create a simple and low-cost casual-viewing design which could be retrofitted and eventually be embedded into smartphones, instead of using larger spatial viewing accessories. We explore different designs and implemented several prototypes. One prototype uses thin and light near-to-eye optics with a smartphone display, thus providing the user with the functionality of a large, high-resolution virtual display. Our designs also enable 3D user interfaces. Easy interaction through various gestures and other modalities is possible by using the inertial and other sensors and camera of the smartphone. Our preliminary concepts are a starting point for exploring useful constructions and designs for such usage.

Bidirectional touch interaction for immaterial displays

In this paper, we present an immaterial mid-air display with touchscreen capability and aerial tactile feedback. By augmenting a projected desktop fogscreen display with a 3D camera for touch detection and a focusable, modulated ultrasonic phased array for tactile feedback generation, very convincing yet completely immaterial user interfaces were constructed. A typical feedback response is delivered within 2 ms with a positional resolution of 2 mm and an active spot size of 10 mm at 100 mm distance. By combining a number of the tileable ultrasonic arrays, a tighter focus with much higher intensity can be produced.

A Fluorescent Mid-air Screen

The Fog Screen can create high-quality walk-through images onto a translucent fog plane with rear-projection, so that they appear to float in mid-air. Projectors hotspot may distract viewers, but here we present a method based on fluorescence and front-projection to apparently remove the projectors light altogether. The fluorescent Fog Screen can create even more magical images than the standard Fog Screen.

Ballistic Tracking - A True Zero Latency Gaming Interface

In this paper, we describe a new gaming interface, enabling human-computer interaction mediated by flying objects thrown at the computer display. The objects are tracked midair and the trajectories calculated before the actual hit. By extrapolating these trajectories, true zero latency for the interaction has been demonstrated in proof-of-concept games. The full spatiotemporal accuracy of this very low-cost system is shown to be of the order of 10 mm and 1 ms, with up to 50 ms predictability.

A superwide-FOV optical design for head-mounted displays

We present a new optical design for head-mounted displays (HMD) that has an exceptionally wide field of view (FOV). It can cover even the full human FOV. It is based on seamless lenses and screens curved around the eyes. We constructed several compact and lightweight proof-of-concept prototypes of the optical design. One of them far exceeds the human FOV, although the anatomy of the human head limits the effective FOV. The presented optical design has advantages such as compactness, light weight, low cost and superwide FOV with high resolution. The prototypes are promising, and though this is still work-in-progress and display functionality is not yet implemented, it suggests a feasible way to significantly expand the FOV of HMDs.

Midair User Interfaces Employing Particle Screens

Recent developments with low-cost optical sensors and ultralight, mobile fogscreens enable new applications such as midair volumetric screens and augmented reality. The major advantages of the fogscreen are its walk-through and immaterial nature, translucency, good image quality, possibility for direct interaction, and the intriguing appearance of the screen. Fogscreens can create a collaborative visualization space or even personal views for each viewer. Dual-sided face-to-face interaction mimics most real-world competitive games and sports and enables body language and free switching of sides. This article reports the authors experiments with these emerging technologies and their suitability for novel midair user interfaces.