Benjamin Tag

GazeSim: simulating foveated rendering using depth in eye gaze for VR

We present a novel technique of implementing customized hardware that uses eye gaze focus depth as an input modality for virtual reality applications. By utilizing eye tracking technology, our system can detect the point in depth the viewer focusses on, and therefore promises more natural responses of the eye to stimuli, which will help overcoming VR sickness and nausea. The obtained information for the depth focus of the eye allows the utilization of foveated rendering to keep the computing workload low and create a more natural image that is clear in the focused field, but blurred outside that field.

In the Eye of the Beholder: The Impact of Frame Rate on Human Eye Blink

We introduce a study investigating the impact of high frame rate videos on viewers eye blink frequency. A series of videos with varying combinations of motion complexities and frame rates were shown to participants, while their eye blinks were counted with J!NS MEME (smart eye wear). Lower frame rates and lower motion complexity caused higher blink frequencies, which are markers for stress and emotional arousal.

GazeSphere: navigating 360-degree-video environments in VR using head rotation and eye gaze

Viewing 360-degree-images and videos through head-mounted displays (HMDs) currently lacks a compelling interface to transition between them. We propose GazeSphere; a navigation system that provides a seamless transition between 360-degree-video environment locations through the use of orbit-like motion, via head rotation and eye gaze tracking. The significance of this approach is threefold: 1) It allows navigation and transition through spatially continuous 360-video environments, 2) It leverages the humans proprioceptive sense of rotation for locomotion that is intuitive and negates motion sickness, and 3) it uses eye tracking for a completely seamless, hands-free, and unobtrusive interaction. The proposed method uses an orbital motion technique for navigation in virtual space, which we demonstrate in applications such as navigation and interaction in computer aided design (CAD), data visualization, as a game mechanic, and for virtual tours.

Facial Thermography for Attention Tracking on Smart Eyewear: An Initial Study

We are describing the first step towards the development of an unobtrusive open eyewear system for attention tracking in daily life situations. We are logging thermographic data from infrared imaging and electrooculographic readings from off-the-shelf smart glasses and measure cognitive engagement of people in different situations. We are identifying new potential areas on the face for contactless IR temperature sensing. Attached to smart glasses and in combination with the EOG potentials we can monitor the wearers facial temperature changes, eye movement and eye blink in everyday situations, which is a major step towards becoming able to measure attention in unconstrained settings, and thus make it manageable.

Eye blink as an input modality for a responsive adaptable video system

We propose a unique system that allows real-time adaption of video settings to a viewers physical state. A custom made program toggles between videos according to the average eye blink frequency of each viewer. The physical data is harnessed with J!NS MEME smart glasses that utilize electrooculography (EOG). To the best of our belief, this is the first adaptable multimedia system that responds in real time to physical data and alters technical settings of video contents.

atmoSphere: mindfulness over haptic-audio cross modal correspondence

We explore cross-modal correspondence between haptic and audio output for meditation support. To this end, we implement atmoSphere, a haptic ball to prototype several haptic/audio designs. AtmoSphere consists of a sphere shaped device which provides haptic feedback. The users can experience the design aimed at instructing them in breathing techniques shown to enhance meditation. The aim of the haptic/audio design is to guide the user into a particular rhythm of breathing. We detect this rhythm using smart eyewear (J!NS MEME) that estimates cardiac and respiratory parameters using embedded motion sensors. Once this rhythm is achieved the feedback stops. If the user drops out of the rhythm, the haptic/audio feedback starts again.

Wearable aura: an interactive projection on personal space to enhance communication

This study focuses on how technology can encourage and ease awkwardness-free communications between people in real-world scenarios. We propose a device, The Wearable Aura, able to project a personalized animation onto ones Personal Distance zone. This projection, as an extension of one-self is reactive to users cognitive status, aware of its environment, context and users activity. Our user study supports the idea that an interactive projection around an individual can indeed benefit the communications with other individuals.

Shape memory alloy wire actuators for soft, wearable haptic devices

This paper presents a new approach to implement wearable haptic devices using Shape Memory Alloy (SMA) wires. The proposed concept allows building silent, soft, flexible and lightweight wearable devices, capable of producing the sense of pressure on the skin without any bulky mechanical actuators. We explore possible design considerations and applications for such devices, present user studies proving the feasibility of delivering meaningful information and use nonlinear autoregressive neural networks to compensate for SMA inherent drawbacks, such as delayed onset, enabling us to characterize and predict the physical behavior of the device.

Towards Enhancing Emotional Responses to Media using Auto-Calibrating Electric Muscle Stimulation (EMS)

We evaluate the use of Electric Muscle Stimulation (EMS) as a method of amplifying emotional responses to multimedia content. This paper presents an auto-calibration method to stimulate two facial expressions using EMS. We focus on two expressions: frown and smile. We attempted control of facial muscles with facial feedback for automatically calibrating these facial expressions: our computer vision system detects the facial expression and auto-calibrates the EMS parameters (intensity and duration) based on the users current facial expression. We present results from a pilot study with four participants evaluating the auto-calibration system and collecting initial feedback on the use of EMS to augment, for example, media experiences: while watching movies we can enhance the emotional response of the users during happy and sad scenes by stimulating corresponding face muscles.

Seamless Multithread Films in Virtual Reality

In this paper we are proposing a new system for the production of VR stories that allow users seamless interaction with the content. Our system plays real life footage rather than animation allowing for interactive live-action experiences. The unaware empowered users take subliminal decisions by focusing their attention at predefined ROIs. The moment a decision is taken is always placed seconds before the actual video branching happens to allow the system to preload only the chosen storyline in order to keep the computational workload as low as possible. We believe that this unique system will not only change the way we experience VR contents, but will rather lead to a paradigm shift in film production.