Christian Holz

NormalTouch and TextureTouch: High-fidelity 3D Haptic Shape Rendering on Handheld Virtual Reality Controllers

We present an investigation of mechanically-actuated hand-held controllers that render the shape of virtual objects through physical shape displacement, enabling users to feel 3D surfaces, textures, and forces that match the visual rendering. We demonstrate two such controllers, NormalTouch and TextureTouch, which are tracked in 3D and produce spatially-registered haptic feedback to a users finger. NormalTouch haptically renders object surfaces and provides force feedback using a tiltable and extrudable platform. TextureTouch renders the shape of virtual objects including detailed surface structure through a 4×4 matrix of actuated pins. By moving our controllers around while keeping their finger on the actuated platform, users obtain the impression of a much larger 3D shape by cognitively integrating output sensations over time. Our evaluation compares the effectiveness of our controllers with the two de-facto standards in Virtual Reality controllers: device vibration and visual feedback only. We find that haptic feedback significantly increases the accuracy of VR interaction, most effectively by rendering high-fidelity shape output as in the case of our controllers.

DuoSkin: rapidly prototyping on-skin user interfaces using skin-friendly materials

Miniature devices have become wearable beyond the form factor of watches or rings---functional devices can now directly affix to the users skin, unlocking a much wider canvas for electronics. However, building such small and skin-friendly devices currently requires expensive materials and equipment that is mostly found in the medical domain. We present DuoSkin, a fabrication process that affords rapidly prototyping functional devices directly on the users skin using gold leaf as the key material, a commodity material that is skin-friendly, robust for everyday wear, and user-friendly in fabrication. We demonstrate how gold leaf enables three types of interaction modalities on DuoSkin devices: sensing touch input, displaying output, and communicating wirelessly with other devices. Importantly, DuoSkin incorporates aesthetic customizations found on body decoration, giving form to exposed interfaces that so far have mostly been concealed by covers. Our technical evaluation confirmed that gold leaf was more durable and preferable when affixed to skin than current commodity materials during everyday wear. This makes gold leaf a viable material for users to build functional and compelling on-skin devices. In our workshop evaluation, participants were able to customize their own on-skin music controllers that reflected personal aesthetics.

Finding Common Ground: A Survey of Capacitive Sensing in Human-Computer Interaction

For more than two decades, capacitive sensing has played a prominent role in human-computer interaction research. Capacitive sensing has become ubiquitous on mobile, wearable, and stationary devices - enabling fundamentally new interaction techniques on, above, and around them. The research community has also enabled human position estimation and whole-body gestural interaction in instrumented environments. However, the broad field of capacitive sensing research has become fragmented by different approaches and terminology used across the various domains. This paper strives to unify the field by advocating consistent terminology and proposing a new taxonomy to classify capacitive sensing approaches. Our extensive survey provides an analysis and review of past research and identifies challenges for future work. We aim to create a common understanding within the field of human-computer interaction, for researchers and practitioners alike, and to stimulate and facilitate future research in capacitive sensing.

Project Zanzibar: A Portable and Flexible Tangible Interaction Platform

We present Project Zanzibar: a flexible mat that can locate, uniquely identify and communicate with tangible objects placed on its surface, as well as sense a users touch and hover hand gestures. We describe the underlying technical contributions: efficient and localised Near Field Communication (NFC) over a large surface area; object tracking combining NFC signal strength and capacitive footprint detection, and manufacturing techniques for a rollable device form-factor that enables portability, while providing a sizable interaction area when unrolled. In addition, we detail design patterns for tangibles of varying complexity and interactive capabilities, including the ability to sense orientation on the mat, harvest power, provide additional input and output, stack, or extend sensing outside the bounds of the mat. Capabilities and interaction modalities are illustrated with self-generated applications. Finally, we report on the experience of professional game developers building novel physical/digital experiences using the platform.

Glabella: Continuously Sensing Blood Pressure Behavior using an Unobtrusive Wearable Device

We propose Glabella, a wearable device that continuously and unobtrusively monitors heart rates at three sites on the wearer’s head. Our glasses prototype incorporates optical sensors, processing, storage, and communication components, all integrated into the frame to passively collect physiological data about the user without the need for any interaction. Glabella continuously records the stream of reflected light intensities from blood flow as well as inertial measurements of the user’s head. From the temporal differences in pulse events across the sensors, our prototype derives the wearer’s pulse transit time on a beat-to-beat basis. Numerous efforts have found a significant correlation between a person’s pulse transit time and their systolic blood pressure. In this paper, we leverage this insight to continuously observe pulse transit time as a proxy for the behavior of systolic blood pressure levels—at a substantially higher level of convenience and higher rate than traditional blood pressure monitors, such as cuff-based oscillometric devices. This enables our prototype to model the beat-to-beat fluctuations in the user’s blood pressure over the course of the day and record its short-term responses to events, such as postural changes, exercise, eating and drinking, resting, medication intake, location changes, or time of day. During our in-the-wild evaluation, four participants wore a custom-fit Glabella prototype device over the course of five days throughout their daytime job and regular activities. Participants additionally measured their radial blood pressure three times an hour using a commercial oscillometric cuff. Our analysis shows a high correlation between the pulse transit times computed on our devices with participants’ heart rates (mean r = 0.92, SE = 0.03, angular artery) and systolic blood pressure values measured using the oscillometric cuffs (mean r = 0.79, SE = 0.15, angular-superficial temporal artery, considering participants’ self-administered cuff-based measurements as ground truth). Our results indicate that Glabella has the potential to serve as a socially-acceptable capture device, requiring no user input or behavior changes during regular activities, and whose continuous measurements may prove informative to physicians as well as users’ self-tracking activities.

DualBlink: A Wearable Device to Continuously Detect, Track, and Actuate Blinking For Alleviating Dry Eyes and Computer Vision Syndrome

Increased visual attention, such as during computer use leads to less blinking, which can cause dry eyes—the leading cause of computer vision syndrome. As people spend more time looking at screens on mobile and desktop devices, computer vision syndrome is becoming epidemic in todays population, leading to blurry vision, fatigue, and a reduced quality of life. One way to alleviate dry eyes is increased blinking. In this paper, we present a series of glasses-mounted devices that track the wearers blink rate and, upon absent blinks, trigger blinks through actuation: light flashes, physical taps, and small puffs of air near the eye. We conducted a user study to evaluate the effectiveness of our devices and found that air puff and physical tap actuations result in a 36% increase in participants’ average blink rate. Air puff thereby struck the best compromise between effective blink actuations and low distraction ratings from participants. In a follow-up study, we found that high intensity, short puffs near the eye were most effective in triggering blinks while receiving only low-rated distraction and invasiveness ratings from participants. We conclude this paper with two miniaturized and self-contained DualBlink prototypes, one integrated into the frame of a pair of glasses and the other one as a clip-on for existing glasses. We believe that DualBlink can serve as an always-available and viable option to treat computer vision syndrome in the future.

SurfaceConstellations: A Modular Hardware Platform for Ad-Hoc Reconfigurable Cross-Device Workspaces

We contribute SurfaceConstellations, a modular hardware platform for linking multiple mobile devices to easily create novel cross-device workspace environments. Our platform combines the advantages of multi-monitor workspaces and multi-surface environments with the flexibility and extensibility of more recent cross-device setups. The SurfaceConstellations platform includes a comprehensive library of 3D-printed link modules to connect and arrange tablets into new workspaces, several strategies for designing setups, and a visual configuration tool for automatically generating link modules. We contribute a detailed design space of cross-device workspaces, a technique for capacitive links between tablets for automatic recognition of connected devices, designs of flexible joint connections, detailed explanations of the physical design of 3D printed brackets and support structures, and the design of a web-based tool for creating new SurfaceConstellation setups.

CLAW: A Multifunctional Handheld Haptic Controller for Grasping, Touching, and Triggering in Virtual Reality

CLAW is a handheld virtual reality controller that augments the typical controller functionality with force feedback and actuated movement to the index finger. Our controller enables three distinct interactions (grasping virtual object, touching virtual surfaces, and triggering) and changes its corresponding haptic rendering by sensing the differences in the users grasp. A servo motor coupled with a force sensor renders controllable forces to the index finger during grasping and touching. Using position tracking, a voice coil actuator at the index fingertip generates vibrations for various textures synchronized with finger movement. CLAW also supports a haptic force feedback in the trigger mode when the user holds a gun. We describe the design considerations for CLAW and evaluate its performance through two user studies. The first study obtained qualitative user feedback on the naturalness, effectiveness, and comfort when using the device. The second study investigated the ease of the transition between grasping and touching when using our device.

Haptic Revolver: Touch, Shear, Texture, and Shape Rendering on a Reconfigurable Virtual Reality Controller

We present Haptic Revolver, a handheld virtual reality controller that renders fingertip haptics when interacting with virtual surfaces. Haptic Revolvers core haptic element is an actuated wheel that raises and lowers underneath the finger to render contact with a virtual surface. As the users finger moves along the surface of an object, the controller spins the wheel to render shear forces and motion under the fingertip. The wheel is interchangeable and can contain physical textures, shapes, edges, or active elements to provide different sensations to the user. Because the controller is spatially tracked, these physical features can be spatially registered with the geometry of the virtual environment and rendered on-demand. We evaluated Haptic Revolver in two studies to understand how wheel speed and direction impact perceived realism. We also report qualitative feedback from users who explored three application scenarios with our controller.

Haptic Links: Bimanual Haptics for Virtual Reality Using Variable Stiffness Actuation

We present Haptic Links, electro-mechanically actuated physical connections capable of rendering variable stiffness between two commodity handheld virtual reality (VR) controllers. When attached, Haptic Links can dynamically alter the forces perceived between the users hands to support the haptic rendering of a variety of two-handed objects and interactions. They can rigidly lock controllers in an arbitrary configuration, constrain specific degrees of freedom or directions of motion, and dynamically set stiffness along a continuous range. We demonstrate and compare three prototype Haptic Links: Chain, Layer-Hinge, and Ratchet-Hinge. We then describe interaction techniques and scenarios leveraging the capabilities of each. Our user evaluation results confirm that users can perceive many two-handed objects or interactions as more realistic with Haptic Links than with typical unlinked VR controllers.