Vincent Lévesque

A role for haptics in mobile interaction: initial design using a handheld tactile display prototype

Mobile interaction can potentially be enhanced with well-designed haptic control and display. However, advances have been limited by a vicious cycle whereby inadequate haptic technology obstructs inception of vitalizing applications. We present the first stages of a systematic design effort to break that cycle, beginning with specific usage scenarios and a new handheld display platform based on lateral skin stretch. Results of a perceptual device characterization inform mappings between device capabilities and specific roles in mobile interaction, and the next step of hardware re-engineering.

Finger pad friction and its role in grip and touch

Many aspects of both grip function and tactile perception depend on complex frictional interactions occurring in the contact zone of the finger pad, which is the subject of the current review. While it is well established that friction plays a crucial role in grip function, its exact contribution for discriminatory touch involving the sliding of a finger pad is more elusive. For texture discrimination, it is clear that vibrotaction plays an important role in the discriminatory mechanisms. Among other factors, friction impacts the nature of the vibrations generated by the relative movement of the fingertip skin against a probed object. Friction also has a major influence on the perceived tactile pleasantness of a surface. The contact mechanics of a finger pad is governed by the fingerprint ridges and the sweat that is exuded from pores located on these ridges. Counterintuitively, the coefficient of friction can increase by an order of magnitude in a period of tens of seconds when in contact with an impermeably smooth surface, such as glass. In contrast, the value will decrease for a porous surface, such as paper. The increase in friction is attributed to an occlusion mechanism and can be described by first-order kinetics. Surprisingly, the sensitivity of the coefficient of friction to the normal load and sliding velocity is comparatively of second order, yet these dependencies provide the main basis of theoretical models which, to-date, largely ignore the time evolution of the frictional dynamics. One well-known effect on taction is the possibility of inducing stick–slip if the friction decreases with increasing sliding velocity. Moreover, the initial slip of a finger pad occurs by the propagation of an annulus of failure from the perimeter of the contact zone and this phenomenon could be important in tactile perception and grip function.

Effect of skin hydration on the dynamics of fingertip gripping contact.

The dynamics of fingertip contact manifest themselves in the complex skin movements observed during the transition from a stuck state to a fully developed slip. While investigating this transition, we found that it depended on skin hydration. To quantify this dependency, we asked subjects to slide their index fingertip on a glass surface while keeping the normal component of the interaction force constant with the help of visual feedback. Skin deformation inside the contact region was imaged with an optical apparatus that allowed us to quantify the relative sizes of the slipping and sticking regions. The ratio of the stuck skin area to the total contact area decreased linearly from 1 to 0 when the tangential force component increased from 0 to a maximum. The slope of this relationship was inversely correlated to the normal force component. The skin hydration level dramatically affected the dynamics of the contact encapsulated in the course of evolution from sticking to slipping. The specific effect was to reduce the tendency of a contact to slip, regardless of the variations of the coefficient of friction. Since grips were more unstable under dry skin conditions, our results suggest that the nervous system responds to dry skin by exaggerated grip forces that cannot be simply explained by a change in the coefficient of friction.

Refreshable tactile graphics applied to schoolbook illustrations for students with visual impairment

This article presents research on making schoolbook illustrations accessible for students with visual impairment. The MaskGen system was developed to interactively transpose illustrations of schoolbooks into tactile graphics. A methodology was designed to transpose the graphics and prepare them to be displayed on the STReSS2, a refreshable tactile device. We experimented different associations of tactile rendering and audio feedbacks to find a model that children with visual impairment could use. We experimented with three scientific graphics (diagram, bar-chart and map) with forty participants: twenty sighted adults, ten adults with visual impairment, and ten children with visual impairment. Results show that the participants with visual impairment liked the tactile graphics and could use them to explore illustrations and answer questions about their content.

Braille Display by Lateral Skin Deformation with the STReSS2 Tactile Transducer

Earlier work with a 1D tactile transducer demonstrated that lateral skin deformation is sufficient to produce sensations similar to those felt when brushing a finger against a line of Braille dots. Here, we extend this work to the display of complete 6-dot Braille characters using a general purpose 2D tactile transducer called STReSS2. The legibility of the produced Braille was evaluated by asking seven expert Braille readers to identify meaningless 5-letter strings as well as familiar words. Results indicate that reading was difficult but possible for most individuals. The superposition of texture to the sensation of a dot improved performance. The results contain much information to guide the design of a specialized Braille display operating by lateral skin deformation. They also suggest that rendering for contrast rather than realism may facilitate Braille reading when using a weak tactile transducer

Haptically Enabled Handheld Information Display With Distributed Tactile Transducer

This paper describes the design, construction, and initial evaluation of a handheld information device that supports combined tactile and graphical interaction. The design comprises a liquid crystal graphic display co-located with a miniature, low-power, distributed tactile transducer. This transducer can create electronically-controlled lateral skin deformation patterns which give the sensation of sliding over small shapes. It is integrated within a slider mechanism to control scrolling. It also functions as a detent when pushing on it. Tactile feedback and the combination of visual and tactile feedback in a mobile context enable the development of new functions, such as multimodal navigation within large graphic spaces

A haptic memory game using the STRESS 2 tactile display

A computer implementation of a classic memory card game was adapted to rely on touch rather than vision. Instead of memorizing pictures on cards, players explore tactile graphics on a computer-generated virtual surface. Tactile sensations are created by controlling dynamic, distributed lateral strain patterns on a fingerpad in contact with a tactile display called STRESS2. The tactile graphics are explored by moving the device within the workspace of a 2D planar carrier. Three tactile rendering methods were developed and used to create distinct tactile memory cards. The haptic memory game showcases the capabilities of this novel tactile display technology.

Tactile Graphics Rendering Using Three Laterotactile Drawing Primitives

This paper presents preliminary work towards the development and evaluation of a practical refreshable tactile graphics system for the display of tactile maps, diagrams and graphs for people with visual impairments. Refreshable tactile graphics were dynamically produced by laterally deforming the skin of a finger using the STReSS2 tactile display. Tactile features were displayed over an 11x6 cm virtual surface by controlling the tactile sensations produced by the fingerpad-sized tactile display as it was moved on a planar carrier. Three tactile rendering methods were used to respectively produce virtual gratings, dots and vibrating patterns. These tactile features were used alone or in combination to display shapes and textures. The ability of the system to produce tactile graphics elements was evaluated in five experiments, each conducted with 10 sighted subjects. The first four evaluated the perception of simple shapes, grating orientations, and grating spatial frequencies. The fifth experiment combined these elements and showed that tactile icons composed of both vibrating contours and grated textures can be identified. The fifth experiment was repeated with 6 visually impaired subjects with results suggesting that similar performance should be expected from that user group.

Exploring the design space of programmable friction for scrolling interactions

Scrolling interactions are an important aspect of the design of usable touchscreen interfaces, particularly for handheld devices that can only display a limited amount of information at once. Using a touchscreen capable of dynamically altering its surface friction, we explore the design space of haptically-augmented scrolling interactions and investigate programmable frictions ability to provide appropriate feedback in envisioned usage scenarios. We performed five user experiments to evaluate respectively the identifiability of a set of iconic detents, the countability of detents, the perception of detent density, the synchronization of tactile feedback to on-screen events, and the optimal friction pattern for a spring-like resistance. The results of these experiments provide valuable information that will inform the design of scrolling interactions that leverage programmable friction for an improved user experience.

Adaptive level of detail in dynamic, refreshable tactile graphics

We investigate gains in user appreciation and performance when the level of detail of tactile graphics is dynamically altered either at the press of a button or automatically, as a function of exploration speed. This concept was evaluated by asking 9 visually impaired participants to perform hierarchical spatial search tasks in a concert hall illustration. The tasks could be simplified by first searching for a section in a sparse illustration, and then a seat in a detailed illustration. The results show no improvement in task performance but indicate a user preference for explicitly controlling the level of details with the manual toggle.