Yeongmi Kim

Augmented white cane with multimodal haptic feedback

This paper proposes an instrumented handle with multimodal augmented haptic feedback, which can be integrated into a conventional white cane to extend the haptic exploration range of visually impaired users. The information extracted from the environment through a hybrid range sensor is conveyed to the user in an intuitive manner over two haptic feedback systems. The first renders impulses that imitate the impact of the real cane with a distant obstacle. In combination with the range sensors, this system allows to “touch” and explore remote objects, thus compensating for the limited range of the conventional white cane without altering its intuitive usage. The impulses are generated by storing kinetic energy in a spinning inertia wheel, which is released by abruptly braking the wheel. Furthermore, a vibrotactile interface integrated into the ergonomic handle conveys the distance to obstacles to the user. Three vibrating motors located along the index finger and hand are activated in different spatiotemporal patterns to induce a sense of distance through apparent movement. The realized augmented white cane not only increases the safety of the user by detecting obstacles from a further distance and alerting about those located at the head level, but also allows the user to build extended spatial mental models by increasing the sensing range, thereby allowing anticipated decision making and thus more natural navigation.

Prototype of a VR upper-limb rehabilitation system enhanced with motion-based tactile feedback

Stroke is the leading cause of long-term disability in adults, with increasing incidence as the population ages. Physical and occupational therapy support the recovery process, with therapy duration and intensity having been identified as critical factors. However, conventional therapy is limited to a short period following the stroke, with only limited possibilities to continue therapy at home. Increasing health costs and lack of trained personnel further strengthen this problem. There is thus a great need for home-based therapy systems, allowing to engage patients and assess the training with sufficient precision to provide meaningful exercises. In this paper, we suggest expandable software architecture for virtual reality rehabilitation applications. This system utilizes various sensors and actuators (i.e. the Kinect for upper limb tracking and a haptic glove). Based on this system, we propose a motion based tactile rendering algorithm. It generates interactive vibration patterns employing the tactile apparent movement phenomenon based on users movement to enhance the immersion and provide sensory feedback during rehabilitation training in the virtual reality environment. Three rehabilitation game applications are presented as proof of concept of the proposed system.

Design of a robotic device for assessment and rehabilitation of hand sensory function

This paper presents the design and implementation of the Robotic Sensory Trainer, a robotic interface for assessment and therapy of hand sensory function. The device can provide three types of well controlled stimuli: (i) angular displacement at the metacarpophalangeal (MCP) joint using a remote-center-of-motion double-parallelogram structure, (ii) vibration stimuli at the fingertip, proximal phalange and palm, and (iii) pressure at the fingertip, while recording position, interaction force and feedback from the user over a touch screen. These stimuli offer a novel platform to investigate sensory perception in healthy subjects and patients with sensory impairments, with the potential to assess deficits and actively train detection of specific sensory cues in a standardized manner. A preliminary study with eight healthy subjects demonstrates the feasibility of using the Robotic Sensory Trainer to assess the sensory perception threshold in MCP angular position. An average just noticeable difference (JND) in the MCP joint angle of 2.46° (14.47%) was found, which is in agreement with previous perception studies.

Advanced Augmented White Cane with obstacle height and distance feedback

The white cane is a widely used mobility aid that helps visually impaired people navigate the surroundings. While it reliably and intuitively extends the detection range of ground-level obstacles and drop-offs to about 1.2 m, it lacks the ability to detect trunk and head-level obstacles. Electronic Travel Aids (ETAs) have been proposed to overcome these limitations, but have found minimal adoption due to limitations such as low information content and low reliability thereof. Although existing ETAs extend the sensing range beyond that of the conventional white cane, most of them do not detect head-level obstacles and drop-offs, nor can they identify the vertical extent of obstacles. Furthermore, some ETAs work independent of the white cane, and thus reliable detection of surface textures and drop-offs is not provided. This paper introduces a novel ETA, the Advanced Augmented White Cane, which detects obstacles at four vertical levels and provides multi-sensory feedback. We evaluated the device in five blindfolded subjects through reaction time measurements following the detection of an obstacle, as well as through the reliability of dropoff detection. The results showed that our aid could help the user successfully detect an obstacle and identify its height, with an average reaction time of 410 msec. Drop-offs were reliably detected with an intraclass correlation > 0.95. This work is a first step towards a low-cost ETA to complement the functionality of the conventional white cane.

Identification of Vibrotactile Patterns Encoding Obstacle Distance Information

Delivering distance information of nearby obstacles from sensors embedded in a white cane - in addition to the intrinsic mechanical feedback from the cane - can aid the visually impaired in ambulating independently. Haptics is a common modality for conveying such information to cane users, typically in the form of vibrotactile signals. In this context, we investigated the effect of tactile rendering methods, tactile feedback configurations and directions of tactile flow on the identification of obstacle distance. Three tactile rendering methods with temporal variation only, spatio-temporal variation and spatial/temporal/intensity variation were investigated for two vibration feedback configurations. Results showed a significant interaction between tactile rendering method and feedback configuration. Spatio-temporal variation generally resulted in high correct identification rates for both feedback configurations. In the case of the four-finger vibration, tactile rendering with spatial/temporal/intensity variation also resulted in high distance identification rate. Further, participants expressed their preference for the four-finger vibration over the single-finger vibration in a survey. Both preferred rendering methods with spatio-temporal variation and spatial/temporal/intensity variation for the four-finger vibration could convey obstacle distance information with low workload. Overall, the presented findings provide valuable insights and guidance for the design of haptic displays for electronic travel aids for the visually impaired.

Combined tendon vibration and virtual reality for post-stroke hand rehabilitation

Sensory function is essential for functional post-stroke recovery and control of basic hand movements like grasping. Despite this fact, therapy focuses strongly on motor aspects of rehabilitation, requiring active participation and thus excluding stroke patients with severe paresis. The aim of our novel therapeutic approach combining virtual reality, based on clinically proven mirror therapy, and tendon vibration of hand and wrist muscles is to induce neuroplastic changes leading to improved hand function. This paper presents the further development and evaluation of a robotic device, which can apply vibrations at precise locations on the finger flexor tendons to create illusions of extension movements and visualize the movements with a virtual hand. A preliminary study including 16 healthy subjects investigated the influence of the virtual reality on the perception of proprioceptive illusory movements. The experimental results provided evidence that the addition of the virtual reality enhanced the perception of the illusory movement generated by tendon vibration, by inducing movements with significantly higher extension (+4.5%, p <; 0.05). Furthermore, the virtual reality allowed a better controlled temporal elicitation of the illusion. These findings indicate the potential of this novel strategy for a more effective therapy, especially for severely impaired patients.

Head-Mounted Sensory Augmentation Device: Comparing Haptic and Audio Modality

This paper investigates and compares the effectiveness of haptic and audio modality for navigation in low visibility environment using a sensory augmentation device. A second generation head-mounted vibrotactile interface as a sensory augmentation prototype was developed to help users to navigate in such environments. In our experiment, a subject navigates along a wall relying on the haptic or audio feedbacks as navigation commands. Haptic/audio feedback is presented to the subjects according to the information measured from the walls to a set of 12 ultrasound sensors placed around a helmet and a classification algorithm by using multilayer perceptron neural network. Results showed the haptic modality leads to significantly lower route deviation in navigation compared to auditory feedback. Furthermore, the NASA TLX questionnaire showed that subjects reported lower cognitive workload with haptic modality although both modalities were able to navigate the users along the wall.

Robot-assisted assessment of vibration perception and localization on the hand

Sensory perception is crucial for motor learning and the control of fine manipulations. However, therapy after stroke still strongly focuses on the training of motor skills. Sensory assessments are often left out or provide only very subjective data from poorly controlled stimuli. This paper presents a vibration detection/localization test with the Robotic Sensory Trainer, a device that focuses entirely on the assessment and training of sensory function of the hand, with the aim of gaining insights into the prevalence and severity of sensory deficits after stroke, and to provide semiobjective data on absolute and difference perception thresholds in patients. An initial feasibility study investigated localization performance and reaction time during the discrimination of vibration stimuli presented in four locations on the dominant and nondominant hands of 13 healthy volunteers. High correct detection rates were observed (mean ± SD of 99.6% ± 0.6%), touch screen PC Robotic Sensory Trainer which were found to be significantly different between stimulus locations on the dominant hand. Average correct detection rates were not statistically different between dominant and nondominant hand. These data will serve as baseline for future measurements on elderly and stroke subjects, and suggest that data from the nonimpaired hand could be used to identify and assess sensory deficits in the affected hand of stroke patients. Implications for Rehabilitation Sensory deficits may limit functional recovery after stroke, are poorly documented and often neglected in current therapeutic programs. Clinical sensory assessments are not well standardized, lack sensitivity and suffer from high variability. We present a novel device for the semiobjective assessment of hand sensory function, capable of providing vibration, proprioception and pressure stimuli. Vibration perception and localization on the hand were tested on 13 healthy subjects to serve as baseline data for future measurements with stroke patients.

Quantification of long cane usage characteristics with the constant contact technique.

While a number of Electronic Travel Aids (ETAs) have been developed over the past decades, the conventional long cane remains the most widely utilized navigation tool for people with visual impairments. Understanding the characteristics of long cane usage is crucial for the development and acceptance of ETAs. Using optical tracking, cameras and inertial measurement units, we investigated grasp type, cane orientation and sweeping characteristics of the long cane with the constant contact technique. The mean cane tilt angle, sweeping angle, and grip rotation deviation were measured. Grasp type varied among subjects, but was maintained throughout the experiments, with thumb and index finger in contact with the cane handle over 90% of the time. We found large inter-subject differences in sweeping range and frequency, while the sweeping frequency showed low intra-subject variability. These findings give insights into long cane usage characteristics and provide critical information for the development of effective ETAs.

SurfTics — Kinesthetic and tactile feedback on a touchscreen device

This paper presents a novel touchscreen-based haptic device, which provides access to graphical media for the visually impaired. The proposed device allows the exploration of shapes and the texture of images displayed on the touchscreen. The hardware design and rendering algorithms are described. Furthermore, when determining the size of objects the effectiveness of texture information combined with shape rendering is investigated. The result of the study, which is conducted with nine subjects, shows a significant improvement in the identification rate when additional texture is present.