Dongseok Ryu

Multi-modal user interface for teleoperation of ROBHAZ-DT2 field robot system

This paper describes a multi-modal interface design and its implementation to a teleoperated field robot system. The ROBHAZ-DT2 is developed as a teleoperated mobile manipulator for hazard environment applications (e.g. rescue, explosive ordnance disposal, security). To complete these missions in outdoor environment, the robot system must have appropriate functions, accuracy and reliability. However, the more functions it has, the more difficulties for the operator to control the functions. To cope up with this problem, an effective user interface should be developed. The main challenge of this research is to make a simple and intuitive user interface and teleoperate the slave robot easily. This paper provides multi-modalities such as visual, auditory and haptic sense. It enables an operator to control every functions of a field robot, ROBHAZ-DT2 more easily. In this paper, an EOD (explosive ordinance disposal) demonstration is conducted to verify the validity of the proposed multimodal interface.

ROBHAZ-DT2: design and integration of passive double tracked mobile manipulator system for explosive ordnance disposal

The design and integration of ROBHAZ-DT2 is presented which is a newly developed mobile manipulator system. It is designed to carry out military and civilian missions in various hazardous environments. In developing the integrated ROBHAZ-DT2 system, we have focused on two issues: 1) novel mechanism design for mobility and manipulation and 2) intuitive user interface for teleoperation. In mechanism design, a double tracks connected by a passive joint has been designed to achieve high speed and rugged mobility on uneven terrain. In addition, a six-dof foldable manipulator suitable for the mobile manipulation has been designed. Secondly, a new compact 6-dof haptic device has been developed for teleoperation of the ROBHAZ-DT2. This haptic device is specially designed for simultaneous control both of the mobile base and the manipulator of ROBHAZ-DT2. As a result of integration of ROBHAZ-DT2 and the user interface unit including the haptic device, we could successfully demonstrate a typical EOD task requiring abilities of mobility and manipulation in outdoor environment.

T-less : A novel touchless human-machine interface based on infrared proximity sensing

In todays industry, intuitive gesture recognition, as manifested in numerous consumer electronics devices, becomes a main issue of HMI device research. Although finger-tip touch based user interface has paved a main stream in mobile electronics, we envision touch-less HMI as a promising technology in futuristic applications with higher potential in areas where sanity or outdoor operation become of importance. In this paper, we introduce a novel HMI device for non-contact gesture input for intuitive HMI experiences. The enabling technology of the proposed device is the IPA (infrared Proximity Array) sensor by which realtime 3 dimensional depth information can be captured and realized for machine control. For the usability study, two different operating modes are adopted for hand motion inputs: one is a finger tip control mode and the other is a palm control mode. Throughput of the proposed device has been studied and compared to a traditional mouse device for usability evaluation. During the human subject test, the proposed device is found to be useful for PC mouse pointer control. The experimental results are shared in the paper as well.

T-hive : Vibrotactile interface presenting spatial information on handle surface

Many recent studies have explored the use of tactile cues, however they were confined to the unilateral display device. Although lots of bilateral haptic devices have been developed to provide a guiding force on an input handle, however, a vibrotactile stimuli has not been tried to present directional information on the handle. This research introduces an attempt to combine a tactile display with an input device. A new 6DOF bilateral haptic device, which provides a spatial sensation on the handle using vibrotactile display, is proposed in this research. The sphere-shape handle is specially designed to be covered with several pieces of vibrating panels. When a specific panel is activated, the user perceives the spatial location of the vibrotactile stimulus during an input operation. This paper introduces the design of the proposed device, including the selection guide of the dimension, location, and number of vibrotactile panels. The method for combination of vibrotactile stimulus and the way to achieve fine resolution with small number of tactors are discussed. Experimental results show that the users can reliably perceive the directional information using the proposed device. An application for teleoperation of a robot proves the effectiveness and the usefulness of the proposed bilateral device.

Development of a Slim Haptic Glove Using McKibben Artificial Muscles

In this paper, a light-weight and wearable haptic glove system is introduced which is designed for virtual environments. In order to reduce the weight of the system, micro McKibben artificial muscles are used and 2-port solenoid valve pneumatic system was developed for faster response. Hydraulic system was also developed to actuate artificial muscles, so that we can overcome softness and inaccuracy of pneumatic system due to the compressibility of air. We verify the functionality and usefulness of the proposed system by synchronizing it with a virtual environment

Wearable haptic-based multi-modal teleoperation of field mobile manipulator for explosive ordnance disposal

This paper describes a wearable multi-modal user interface design and its implementation for a teleoperated field robot system. Recently some teleoperated field robots are employed for hazard environment applications (e.g. rescue, explosive ordnance disposal, security). To complete these missions in outdoor environment, the robot system must have appropriate functions, accuracy and reliability. However, the more functions it has, the more difficulties occur in operation of the functions. To cope up with this problem, an effective user interface should be developed. Furthermore, the user interface is needed to be wearable for portability and prompt action. This research starts at the question: how to teleoperate the complicated slave robot easily. The main challenge is to make a simple and intuitive user interface with a wearable shape and size. This research provides multi-modalities such as visual, auditory and haptic sense. It enables an operator to control every functions of a field robot more intuitively. As a result, an EOD (explosive ordnance disposal) demonstration is conducted to verify the validity of the proposed wearable multi-modal user interface.

Vibrotactile display for hand-held input device providing spatial and directional information

A hand-held input device providing vibrotactile cues is developed for transmitting spatial and directional information to the users controlling a mobile robot or a manipulator arm. A sphere-shaped handle is applied to make comfortable for gripping the handle, and the input device can provide six degrees of freedom motion of the handle. For maintaining isolation among vibrotactile actuators, surface of the sphere is divided into several pieces. Each vibrating module consists of a vibration motor, a piece of sphere surface and a vibration absorber. Spatial and directional information can be generated using sensory saltation and phantom sensation. As an evaluation of the developed device, two experiments were conducted. Experimental results show that the developed device is useful for displaying intuitive information for controlling mobile robot with vibrotactile feedback.

Real-time eye tracking using a smart camera

Real-time eye and iris tracking is important for handsoff gaze-based password entry, instrument control by paraplegic patients, Internet user studies, as well as homeland security applications. In this project, a smart camera, LabVIEW and vision software tools are utilized to generate eye detection and tracking algorithms. The algorithms are uploaded to the smart camera for on-board image processing. Eye detection refers to finding eye features in a single frame. Eye tracking is achieved by detecting the same eye features across multiple image frames and correlating them to a particular eye. The algorithms are tested for eye detection and tracking under different conditions including different angles of the face, head motion speed, and eye occlusions to determine their usability for the proposed applications. This paper presents the implemented algorithms and performance results of these algorithms on the smart camera.

Design of a Haptic Stability Observer in Frequency Domain for Stable Haptic Interaction

Stable haptic interaction has been studied extensively by an energy-based approach. However, the energy in the haptic system is not directly measurable, but estimated from some measured quantities such as force and velocity; therefore, the estimated energy is occasionally inaccurate. Furthermore, haptic stability is difficult to observe by the energy-based approach for complicated virtual environments. To resolve this problem, a new observer, called a haptic stability observer, is proposed in this research. The empirical investigation of unstable haptic interaction in the frequency domain showed that the typical unstable behavior of a haptic system is different from random oscillations introduced by the hand motion of a human operator. The haptic stability observer working in the frequency domain quantifies the degree of instability of a haptic system. A double layered wall was implemented in the virtual environment to verify the validity of the HSO. Experimental results of the double layered wall show that the HSO is much faster than the passivity observer in detecting haptic instability

Micro hydraulic system using slim artificial muscles for a wearable haptic glove

Over the past few decades, various haptic gloves have been developed for use in virtual environments. The actuating systems for most existing haptic gloves require lots of external auxiliary equipment. Because of this, the motion of the user is restricted by the length of the electric wires or pneumatic tubes attached to this equipment. A compact actuation system, including related equipment, is thus indispensable for a wearable haptic glove to be truly effective. To resolve the problem of hampered motion and reach, a micro hydraulic actuating system was developed in this research. It was composed of a slim, flexible artificial muscle, a compact hydraulic module for actuating the muscle, and a micro pressure sensor for measuring without flux loss. The characteristics of the muscle were investigated for their control capacity. The step and sinusoidal responses were analyzed to evaluate the performance of the micro hydraulic system. Once these analyses were completed, a lightweight and compact actuation system was built incorporating a wearable haptic glove. By virtue of the developed micro hydraulic system, the wearable haptic glove was able to operate independently of any external equipment, and movement was completely free of any restrictions from wires or tubes.