Hyoung Il Son

Bilateral Teleoperation of Groups of Mobile Robots With Time-Varying Topology

In this paper, a novel decentralized control strategy for bilaterally teleoperating heterogeneous groups of mobile robots from different domains (aerial, ground, marine, and underwater) is proposed. By using a decentralized control architecture, the group of robots, which is treated as the slave side, is made able to navigate in a cluttered environment while avoiding obstacles, interrobot collisions, and following the human motion commands. Simultaneously, the human operator acting on the master side is provided with a suitable force feedback informative of the group response and of the interaction with the surrounding environment. Using passivity-based techniques, we allow the behavior of the group to be as flexible as possible with arbitrary split and join events (e.g., due to interrobot visibility/packet losses or specific task requirements) while guaranteeing the stability of the system. We provide a rigorous analysis of the system stability and steady-state characteristics and validate performance through human/hardware-in-the-loop simulations by considering a heterogeneous fleet of unmanned aerial vehicles (UAVs) and unmanned ground vehicles as a case study. Finally, we also provide an experimental validation with four quadrotor UAVs.

A passivity-based decentralized approach for the bilateral teleoperation of a group of UAVs with switching topology

In this paper, a novel distributed control strategy for teleoperating a fleet of Unmanned Aerial Vehicles (UAVs) is proposed. Using passivity based techniques, we allow the behavior of the UAVs to be as flexible as possible with arbitrary split and join decisions while guaranteeing stability of the system. Furthermore, the overall teleoperation system is also made passive and, therefore, characterized by a stable behavior both in free motion and when interacting with unknown passive obstacles. The performance of the system is validated through semi-experiments.

Human-Centered Design and Evaluation of Haptic Cueing for Teleoperation of Multiple Mobile Robots

In this paper, we investigate the effect of haptic cueing on a human operators performance in the field of bilateral teleoperation of multiple mobile robots, particularly multiple unmanned aerial vehicles (UAVs). Two aspects of human performance are deemed important in this area, namely, the maneuverability of mobile robots and the perceptual sensitivity of the remote environment. We introduce metrics that allow us to address these aspects in two psychophysical studies, which are reported here. Three fundamental haptic cue types were evaluated. The Force cue conveys information on the proximity of the commanded trajectory to obstacles in the remote environment. The Velocity cue represents the mismatch between the commanded and actual velocities of the UAVs and can implicitly provide a rich amount of information regarding the actual behavior of the UAVs. Finally, the Velocity+Force cue is a linear combination of the two. Our experimental results show that, while maneuverability is best supported by the Force cue feedback, perceptual sensitivity is best served by the Velocity cue feedback. In addition, we show that large gains in the haptic feedbacks do not always guarantee an enhancement in the teleoperators performance.

Gain-Scheduling Control of Teleoperation Systems Interacting With Soft Tissues

Surgical teleoperation systems are being increasingly deployed recently. There are, however, some unsolved issues such as nonlinear characteristics of the interaction between the slave robot and soft tissues and difficulty in employing force sensors in the surgical end-effectors of the slave. These issues make it difficult to generalize any approach to develop a control for the system. This paper addresses these issues by proposing a H∞ suboptimal controller preserving robust stability and performance. The environment, i.e., soft tissues, is characterized with the nonlinear Hunt-Crossley model. This nonlinear characteristics of soft tissues are expressed with an affine combination of linear models within a predefined parameter polytope. For this linear parameter-varying system, a gain-scheduling control scheme is employed to design a suboptimal controller while guaranteeing its stability. To avoid using any force measurement in slave, we used position-position (PP) control architecture. The developed gain-scheduling control is validated with quantitative experimental results. The developed gain-scheduling PP control scheme shows good tracking capacity and high transparency for varied experimental conditions. Error of the transmitted impedance is significantly lower compared with other conventional control schemes for frequencies less than 2 Hz, which is frequently recommended for surgical teleoperation.

Enhancement in Operator's Perception of Soft Tissues and Its Experimental Validation for Scaled Teleoperation Systems

This paper focuses on scaled teleoperation systems interacting with soft tissues and presents an optimal control scheme to maximize the operators kinesthetic perception of remote soft environments while maintaining the stability in macro-micro interactions. Two performance metrics are defined to quantify the kinesthetic perception of the surgeons and the position tracking ability of the master-slave system. Kinesthetic perception is defined based on psychophysics by using two metrics, which relate to the detection and discrimination of stimulus. This paper then employs a multiconstrained optimization approach to get an optimal solution in the presence of the stability-performance tradeoff wherein the objective is to enhance the kinesthetic perception while maintaining the tracking and robust stability for interactions between macro and microworlds. Simplified stability constraints for scaled teleoperation systems are designed based on Llewellyns absolute stability criterion for the optimization procedure, which provides easy and effective design guidelines for selecting control gains. Experiments with phantom soft tissues have been conducted using scaled force-position control architecture, scaled position-position control architecture, and scaled four-channel control architecture to verify the proposed control scheme. Results prove the effectiveness of this algorithm in enhancing the kinesthetic perception of surgeons for scaled teleoperation systems. Psychophysical experiments were then performed to compare our approach with similar contemporary research methods that further validated its efficacy.

Estimation of environmental force for the haptic interface of robotic surgery

The success of a telerobotic surgery system with haptic feedback requires accurate force‐tracking and position‐tracking capacity of the slave robot. The two‐channel force‐position control architecture is widely used in teleoperation systems with haptic feedback for its better force‐tracking characteristics and superior position‐tracking capacity for the maximum stability margin. This control architecture, however, requires force sensors at the end‐effector of the slave robot to measure the environment force. However, it is difficult to attach force sensors to slave robots, mainly due to their large size, insulation issues and also large currents often flowing through the end‐effector for incision or cautery of tissues.

Effect of Impedance-Shaping on Perception of Soft Tissues in Macro-Micro Teleoperation

This paper aims at analyzing the effect of widely known impedance-shaping (IS) control method on the perception of soft tissues in telemicrosurgical applications. The generalized teleoperation control architecture has been modified to include the IS term. New performance index has been defined based on the two proposed indices for the detection and the discrimination of the soft environments to analyze the effect of this modified control on the kinesthetic perception of soft tissues. The effect is then theoretically analyzed on the conventional position-position, force-position, and four-channel control architectures based on the newly defined index. The effectiveness of this newly proposed kinesthetic perception index is also verified using psychophysics experiments. The theoretical analysis of the effects of the IS method on the perception of soft tissues is then validated using the proposed index by experiments with phantom soft tissues for conventional teleoperation architectures.

Experiments on Intercontinental Haptic Control of Multiple UAVs

In this paper we propose and experimentally validate a bilateral teleoperation framework where a group of UAVs are controlled over an unreliable network with typical intercontinental time delays and packet losses. This setting is meant to represent a realistic and challenging situation for the stability the bilateral closed-loop system. In order to increase human telepresence, the system provides the operator with both a video stream coming from the onboard cameras mounted on the UAVs, and with a suitable haptic cue, generated by a force-feedback device, informative of the UAV tracking performance and presence of impediments on the remote site.

Measuring an operator's maneuverability performance in the haptic teleoperation of multiple robots

In this paper, we investigate the maneuverability performance of human teleoperators on multi-robots. First, we propose that maneuverability performance can be assessed by a frequency response function that jointly considers the input force of the operator and the position errors of the multi-robot system that is being maneuvered. Doing so allows us to evaluate maneuverability performance in terms of the human teleoperators interaction with the controlled system. This allowed us to effectively determine the suitability of different haptic cue algorithms in improving teleoperation maneuverability. Performance metrics based on the human teleoperators frequency response function indicate that maneuverability performance is best supported by a haptic feedback algorithm which is based on an obstacle avoidance force.

Analytical and Psychophysical Comparison of Bilateral Teleoperators for Enhanced Perceptual Performance

This paper focuses on the human perception capabilities for haptic interaction with remote environments. The perception capabilities are compared for two well-known control methods with two kinds of haptic cues. Analytical and psychophysical methods are used to analyze the performance. The first control method aims at maximizing the transparency of the remote interactions (i.e., transparency-based method), whereas the second one aims at maximizing the detection and discrimination abilities of the human operator (i.e., perception-based method). For each of these two control methods, two kinds of haptic cues are studied, which use position and force cues from remote environments. Hybrid matrix formulation is employed, and it is analyzed in the frequency domain for these studies. Psychophysical experiments are then conducted for human-centered evaluation and comparison of the control methods. Analytical and experimental results clearly show that the perception-based method, when compared with the transparency-based method, enhances the human operators perceptual capabilities of remote environments irrespective of force cues. For each of the two control methods, the force cues always contribute more to the increase in perceptual sensitivity when compared with the case of position cues.