Jian Huang

Study on Motion Forms of a Two-dimensional Mobile Robot by Using Reinforcement Learning

The main advantage of reinforcement learning is that it provides unexpected solutions for a designer. This study shows how a mobile robot can obtain unexpected motion forms by using reinforcement learning. Results show that the mobile robot with two-dimensional mobile ability can obtain unexpected motion forms for both advance motion and rotation motion. The mechanisms for these motions were investigated in order to understand how to obtain these motions. Moreover, since this system has a two-dimensional factor, this study examines the learning characteristic for the oblivion of the learning knowledge. In addition, this study examines the learning of the knowledge manipulation method to obtain new learning results with respect to the two-dimensional factor

Consideration on robotic giant-swing motion generated by reinforcement learning

This study attempts to make a compact humanoid robot acquire a giant-swing motion without any robotic models by using reinforcement learning; only the interaction with environment is available. Generally, it is widely said that this type of learning method is not appropriated to obtain dynamic motions because Markov property is not necessarily guaranteed during the dynamic task. However, in this study, we try to avoid this problem by embedding the dynamic information in the robotic state space; the applicability of the proposed method is considered using both the real robot and dynamic simulator. This paper, in particular, discusses how the robot with 5-DOF, in which the Q-Learning algorithm is implemented, acquires a giant-swing motion. Further, we describe the reward effects on the Q-Learning. Finally, this paper demonstrates that the application of the Q-Learning enable the robot to perform a very attractive giant-swing motion.

Integration of impedance control and manipulability regulation for a finger-arm robot

Motion control algorithms were proposed for a 9-DOF finger-arm robot by using the finger manipulability obtained in a previous study. However, in the previous study, only methods for achieving unconstrained movement of the finger-arm robot were discussed. In this paper, the authors propose a novel method for having a finger-arm robot complete a constrained task by integrating impedance control with manipulability control of the finger. First, methods of applying the previous heuristic method to the passive impedance control and the active impedance control were developed; and experiments were performed to demonstrate the features. Then, an impedance control combined with the steepest ascent method to modulate the manipulability of the finger was proposed. The proposed method demonstrates a strong performance even when a dynamic external force is applied to the finger. By using the proposed method, the arm actively moves along a direction to effectively maintain the moving potential of the finger during both the unconstrained and the constrained tasks.

Emulating the Motion of a Human Upper Limb: Controlling a Finger-arm Robot by using the Manipulability of its Finger

In this paper, a 3-DOF robot finger is fixed onto the end-effector of a 6-DOF robot arm to realize a finger-arm robot. As in the case of the upper limb of a human being, the finger- arm robot possesses a high degree of freedom. This paper describes a new method to control the finger-arm robot so as to simply generate a natural motion emulating the movement of a human upper limb. To achieve this, a control method based on the manipulability of the finger is proposed. By using the proposed method, the finger primarily moves to complete a delicate local task, while the arm only moves to assist the finger when the manipulability of the finger falls below a reference value. The proposed method was applied to the task of drawing two 3D figures. The obtained results clarify the relation between manipulability and kinetic energy and also demonstrate the effectiveness of the proposed method.

Analysis of Human Weight Perception for Sudden Weight Changes during Lifting Task Using a Force Display Device

The main purpose of this study is to apply the mechanism of human weight perception to multimedia applications. When a start and the goal points are given in the human motion, the velocity profile has a unimodal profile including the acceleration and deceleration zones. In such case, it is widely said that the motion control becomes more accurate in the deceleration zone for the final approach. Hence, it is assumed that the sensing ability also improves. This study challenges to confirm the hypothesis with weight stimuli generated by a force display device. As a preliminary experiment, this study confirms that our device enables the evaluation of human perception by tracing Webers experiment in virtual reality; the obtained difference threshold (DL: Differenz Limen) almost agrees with the previous results. The difference threshold is a very important factor, which means perceptual threshold for weight changes. We further attempt to clarify the mechanism of the human weight perception for sudden weight changes during the lifting process. Based on individual lifting profiles, this study suddenly changes the load force in the two zones, and examines the difference thresholds of each subject. The results demonstrate that the human weight perception tends to become more sensitive in the deceleration zone during the lifting process. These results also imply the effectiveness of force display devices for physical experiments.

Study on Motion Forms of Mobile Robots Generated by Q-Learning Process Based on Reward Databases

This paper investigates the motion forms of robots generated by the Q-Learning algorithm during the learning process. We analyzed the manner in which a caterpillar robot, which performs looping motions using two actuators, acquires advance actions by focusing on the process. By observing a series of processes, we confirmed that various motion forms appeared or disappeared as a result of their interactions with the learning process and approach an optimum motion form. In most algorithms, such motion forms cannot appear in the learning process because its framework is almost predetermined by the teacher data, and the cost functions for learning cannot be usually considered as a continuous process. The characteristics of reinforcement learning are very interesting from the viewpoint of biological evolution. This paper describes the effects of the interaction between the robot kinematics and the environment as a direct result of changing the environment. In addition, this study challenged the acquisition of two-dimensional motions with a starfish robot having four actuators. The result demonstrates that the robot can obtain a reasonable motion from the complicated relationships with the environment by skillfully employing its structure. Moreover, this paper implies that the reward manipulation may give a new insight for the learning process by the investigations performed in this study. This paper examines the possibility of the reward combinations for generating arbitrary motions.

Analysis of weight perceptual mechanism based on muscular motion using virtual reality

In this paper, by focusing on weight perception, we attempted to clarify a perceptual mechanism using functions of the haptic interface. We especially examined the weight perceptual mechanism when the weight of an object suddenly changed, which does not take place in our natural life. This experimental condition can only be realized by using the advantage of the haptic interface. The results show the difference in the perceptual threshold due to addition and reduction of the weight. Moreover, the results indicate that subjects can also correct the supporting force corresponding to the sudden change in the weight when they could not sense the change. Namely, it implies that subjects adjust the supporting force under an imperceptible state, which may indicate an interesting muscular motion under a state of unconsciousness.

Controlling a Finger-Arm Robot to Emulate the Motion of the Human Upper Limb by Regulating Finger Manipulability

The human upper limb possesses a high degree of freedom (DOF) and its redundant structure permits greater flexibility in various dexterous manipulations. The simplest structure of a multifingered robot arm is constructed by fixing a robot finger onto the end effector of a robot arm. A robot with such a structure is also called a macro-micro manipulator (Nagai & Yoshigawa, 1994, 1995; Yoshikawa, et al. 1993). Similar to the human upper limb, the finger-arm robot exhibits a high redundancy. The movement of the robots with such high redundancies creates the problem of how to determine the numerous DOFs of its joints. Controlling a robot with a high degree of redundancy is a fundamental problem in the field of robotics. A large number of studies have been published on the methodology for determining the redundant DOFs of a robot. Avoidance control of kinematics singularity (Nakamura & Hanafusa, 1986; Furusho & Usui 1989) and obstacle collision avoidance (Khatib, 1986; Maciejewski & Klein, 1985; Loeff & Soni, 1975; Guo & Hsia, 1993; Glass et. Al, 1995) by using redundant DOFs has been mostly investigated. In order to realize desired solutions for the above mentioned problems, methods involving null space (Vannoy & Xiao, 2004) and the criterion function (Kim & Kholsa, 1992; Ma & Nechev, 1995; Ma et al, 1996) have been typically applied. The finger-arm robot is unlike conventional redundant manipulators. The finger is usually lightweight and has a small link size as compared to the arm. Therefore, it is inappropriate to directly apply the methods developed for controlling a redundant manipulator to the finger-arm robot. To achieve the dexterity like the human hand-arm, a lightweight finger should be actively moved whereas the arm cooperate the movement of the finger, which will greatly improve the performance of a robot (Khatib, 1995; Melchiorri & Salisbury, 1995). The human hand-arm system exhibits similar features. The human hand is obviously lighter, smaller and more sensitive as compared to the arm. The hand-arm coordination is well organized by the central nervous system so as to generate a natural motion. The motivation of this study is to develop a control method emulating a natural movement similar to that of a human upper limb.

Basic Study on Sensory Aspects of a Master/Slave System for Force Telecommunication

The advancement of multimedia services including audio/visual media has been rapid. The applications of cellular phones, in particular, have increased by using these technologies. In addition, if force telecommunication were realized, force information would become a next leading factor in the multimedia service. In this field, a robotic master/slave system is a leading candidate. This system has been studied as a teleoperation system between an operator and an environment; the operator controls only the master device whereas the slave device is never controlled. For multimedia applications, such systems require flexibility and bidirectionality because force telecommunication must consider the interactions between users. This paper details the fundamental issues of force communication from a viewpoint of human perception. With regard to the control system, this study uses an integrated hybrid control master/slave system that helps to find a key factor of equal bilateral force telecommunication in the sensory aspect. The employed system has a function to change the proportion of force display to position display by varying a weight parameter. We discuss the basic control performance of a force telecommunication system and evaluate qualitative feelings of users by using a psychological evaluation method

Dynamic movement by admittance control of a multi-finger-arm robot with manipulability control of fingers

Cooperative control algorithms for a redundant finger-arm robot were proposed in a previous study, based on the movement of the human hand-arm system. With these algorithms, the finger-arm robot arm was able to complete a constrained task by integrating admittance control and impedance control with the manipulability control of the fingers. The manipulability of the fingers was controlled using both an approximate global search (i.e., the top search method) and a local optimization method. However, the dynamic characteristics of these methods have not yet been studied. In this study, the conventional cooperative control algorithms were developed further by extending them to dynamic movement in the admittance control of a redundant multi-finger-arm robot with manipulability control of the fingers. We clarified the relationship between the feasibility of the virtual dynamics and the motion speed, as well as the range of the achievable virtual dynamics. This paper evaluates and discusses the proposed methods in terms of the finger manipulability control. The experimental results show that the algorithms responded to more dynamic movement, and a broader range of virtual dynamics was achievable.