Haobin Shi

Decoupled Visual Servoing With Fuzzy Q -Learning

The objective of visual servoing aims to control an objects motion with visual feedbacks and becomes popular recently. Problems of complex modeling and instability always exist in visual servoing methods. Moreover, there are few research works on selection of the servoing gain in image-based visual servoing (IBVS) methods. This paper proposes an IBVS method with Q-Learning, where the learning rate is adjusted by a fuzzy system. Meanwhile, a synthetic preprocess is introduced to perform feature extraction. The extraction method is actually a combination of a color-based recognition algorithm and an improved contour-based recognition algorithm. For dealing with underactuated dynamics of the unmanned aerial vehicles (UAVs), a decoupled controller is designed, where the velocity and attitude are decoupled through attenuating the effects of underactuation in roll and pitch and two independent servoing gains, for linear and angular motion servoing, respectively, are designed in place of single servoing gain in traditional methods. For further improvement in convergence and stability, a reinforcement learning method, Q-Learning, is taken for adaptive servoing gain adjustment. The Q-Learning is composed of two independent learning agents for adjusting two serving gains, respectively. In order to improve the performance of the Q-Learning, a fuzzy-based method is proposed for tuning the learning rate. The results of simulations and experiments on control of UAVs demonstrate that the proposed method has better properties in stability and convergence than the competing methods.

An adaptive decision-making method with fuzzy Bayesian reinforcement learning for robot soccer

Abstract A robot soccer system is a typical complex time-sequence decision-making system. Problems of uncertain knowledge representation and complex models always exist in robot soccer games. To achieve an adaptive decision-making mechanism, a method with fuzzy Bayesian reinforcement learning (RL) is proposed in this paper. To extract the features utilized in the proposed learning method, a fuzzy comprehensive evaluation method (FCEM) is developed. This method classifies the situations in robot soccer games into a set of features. With the fuzzy analytical hierarchy process (FAHP), the FCEM can calculate the weights according to defined factors for these features, which comprise the dimensionality of the state space. The weight imposed on each feature determines the range of each dimension. Through a Bayesian network, the comprehensively evaluated features are transformed into decision bases. An RL method for strategy selection over time is implemented. The fuzzy mechanism can skillfully adapt experiences to the learning system and provide flexibility in state aggregation, thus improving learning efficiency. The experimental results demonstrate that the proposed method has better knowledge representation and strategy selection than other competing methods.

A novel fuzzy omni-directional gait planning algorithm for biped robot

Aiming at the problems in gait planning of the biped robots, including the complex model, low stability, etc., a novel fuzzy omni-directional gait planning algorithm (FOGPA) is proposed. At first, this method puts forward a new separated omni-directional gait planning model, which combines the straight walking planning algorithm based on the improved Hermite interpolation and the rotation motion together. And then, a fuzzy gait parameter adjustment algorithm is put forward to control the gait parameters including the step size and rotation speed dynamically. At last, the fuzzy control results are used to get the gait data of robot real-timely. The experiment results show that the FOGPA improves the stability and robustness of gait in a certain degree and also improves the adaptability to the complex environment of the robot.

Research on collaborative allocation of the multi-agent based on equivalent time

According to the problems of complex allocation model, large calculation amount, poor real time capability of current robots co-allocation, this paper puts forward a collaborative allocation method of the multi-agent micro queue system based on equivalent time. Firstly, this method makes use of the improved single camera field depth algorithm for an accurate self-localization of the robot. Then, this method puts forward the concept of equivalent time based on the robots self status and the obstacle-avoidance status. Finally, this method looks for the optimal role assignment of robots with the total equivalent execution time being regarded as the optimization objective. The experiment results show that, the method successfully achieves robots role assignment with higher execution efficiency on the premise of strong real-time performance.