Rafiuddin Syam

Adaptive actor-critic learning for the control of mobile robots by applying predictive models

In this paper, we propose two methods of adaptive actor-critic architectures to solve control problems of nonlinear systems. One method uses two actual states at time k and time k+1 to update the learning algorithm. The basic idea of this method is that the agent can directly take some knowledge from the environment to improve its knowledge. The other method only uses the state at time k to update the algorithm. This method is called, learning from prediction (or simulated experience). Both methods include one or two predictive models, which are assumed to be applied to construct predictive states and a model-based actor (MBA). Here, the MBA as an actor can be viewed as a network where the connection weights are the elements of the feedback gain matrix. In the critic part, two value-functions are realized as a pure static mapping, which can be reduced to a nonlinear current estimator by using the radial basis function neural networks (RBFNNs). Simulation results obtained for a dynamical model of nonholonomic mobile robots with two independent driving wheels are presented. They show the effectiveness of the proposed approaches for the trajectory tracking control problem.

Control of nonholonomic mobile robot by an adaptive actor-critic method with simulated experience based value-functions

An adaptive actor-critic algorithm is proposed under the assumption that a predictive model is available and only the measurement at time k is used to update the learning algorithms. Two value-functions are realized as a pure static mapping, according to the fact that they can be reduced to nonlinear current estimators, which can be easily constructed by using any artificial neural networks (NNs) with sigmoidal function or radial basis function (RBF), if all the inputs to the present value-functions are based on simulated experiences generated from the predictive model. In addition, if a predictive model is assumed to be used to construct a model-based actor (MBA) in the framework of adaptive actor-critic approach, then this type of MBA can be viewed as a network whose connection weights are composed of the elements of feedback gain matrix, so that the temporal difference (TD) learning can also be naturally applied to update the weights of the actor. Since the present method can update the learning by using only one measurement at time k, a relatively fast learning is expected, compared with the previous approach that needs two measurements at times k and k + 1 to update the actor-critic networks. The effectiveness of the proposed approach is illustrated by simulating a trajectory-tracking control problem for a nonholonomic mobile robot.

Discontinuous Control and Backstepping Method for the Underactuated Control of VTOL Aerial Robots with Four Rotors

A control strategy is proposed here for four-rotor vertical take-off and landing (VTOL) aerial robot called X4-flyer. Since the X4-flyer has underactuated and nonholonomic features, a kinematics control law is first derived using Astolfi’s discontinuous control. A backstepping method that is one of adaptive control methods based on Lyapunov methods, then provides the kinematic based inputs, to construct the torque control of X4-flyer. Finally, computer simulations are given to demonstrate the effectiveness of our approach.

A study on constructing a neuro-interface using the concept of a virtual master–slave system

To ease the control of a nonholonomic robot by a non-expert, a neuro-interface is proposed by using the concept of a virtual master–slave system. The design procedure for the interface is elaborated for the control of nonholonomic two-wheeled robots. In particular, it is shown that if the coordinate transformation from the slave to the master is assumed to be known, the resultant inverse mapping of the master robot can be simply realized by a neural network (NN) with all linear units. The training of the NN is performed by an offline method. As a result, the effectiveness of the proposed method is shown for some simulations to solve a trajectory tracking control problem with a nonholonomic mobile robot.

An Adaptive Actor-critic Algorithm with Multi-step Simulated Experiences for Controlling Nonholonomic Mobile Robots

In this paper, we propose a new algorithm of an adaptive actor-critic method with multi-step simulated experiences, as a kind of temporal difference (TD) method. In our approach, the TD-error is composed of two value- functions and m utility functions, where m denotes the number of multi-steps in which the experience should be simulated. The value-function is constructed from the critic formulated by a radial basis function neural network (RBFNN), which has a simulated experience as an input, generated from a predictive model based on a kinematic model. Thus, since our approach assumes that the model is available to simulate the m-step experiences and to design a controller, such a kinematic model is also applied to construct the actor and the resultant model based actor (MBA) is also regarded as a network, i.e., it is just viewed as a resolved velocity control network. We implement this approach to control nonholonomic mobile robot, especially in a trajectory tracking control problem for the position coordinates and azimuth. Some simulations show the effectiveness of the proposed method for controlling a mobile robot with two-independent driving wheels.

A Neuro-interface with fuzzy compensator for controlling nonholonomic mobile robots

This paper describes a control method for mobile robots represented by a nonlinear dynamical system, which is subjected to an output deviation caused by drastically changed disturbances. We here propose some controllers in the framework of neuro-interface. It is assumed that a neural network (NN)-based feedforward controller is construcetd by following the concept of virtual master-slave robot, in which a virtual master robot as a feedforward controller is used to control the slave (i.e., actual) robot. The whole system of the present neuro-interface consists of an NN-based feedforward controller, a feedback PD controller and an adaptive fuzzy feedback compensator. The NN-based feedforward controller is trained offline by using a gradient method, the gains of the PD controller are to be chosen constant, and the adaptive fuzzy compensator is constructed with a simplified fuzzy reasoning. Some simulations are presented to confirm the validity of the present approach, where a nonholonomic mobile robot with two independent driving wheels is assmued to have a disturbance due to the change of mass for the robot.

Design of Wheeled Mobile Robot with Tri-Star Wheel as Rescue Robot

This study aims to design, and analyze a mobile robot that can handle some of the obstacles, they are uneven surfaces, slopes, can also climb stairs. WMR in this study is Tristar wheel that is containing three wheels for each set. On average surface only two wheels in contact with the surface, if there is an uneven surface or obstacle then the third wheel will rotate with the rotation center of the wheel in contact with the leading obstacle then only one wheel in contact with the surface. This study uses the C language program. Furthermore, the minimum thrust to be generated torque of the motor and transmission is 9.56 kg. The results obtained by calculation and analysis of DC motors used must have a torque greater than 14.67 kg.cm. Minimum thrust to be generated motor torque and the transmission is 9.56 kg. The experimental results give good results for robot to moving forward, backward, turn left, turn right and climbing the stairs

Simulation and Experimental Works of Quadcopter Model for Simple Maneuver

This study aims to create a simulated and experimental of aircraft movements for multirotor quadcopter. The research method is theoretical and experimental methods. For theoretical method consists of calculating the dynamics and kinematics. While the experimental method consists of the aircraft testing and processing of GPS data recorded aircraft. The results showed that the acceleration acting on the aircraft is large enough that x ̈ = 1.751 m/s2, y = 2.038 m /s2 = 1.6371 m danz ̈ / s2, (2) the value of the maximum error between the theoretical and the actual movement is ex = 0.682 m; ey and ez = 0.353 m = 0.546 m. Theoretical movement pattern already resembles the actual movement..

A neurointerface with an adaptive fuzzy compensator for controlling nonholonomic mobile robots

This paper describes an adaptive control for nonholonomic mobile robots, which are subjected to a suddenly changed disturbance due to the change of payloads. We adopt a control architecture based on a two-degrees-of-freedom design, where the feedforward controller is constructed by a neural network (NN) to acquire an inverse dynamical model of the robot, whereas the feedback controller is designed by two methods: one is a conventional PD compensator and the other is an adaptive fuzzy compensator. A concept of virtual master-slave robots is applied to obtain an inverse model of a nonholonomic robot. A compensator needs to be used to reduce the effect of the NN mapping errors or to suppress the effect of a sudden change of payloads. It is demonstrated by several simulations that the present approach is effective for controlling a nonholonomic mobile robot in a navigation of trajectory tracking problem for the positions and azimuth.