Marcin Szuster

Discrete model-based adaptive critic designs in wheeled mobile robot control

In this paper a discrete tracking control algorithm for a nonholonomic two-wheeled mobile robot (WMR) is presented. The basis of the control algorithm is an Adaptive Critic Design (ACD) in two model-based configurations: Heuristic Dynamic Programming (HDP) and Dual Heuristic Programming (DHP). In proposed control algorithm Actor-- Critic structure, composed of two neural networks (NN), is supplied by a PD controller and a supervisory term derived from the Lyapunov stability theorem. The control algorithm works on-line and does not require preliminary learning. Verification of the proposed control algorithm was realized on a WMR Pioneer-2DX.

Neural dynamic programming in reactive navigation of wheeled mobile robot

In the article a new approach to a reactive navigation of a wheeled mobile robot (WMR), using a neural dynamic programming algorithm (NPD), is presented. A proposed discrete hierarchical control system consists of a trajectory generator and a tracking control system. In the trajectory generator we used a sensor-based approach to path design for the WMR in an unknown 2-D environment with static obstacles. The main part of the navigator is an action dependant heuristic dynamic programming algorithm (ADHDP), that generates control signals used to design a collision-free trajectory, that makes reaching a goal possible. ADHDP is the discrete algorithm of actor-critic architecture, that works on-line and does not require a preliminary learning or a controlled system knowledge. The tracking control system realises the generated trajectory, it consists of dual-heuristic dynamic programming (DHP) structure, PD controller and the supervisory term derived from the Lyapunov stability theorem. Computer simulations have been conducted to illustrate the performance of the algorithm.

Discrete Action Dependant Heuristic Dynamic Programming in Control of a Wheeled Mobile Robot

In presented paper we propose a discrete tracking control algorithm for a two-wheeled mobile robot. The control algorithm consists of discrete Adaptive Critic Design (ACD) in Action Dependant Heuristic Dynamic Programming (ADHDP) configuration, PD controller and a supervisory term, derived from the Lyapunov stability theorem and based on the variable structure systems theory. Adaptive Critic Designs are a group of algorithms that use two independent structures for estimation of optimal value function from Bellman equation and estimation of optimal control law. ADHDP algorithm consists of Actor (ASE - Associate Search Element) that estimates the optimal control law and Critic (ACE - Adaptive Critic Element) that evaluates quality of control by estimation of the optimal value function from Bellman equation. Both structures are realized in a form of Neural Networks (NN). ADHDP algorithm does not require a plant model (the wheeled mobile robot (WMR) model) for ACE or ASE neural network weights update procedure (in contrast with other ACD configurations e.g. Heuristic Dynamic Programming or Dual Heuristic Programming that use the plant model). In presented control algorithm Actor-Critic structure is supported by PD controller and the supervisory term, that guarantee stable implementation of tracking in an initial adaptive critic neural networks learning phase, and robustness in a face of disturbances. Verification of proposed control algorithm was realized on the two-wheeled mobile robot Pioneer-2DX.

Heuristic Dynamic Programming in Wheeled Mobile Robot Control

Abstract In this paper a discrete algorithm for tracking control of a two-wheeled mobile robot is presented. The basis of the control algorithm is an Adaptive Critic Design in Heuristic Dynamic Programming (HDP) configuration. HDP is a model-based discrete reinforcement learning algorithm. In proposed control algorithm Actor - Critic structure is supplied by a PD controller and a supervisory element. The algorithm does not require preliminary learning, works on-line and uses a dynamics model of the mobile robot for a state prediction in Actor - Critic structure. The performance of control algorithm was tested by experiments on the mobile robot Pioneer-2DX.

Fuzzy Sensor-Based Navigation with Neural Tracking Control of the Wheeled Mobile Robot

Navigation of the wheeled mobile robot in the unknown environment with simultaneous realisation of the generated trajectory, is one of the most challenging and up to date problems in the modern mobile robotics. In the article a new approach is presented to a collision-free trajectory generating for a wheeled mobile robot, realised in a form of the hierarchical control system with two layers. The first layer is a tracking control system, where the Neuro-Dynamic Programming algorithm in the Dual Heuristic Dynamic Programming configuration was applied. The second layer is a trajectory generator where the Fuzzy Logic systems were used. The presented control system generates and realises trajectory of the wheeled mobile robot within the complex task of goal-seeking and obstacle avoiding. The proposed hierarchical control system works on-line, its performance was verified using the wheeled mobile robot Pioneer 2-DX.

Reinforcement Learning in Discrete Neural Control of the Underactuated System

The article presents a new approach to the problem of a discrete neural control of an underactuated system, using reinforcement learning method to an on-line adaptation of a neural network. The controlled system is of the ball and beam type, which is the nonlinear dynamical object with the number of control signals smaller than the number of degrees of freedom. The main part of the neural control system is the actor-critic structure, that comes under the Neural Dynamic Programming algorithms family, realised in the form of Dual Heuristic Dynamic Programming structure. The control system includes moreover the PD controller and the supervisory therm, derived from the Lyapunov stability theorem, that ensures stability. The proposed neural control system works on-line and does not require a preliminary learning. Computer simulations have been conducted to illustrate the performance of the control system.

Conventional and fuzzy force control in robotised machining

This article presents an application of the hybrid position-force control of the robotic manipulator with use of artificial neural networks and fuzzy logic systems in complex control system. The mathematical description of the manipulator and a closed-loop system are presented. In the position control were used the PD controller and artificial neural networks, which compensate nonlinearities of the manipulator. The paper presents mainly the application of various strategies of the force control. The force control strategies using conventional controllers P, PI, PD, PID and fuzzy controllers are presented and discussed. All of the control methods were verified on the real object in order to make a comparison of a control quality.

Discrete Globalised Dual Heuristic Dynamic Programming in Control of the Two-Wheeled Mobile Robot

Network-based control systems have been emerging technologies in the control of nonlinear systems over the past few years. This paper focuses on the implementation of the approximate dynamic programming algorithm in the network-based tracking control system of the two-wheeled mobile robot, Pioneer 2-DX. The proposed discrete tracking control system consists of the globalised dual heuristic dynamic programming algorithm, the PD controller, the supervisory term, and an additional control signal. The structure of the supervisory term derives from the stability analysis realised using the Lyapunov stability theorem. The globalised dual heuristic dynamic programming algorithm consists of two structures: the actor and the critic, realised in a form of neural networks. The actor generates the suboptimal control law, while the critic evaluates the realised control strategy by approximation of value function from the Bellman’s equation. The presented discrete tracking control system works online, the neural networks’ weights adaptation process is realised in every iteration step, and the neural networks preliminary learning procedure is not required. The performance of the proposed control system was verified by a series of computer simulations and experiments realised using the wheeled mobile robot Pioneer 2-DX.

Discrete dual-heuristic programming in 3DOF manipulator control

In this paper we propose a discrete tracking control system for 3 degrees of freedom (DOF) robotic manipulator control. The control system is composed of Adaptive Critic Design (ACD), a PD controller and a supervisory term derived from the Lyapunov stability theory. ACD in Dual-Heuristic Programming (DHP) configuration consists of two structures realized in a form of neural networks (NN): actor - generates a control signal and critic approximates a derivative of the cost function with respect to the state. The control system works on-line, does not require a preliminary learning and uses the 3DOF manipulator dynamicsmodel for a state prediction in ACD structure. Verification of the proposed control algorithm was realized on a SCORBOT 4PC manipulator.

Adaptive position/force control for robot manipulator in contact with a flexible environment

The subject of the article is the adaptive position and force control of a robotic manipulator in interaction with flexible environment. The aim of the study is to provide a solution that takes into account the essential aspects of operation of the manipulator with the environment and at the same time can be actually implemented. A manipulatorenvironment system model taking into account motion resistance and environment elasticity. The position and force control task has been defined considering the manipulator and environment models. Asymptotic stability of the control system has been demonstrated considering the adaptation of parameters of the manipulator and the environment. Practical stability of the system has been demonstrated in the case of interference with the guaranteed stability of the adaptation of parameters without requiring persistence of excitation. Numerical analysis and experimental study of the issue has been presented. An interaction of robot manipulator with flexible environment is considered.A force/position tracking controller is proposed.No information on robot parameters is required.Practical stability is guaranteed by the adaptive controller.System stability is proved by using Lyapunov stability theory.