Zenon Hendzel

Discrete model-based adaptive critic designs in wheeled mobile robot control

In this paper a discrete tracking control algorithm for a nonholonomic two-wheeled mobile robot (WMR) is presented. The basis of the control algorithm is an Adaptive Critic Design (ACD) in two model-based configurations: Heuristic Dynamic Programming (HDP) and Dual Heuristic Programming (DHP). In proposed control algorithm Actor-- Critic structure, composed of two neural networks (NN), is supplied by a PD controller and a supervisory term derived from the Lyapunov stability theorem. The control algorithm works on-line and does not require preliminary learning. Verification of the proposed control algorithm was realized on a WMR Pioneer-2DX.

Neural dynamic programming in reactive navigation of wheeled mobile robot

In the article a new approach to a reactive navigation of a wheeled mobile robot (WMR), using a neural dynamic programming algorithm (NPD), is presented. A proposed discrete hierarchical control system consists of a trajectory generator and a tracking control system. In the trajectory generator we used a sensor-based approach to path design for the WMR in an unknown 2-D environment with static obstacles. The main part of the navigator is an action dependant heuristic dynamic programming algorithm (ADHDP), that generates control signals used to design a collision-free trajectory, that makes reaching a goal possible. ADHDP is the discrete algorithm of actor-critic architecture, that works on-line and does not require a preliminary learning or a controlled system knowledge. The tracking control system realises the generated trajectory, it consists of dual-heuristic dynamic programming (DHP) structure, PD controller and the supervisory term derived from the Lyapunov stability theorem. Computer simulations have been conducted to illustrate the performance of the algorithm.

Discrete Action Dependant Heuristic Dynamic Programming in Control of a Wheeled Mobile Robot

In presented paper we propose a discrete tracking control algorithm for a two-wheeled mobile robot. The control algorithm consists of discrete Adaptive Critic Design (ACD) in Action Dependant Heuristic Dynamic Programming (ADHDP) configuration, PD controller and a supervisory term, derived from the Lyapunov stability theorem and based on the variable structure systems theory. Adaptive Critic Designs are a group of algorithms that use two independent structures for estimation of optimal value function from Bellman equation and estimation of optimal control law. ADHDP algorithm consists of Actor (ASE - Associate Search Element) that estimates the optimal control law and Critic (ACE - Adaptive Critic Element) that evaluates quality of control by estimation of the optimal value function from Bellman equation. Both structures are realized in a form of Neural Networks (NN). ADHDP algorithm does not require a plant model (the wheeled mobile robot (WMR) model) for ACE or ASE neural network weights update procedure (in contrast with other ACD configurations e.g. Heuristic Dynamic Programming or Dual Heuristic Programming that use the plant model). In presented control algorithm Actor-Critic structure is supported by PD controller and the supervisory term, that guarantee stable implementation of tracking in an initial adaptive critic neural networks learning phase, and robustness in a face of disturbances. Verification of proposed control algorithm was realized on the two-wheeled mobile robot Pioneer-2DX.

Heuristic Dynamic Programming in Wheeled Mobile Robot Control

Abstract In this paper a discrete algorithm for tracking control of a two-wheeled mobile robot is presented. The basis of the control algorithm is an Adaptive Critic Design in Heuristic Dynamic Programming (HDP) configuration. HDP is a model-based discrete reinforcement learning algorithm. In proposed control algorithm Actor - Critic structure is supplied by a PD controller and a supervisory element. The algorithm does not require preliminary learning, works on-line and uses a dynamics model of the mobile robot for a state prediction in Actor - Critic structure. The performance of control algorithm was tested by experiments on the mobile robot Pioneer-2DX.

Fuzzy Sensor-Based Navigation with Neural Tracking Control of the Wheeled Mobile Robot

Navigation of the wheeled mobile robot in the unknown environment with simultaneous realisation of the generated trajectory, is one of the most challenging and up to date problems in the modern mobile robotics. In the article a new approach is presented to a collision-free trajectory generating for a wheeled mobile robot, realised in a form of the hierarchical control system with two layers. The first layer is a tracking control system, where the Neuro-Dynamic Programming algorithm in the Dual Heuristic Dynamic Programming configuration was applied. The second layer is a trajectory generator where the Fuzzy Logic systems were used. The presented control system generates and realises trajectory of the wheeled mobile robot within the complex task of goal-seeking and obstacle avoiding. The proposed hierarchical control system works on-line, its performance was verified using the wheeled mobile robot Pioneer 2-DX.

Reinforcement Learning in Discrete Neural Control of the Underactuated System

The article presents a new approach to the problem of a discrete neural control of an underactuated system, using reinforcement learning method to an on-line adaptation of a neural network. The controlled system is of the ball and beam type, which is the nonlinear dynamical object with the number of control signals smaller than the number of degrees of freedom. The main part of the neural control system is the actor-critic structure, that comes under the Neural Dynamic Programming algorithms family, realised in the form of Dual Heuristic Dynamic Programming structure. The control system includes moreover the PD controller and the supervisory therm, derived from the Lyapunov stability theorem, that ensures stability. The proposed neural control system works on-line and does not require a preliminary learning. Computer simulations have been conducted to illustrate the performance of the control system.

Conventional and fuzzy force control in robotised machining

This article presents an application of the hybrid position-force control of the robotic manipulator with use of artificial neural networks and fuzzy logic systems in complex control system. The mathematical description of the manipulator and a closed-loop system are presented. In the position control were used the PD controller and artificial neural networks, which compensate nonlinearities of the manipulator. The paper presents mainly the application of various strategies of the force control. The force control strategies using conventional controllers P, PI, PD, PID and fuzzy controllers are presented and discussed. All of the control methods were verified on the real object in order to make a comparison of a control quality.

Discrete Globalised Dual Heuristic Dynamic Programming in Control of the Two-Wheeled Mobile Robot

Network-based control systems have been emerging technologies in the control of nonlinear systems over the past few years. This paper focuses on the implementation of the approximate dynamic programming algorithm in the network-based tracking control system of the two-wheeled mobile robot, Pioneer 2-DX. The proposed discrete tracking control system consists of the globalised dual heuristic dynamic programming algorithm, the PD controller, the supervisory term, and an additional control signal. The structure of the supervisory term derives from the stability analysis realised using the Lyapunov stability theorem. The globalised dual heuristic dynamic programming algorithm consists of two structures: the actor and the critic, realised in a form of neural networks. The actor generates the suboptimal control law, while the critic evaluates the realised control strategy by approximation of value function from the Bellman’s equation. The presented discrete tracking control system works online, the neural networks’ weights adaptation process is realised in every iteration step, and the neural networks preliminary learning procedure is not required. The performance of the proposed control system was verified by a series of computer simulations and experiments realised using the wheeled mobile robot Pioneer 2-DX.

Neural Network Identifier of a Four-wheeled Mobile Robot Subject to Wheel Slip

The paper presents a sequential neural network (NN) identification scheme for the four-wheeled mobile robot subject to wheel slip. The sequential identification scheme, different from conventional methods of optimization of a cost function, attempts to ensure stability of the overall system while the neural network learns the nonlinearities of the mobile robot. An on-line weight learning algorithm is developed to adjust the weights so that the identified model can adapt to variations of the characteristics and operating points in the four-wheeled mobile robot. The proposed identification system that can guarantee stability is derived from the Lyapunov stability theory. Computer simulations have been conducted to illustrate the performance of the proposed solution by a series of experiments on the emulator of the wheeled mobile robot.

Approximate Dynamic Programming in the Sensor-Based Navigation of the Wheeled Mobile Robot

The article presents a new approach to the sensor-based navigation of wheeled mobile robot Pioneer 2-DX in the unknown 2-D environment with static obstacles. The navigation task has been developed using a discrete hierarchical control system with a path planning layer and a tracking control layer designed using approximate dynamic programming algorithms. The navigator realises a behavioural control approach to the path planning process using the adaptive coordination of two simple behaviours: “goal-seeking” and “obstacle avoiding”. The main part of the navigator is the Action-Dependant Heuristic Dynamic Programming structure realised in a form of the actor and critic neural networks. To avoid the time consuming trial and error learning, additional proportional controllers generating signals that prompt the direction of the sub-optimal control law seeking process at the beginning of the NNs adaptation process are arranged in the navigator. The tracking control layer is composed of a PD controller, the Dual Heuristic Dynamic Programming algorithm and a supervisory term. It generates control signal for DC motors of the robot. The performance of the proposed discrete control system was verified by a series of experiments conducted using wheeled mobile robot Pioneer 2-DX equipped with one laser and eight ultrasonic range finders that provide object detection.