Marko Mitić

Neural network Reinforcement Learning for visual control of robot manipulators

It is known that most of the key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors. In this paper, the development and performance evaluation of a novel intelligent visual servo controller for a robot manipulator using neural network Reinforcement Learning is presented. By implementing machine learning techniques into the vision based control scheme, the robot is enabled to improve its performance online and to adapt to the changing conditions in the environment. Two different temporal difference algorithms (Q-learning and SARSA) coupled with neural networks are developed and tested through different visual control scenarios. A database of representative learning samples is employed so as to speed up the convergence of the neural network and real-time learning of robot behavior. Moreover, the visual servoing task is divided into two steps in order to ensure the visibility of the features: in the first step centering behavior of the robot is conducted using neural network Reinforcement Learning controller, while the second step involves switching control between the traditional Image Based Visual Servoing and the neural network Reinforcement Learning for enabling approaching behavior of the manipulator. The correction in robot motion is achieved with the definition of the areas of interest for the image features independently in both control steps. Various simulations are developed in order to present the robustness of the developed system regarding calibration error, modeling error, and image noise. In addition, a comparison with the traditional Image Based Visual Servoing is presented. Real world experiments on a robot manipulator with the low cost vision system demonstrate the effectiveness of the proposed approach.

Chaotic fruit fly optimization algorithm

Display Omitted Development of new method named chaotic fruit fly optimization algorithm (CFOA).Fruit fly algorithm (FOA) is integrated with ten different chaos maps.Novel algorithm is tested on ten different well known benchmark problems.CFOA is compared with FOA, FOA with Levy distribution, and similar chaotic methods.Experiments show superiority of CFOA in terms of obtained statistical results. Fruit fly optimization algorithm (FOA) is recently presented metaheuristic technique that is inspired by the behavior of fruit flies. This paper improves the standard FOA by introducing the novel parameter integrated with chaos. The performance of developed chaotic fruit fly algorithm (CFOA) is investigated in details on ten well known benchmark problems using fourteen different chaotic maps. Moreover, we performed comparison studies with basic FOA, FOA with Levy flight distribution, and other recently published chaotic algorithms. Statistical results on every optimization task indicate that the chaotic fruit fly algorithm (CFOA) has a very fast convergence rate. In addition, CFOA is compared with recently developed chaos enhanced algorithms such as chaotic bat algorithm, chaotic accelerated particle swarm optimization, chaotic firefly algorithm, chaotic artificial bee colony algorithm, and chaotic cuckoo search. Overall research findings show that FOA with Chebyshev map show superiority in terms of reliability of global optimality and algorithm success rate.

Bio-inspired approach to learning robot motion trajectories and visual control commands

We propose a robust bio-inspired learning control approach (BILCA) for mobile robots.Novel approach treats the robot trajectory learning and visual homing problems.First paper to integrate metaheuristic algorithm and trajectory learning problem in robots.First paper to integrate metaheuristic technique and visual homing strategy in robots.Various simulations and a real world experiment confirm applicability and usefulness of BILCA. In this paper, a novel bio-inspired learning control approach (BILCA) for mobile robots based on Learning from Demonstration (LfD), Firefly Algorithm (FA), and homography between current and target camera view is developed. BILCA consists of two steps: (i) first step in which the actuator commands are learned using FA and demonstrations of desired behavior, and (ii) second step in which the obtained wheel commands are evaluated through the real world experiment. Two different problems are considered in this study: trajectory reproduction, and generation of visual control commands for correction of robot orientation. Developed simulations are used to evaluate BILCA in the domain of learning actuator commands for reproduction of different complex trajectories. Results show that the bigger firefly swarms produce better results in terms of accuracy in the final mobile robot pose, and that the desired trajectory is reproduced with minimal error in final control iteration. Likewise, simulations prove that the FA outperforms other metaheuristic techniques. Experiment conducted on a real mobile robot in indoor environment unifies two considered problems within a single transportation task. Depending of the feature position in the image plane, the homography controller for forward motion or the BILCA based controller for robot orientation correction is employed. Experimental results show the applicability and effectiveness of the developed intelligent approach in real world conditions.

Integration of process planning and scheduling using chaotic particle swarm optimization algorithm

Chaotic PSO algorithm is proposed to solve NP-hard IPPS problem.Ten chaotic maps are implemented to avoid premature convergence to local optimum.Makespan, balanced level of machine utilization and mean flow time are observed.Five experimental studies show that cPSO outperforms GA, SA, and hybrid algorithm.Scheduling plans are tested by mobile robot within a laboratory environment. Process planning and scheduling are two of the most important manufacturing functions traditionally performed separately and sequentially. These functions being complementary and interrelated, their integration is essential for the optimal utilization of manufacturing resources. Such integration is also significant for improving the performance of the modern manufacturing system. A variety of alternative manufacturing resources (machine tools, cutting tools, tool access directions, etc.) causes integrated process planning and scheduling (IPPS) problem to be strongly NP-hard (non deterministic polynomial) in terms of combinatorial optimization. Therefore, an optimal solution for the problem is searched in a vast search space. In order to explore the search space comprehensively and avoid being trapped into local optima, this paper focuses on using the method based on the particle swarm optimization algorithm and chaos theory (cPSO). The initial solutions for the IPPS problem are presented in the form of the particles of cPSO algorithm. The particle encoding/decoding scheme is also proposed in this paper. Flexible process and scheduling plans are presented using AND/OR network and five flexibility types: machine, tool, tool access direction (TAD), process, and sequence flexibility. Optimal process plans are obtained by multi-objective optimization of production time and production cost. On the other hand, optimal scheduling plans are generated based on three objective functions: makespan, balanced level of machine utilization, and mean flow time. The proposed cPSO algorithm is implemented in Matlab environment and verified extensively using five experimental studies. The experimental results show that the proposed algorithm outperforms genetic algorithm (GA), simulated annealing (SA) based approach, and hybrid algorithm. Moreover, the scheduling plans obtained by the proposed methodology are additionally tested by Khepera II mobile robot using a laboratory model of manufacturing environment.

Neural network learning from demonstration and epipolar geometry for visual control of a nonholonomic mobile robot

The control of a robot system using camera information is a challenging task regarding unpredictable conditions, such as feature point mismatch and changing scene illumination. This paper presents a solution for the visual control of a nonholonomic mobile robot in demanding real world circumstances based on machine learning techniques. A novel intelligent approach for mobile robots using neural networks (NNs), learning from demonstration (LfD) framework, and epipolar geometry between two views is proposed and evaluated in a series of experiments. A direct mapping from the image space to the actuator command is conducted using two phases. In an offline phase, NN–LfD approach is employed in order to relate the feature position in the image plane with the angular velocity for lateral motion correction. An online phase refers to a switching vision based scheme between the epipole based linear velocity controller and NN–LfD based angular velocity controller, which selection depends on the feature distance from the pre-defined interest area in the image. In total, 18 architectures and 6 learning algorithms are tested in order to find optimal solution for robot control. The best training outcomes for each learning algorithms are then employed in real time so as to discover optimal NN configuration for robot orientation correction. Experiments conducted on a nonholonomic mobile robot in a structured indoor environment confirm an excellent performance with respect to the system robustness and positioning accuracy in the desired location.

Trajectory learning and reproduction for differential drive mobile robots based on GMM/HMM and dynamic time warping using learning from demonstration framework

In this paper, we present new Learning from Demonstration-based algorithm that generalizes and extracts relevant features of desired motion trajectories for differential drive mobile robots. The algorithm is tested through series of simulations and real world experiments in which desired task is demonstrated by the human teacher while teleoperating the mobile robot in the working environment. In the first step of the developed method, Gaussian Mixture Model (GMM) is built for incremental motions of the mobile robot between two consecutive poses. After this, the hidden Markov model is used to capture transitions between states (temporal variations of the data between clusters) which are missing from static GMM representation. Generalization of the motion is achieved by using the concept of keyframes, defined as points in which significant changes between GMM/HMM states occur. In the second step, the resulting GMM/HMM representation is used to generate optimal state sequences for each demonstration and to temporally align them, using 1D dynamic time warping, with respect to the one most consistent with the GMM/HMM model. This phase implies extraction of keyframes along all state sequences and projecting them into control space, in which controls are aligned in time as well. Finally, the generalized controls are obtained by averaging over all controls at the keyframes; simple piecewise cubic spline method is used for interpolation between generated control values. The main advantage of the developed algorithm is its ability to learn and generalize from all demonstrated examples which results in high quality reproductions of the motion. The proposed approach is verified both in simulated environment and using real mobile robot.

Neural networks for prediction of robot failures

It is known that the supervision and learning of robotic executions is not a trivial problem. Nowadays, robots must be able to tolerate and predict internal failures in order to successfully continue performing their tasks. This study presents a novel approach for prediction of robot execution failures based on neural networks. Real data consisting of robot forces and torques recorded immediately after the system failure are used for the neural network training. The multilayer feedforward neural networks are employed in order to find optimal solution for the failure prediction problem. In total, 7 learning algorithms and 24 neural architectures are implemented in two environments – Matlab and specially designed software titled BPnet. The results show that the neural networks can successfully be applied for the problem in hand with prediction rate of 95.4545%, despite having the erroneous or otherwise incomplete sensor measurements invoked in the dataset. Additionally, the real-world experiments are conducted on a mobile robot for obstacle detection and trajectory tracking problems in order to prove the robustness of the proposed prediction approach. In over 96% for the detection problem and 99% for the tracking experiments, neural network successfully predicted the failed information, which evidences the usefulness and the applicability of the developed intelligent method.

Neural extended Kalman filter for monocular SLAM in indoor environment

The extended Kalman filter (EKF) has become a popular solution for the simultaneous localization and mapping (SLAM). This paper presents the implementation of the EKF coupled with a feedforward neural network for the monocular SLAM. The neural extended Kalman filter (NEKF) is applied online to approximate an error between the motion model of the mobile robot and the real system performance. Inadequate modeling of the robot motion can jeopardize the quality of estimation. The paper shows integration of EKF with feedforward neural network and simulation analysis of its consistency and implementation of the NEKF with a mobile robot, laboratory experimental environment, and a simple USB camera. The simulation and experimental results show that integration of neural network into EKF prediction–correction cycle results in improved consistency and accuracy.

The Ant Lion Optimization Algorithm for Integrated Process Planning and Scheduling

Process planning and scheduling are two of the most important manufacturing functions which are usually performed sequentially in traditional approaches. Considering the fact that these functions are usually complementary, it is necessary to integrate them so as to improve performance of a manufacturing system. This paper presents implementation of novel nature-inspired Ant Lion Optimization (ALO) algorithm for solving this combinatorial optimization problem effectively. As the ALO algorithm mimics the intelligent behavior of antlions in hunting ants, the main steps of hunting prey, its mathematical modeling, and optimization procedure for integration of process planning and scheduling is proposed. The algorithm is implemented in Matlab environment and run on the 3.10 GHz processor with 2 GBs of RAM memory. Experimental results show applicability of the proposed approach in solving integrated process planning and scheduling problem.

Chaotic metaheuristic algorithms for learning and reproduction of robot motion trajectories

Most of today’s mobile robots operate in controlled environments prone to various unpredictable conditions. Programming or reprogramming of such systems is time-consuming and requires significant efforts by number of experts. One of the solutions to this problem is to enable the robot to learn from human teacher through demonstrations or observations. This paper presents novel approach that integrates Learning from Demonstrations methodology and chaotic bioinspired optimization algorithms for reproduction of desired motion trajectories. Demonstrations of the different trajectories to reproduce are gathered by human teacher while teleoperating the mobile robot in working environment. The learning (optimization) goal is to produce such sequence of mobile robot actuator commands that generate minimal error in the final robot pose. Four different chaotic methods are implemented, namely chaotic Bat Algorithm, chaotic Firefly Algorithm, chaotic Accelerated Particle Swarm Optimization and newly developed chaotic Grey Wolf Optimizer (CGWO). In order to determine the best map for CGWO, this algorithm is tested on ten benchmark problems using ten well-known chaotic maps. Simulations compare aforementioned algorithms in reproduction of two complex motion trajectories with different length and shape. Moreover, these tests include variation of population in swarm and demonstration examples. Real-world experiment on a nonholonomic mobile robot in indoor environment proves the applicability of the proposed approach.