John Nassour

Multi-layered multi-pattern CPG for adaptive locomotion of humanoid robots

In this paper, we present an extended mathematical model of the central pattern generator (CPG) in the spinal cord. The proposed CPG model is used as the underlying low-level controller of a humanoid robot to generate various walking patterns. Such biological mechanisms have been demonstrated to be robust in locomotion of animal. Our model is supported by two neurophysiological studies. The first study identified a neural circuitry consisting of a two-layered CPG, in which pattern formation and rhythm generation are produced at different levels. The second study focused on a specific neural model that can generate different patterns, including oscillation. This neural model was employed in the pattern generation layer of our CPG, which enables it to produce different motion patterns—rhythmic as well as non-rhythmic motions. Due to the pattern-formation layer, the CPG is able to produce behaviors related to the dominating rhythm (extension/flexion) and rhythm deletion without rhythm resetting. The proposed multi-layered multi-pattern CPG model (MLMP-CPG) has been deployed in a 3D humanoid robot (NAO) while it performs locomotion tasks. The effectiveness of our model is demonstrated in simulations and through experimental results.

Qualitative Adaptive Reward Learning With Success Failure Maps: Applied to Humanoid Robot Walking

In the human brain, rewards are encoded in a flexible and adaptive way after each novel stimulus. Neurons of the orbitofrontal cortex are the key reward structure of the brain. Neurobiological studies show that the anterior cingulate cortex of the brain is primarily responsible for avoiding repeated mistakes. According to vigilance threshold, which denotes the tolerance to risks, we can differentiate between a learning mechanism that takes risks and one that averts risks. The tolerance to risk plays an important role in such a learning mechanism. Results have shown the differences in learning capacity between risk-taking and risk-avert behaviors. These neurological properties provide promising inspirations for robot learning based on rewards. In this paper, we propose a learning mechanism that is able to learn from negative and positive feedback with reward coding adaptively. It is composed of two phases: evaluation and decision making. In the evaluation phase, we use a Kohonen self-organizing map technique to represent success and failure. Decision making is based on an early warning mechanism that enables avoiding repeating past mistakes. The behavior to risk is modulated in order to gain experiences for success and for failure. Success map is learned with adaptive reward that qualifies the learned task in order to optimize the efficiency. Our approach is presented with an implementation on the NAO humanoid robot, controlled by a bioinspired neural controller based on a central pattern generator. The learning system adapts the oscillation frequency and the motor neuron gain in pitch and roll in order to walk on flat and sloped terrain, and to switch between them.

Experience-based learning mechanism for neural controller adaptation: Application to walking biped robots

Neurobiology studies showed that the role of the anterior cingulate cortex of the brain is primarily responsible for avoiding repeated mistakes. According to vigilance threshold, which denotes the tolerance to risks, we can differentiate between a learning mechanism that takes risks, and one that averts risks. The tolerance to risk plays an important role in such learning mechanism. Results have shown the differences in learning capacity between risk-taking and risk avert behaviors. In this paper, we propose a learning mechanism that is able to learn from negative and positive feedback. It is composed of two phases, evaluation and decision-making phase. In the evaluation phase, we use a Kohonen Self Organizing Map technique to represent success and failure. Decision-making is based on an early warning mechanism that enables to avoid repeating past mistakes. Our approach is presented with an implementation on a simulated planar biped robot, controlled by a reflexive low-level neural controller. The learning system adapts the dynamics and range of a hip sensor neuron of the controller in order for the robot to walk on flat or sloped terrain. Results show that success and failure maps can learn better with a threshold that is more tolerant to risk. This gives rise to robustness to the controller even in the presence of slope variations.

Learning diverse motor patterns with a single multi-layered multi-pattern CPG for a humanoid robot

This paper presents a Multi-Layered Multi-Pattern Central Pattern Generator (CPG) that provides humanoid robots the ability to generate motor patterns in order to perform various upper body tasks (like: reaching and writing). This CPG has two control levels: 1) one for pattern formation (coordination); and 2) another for pattern generation (selection). A unique feature of this CPG is its ability to generate oscillatory, semi-oscillatory, and non-periodic patterns locally, simply through descending control. With a simple learning method the NAO humanoid robot was able to learn how to coordinate motor patterns at different joints in writing numbers from 0 to 9. With a neural-based structure, which separate between the coordination and the selection control levels, our approach is shown to be robust during the execution even with a noisy proprioception (sensory) feedback and also with noisy coordination (pattern formation descending control) signals.

Extending cortical-basal inspired reinforcement learning model with success-failure experience

Neurocognitive studies showed that neurons of the orbitofrontal cortex get activated for expectation of immediate reward. Therefore they are the key reward structure in the brain. It was also shown that neurons in the anterior cingulate cortex work as an early warning system that prevents repeating mistakes. This paper introduces an extended model of reinforcement learning in the cortex-basal ganglia network by the hypothetical involvement of two cortical regions, the orbitofrontal cortex and the anterior cingulate cortex. In order to prove the effectiveness of the approach, we propose an enhanced actor-critic method that is guided by experiences of success and failure. Failures help the agent to explore regions by avoiding past mistakes. Successful experiences allow to exploit those regions that guarantee the agent to reach its goal. First, the method was applied to a 2-D grid problem, where an agent had to reach its goal by avoiding obstacles in its path. Second, the proposed RL model was used to optimize the learning policy of how to play bowling by the NAO humanoid robot. The results showed significant improvement using the enhanced actor-critic method both in terms of performance and rate of learning compared with the standard actor-critic method.

Gait Transition Between Simple and Complex Locomotion in Humanoid Robots

In this paper, we present the gait transition between rhythmic and non-rhythmic behaviors during walking of a humanoid robot Nao. In biological studies, two kinds of locomotion were observed in cat during walking on a flat terrain and on a ladder (simple and complex walking). Both behaviors were obtained on the robot thanks to the multi-layers multi-patterns central pattern generator model. We generate the rhythmic behavior from the non-rhythmic one based on the frequency of interaction between the robot feet and the ground surface during the complex locomotion. Although the complex locomotion requires a sequence of descending control signals to drive each robot step, the simple one requires only a triggering signal to generate the periodic movement. The overall system behavior fits with the biological findings in cat locomotion.

Minimized-Torque-Oriented Design of Parallel Modular Mechanism for Humanoid Waist

This article focuses on the design and integration of a parallel modular mechanism inside the waist of a human-sized biped robot to enable tilting motion of the torso. The mechanism for each tier is adapted from the parallel 2-degree-of-freedom tilting part of an existing 3-rotation flight simulator structure. The main contribution of this work is the design of a minimized-torque-oriented optimization process that takes into account the upper mass load to be supported by the mechanism, the constrained volume of the waist, a minimal dexterity threshold, and the tilting range required. The design process aims to determine the relative size and position of the different parts of the mechanism. The objective consists of minimizing the actuator average torque over the entire tilt range, and to evaluate how much torque reduction this parallel mechanism can bring compared with the use of a serial mechanism. Up to three modules can be stacked inside the waist to limit the actuator torques and to reach the required tilting range for sitting and bending movements.

Tactile and Proximity Servoing by a Multi-modal Sensory Soft Hand

We present the manufacturing and the implementation of a multi-modal sensory soft hand for the interaction with conductive and non-conductive objects. The hand sensors were mounted on two fabric layers with three sensory modularities: touch, proximity, and curvature. Servoing behavior is generated based on the estimation of the center of touch (force sensitive resistor) and the center of proximity (proximity sensitive capacitor). Results are presented on human subjects wear the hand, and a set of vibration motors that work as haptic feedback for the center of stimulation. Driven by vibration, the system guides the subject to explore conductive objects. Servoing behavior is generated based on the estimation of the center of stimulations without visual feedback.

Design of Soft Exosuit for Elbow Assistance Using Butyl Rubber Tubes and Textile

Soft materials show numerous advantages compared to rigid ones in exosuit devices. We present the design of a soft wearable elbow assistance device with flexion and extension actuations. Commercially available butyl rubber tubes have been used as a pneumatic actuator. Tubes are enveloped by a lightweight polyester fabric to eliminate a non-homogeneous expansion. The surrounding fabric in turn is mounted on a clothes fabric as zigzag paths. The exosuit is lightweight, shock resistant, simple to manufacture, and low cost. The subjective experiments show a reduction average of 48% in the Rectified & integrated raw electromyography signal of the brachialis muscle during a rhythmic flexion/extension sequence while lifting weights (3 and 5 [kg]). Results indicate a significant assistance with respect to the other existing soft elbow exosuits.