Neuromorphic adaptive spiking CPG towards bio-inspired locomotion of legged robots
Pablo Lopez-Osorio, Alberto Patino-Saucedo, Juan P. Dominguez-Morales, Horacio Rostro-Gonzalez, Fernando Perez-Peña
NNeuromorphic adaptive spiking CPG towardsbio-inspired locomotion of legged robots
Pablo Lopez-Osorio a , Alberto Patino-Saucedo b ,Juan P. Dominguez-Morales c , Horacio Rostro-Gonzalez b ,Fernando Perez-Pe˜na a a School of Engineering, Universidad de C´adiz, Spain. b Department of Electronics, DICIS-University of Guanajuato, Mexico c Robotics and Technology of Computers Lab. Universidad de Sevilla, Spain.
Abstract
In recent years, locomotion mechanisms exhibited by vertebrate animalshave been the inspiration for the improvement in the performance of roboticsystems. These mechanisms include the adaptability of their locomotion toany change registered in the environment through their biological sensors.In this regard, we aim to replicate such kind of adaptability in legged robotsthrough a Spiking Central Pattern Generator. This Spiking Central Pat-tern Generator generates different locomotion (rhythmic) patterns which aredriven by an external stimulus, that is, the output of a Force Sensitive Resis-tor connected to the robot to provide feedback. The Spiking Central PatternGenerator consists of a network of five populations of Leaky Integrate-and-Fire neurons designed with a specific topology in such a way that the rhyth-mic patterns can be generated and driven by the aforementioned externalstimulus. Therefore, the locomotion of the end robotic platform (any-leggedrobot) can be adapted to the terrain by using any sensor as input. The Spik-ing Central Pattern Generator with adaptive learning has been numericallyvalidated at software and hardware level, using the Brian 2 simulator and theSpiNNaker neuromorphic platform for the latest. In particular, our experi-ments clearly show an adaptation in the oscillation frequencies between thespikes produced in the populations of the Spiking Central Pattern Generatorwhile the input stimulus varies. To validate the robustness and adaptability
Email addresses: [email protected] (Pablo Lopez-Osorio), [email protected] (Fernando Perez-Pe˜na)
Preprint submitted to Neural Networks January 26, 2021 a r X i v : . [ c s . N E ] J a n f the Spiking Central Pattern Generator, we have performed several testsby variating the output of the sensor. These experiments were carried outin Brian 2 and SpiNNaker; both implementations showed a similar behaviorwith a Pearson correlation coefficient of 0.905. Keywords:
Neurorobotics, SpiNNaker, Central Pattern Generator, SpikingNeural Network, Neuromorphic Hardware, Adaptive-learning
1. Introduction
It is well known that, in biology, rhythmic locomotion is produced by aneural structure called Central Pattern Generator (CPG) [1]. This structureis located at the spinal cord and it usually comprises of two neural pop-ulations which produce an alternating output of spikes. Eventually, theseoutput spikes are used to activate the muscles fibers.This approach of using CPGs can be borrowed to create locomotion inrobotics. There are several possibilities to implement a CPG: using coupled-oscillators, using Artificial Neural Networks (ANNs) or using Spiking NeuralNetworks (SNNs). The closest biological implementation is to use an SNN.These networks are based on neuron models and synaptic connections thatimplement biological features. The field of research called neuromorphic en-gineering aims to implement these networks on electronics, mimicking theway living beings have solved complex problems by using both analog anddigital circuits. The neuromorphic robotics field puts together both the neu-romorphic engineering and the roboticists communities [2]. The use of neuralstructures made of spiking neurons, coming from the neuromorphic engineer-ing field, within robotics results in the need for less resources, less powerconsumption and a simplification of the algorithms [3].One of this neural structures is the CPG. This structure generates arhythmic pattern at its output, which can be used within robotics to generatelocomotion. Thus, a CPG creates gaits that are suitable to use within arobotic platform. These structures can generate a very stable pattern evenwithout sensory information or brain activity [4].As briefly shown, there is a growing community of researchers that areexploring the possibility of using a CPG to create locomotion in robots [1].Most of the previous works in the literature present an open-loop CPG whichdoes not include any sensory information: in [5], the actuation of a lamprey-like robot is done by using an open-loop CPG and neuromorphic hardware.2hen, in [6] and [7], the authors proposed a CPG implemented on an FieldProgrammable Gate Array (FPGA) and the SpiNNaker platform [8]; thesethree works do not offer the possibility of changing the originally producedpattern in real time. However, they showed that implementing CPGs usingspiking neurons uses less power. There is a more recent paper which allowsboth real time functioning and pattern variation but without including anysensory information [9]. In [10], the open-loop CPG is implemented on Loihi[11] using an astrocytic network, producing two different gaits with 24 motorneurons. However, the most recent work [12], proposes the implementation ofa CPG with the possibility of changing the amplitude, frequency and phaseonline without any sensory input required. The authors also pointed outthat the architecture should include sensory feedback to modify the behaviorof the CPG. That is the objective presented in this paper. We propose anSNN that, using the sensory information, adapts the behavior of the CPG.Regarding works that include sensory information, in [13], the CPG is builtusing coupled-oscillators instead of spiking neurons and the feedback to theCPG is included into the control loop of the equations of the CPG. In[14], 12 simulated neurons modulated by sensory feedback are used to buildthe CPG. Instead of what we propose, an adaptation SNN, they achievedifferent gaits by either moving the location or increasing the number ofneural structures. The neuron model used is Izhikevich instead of the LeakyIntegrate-and-Fire (LIF) model proposed in this work to reduce the compu-tational resources used. The sensory information is used to adapt the timeduration of each phase of the frequency switching to enable the actuators toreach the commanded position. Finally, in [15], a neuromorphic sensor hasbeen used to select which predefined gait of the CPG should be activated.In most of these works, the use of an external input to the CPG changes theperformance of it. This performance can be defined as a neuromodulation.It has been shown that this modulation is essential to alter the behavior ofa neural structure by modifying the synaptic connections [16].Finally, there are works where the main focus is on the learning processof the robotic platform: in [17], the authors proposed a rewarding-learningprocess to teach a hexapod robot how to walk without any previous knowl-edge. A couple of sensors (a standard camera and a gyroscope) are used toprovide the rewarding signal to the neural network based on a CPG. Al-though they used a digital version model neuron of the LIF, they do not useneuromorphic hardware to implemented the neural architecture; a RaspberryPi is used instead. A similar approach based on reinforcement learning, but3ithout using spiking neurons, was proposed to improve the locomotion ofthe NAO robot [18]. Another approach is used in [19], where the authorshave two hexapod robots: an expert and a student. The student learns orimitates the gait of the expert by using a one-layer feedforward SNN and aDynamic Vision Sensor (DVS) camera as input. However, the possibility forthe robot to adapt to its environment is not implemented.The objective of this paper is to design and deploy a spiking architecturethat makes the interaction of a spiking CPG with its environment possible.An external agent. i.e. a Force Sensitive Resistor (FSR), is introduced as thefeedback stimulus to the network. This agent can modify the gait generatedby the CPG. Therefore, the locomotion of the end robotic platform (anylegged robot) can be adapted to the terrain. Furthermore, the spiking net-work presented in this paper allows the introduction of the feedback sensoryinformation on the loop of the CPG to provide adaptation. This adaptationSNN could be used with any sensor as stimulus.The rest of the paper is structured as follows: section 2.1 introduces thematerials used in this work, including the simulator and hardware used. Theimplemented methods are described in 2.2, together with the SNN model.Then, the results obtained are presented and discussed in section 3. Thissection is divided into two different subsections: first, the experiments runusing the software simulator and then, the same experiments run on on thehardware platform. Finally, the conclussions are presented.
2. Materials and Methods
This section describes both the software and hardware used to performthe experiments.
Brian 2 [20] is a neural simulator for SNNs written in Python program-ming language. Thus, it is a cross-platform which is available in differentoperating systems. In contrast to other SNN simulators such as NEURON[21] or PyNN [22], Brian 2 is highly flexible and it is easily adaptable withnew non-standard neuron models and synapses. Brian 2 can be used tomodel and simulate complex problems faced by neuroscientists, as well asgiving faster and more robust results before implementing the solution on ahardware platform. 4 .1.2. SpiNNaker
SpiNNaker [23, 8, 24] is a massively parallel, multi-core computing systemdesigned by the Advanced Processor Technologies (APT) Research Groupfrom the University of Manchester. It was designed under the Human BrainProject (HBP) [25] for simulating parts of the brain by using SNNs. SpiN-Naker machines consist of SpiNNaker chips, which have eighteen 200-HzARM968 processor cores each [26]. This allows an asynchronous commu-nication infrastructure for sending short packages (each of them representinga particular neuron firing) [27] identified using Address Event Representa-tion (AER) [28]. Different SpiNNaker machines were built and commercial-ized, including SpiNN-3 and SpiNN-5, with 4 and 48 SpiNNaker chips each,respectively. They also include the spinnlinks [29], which allow real-time in-put/output interfacing with neuromorphic sensors and other neuromorphicplatforms such as FPGAs [30, 31, 32]. A PyNN-based [22] software pack-age called sPyNNaker [33] can be used to design and implement SNNs onthese machines. The recently built million-core machine is at the School ofComputer Science at the University of Manchester that can be used throughthe HBP portal. In this work, a SpiNN-5 machine was used to run thesimulations proposed.
We simulated our Spiking Central Pattern Generator (sCPG) model on astandard computer using the Brian 2 Simulator to characterize the networkdynamics, to analyse the number of neurons per population needed and toadjust the network parameters. The simulation results guided the subsequentneuromorphic implementation using the SpiNNaker platform.
The LIF model is used to implement the neuron on both the softwaresimulator and the SpiNNaker hardware platform [34]. The model is definedwithin the equations (1) and (2). τ m dVdt = − ( V − V r ) + RI ( t ) (1) if V ( t ) = V th then lim δ → δ> V ( t + δ ) = V r (2)where V is the membrane potential of the neuron, R represents the resistanceof the membrane, τ m the time constant of the neuron, V r the resting potential, V th the threshold and I ( t ) is the stimulus.5 .2.2. Network Model The SNN model depicted in Figure 1 was designed based on the neuronmodel presented in section 2.2.1. The main objective of the proposed model isto generate a constant oscillation between the spikes produced in ensembles Aand B, whose frequency varies depending on the value read from the FSRsensor. Thus, the proposed CPG is able to automatically adapt its behaviordepending on the input stimulus.The SNN consists of different parts which are described next. It is im-portant to note that each of the ensembles (also called populations) have thesame number of neurons. The number of neurons in each population was setbased on different experiments, which are presented in section 3.
FSR
A B21 Ref
Input stimulus Id Ensemble of neuronsInhibitory projections from X to Y
Excitatory projections from X to Y
X YX Y . n A . n A . n A . n A Figure 1: Diagram of the proposed spiking neural network architecture.
The main block of the architecture is the so-called
CP G AB , which consistsof the populations A and B represented in Figure 1. These populationsare self-excited and self-inhibited with a probability of 25% and 75% andweights of 4 nA and 1.5 nA, respectively. Moreover, a 75% probability ofhaving inhibitory synapses between neurons from the two aforementionedpopulations is also present. Thus, when one of the populations is producingspikes, the opposite is inhibited and, thus, generating the desired oscillationpattern. Populations A and B are injected with a constant external current inorder to start generating the oscillation. Therefore, for these two populationsequation 3 is used instead of equation 1.6 Vdt = V r − V + R ( Iexc − Iinh + Ist ) τ m (3)Where I st is the current injected to neurons in populations A and B. Thisvalue was set to 2.2 nA, which is sufficient for producing the desired rhythmicpattern.Furthermore, populations 1 and 2 ( CP G ) both have the same numberof neurons as those in CP G AB , and are also interconnected in the same way.The projections between CP G and CP G AB are depicted in Figure 1, andfollow the same aforementioned probabilities (25% for excitatory and 75%for inhibitory projections). The weights between the different populations ofthe proposed model are specified in the same figure.Finally, the reference population ( Ref ) was implemented as a Poissondistribution with variable frequency. In contrast to the populations from
CP G AB and CP G , the number of neurons in Ref was set to 50. Theseneurons are connected to populations 1 and 2 following the scheme presentedin Figure 1. This number of neurons was set to 50 since it was the opti-mum for producing biologically-plausible spiking rates in the population,with maximum and minimum frequencies close to the biological counterpart[35]. The spiking rate of the Poisson distribution depends on the values ob-tained from the FSR sensor used as input to population
Ref . This sensorshould be placed at the end of the leg of the robot. This sensor providesvalues between 0 and 5 V. Since the spinal cord ventral horn motor neuronalpha, which is the biological neuron taken as reference, has a spike ratebetween 10 and 171 Hz [35], a linear regression was established in order tomatch the frequency of the Poisson distribution with the voltage value readfrom the sensor. Equation 4, where V s is the voltage value provided by thesensor, shows this relation to provide the desired rates. f Ref = 280 V s − CP G AB , CP G and Ref ) and the con-nections between them, the proposed behavior explained at the begining ofthis section was achieved. Therefore, following Fig. 1, different scenarios canbe analyzed. In the case where the spike rate of population
Ref is greaterthan the oscillation frequency of
CP G AB , population 1 will be inhibited andpopulation 2 will be excited. Since population 1 is excited from A, but the7 arameter Value u reset -55.0 mV u rest -55.0 mV u th τ m τ syn e τ syn i c m t I bias Table 1: Neuron parameters for the proposed CPG in both the Brian 2 simulator andthe SpiNNaker hardware platform. number of spikes is lower than the ones that are inhibiting the same popula-tion from
Ref , population 1 will have very low activity. The opposite occursin population 2, which will be inhibited from B but excited from
Ref in astronger way and, thus, having considerable more activity than population 1.The activity from population 2 will excite
CP G AB , increasing its oscillationfrequency. Conversely, the opposite happens when the spike rate of Ref islower than the oscillation frequency of
CP G AB , where population 1 will havemore activity than 2 and, therefore, inhibit CP G AB in order to reduce itsfrequency. As a result, the proposed network is able to adapt the frequencyof the CPG based on an input stimulus. This model was simulated in Brian 2and emulated using SpiNNaker, and the results can be seen in section 3.
3. Results and Discussion
The first experiment was performed to determine the number of neuronsper population of the CPG that was needed to achieve a stable rate valuealong the simulations. The parameters used for all the experiments are shownin Table 1. Figure 2 shows the maximum, minimun and mean values obtainedfor the rate of the simulations performed. A thousand simulations per numberof neurons were run. As the number of neurons increased, the rate achievedis more stable and the standard deviation becomes lower. Starting from 40neurons per population in the
CP G AB , the standard deviation is less than 1.58 R a t e ( H z ) Figure 2: Average, maximum and minimum rate values obtained for each number ofneurons per population. The trace shows the mean, and, its shadow, the maximum andminimun rate of all the one thousand simulations performed. and the bahavior is more stable. A hundred neurons per population shows themost stable output rate with the least deviation. Once the number of neuronsper population was fixed, to verify if the architecture presented in Section 2.2behaved as expected, the operation of the
CP G AB in isolation was analyzed.This experiment ensured that it was able to produce a constant oscillation.Then, the operation of the same CPG was examined once interconnected withthe CP G , performing tests with different stimuli to analyze the resultsobtained. Finally, the entire architecture was connected and analyzed indifferent scenarios based on the external input from the FSR sensor. CP G AB analysis The topology of the
CP G AB can be seen in Figure 1, where green arrowsdenote excitatory connections and red arrows denote inhibitory connections.As mentioned in section 2.2.2, I St is a constant current injected to all neuronsin populations A and B, with a fixed value of 2.2 nA.This current is the minimum value required to produce the oscillatorypattern in the CPG. While the proposed topology can work with highervalues of I St , this generates a higher frequency of oscillation and a noticeableincrease in the noise introduced in the simulation (see figures 3 and 4). Thus,this value was set to 2.2 nA in order to be able to more easily appreciate the9 igure 3: Simulation of the CP G AB with a I St value of 10 nA. Increased oscillationfrequency may be observed along with an increase in the amount of noise when comparedto Figure 4.Figure 4: Simulation of the CP G AB with a I St value of 2.2 nA. A lower oscillationfrequency and almost total noise removal can be observed when compared to Figure 3. effect of feedback on the CPG. The results of the simulation performed areshown in Figure 4, where a frequency of, approximately, 5.8 Hz was obtainedfor each population, while the total frequency of the CPG was 11.6 Hz.10 igure 5: Results of the simulation of the proposed SNN model when having a 5 V inputfrom the FSR sensor for 1000 ms between t = 1000 ms and t = 2000 ms and between t = 3000 ms and t = 4000 ms (generating a Poisson distribution of 171 Hz in Ref ). Forthe rest of the simulation time, the sensor’s output was set to 0 V.
In order to study the robustness of the network against sudden changesin the oscillation frequency, an experiment where the values read from thesensor were alternating between maximum and minimum voltage peaks wasperformed. Initially, a 5 V input was simulated in 1 s and 2 s, both with aduration of 1 s. This made neurons in population
Ref to fire at a frequencyof, approximately, 171 Hz during this period. Before 1 s, between 2 s and3 s, and after 4 s the sensor readings corresponded to 0 V. Figure 5 showsthe results of this simulation. It can be observed that, at time zero, since noinformation was being received from the sensor,
Ref had no activity. There-fore, population 1 was excited by
CP G AB and population 2 was inhibited.In turn, population 1 slightly inhibited CP G AB . At 1 s, Ref started firingat a frequency of approximately 171 Hz, exciting population 2 and inhibitingpopulation 1. During this period, the former excited
CP G AB , increasing itsoscillation frequency. After 2 s, population 1 started dominating population 2again, slightly inhibiting CP G AB again. Exactly the same behavior can beseen again starting at 3 s. In this figure it can be observed how the frequencyof CP G AB increased considerably between 1 s and 2 s and between 3 s and11 igure 6: Results obtained when simulating random voltage readings from the FSR sensor. Ref were (171,40, 80, 30, 5, 130, 50, 76, 20, 150) Hz. In Figure 6 it can be seen how
CP G AB adapts its oscillation frequency based on the inputs stimuli, obtaining maxi-mum and minimum frequency peaks of 15 Hz and 8 Hz, respectively.After this, a constant increase in the values of the readings from thesensor was simulated. In particular, increases in steps of 20 Hz per 500 mswere introduced in the frequency of Ref . To check the performance limitsof the
CP G AB , the last injected frequency value exceeded up to 17% of themaximum theoretical value of 171 Hz. Figure 7 shows the results of thisexperiment.As can be seen in the figure, although there is a significant increase inthe amount of noise, the oscillation frequency of CP G AB does not increase,achieving a maximum frequency of 15 Hz.12 igure 7: Results obtained when simulating a continuous increase in the frequency of Ref . The proposed sCPG model was tested in the SpiNNaker neuromorphichardware. The neuronal model is defined in sPyNNaker [33], a PyNN-basedsoftware interface that allows a quick prototyping and implementation ofspiking neural networks in the SpiNNaker platform.The spiking neuron model is the LIF with fixed threshold and decaying-exponential post-synaptic current, whose parameters are given in table 1.These parameters were chosen so as to emulate the neuron dynamics of thesimulations in Brian 2.In order to show that the model achieves a good performance in SpiN-Naker, we performed three tests: first, by verifying that the constant os-cillations of the
CP G AB were observed and matched the rates presented inBrian 2. Then, by implementing the whole sCPG with increasing rates ofthe input sensor represented by the reference population. Finally, by testingthe sCPG under random stimuli. CP G AB implementation For the implementation of the CPG in Spinnaker (see Figure 8) 100 neu-rons were used, each with a constant current I St equal to 2.2 nA, like in theequivalent Brian experiment. The measured rate of the oscillatory patternof populations A and B was 11.62 Hz, which matches the rate measured in13 igure 8: SpiNNaker simulation of the CP G AB .Figure 9: SpiNNaker simulation of the CP G for extreme values of the stimulus spikingrate (0.1 Hz and 171 Hz). igure 10: SpiNNaker simulation of the CP G for ten increasing values of the stimulusrate (from 0 Hz to 50 Hz). the Brian simulation. Although some spikes were lost in the raster plot com-pared to Brian, which can be attributed to limited support of floating-pointcalculations in sPyNNaker, the pattern appears consistent and with littlenoise.
CP G under different stimuli
The SpiNNaker implementation of the full CPG, including the feedbacknetwork was tested under different conditions of the stimulus. First, wesimulated a sudden change of the value of the sensor, represented by the rateof the Poisson generator in the
Ref population (see figure 9). This rate wasset to 0.1 Hz for the first 2 seconds of the simulation and oscillated between171 Hz and 0.1 Hz for the next four seconds, in order to appreciate bothregimes of the CPG. It can be seen how, with a low
P ref frequency, the firstfeedback population dominates and the rate of the output oscillatory is low,at around 12.5 Hz. With a high
P ref frequency, it is the second feedbackpopulation which dominates and the measured oscillatory pattern displays apeak frequency of 27.5 Hz. 15 igure 11: SpiNNaker simulation of the
CP G for ten random values of the stimulus rate(from 0 Hz to 50 Hz).
Figures 10 and 11 show the spiking response of the SpiNNker CPG toincreasing rates of
Ref and to random values of
Ref , respectively. The tworegimes can be clearly observed in the spiking response of populations 1 and 2and in the measured oscillatory rates of Populations A and B, proving thatthe feedback mechanism works correctly.
Figure 12 shows the comparison made between both approaches: thesimulations on Brian 2 and the implementation on the SpiNNaker platform.In this experiment, the same input stimulus from the sensor was used forboth approaches, which went from 0 Hz to 180 Hz. The rates generated byboth sCPGs were very similar. In the case of the Brian 2, the operationfrequency of the sCPG ranged from 9.5 Hz to 14.9 Hz, and from 11.62 Hzto 26.66 Hz in the case of the SpiNNaker platform. The calculated Pearsoncorrelation coefficient between both is 0.905.16 igure 12: Comparison of the results obtained in both Brian 2 simulation and SpiNNakerimplementation. The plot shows the rate generated by the sCPG when the input stimulus(population
Ref ) is changed from 0 Hz to 180 Hz. The blue trace shows Brian 2 results(left y-axis) and the red trace the SpiNNaker results (right y-axis).
4. Conclusions
In this paper, we have presented what, to the best of our knowledge is,the first Spiking Central Pattern Generator that incorporates a feedback inthe loop to modify the locomotion frequencies of a legged robot through anadaptive learning mechanism. The feedback was provided by a FSR (but anysensor might be used) to simulate the force exerted on each limb of a futurerobot. With the use of this sensor, we injected different stimuli to the sCPG.Firstly, we performed some tests to find the optimal (minimum) input value,where the feedback effect could be easily observed. Then, some tests wereperformed to observe the effect in the oscillation frequencies under differentstimuli. Finally, we carried out some experiments where the values read fromthe sensor were alternating between maximum and minimum voltage peaksin order to study the robustness of the sCPG against sudden changes in theoscillation frequency. From these experiments, we conclude that our sCPGpresents a robust behavior in the adaptability of the oscillation frequencies.These frequencies can be used further in the generation of locomotion gaitsfor legged robots with the advantage that they can be modified dependingon the terrain conditions.The implementation of the sCPG was firstly done by using the Brian 217imulator and then, considering the same parameters, it was migrated toSpiNNaker. The results in both cases are highly the same ones as it is shownin Figure 12; this fact demonstrates the reproducibility of our architectureon any platform.Compared to the biological locomotion mechanism, we considered thatour approach is highly plausible in two senses, the first one is that the pro-posed network model is based on spiking neurons, which are considered theneuron models that mimic the best the behavior of biological neurons. Thesecond aspect is that the implementation performed in SpiNNaker allowedus to improve it in terms of the power consumption, and the hardware byitself attempts to be an artificial representation of the brain.As a future work, we propose to embed the SpiNNaker system into dif-ferent legged robots (e.g. biped, quadruped and hexapod) to validate ourapproach on a real robotic platform. Also, it could directly process spa-tiotemporal patterns with the same sCPG by incorporating a neuromorphicsensor.
Acknowledgements
This work was partially supported by the Interreg Atlantic Area Pro-gramme through the European Regional Development Fund (TIDE - At-lantic network for developing historical maritime tourism, EAPA 630/2018),by the Spanish grant (with support from the European Regional Develop-ment Fund) MIND-ROB (PID2019-105556GB-C33) and by the EU H2020project CHIST-ERA SMALL (PCI2019-111841-2).