A Spiking Central Pattern Generator for the control of a simulated lamprey robot running on SpiNNaker and Loihi neuromorphic boards
Emmanouil Angelidis, Emanuel Buchholz, Jonathan Patrick Arreguit O'Neil, Alexis Rougè, Terrence Stewart, Axel von Arnim, Alois Knoll, Auke Ijspeert
AA S
PIKING C ENTRAL P ATTERN G ENERATOR FOR THE CONTROLOF A SIMULATED LAMPREY ROBOT RUNNING ON S PI NN AKERAND L OIHI NEUROMORPHIC BOARDS
A P
REPRINT
Emmanouil Angelidis ∗ Department of Neuromorphic Computingfortiss - Research Institute of the Free State of BavariaMunich, Germany [email protected]
Emanuel Buchholz † Department of Neuromorphic Computingfortiss - Research Institute of the Free State of BavariaMunich, Germany
Jonathan Patrick Arreguit O’Neil
Biorobotics Laboratory, Institute of BioengineeringÉcole Polytechnique Fédérale de LausanneLausanne, Switzerland
Alexis Rougé
Department of Neuromorphic Computingfortiss - Research Institute of the Free State of BavariaMunich, Germany
Terrence Stewart
Computational Neuroscience Research GroupUniversity of Waterloo Centre for Theoretical NeuroscienceWaterloo, Canada
Axel von Arnim
Department of Neuromorphic Computingfortiss - Research Institute of the Free State of BavariaMunich, Germany
Alois Knoll
Chair of Robotics, Artificial Intelligence and Embedded SystemsTechnical University of MunichMunich, Germany
Auke Ijspeert
Biorobotics Laboratory, Institute of BioengineeringÉcole Polytechnique Fédérale de LausanneLausanne, SwitzerlandJanuary 19, 2021 A BSTRACT
Central Pattern Generators (CPGs) models have been long used to investigate both the neuralmechanisms that underlie animal locomotion as well as a tool for robotic research. In this workwe propose a spiking CPG neural network and its implementation on neuromorphic hardware as ameans to control a simulated lamprey model. To construct our CPG model, we employ the naturally ∗ Also Chair of Robotics, Artificial Intelligence and Embedded Systems, Technical University of Munich † Also Chair of Robotics, Artificial Intelligence and Embedded Systems, Technical University of Munich a r X i v : . [ c s . N E ] J a n PREPRINT - J
ANUARY
19, 2021emerging dynamical systems that arise through the use of recurrent neural populations in the NeuralEngineering Framework (NEF). We define the mathematical formulation behind our model, whichconsists of a system of coupled abstract oscillators modulated by high-level signals, capable ofproducing a variety of output gaits. We show that with this mathematical formulation of the CentralPattern Generator model, the model can be turned into a Spiking Neural Network (SNN) that can beeasily simulated with Nengo, an SNN simulator. The spiking CPG model is then used to producethe swimming gaits of a simulated lamprey robot model in various scenarios. We show that bymodifying the input to the network, which can be provided by sensory information, the robot canbe controlled dynamically in direction and pace. The proposed methodology can be generalized toother types of CPGs suitable for both engineering applications and scientific research. We test oursystem on two neuromorphic platforms, SpiNNaker and Loihi. Finally, we show that this category ofspiking algorithms shows a promising potential to exploit the theoretical advantages of neuromorphichardware in terms of energy efficiency and computational speed. K eywords Neurorobotics, Central Pattern Generators, Spiking Neural Networks, Nengo, Neurorobotics Platform,SpiNNaker, Intel Loihi, Neuromorphic Computing, Robotic Control, Hopf Oscillators
Our work can be placed in the emerging field of Neurorobotics, a field that combines knowledge acquired from differentscientific fields and applies them to the study and the control of animal models and robots. Within the context ofNeurorobotics, an artificial brain, either biologically or AI inspired, is interacting with a robot model in physical orvirtual experiments [1]. This enables the testing of hypotheses on virtual embodiment, a concept which encompassesthe idea that a brain is not a system isolated from the outer world, but one that constantly receives and processes stimuliand acts according to them. Neurorobotics problems can fall into various categories, for example robotic control basedon cerebellar models [2, 3], dynamic vision systems based on event-based cameras [4, 5], visual perception [6], motorcontrol and locomotion tasks [7, 8] and action selection [9].A major limitation of existing neuronal models that are often used as artificial brains is that they are both energy andcomputationally demanding, since they are usually running on conventional CPUs. Even though spiking neural network(SNN) models are computationally sparse by definition [10], this characteristic is not taken into account when runningthem on conventional hardware. Thus specialized hardware that is optimized to run these models has been researchedand developed, among others Intel Loihi [11], IBM TrueNorth [12], SpiNNaker [13] and BrainScale [14], the latter twodeveloped within the context of the Human Brain Project. Our work makes use of a SpiNNaker and a Loihi chip thatruns the spiking neural network that we developed.Many fields of robotics have taken inspiration from biological systems, and particularly from the locomotor system.Locomotion of animals is hypothesized to be controlled to a large extent by functional units in the central nervoussystem (CNS) called called Central Pattern Generators (CPGs) [15, 16], which are usually described as neuronalsystems that create rhythmic activity patterns with minimal sensory feedback. In vertebrates, these locomotor circuitsare located mostly in the spinal cord, and receive stimulation from the brainstem and other areas of the brain such asthe motor cortex, the cerebellum and the basal ganglia [17]. One interesting finding is that these networks are capableof producing rhythmic output in the absence of feedback with minimal stimulation, even if the spinal cord has beencompletely isolated from the body [18]. The investigation of CPG based locomotion control is motivated by the insightthat it can give on animals locomotion systems and by the fact that these kind of bio-inspired controllers present goodcapabilities in terms of autonomy and modulation [19]. So far the CPG approach has been largely validated for thelocomotion of snake-like robots [20, 21, 22, 23]. On an implementation level there exist several CPG models which areformulated as SNNs, and and these spiking CPGs (SCPGs) are often running on specialized or generic neuromorphichardware. It was shown that such SCPGs running on Neuromorphic hardware such as FPGAs, SpiNNaker or VLSIare providing a robust and efficient way to control a complex movement [24] including sensory feedback, namely forbipedal walking [25, 26], for the movement of an arm [27, 28] or to control a six-legged robot [29, 30].The mathematical modelling of CPGs can be categorized into roughly 3 approaches. The first treats the neural circuitryto the abstraction level of biophysical models and incorporates information about ion pumps and ion channels locatedin the neural cells membrane and their influence on membrane potentials and the generation of action potentials,frequently modelled by Hodgkin-Huxley neuron models. The second approach uses simpler leaky integrate-and-fireneurons as the basis of computation, abstracting away low-level biological information. The third category whichis also our starting point is deprived of lower level biological information and treats CPGs as systems of nonlinearcoupled oscillators, where one oscillator models the activity of a whole oscillatory neural network at an abstract level.Although conceptually the latter is a phenomenological approach based on the observation of the emerging locomotor2
PREPRINT - J
ANUARY
19, 2021patterns, it still offers many explanations of the underlying mechanisms of rhythmic pattern generation. One of the firstsuccessful attempts to use a high-level mathematical formulation of a CPG and model it as a dynamical system whichcan be simulated with spiking neurons was the work of Eliasmith and Anderson [31]. Many of the described models areaccompanied with neuromechanical simulations that close the loop between body and brain. For an extensive review onCPGs in robotics and biology we refer to [16].In this article, we present a high-level SCPG for a lamprey robot that was trained to replicate the dynamics of a systemof coupled Hopf-like oscillators. This model is able to produce a set of travelling waves with high-level modulationwhich correspond to a continuous space of swimming gaits. It can run directly on the neuromorphic SpiNNaker andLoihi boards. It builds on the core Neurorobotics idea of interaction between a virtual robot or animal agent and avirtual brain that runs on neuromorphic hardware and achieves a complex locomotion task. In Section 2, we present theunderlying mathematical formulation of the system of coupled Hopf-like oscillators as a first step of the modeling, inSection 2.3 we present the spiking version of the CPG and its performance on the two boards. We provide simulationsof both the isolated spiking CPG model as well as neuromechanical simulations under different scenarios in 3. We thenpresent our future work (4.1) and a conclusion (4).
Locomotor CPGs are modulated by higher level control centers of the brain with low-dimensional control signals, aproperty which makes CPG models good candidates for robotic control problems. This property of CPGs gives them arole similar to a feed-forward controller inside a control framework, of producing oscillatory signals that are modulatedby external stimulation. To test whether our CPG model can successfully control a lamprey robot we implemented aneuromechanical simulation for which we employed an accurate 3D model of a lamprey robot that is composed of ninebody parts similar to the Amphibot robot in [32]. These parts are bound together by eight joints that have one degree offreedom: the rotation around the vertical axis. To produce the swimming patterns, the angular positions of these jointsoscillate with amplitudes, frequencies and phases prescribed by the CPG model. The complete controller architecturecan then be divided in three components (see Figure 1):1. the mesencephalic locomotor region (MLR), that emits high level signals on each side of the spinal cord: thedrives;2. the central pattern generator (CPG), that generates travelling waves for each joint corresponding to the receiveddrives;3. the proportional derivative (PD) controller, that controls the torques applied to the joints to reach the time-varying target angle positions. .
In order to explain the synchronization phenomena between the different oscillatory centers in the vertebrate spinalcord, Ijspeert [7] proposed a model of nonlinear coupled oscillators, and used this model to control a salamander robot.This model proposes a coupling between different oscillatory centers based on coupling weights that dictate the phasedifference and frequency of the oscillatory centers. The oscillators can be chained either in a single or double chain. Inthe double chain model, the one that we employ here, the activity of the one side of the spinal cord is in antiphase withthe activity of the other side, a phenomenon which is also observed in measurements of muscle activity of lampreys.Providing different stimuli, coming from the high-level control centers, between the oscillators found on each sidecan lead to a shift of the overall oscillatory patterns, which when applied to a robot model induces turning due to thechange of the overall curvature of the robot. This dynamical system can be described by the following differentialequations which describe a system of phase oscillators with controlled amplitude. The oscillators are described first inphase space, which gives an intuition of how the coupling is induced, and then rewritten in Cartesian space which as weexplain is a form suitable for modelling with an SNN: ˙ θ i = 2 πν i + (cid:88) j r j w i,j sin ( θ i − θ j − Φ i,j ) (1) ¨ r i = a i (cid:16) a i R i − r i ) − ˙ r i (cid:17) (2) x i = r i (1 + cos θ i ) (3)3 PREPRINT - J
ANUARY
19, 2021 Ψ i = α ( x i,right − x i,left ) (4)In this system the θ i , v i are the phase and the preferred frequency of the i -th oscillator, r i , the amplitude, x i is theoutput of the i -th oscillator which represents motoneuron activity, and Ψ i is the output of the model that is appliedto the robot and combines the activity of the oscillators of left and the right side of the double chained model. Fromequation 1 one can observe that the first derivative with respect to time of the phase of each oscillator, is modulatedby the coupling weights w ij and the amplitude of the oscillators it is connected to. It is interesting to note that whenthe phase differences Φ ij are reached between the coupled oscillators the term θ j - θ i - Φ ij becomes zero, and thusthe oscillator oscillates with the preferred frequency πν i . This is indeed the case when the steady state is reached,which takes place when certain convergence criteria are met. Equation 2 describes how the amplitude of each oscillatorconverges to the preferred amplitude R i , with parameter a i dictating the speed of convergence. This ensures smoothtransitions of the amplitude when abrupt changes of the high-level drive occur. Even though this system fully describesa CPG in phase space, it is not suitable for approximation with an SNN, as integrating equation 1 in time, leads toa constantly increasing phase. This constantly increasing value quickly saturates the representational capabilities ofneural populations, as they excel in approximating values within a subset of a larger space. The solution for this problemis to reformulate the problem in Cartesian space as follows [33]: ˙ x i = a ( R i − r i ) x i − ω i y i (5) ˙ y i = a ( R i − r i ) y i + ω i x i (6) ω i = ω i + (cid:88) j w ij r i [( x i y j − x j y i ) cos Φ i,j − ( x i x j − y i y j ) sin Φ i,j ] (7)where x i , y i denote the x and y-coordinates of a point in 2-D space moving in a circle through time, with frequencycontrolled by equation 7. The parameter a dictates the speed of convergence of the amplitude to the steady state, and r i it the norm of the [x,y] vector. This formulation is close to the standard form of coupled Hopf oscillators with couplingto other oscillators. This equation has the advantage that the x,y values stay within a limit cycle, whose radius is dictatedby the amplitude of the oscillation, solving the problem of continuously increasing phase when one attempts to use thephase representation.To incorporate the drive corresponding to the high-level stimulation we use two piece-wise linear functions, whichsaturate when the stimulation is outside of a certain range. These two functions control the target frequency and thetarget amplitude of each oscillator according to the relations: ω i ( d ) = (cid:26) c ω, d + c ω, , if d low ≤ d ≤ d high , otherwise (8) R i ( d ) = (cid:26) c R, d + c R, , if d low ≤ d ≤ d high , otherwise (9)These two equations replicate biological observations that the frequency and amplitude of muscle contraction increasetogether with increased stimulation, hence leading to faster locomotion. They complement the CPG with high-levelmodulation, and with them we have a complete mathematical formulation of the control framework, which we implementin an SNN. The model that we introduced in the previous section is a mathematical formulation of a system of coupled abstractHopf-like oscillators, modulated in frequency and amplitude by high-level stimulation. We show that such a system canbe easily simulated with an SNN simulator. To do so we designed a modular SNN architecture where one oscillatorycenter is represented by one population of spiking neurons and computes the equations described in (5 - 7). Thispopulation at the same time encodes equation 9. For the coupling between the neural oscillators we introduce anintermediate population which receives the x,y values from neighbor oscillators, and computes the coupling term of4
PREPRINT - J
ANUARY
19, 2021equation 7. This intermediate population facilitates the exchange of data between the neural oscillators, and it’s presenceis dictated purely by the framework that we chose to implement the SNN. The overall architecture of the model can beseen in Figure 2. At the same time each of the oscillatory centers is receiving input from the high-level drive throughequations 8 - 9.
In order to replicate the system of modulated oscillators with a spiking neural network the choice of a frameworkthat can perform such numerical computations was necessary. A characteristic shared by most neural simulators isthat they allow the simulation of simple leaky integrate-and-fire neuron models (LIF). According to this model [34]the neuron spikes when its membrane potential reaches a certain threshold. Each neuron is excited by the neuronsthat are connected to it either in an excitatory or inhibitory fashion, increasing or decreasing the membrane potentialrespectively. After a period of inactivity the membrane potential is reset -leaks- to a base value. A neuron is usuallyconnected with multiple other neurons via junctions called synapses. The information flow from one neuron to the otheris dictated among other factors by the level of present in the synapse neurotransmitters and whose release is regulatedby dedicated proteins. The overall strength of the connection between neurons is dictated by the synaptic weight.From a computational perspective, the adaptation of the synaptic weights through synaptic plasticity mechanisms isthe process which allows these networks of neurons to learn a representation. Synaptic plasticity mechanisms can beeither biologically accurate, i.e. STDP [35], or variations of some machine learning inspired approach such as the onesmaking use of backpropagation algorithms [36], or biologically plausible mechanisms such as the e-prop algorithm[37]. Most computational models of spiking neurons employ the simple Leaky integrate-and-fire neuron model. We usethese types of neurons for our study as well. Several simulation platforms were suitable for the task of simulating suchneurons, but Nengo [38] was chosen for two reasons. First, it has built-in methods for generating neural networks thatapproximate differential equations. This approach is described in section 2.3.3. Second, it can generate versions ofthese networks that can run on dedicated neuromorphic hardware, as we discuss in section 2.5.
In this section we give an overview of the Neural Engineering Framework (NEF), which is a general methodology forcreating neural networks that approximate differential equations [39]. Importantly, it generalizes to any neuron model,including LIF spiking neurons, and takes into account the timing of synapses.To understand the NEF, we start with the standard observation that a normal feed-forward neural network is a functionapproximator. That is, if we have some input x and some output y , we can train a neural network produce the desiredoutput y = f ( x ) . While this training can be done using any neural network learning algorithm, here we just use thesimple method of having a network with a single hidden layer of LIF neurons (no non-linearities at the input or output),randomly generate the first layer of weights, and use least-squares minimization to solve for the second layer of weights.This method works for a large range of functions and is robust to spiking neuron models [39].However, to generate the CPG model described here, we need networks that approximate differential equations. Here,the NEF applies the following method. Suppose we want the differential equation ˙ x = f ( x, u ) . We build a feed-forwardnetwork where the inputs are x and u and the output approximates τ f ( x, u ) + x . We introduce the variable τ here,which will be used as the time constant of a simple exponential low-pass filter synapse that will connect the neurons.Now to generate the recurrent neural network, we simply connect the output of that network back to itself, and scale the u input by τ . The resulting network will approximate ˙ x = f ( x, u ) . See [39] for a full proof, which is based on theobservation that the Laplace transform of the low-pass filter is F ( s ) = 1 / (1 + sτ ) . Similar transformations can bedone for more complex synaptic filters, but we do not use those here.As an example of this process, Figure 4 shows an NEF model of a single Hopf-style oscillator. This was formedby creating a feed-forward single-hidden-layer neural network with three inputs ( x , y , and ω ) and two outputs( τ ( a ( R − r ) x − ωy ) + x and τ ( a ( R − r ) y + ωx ) + y ). The weights for this network were found by randomlysampling the inputs ( x , y , and ω ), computing the desired outputs for each input, and then training the network given thisdata. Afterwards, the resulting input and output connection weights were multiplied together to create the recurrentneural network shown.The Nengo software toolkit [38], which is the software implementation of the more general Neural EngineeringFramework, provides high-level tools for creating such networks for a variety of neuron models. Crucially, it alsoprovides facilities for linking networks together, so that large systems can be built out of these components. Futhermore,the resulting systems can be automatically compiled to run on CPUs, GPUs, or a variety of neuromorphic hardware.5 PREPRINT - J
ANUARY
19, 2021
Based on the third principle of the NEF we employ the dynamical systems that emerge through the use of recursiveneurons to implement the oscillators in our model. It is worth noting that recurrent neural populations can implementvarious dynamical systems, such as integrators, oscillators, even chaotic systems such as Lorenz attractors. The networkcomputes each function from equations(5-9) according to the NEF principles. By doing so the decoded spiking activityof each neural population can be seen as a real-valued vector with the appropriate dimensions. For the populations thatencode the oscillators (depicted with theta i in Figure 2) this 4-dimensional vector represents the values [ ˙ x, ˙ y, ω, R ] . Forthe intermediate neuron populations that compute the coupling part of equation 7 the 4-dimensional vector representedis [ ˙ x i , ˙ y i , ˙ x j , ˙ y j ] . The high-level drive is approximated by the decoded activity of a neuronal population dedicated inreceiving the drive and translating it to neural activity. A dedicated readout output node (non-spiking) can be used toread the decoded output of the system, that corresponds to the x-coordinate of the Hopf-like oscillator. The completesystem with input and output for 4 oscillatory centers can be seen in Figure 3. As will be shown the system can scale toa larger number of oscillatory centers but the scaling can be limited by the capabilities of the neuromorphic hardwarethat it is running on.As mentioned in 2.3.3 the Neural Engineering Framework can be used to approximate any linear or non-linear functionwith spiking activity by computing the connection weights between the different components of a spiking neuralnetwork, acting as a neural compiler. This alleviates the need for explicit training of the SNN, as in the NEF theinformation that needs to be provided is limited to the properties of the neurons (i.e. membrane threshold potential,neuron types), the values that the neural populations need to represent and the functions that they compute, and theNEF solves for the connection weights that will compute the desired functions. This enables specifying the high-levelmathematical functions that are encoded by the SNN and that works both for feed-forward as well as for recurrentconnections. The latter is particularly relevant for our work as it enables dynamical systems such as the oscillatorsystem that we employ to emerge from the neuronal activity. In order for the connection weights to be computed bythe NEF, during the initialization phase of the simulation a random selection of sampling points to be used as inputsto the function to approximate is selected. These points are based on the input space that the neuronal populationapproximates, f.e. points in the space [0,1] for a population that encodes 1-D values. Then these points are used togenerate training data from the functions, by providing the points as inputs to the desired functions and collecting theoutput. Subsequently a least-squares optimization computes the weights that best fit the decoded neuronal activity tothe training data. For a more detailed technical overview of this method we refer the viewer to [40]. Animal CPGs have been documented to adapt to various perturbations (i.e. external application of a force), by reactingsmoothly and exhibiting stable limit cycle behavior, i.e. recovering the gait patterns without losing synchronization.Furthermore different degrees of stimulation of the oscillatory centers on the spinal cord can lead to different gaits.Simple asymmetrical stimulation between the right and left side drive of the spinal cord can induce a shift of the gaitpatterns to the left or to the right, and can induce turning. We show that these characteristics are exhibited by our modelunder the following scenarios:1. Perturbation of a single oscillatory center by external stimulation2. Asymmetrical stimulation of the spinal cord from left to right side of the spinal cordThese scenarios show the CPG model’s ability to quickly recover under external perturbations as well as to modulateswimming gaits.
To test the output and the high-level adaptation of the control signals we performed a closed-loop neuromechanicalsimulation of our model with a robot model as a body. The motivation behind simulating our model within a physicalsimulation framework comes from the fact that neural circuits and control algorithms cannot be separated from theirnatural habitat, the body. Only within an embodied simulation can we test whether the system that we proposecan successfully control a robot. For such a full closed-loop robot-brain interaction simulation we made use of aframework built exactly for this purpose, the Neurorobotics Platform. The Neurorobotics Platform (NRP) is a softwaresimulator developed within the Human Brain Project [41] that enables the synchronization and exchange of data betweenmodelled brains and virtual robots within a physical simulation environment. The Robotic Operating System [42] is themiddleware which enables the communication between the different software components, which is also supported by amultitude of physical robots. Within the NRP there is no need for an explicit synchronization mechanism between thephysical world and the modelled brain, as such a mechanism is built into the framework. The physical simulation is6
PREPRINT - J
ANUARY
19, 2021provided by Gazebo [43], which interfaces with multiple physics engines. It supports directly many different brainsimulators such as NEST [44], Nengo and SpiNNaker, and through Nengo one can run models on Loihi. We used thisframework to connect the Nengo model presented in section 2.3.4 with the lamprey robot (Figure 1).To complement the simulation with a simplified fluid dynamics model, we implemented a drag model, which iscomputing the forces produced by the swimming motion, forcing the robot to move forward. The drag model is the onepresented in [45], and computes the forces applied on each robot link based on the formulas: E i (cid:107) = λ i (cid:107) υ i (cid:107) (10) E i ⊥ = λ i ⊥ υ i ⊥ (11)and the coefficients λ can be computed by λ i (cid:107) = 12 C i (cid:107) S i ρ (12) λ i ⊥ = 12 C i ⊥ S i ρ (13)where υ i (cid:107) and υ i ⊥ are the velocity components of each link relative to the water in the parallel and perpendiculardirections. The parameter λ depends on the fluid density ρ and the parameter S i is the surface of the link perpendicularto the link movement. This drag model is only a simple approximation of the fluid forces applied on the robot, butoffers simplicity and computational speed compared to the 3D Navier-Stokes equations. We tested the arising swimming gaits under different simulation scenarios. Firstly we show that the spiking CPG canproduce swimming even with a low number of neurons. Secondly we show unperturbed swimming with no high-levelmodulation. Thirdly, we present modulation of the swimming by the high-level drive with control of direction andspeed. To show the ability of the controller to incorporate sensory feedback from the simulation dynamically we add awater speed barrier to the simulation. This speed barrier forces the robot to move to the side without adaptation of thehigh-level drive, but with modulation the robot manages to overcome it. The water speed barrier is implemented in theform of a global fluid velocity vector opposite to the forward direction. A summary of the scenarios:1. Unperturbed swimming, effect of varying number of neurons per neural population2. Unperturbed swimming, no high-level modulation3. Unperturbed swimming, control of the speed and direction of the robot4. Presence of water speed barrier, no high-level modulation5. Presence of water speed barrier, high-level modulationThe method that we used to modulate the high-level drive of the robot in the presence of a speed barrier consists of ahigh-level feedback loop that modulates the turning commands (i.e. the left-right asymmetry of drive signals) towards adesired target angle (e.g. similarly to a fish aiming to swim towards a particular far away target). This is implementedthrough a linear minimization of the error between a target global angle around the z-axis of the robot’s head and theactual angle of the robot’s head around the z-axis. Thus, when the robot turns i.e. left, the error between the target angleand the measured angle increases and the right drive increases linearly to compensate for the deviation from the targetangle. The equations that we used for this strategy: d right = (cid:40) d right + CF ∗ abs ( (cid:126)R z,target − (cid:126)R z ) , if R z − R z,target ≤ d right otherwise (14) d left = (cid:40) d left + CF ∗ abs ( (cid:126)R z,target − (cid:126)R z ) , if R z − R z,target ≥ d left otherwise (15)7 PREPRINT - J
ANUARY
19, 2021Where the left drive is increased when the error is positive, and the right when negative. (cid:126)u target is the target lateralvelocity, (cid:126)R z is the recorded rotation around the z-axis of the robot’s head, CF is the correction factor that linearlymultiplies the error, and d right and d left provide the baseline of the drive stimulation. This simple error correctionstrategy proves to be enough to correct the deviation of the robot from a target angle by modulating the CPG with thehigh-level drive. As stated in [46], the computational limitations for running spiking models on conventional CPUs are originating in thevon Neumann architecture. Conventional computers are built and optimized to perform Boolean algebra operations andarithmetic on the data stored in memory. Hence, this data needs to be transferred back and forth between the memoryand the CPUs, which can be time consuming. Neuromorphic hardware on the other hand is specialized in runningspiking neural networks. The computation takes place in many small calculators that have access to a small amountof local data. This strategy reveals itself to be more time and energy efficient for neuron oriented computations. Forthis reason, we tested our Nengo model on a SpiNNaker-3 [13] and a Loihi board [11]. Due to the direct connectionof SpiNNaker and Loihi boards to Nengo with a software interface our model remained high-level but could be rundirectly on the boards.It should also be emphasized that, for efficiency reasons, the actual neuron model running on conventional CPUs,SpiNNaker-3, and Loihi, are all slightly different. They can all implement Leaky Integrate-and-Fire neurons (and otherneuron models), but they all make slightly different approximations (e.g. fixed-point rounding). This means that theoptimal neural network connection weights for these different hardware platforms will all be slightly different. However,because we specify our model in Nengo using only the mathematical function to be approximated, this means thatNengo can take the hardware details into account when solving for the connection weights, and the user does not haveto modify their model to adjust for different hardware platforms.That said, there are still some areas where the Nengo-SpiNNaker and Nengo-Loihi interfaces have room for improvement.In particular, the software support for automatically splitting a group of neurons to run across multiple hardware coresis lacking, effectively giving an upper limit on the size of a single group of neurons that is hardware-dependent. Wealso encountered hardware limitations on the amount of data that could be probed (i.e. recorded) during the running ofthe simulation, as discussed in Section 3.2.3.
The first test that we performed on the isolated (i.e. no time-varying external modulation) spinal cord model, showsthat our system can produce oscillations and traveling waves from random initial conditions meaning that it exhibitslimit cycle behavior. For such a scenario there is a clear periodic activation of the spiking neurons inside the oscillatorypopulations as can be seen in 6. In order to provide benchmarks for the neuromorphic platforms vs the CPU as wellas to show the adaptive capabilities of our model we ran the model with different numbers of neurons and differentnumbers of oscillatory centers. An interesting finding is that oscillatory patterns are generated even with low numbersof neurons as can be seen in Figure 8.Furthermore, perturbing the model by providing explicit stimuli on specific oscillatory centers, can lead to someinteresting behaviours which show the stability of the circuit. As can be seen in Figure 7 a single external perturbationon one of the oscillatory centers leads to a temporary disruption of the signals, localized around the neighbouringoscillatory centers. Upon removal of the perturbation the oscillators quickly recover and stabilize. This is the limit cycleproperty of the high-level mathematical model that is captured well by the spiking network, and exhibits the robustnessof the model, a property which is of particular importance for robotics problems.The high-level modulation and control of the signals when varying the input to the network under the scenario describedin 2.3.5 can be seen in Figure 5. In this scenario a simple asymmetrical variation of the input signals between the left andthe right side of the spinal cord leads to a formulation of different travelling wave patterns, which can induce differentswimming behaviours. A variation between the left and right side of the spinal cord leads according to equation 4 to ashift of the center of the signals towards positive or negative angles, which in turn induces a shift of the joints anglestowards one side, causing the robot’s curvature to change, inducing a change of direction.8
PREPRINT - J
ANUARY
19, 2021
As mentioned in section 3.1 swimming patterns arise even with a smaller number of neurons for every neural populationin the spiking neural network, albeit the fewer neurons the less precise the approximation is. A comparison of the threesimulation scenarios with consecutively larger numbers of neurons can be seen in videos (500 neurons), (1000neurons), (2000 neurons). The robot configurations in the scenario of the 2000 neurons can be seen in Figure 9. Thevideos correspond to Figure 8, and as can be observed the less neurons, the less smooth the swimming is. Nevertheless,even the 280 neurons per neural population are enough to provide a swimming pattern.Asymmetry of the driving signals between left and right induces turning as can be seen in video , and providing suchdrives is a simple way to navigate the robot towards one direction. Using a closed loop control method such as the onedescribed in 2.4.1 such asymmetries can be computed and provided automatically to the control loop. As described in section 2.4.1, to demonstrate the controllability of the robot with a closed loop controller we examinethe behaviour of the robot with the presence of a speed barrier, first without adaptation of the high-level signal andthen with high-level adaptation . In the first video, the speed barrier causes the robot to follow a trajectory towards theside, by applying higher drag forces to the robot in the lateral direction. In this scenario the robot does not manage tocompensate for the presence of the speed barrier as the unmodulated oscillatory signals do not induce a correction ofthe direction of the robot. In the second video on the other hand, the error correction mechanism described in 2.4.1is activated, causing the trajectory of the robot to be corrected to compensate for the speed barrier, and eventually itmanages to orient itself and swim forward. We can observe that the model adapts well when the high-level tonic drivesignal is regulated by the error correction mechanism, which conceptually corresponds to the adaptation that a decisionmaking center of the brain would perform in order to follow a certain trajectory. For robotics applications it is important that the control signals are generated in real-time. In order to be able to controla robot with the two neuromorphic boards that we examined, the quality of the generated signals has to be similar to theone coming from the CPU. Such comparison of the quality for a simulation of 10 secs can be seen in Figures 11 and10. As can be observed, the signals are of better quality than the CPU for a low number of neurons. The quality ofthe produced signals depends heavily on the number of neurons that are used to represent them. Due to limitationsarising from the architecture of the two neuromorphic boards we tested, the total number of neurons that we could runon a SpiNNaker board is limited to 30000, for a Loihi board the limitations are reached at a similar number of neuronswhen no probes for measuring the networks output are used. With probes the limit on Loihi is reached at approximately22000 neurons. The concept of a probe corresponds to a software construct that can be used to collect simulation datafrom the neuron activity, energy consumption etc. They are used to record the decoded output value of the neuralpopulation representing the oscillatory centres.A more detailed comparison of the runtime performance for the different platforms can be see in figure 12. What weobserved during the execution on the neuromorphic chips is that most of the time is spent during phases other than thenetwork execution, mostly during the initialization phase where the network configuration is being setup, and duringinput-output(I/O) operations such as the transfer of spikes between the neuromorphic board and the host computer.This is especially true for the Loihi board, as can be observed in figure 13, where the actual execution of the networkis around 1 second for 10 seconds of simulation time, almost 10 times faster than real-time, slightly increasing asthe network’s size increases. In contrast, most of the time during execution is spent on other operations such as theexchange of spikes. It is clear, that this is the main bottleneck of Loihi’s execution time. SpiNNaker on the other hand,and especially the execution of spiking networks on SpiNNaker through Nengo, is already optimized for real-timeexecution. This is the reason why the total operation of SpiNNaker including I/O operations and network execution isstaying almost real-time. It should be noted that this time also includes waiting times induced by Nengo to make sure https://youtu.be/E27Zj1ShI14 https://youtu.be/b1E9EvVRoOw https://youtu.be/n3Q-Sn6jUKU https://youtu.be/DKdtTFdthbI https://youtu.be/hCtUVjqVr5g https://youtu.be/Q58Me79cfSs PREPRINT - J
ANUARY
19, 2021the simulation runs in real-time. The network itself is executed on SpiNNaker at around 2 seconds, marking a slightlyslower execution time than Loihi.A more detailed analysis of the time spent during the execution of the network on Loihi during larger simulation timesis provided in figure 14. To explain the observations it is useful to separate the operation of the board in three distinctphases. The first would be the initialization and setup phase which includes software overhead, overhead to boot theboard, setup of the host server, compilation of neurons and synapses on the board and which is performed only once.The second phase would be the loading of the spikes into the neuromorphic board which can be done in parallel with theexecution of the network, or before the execution of the simulation. The third phase corresponds to the actual executionon the board. From these findings we can conclude that as soon as the execution of the network is separated from thesetup it can perform much faster than real-time. It should be noted that these metrics are relevant for this specific neuralnetwork and do not provide an accurate metric for other types of models.Due to software limitations it was not possible to provide accurate energy benchmarks for the SpiNNaker board.However, a comparison of the energy consumption between a CPU and Loihi is provided in figure 15. On Loihi theenergy consumption was measured with the built in time and energy probes. For measuring the energy consumptionon the CPU, the RAPL interface was used. RAPL is an Intel processor feature that provides the ability of monitoringand controlling the SoC power consumption [47]. As the power measurement control domain we used the
PACKAGE domain which includes the energy consumption of all cores, integrated graphics and other uncore components likecaches and memory controllers. For the actual measurement, a framework developed by [48] was used.As a result, in figure 15 you can see that the energy consumption of the Loihi chip is by three orders of magnitude lowerthan executing the same network with Nengo CPU. This shows neuromorphic hardware can deliver significant energyreductions for executing spiking neural networks when compared to traditional CPU architectures.
In this paper we presented a Spiking Central Pattern Generator based on a high-level system of abstract coupledHopf-like oscillators that can run on both software and neuromorphic hardware. The method which we used can begeneralized to any type of similar CPG controller. Our model is highly parametrizable, and is an excellent candidate foroptimization methods. With different parametrizations it can provide a vast number of possible synchronized gaits,f.e. travelling and standing waves. Our method enables us to smoothly control a lamprey robot that with regulationof the high-level drive adapts to various simulation scenarios. We presented a closed-loop neurorobotics simulationwithin the Neurorobotics Platform achieving multiple locomotor tasks. Lastly, we showed that running the controller onneuromorphic hardware can achieve real-time operation and has potential advantages in terms of energy efficiency andcomputational speed.Our work is related to other works in the field that attempt to provide insight on the performance of neuromorphic hard-ware. In particular, SpiNNaker was benchmarked for its performance in terms of energy efficiency and computationalspeed with similar accuracy, to an HPC system running a full-scale microcircuit of the human cortex model [49]. It wasshown that for such complex models the energy consumption per synaptic event, which provides an estimate of theenergy efficiency is . µ J , close to the . µ J consumed by the HPC system. However for simpler models, closer interms of synaptic connections and number of neurons to the model that we employ, the cost per synaptic event can be aslow as [50]. Similarly, in [12] they compared the performance of an IBM TrueNorth neuromorphic chip runninga set of computer vision neural networks with the performance of a dual 2.4 GHz E5-2440 processor x86 system, aswell as a Blue Gene/Q system with up to 32 compute cards and found two to three orders of execution time speedupand five orders of magnitude less energy consumption compared to the non-neuromorphic systems. Blouw et al. [51]showed that the energy performance of Intel’s Loihi chip compared to the Movidius Neural Compute Stick, Nvidia’sJetson TX1, a CPU, and a GPU was significantly lower (5.3x, 20.5x, 23.2x, 109.1x times respectively), for a keywordspotting task. However it should be noted that generating precise energy consumption benchmarks is a cumbersometask, and often the claims about the theoretical energy efficiency of neuromorphic hardware are not accompanied withthe corresponding metrics. In order to study the challenges presented in animal swimming locomotion, a realistic simulation framework that canmodel all the different aspects of the physical world is necessary. The dynamics of the system, the control part, and theircommunication and synchronization is already solved in the Neurorobotics Platform, but a realistic fluid simulationis still missing. We are planning to address this problem and present a unified framework in our future works. This10
PREPRINT - J
ANUARY
19, 2021would allow providing realistic force feedback in the control loop, thus enabling the generation of more complexcomputational models.Furthermore, our CPG model can be enriched with various form of environmental or sensory feedback, which can beincorporated into the model itself. Sensory data such as stretch receptors, high-level cognitive controllers that regulatethe tonic drive are examples of this type of feedback.One natural continuation of our work would be the transfer of the control framework on a real robot, such as theAmphibot. This is currently limited by the size of the SpiNNaker board that would prevent it from being fitted on therobot. However Loihi comes with a USB stick that is more compact in size and would potentially fit on the robot. Oneimportant consideration would be waterproofing the neuromorphic boards, as well as making sure that the changesinduced in the dynamics of the system by the extra weight would be negligible.
Acknowledgments
The authors would like to thank Peter Blouw and Eric Hunsberger from Applied Brain Research for their valuable helpon setting up the Nengo simulations and David Florey,Yulia Sandamirskaya and Andreas Wild from Intel for their helpwith the Loihi simulation and interpretation of results. PREPRINT - J
ANUARY
19, 2021Figure 1: The control framework. The brainstem component is abstracting the brain areas that are stimulating the spinalcord, separated into two stimulations, one for each side of the spinal cord. The CPG component, comprised of coupledoscillatory centers organised in a double chain, produces the swimming gaits modulated by the high-level brainstemcontrol. A PD controller is receiving the output of the CPG network and applies it to the robot, controlling the angularrotation of each joint. 12
PREPRINT - J
ANUARY
19, 2021Figure 2: Architecture of the spiking CPG model. Each oscillatory center, noted with theta i is coupled with itsneighbours through an intermediate population, depicted with C ij . The intermediate population is computing thecoupling term of equation 7. The x-y diagrams corresponding to each oscillator show the trajectory of a point traversingthe limit circle through time for the ideal mathematical model. As can be observed, the oscillators in each side of thespinal cord have an antiphase relationship between them, whereas the ones upwards or downwards have a fixed phasedifference of π/N umOsc .Figure 3: (Left) The Nengo simulated model where 4 oscillatory centers are shown. In this simulation the high-levelstimulation is driving the oscillations. (Right) The output of each oscillator that corresponds to the decoded spikingactivity, when 2000 neurons per oscillatory center are used, is depicted.13 PREPRINT - J
ANUARY
19, 2021 i n p u t () n e u r o n s o u t p u t Figure 4: The behavior of a single Hopf-like oscillator implemented in spiking neurons using Nengo and the NeuralEngineering Framework (NEF). The model consists of an all-to-all recurrently connected layer of LIF neurons withexponential synapses with 100ms time constants. Their spiking activity is shown in the middle row, sorted bysimilarity. A single input ( ω ) is provided, and the two outputs show that it functions as a controlled oscillator. Theinput weights, recurrent weights, and output weights are found using the NEF such that the network approximates ˙ x = a ( R − r ) x − ωy and ˙ y = a ( R − r ) y + ωx .Figure 5: The output of the CPG network for 16 oscillatory centers, where each oscillator is depicted with θ i . Anasymmetric drive is provided to the network after 5 seconds of simulation, increasing the drive on the right side ofthe spinal cord, and decreasing it on the left. As can be observed the amplitude of the oscillations on the right sideincreases, whereas on the left side decreases. 14 PREPRINT - J
ANUARY
19, 2021Figure 6: Spike train of the first 50 neurons of an oscillatory population with 2000 neurons for 4 secs. The activity of theneurons shows clears signs of periodicity. The neurons are continuously alternating between high and low firing rates.Figure 7: The output of the network when the 5th oscillator is perturbed by an external signal. The perturbation lastingfrom 4.8 to 5 secs causes disturbance of the neighbouring oscillators’ θ , θ , θ wave patterns. The model quicklyrecovers when the perturbation is removed. 15 PREPRINT - J
ANUARY
19, 2021Figure 8: The output of the network for different number of neurons per oscillatory population. Even with 500 neuronsthe network can produce an oscillatory output, of lower quality as some of the oscillators’ waves are not smooth andthere is more high-frequency noise. With 100 neurons there is an improvement of the quality of the signals, whereaswith 2000 neurons the signals are smooth and without high-frequency noise. Even with a low number of neurons thepatterns are capable of producing simulated swimming. The network was trained in Nengo with a random seed of 0.16
PREPRINT - J
ANUARY
19, 2021Figure 9: Swimming with the simulated robot, with snapshots at 160ms intervals for the unperturbed non-adaptivescenario. The network consists of 2000 neurons per neural population. The travelling wave is propagated along therobot’s body from head to tail. 17
PREPRINT - J
ANUARY
19, 2021Figure 10: The output of the network for different number of neurons per oscillatory population when executed onSpiNNaker. On SpiNNaker the output of the network is relatively accurate and better than the CPU even for a smallnumber of neurons. The weights were trained with a random seed of 0. Note that high-frequency filtering is applied bydefault on the output signals.Figure 11: The output of the network for different number of neurons per oscillatory population when executed onLoihi. The results have similar accuracy as SpiNNaker and perform better than the CPU for a low number of neurons.The weights were trained using the random seed 0. Note that high-frequency filtering is applied by default on the outputsignals. 18
PREPRINT - J
ANUARY
19, 2021Figure 12: Runtime of a 10 seconds experiment for various number of neurons per platform. The total execution time inSpiNNaker is referring to the complete execution cycle from the moment the simulation is launched to the moment theexecution data is collected, likewise in Loihi. It is important to note that these values represent the execution of Nengoon the neuromorphic hardware from the perspective of an application developer, treating the hardware as a black box.The SpiNNaker on-chip execution time measures only the time spent on the board for the execution of the network.The Loihi execution measures the execution time reported by Loihi and represents the actual time spent executing thenetwork. The execution + spike transfer represents the execution time plus the time spent during the exchange of spikesbetween the Loihi board and the CPU. The reasoning behind these benchmarks is to demonstrate that the times spenton the chip are very low compared to real-time and the rest of the times is spent on IO operations or other operationsinduced by the software. For a more detailed breakdown of the execution times in Loihi see also Figure 13. It canbe observed that the actual execution time on the boards is much faster than real-time, showing that neuromorphichardware is a great candidate for running the CPG model in real-time.19
PREPRINT - J
ANUARY
19, 2021Figure 13: Breakdown of total execution time on the Loihi chip into different parts for 10 seconds of simulation timeand increasing neurons. Python timings refer to the execution of the network from an application developer’s point ofview and include all the software and IO induced times. The Executing series shows the actual execution time on thechip and is linearly increasing as the number of neurons increase. The Executor series includes both the execution andthe transferring of spikes between the board and the CPU. It should be noted that these two processes can be performedin parallel. The times spent during the setup and initialization phases (Host server up, encoding axons/synapses, bootingthe board, configuring registers) are performed only once and their relative duration is less significant if the simulationtime increases, see also 14 20
PREPRINT - J
ANUARY
19, 2021Figure 14: Nengo Loihi execution times when the simulation time increases. All the benchmarks were performed with anetwork with 450 neurons per oscillatory center. In this figure it is evident that the initialization and setup times play anincreasingly less significant role as the simulation time increases, making it possible to execute the network in real-timeafter roughly 35 secs of simulation time. This is important from the perspective of the application developer as it istaking into account all the software and IO bottlenecks, which usually treats the chips as black boxes and optimizes onthe software and network layer. From the figure we can observe that the times spent during the operation of the chip areon the transfer of spikes and on the actual execution, which increase linearly in time, whereas all the other times remainrelatively stable. 21
PREPRINT - J
ANUARY
19, 2021Figure 15: Energy Benchmark of the CPG with Nengo Loihi and Nengo CPU, measured with built-in energy probesin Loihi and with the RAPL interface on the CPU. Is it clear that the energy consumption on the chip is orders ofmagnitude smaller that the consumption on the CPU. 22
PREPRINT - J
ANUARY
19, 2021
References [1] A. Knoll and Marc-Oliver Gewaltig. Neurorobotics : A strategic pillar of the human brain project. 2016.[2] Marie Claire Capolei, Emmanouil Angelidis, Egidio Falotico, Henrik Lund, and Silvia Tolu. A BiomimeticControl Method Increases the Adaptability of a Humanoid Robot Acting in a Dynamic Environment.
Frontiers inNeurorobotics , 13, August 2019.[3] Jesus A. Garrido Alcazar, Niceto Rafael Luque, Egidio D‘Angelo, and Eduardo Ros. Distributed cerebellarplasticity implements adaptable gain control in a manipulation task: a closed-loop robotic simulation.
Frontiers inNeural Circuits , 7, 2013.[4] Jacques Kaiser, Alexander Friedrich, J. Camilo Vasquez Tieck, Daniel Reichard, Arne Roennau, Emre Neftci, andRüdiger Dillmann. Embodied neuromorphic vision with event-driven random backpropagation, 2019.[5] Jacques Kaiser, Michael Hoff, Andreas Konle, Juan Camilo Vasquez Tieck, David Kappel, Daniel Reichard,Anand Subramoney, Robert Legenstein, Arne Roennau, Wolfgang Maass, et al. Embodied synaptic plasticity withonline reinforcement learning.
Frontiers in Neurorobotics , 13:81, 2019.[6] Alban Bornet, Jacques Kaiser, Alexander Kroner, Egidio Falotico, Alessandro Ambrosano, Kepa Cantero,Michael H. Herzog, and Gregory Francis. Running Large-Scale Simulations on the Neurorobotics Platform toUnderstand Vision – The Case of Visual Crowding.
Frontiers in Neurorobotics , 13, 2019. Publisher: Frontiers.[7] A. J. Ijspeert, A. Crespi, D. Ryczko, and J.-M. Cabelguen. From Swimming to Walking with a Salamander RobotDriven by a Spinal Cord Model.
Science , 315(5817):1416–1420, March 2007.[8] Zhenshan Bing, Long Cheng, Guang Chen, Florian Röhrbein, Kai Huang, and Alois Knoll. Towards autonomouslocomotion: CPG-based control of smooth 3d slithering gait transition of a snake-like robot.
Bioinspiration &Biomimetics , 12(3):035001, apr 2017.[9] Tony J. Prescott, Fernando M. Montes González, Kevin Gurney, Mark D. Humphries, and Peter Redgrave. Arobot model of the basal ganglia: behavior and intrinsic processing.
Neural Networks: The Official Journal of theInternational Neural Network Society , 19(1):31–61, January 2006.[10] Wolfgang Maass. Networks of spiking neurons: The third generation of neural network models.
Neural Networks ,10(9):1659–1671, December 1997.[11] M. Davies, N. Srinivasa, T. Lin, G. Chinya, Y. Cao, S. H. Choday, G. Dimou, P. Joshi, N. Imam, S. Jain,Y. Liao, C. Lin, A. Lines, R. Liu, D. Mathaikutty, S. McCoy, A. Paul, J. Tse, G. Venkataramanan, Y. Weng,A. Wild, Y. Yang, and H. Wang. Loihi: A neuromorphic manycore processor with on-chip learning.
IEEE Micro ,38(1):82–99, January 2018.[12] F. Akopyan, J. Sawada, A. Cassidy, R. Alvarez-Icaza, J. Arthur, P. Merolla, N. Imam, Y. Nakamura, P. Datta,G. Nam, B. Taba, M. Beakes, B. Brezzo, J. B. Kuang, R. Manohar, W. P. Risk, B. Jackson, and D. S. Modha.Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip.
IEEE Transactionson Computer-Aided Design of Integrated Circuits and Systems , 34(10):1537–1557, Oct 2015.[13] Steve B. Furber, Francesco Galluppi, Steve Temple, and Luis A. Plana. The spinnaker project.
IEEE. Proceedings ,102(5):652–665, 2014.[14] J. Schemmel, D. Briiderle, A. Griibl, M. Hock, K. Meier, and S. Millner. A wafer-scale neuromorphic hardwaresystem for large-scale neural modeling. In
Proceedings of 2010 IEEE International Symposium on Circuits andSystems , pages 1947–1950, May 2010.[15] Paolo Arena. The central pattern generator: A paradigm for artificial locomotion.
Soft Computing , 4:251–266, 012000.[16] Auke Jan Ijspeert. Central pattern generators for locomotion control in animals and robots: A review.
NeuralNetworks , 21(4):642–653, May 2008.[17] Sten Grillner. Biological pattern generation: the cellular and computational logic of networks in motion.
Neuron ,52(5):751–766, December 2006.[18] Sten Grillner. The motor infrastructure: from ion channels to neuronal networks.
Nature Reviews Neuroscience ,4(7):573–586, July 2003.[19] Junzhi Yu, M. Tan, Jian Chen, and Jianwei Zhang. A survey on cpg-inspired control models and systemimplementation.
IEEE transactions on neural networks and learning systems , 25:441–56, 03 2014.[20] A. Crespi and A.J. Ijspeert. Amphibot ii: An amphibious snake robot that crawls and swims using a central patterngenerator.
Proceedings of the 9th International Conference on Climbing and Walking Robots (CLAWAR 2006) , 012006. 23
PREPRINT - J
ANUARY
19, 2021[21] K. Inoue, Shugen Ma, and Chenghua Jin. Neural oscillator network-based controller for meandering locomotionof snake-like robots. In
IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA’04. 2004 , volume 5, pages 5064–5069, April 2004. ISSN: 1050-4729.[22] Elisa Donati, Giacomo Indiveri, and Cesare Stefanini. A novel spiking CPG-based implementation system tocontrol a lamprey robot. In , pages 1364–1364, June 2016. ISSN: 2155-1782.[23] Zhelong Wang, Qin Gao, and Hongyu Zhao. CPG-Inspired Locomotion Control for a Snake Robot Basing onNonlinear Oscillators.
Journal of Intelligent & Robotic Systems , 85(2):209–227, February 2017.[24] Brayan Cuevas-Arteaga, Juan Pedro Dominguez-Morales, Horacio Rostro-Gonzalez, Andres Espinal, AngelJiménez-Fernandez, Francisco Gómez-Rodríguez, and Alejandro Linares-Barranco. A spinnaker application:Design, implementation and validation of scpgs. volume 10305, 06 2017.[25] Alex Russell, Garrick Orchard, and Ralph Etienne-Cummings. Configuring of spiking central pattern generatornetworks for bipedal walking using genetic algorthms. pages 1525 – 1528, 06 2007.[26] M. Lewis, Francesco Tenore, and Ralph Etienne-Cummings. Cpg design using inhibitory networks. volume 2005,pages 3682–3687, 01 2005.[27] Alexandros Bouganis and Murray Shanahan. Training a spiking neural network to control a 4-DoF robotic armbased on Spike Timing-Dependent Plasticity. In
The 2010 International Joint Conference on Neural Networks(IJCNN) , pages 1–8, Barcelona, Spain, July 2010. IEEE.[28] Samir Menon, Sam Fok, Alex Neckar, Oussama Khatib, and Kwabena Boahen. Controlling articulated robotsin task-space with spiking silicon neurons. In , pages 181–186, Sao Paulo, Brazil, August 2014. IEEE.[29] Andres Espinal, Horacio Rostro-Gonzalez, Martin Carpio, Erick I. Guerra-Hernandez, Manuel Ornelas-Rodriguez,and Marco Sotelo-Figueroa. Design of Spiking Central Pattern Generators for Multiple Locomotion Gaits inHexapod Robots by Christiansen Grammar Evolution.
Frontiers in Neurorobotics , 10, 2016.[30] Daniel Gutierrez-Galan, Juan Pedro Dominguez-Morales, Fernando Perez-Pena, and Alejandro Linares-Barranco. NeuroPod: a real-time neuromorphic spiking CPG applied to robotics.
Neurocomputing , pageS0925231219315644, November 2019. arXiv: 1904.11243.[31] Chris Eliasmith and Charles H. Anderson. Rethinking central pattern generators: A general approach.
Neurocom-puting , 32-33:735–740, June 2000.[32] Alessandro Crespi, André Badertscher, André Guignard, and Auke Jan Ijspeert. AmphiBot I: an amphibioussnake-like robot.
Robotics and Autonomous Systems , 50(4):163–175, March 2005.[33] Andrej Bicanski, Dimitri Ryczko, Jérémie Knuesel, Nalin Harischandra, Vanessa Charrier, Örjan Ekeberg,Jean-Marie Cabelguen, and Auke Jan Ijspeert. Decoding the mechanisms of gait generation in salamanders bycombining neurobiology, modeling and robotics.
Biological Cybernetics , 107(5):545–564, October 2013.[34] Wulfram Gerstner and Werner Kistler.
Spiking Neuron Models: An Introduction . Cambridge University Press,New York, NY, USA, 2002.[35] H. Markram, W. Gerstner, and P. J. Sjöström. Spike-Timing-Dependent Plasticity: A Comprehensive Overview.
Frontiers in Synaptic Neuroscience , 4, July 2012.[36] Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and Learning-to-learn in networks of spiking neurons. In S. Bengio, H. Wallach, H. Larochelle,K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors,
Advances in Neural Information Processing Systems 31 ,pages 787–797. Curran Associates, Inc., 2018.[37] Guillaume Bellec, Franz Scherr, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass. Biologicallyinspired alternatives to backpropagation through time for learning in recurrent neural nets. arXiv:1901.09049 [cs] ,February 2019. arXiv: 1901.09049.[38] Trevor Bekolay, James Bergstra, Eric Hunsberger, Travis DeWolf, Terrence C. Stewart, Daniel Rasmussen, XuanChoo, Aaron Russell Voelker, and Chris Eliasmith. Nengo: a Python tool for building large-scale functional brainmodels.
Frontiers in Neuroinformatics , 7, 2014.[39] Chris Eliasmith and Charles Anderson. Neural engineering: Computation, representation, and dynamics inneurobiological systems.
IEEE Transactions on Neural Networks , 15(2):528–529, March 2004.[40] Terrence C. Stewart. A technical overview of the neural engineering framework. Technical report, Centre forTheoretical Neuroscience, 2012. 24
PREPRINT - J
ANUARY
19, 2021[41] Egidio Falotico, Lorenzo Vannucci, Alessandro Ambrosano, Ugo Albanese, Stefan Ulbrich, Juan CamiloVasquez Tieck, Georg Hinkel, Jacques Kaiser, Igor Peric, Oliver Denninger, Nino Cauli, Murat Kirtay, ArneRoennau, Gudrun Klinker, Axel Von Arnim, Luc Guyot, Daniel Peppicelli, Pablo Martínez-Cañada, Eduardo Ros,Patrick Maier, Sandro Weber, Manuel Huber, David Plecher, Florian Röhrbein, Stefan Deser, Alina Roitberg,Patrick van der Smagt, Rüdiger Dillman, Paul Levi, Cecilia Laschi, Alois C. Knoll, and Marc-Oliver Gewaltig.Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform.
Frontiers in Neurorobotics , 11, 2017.[42] Morgan Quigley, Brian Gerkey, Ken Conley, Josh Faust, Tully Foote, Jeremy Leibs, Eric Berger, Rob Wheeler,and Andrew Ng. Ros: an open-source robot operating system. In
Proc. of the IEEE Intl. Conf. on Robotics andAutomation (ICRA) Workshop on Open Source Robotics , Kobe, Japan, May 2009.[43] N. Koenig and A. Howard. Design and use paradigms for gazebo, an open-source multi-robot simulator. In ,volume 3, pages 2149–2154 vol.3, 2004.[44] Marc-Oliver Gewaltig and Markus Diesmann. Nest (neural simulation tool).
Scholarpedia , 2(4):1430, 2007.[45] Örjan Ekeberg. A combined neuronal and mechanical model of fish swimming.
Biological Cybernetics , 69(5):363–374, October 1993.[46] Hongyu An, Kangjun Bai, and Yang Yi.
The Roadmap to Realizing Memristive Three-dimensional NeuromorphicComputing System . November 2018.[47] Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3B: System Programming Guide, Part2. page 582.[48] Rui Pereira, Marco Couto, Francisco Ribeiro, Rui Rua, Jácome Cunha, João Paulo Fernandes, and João Saraiva.Energy efficiency across programming languages: how do energy, time, and memory relate? In
Proceedings of the10th ACM SIGPLAN International Conference on Software Language Engineering , pages 256–267, VancouverBC Canada, October 2017. ACM.[49] Sacha J. van Albada, Andrew G. Rowley, Johanna Senk, Michael Hopkins, Maximilian Schmidt, Alan B. Stokes,David R. Lester, Markus Diesmann, and Steve B. Furber. Performance Comparison of the Digital NeuromorphicHardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical MicrocircuitModel.
Frontiers in Neuroscience , 12, 2018. Publisher: Frontiers.[50] Evangelos Stromatias, Francesco Galluppi, Cameron Patterson, and Steve Furber. Power analysis of large-scale,real-time neural networks on SpiNNaker. In
The 2013 International Joint Conference on Neural Networks(IJCNN) , pages 1–8, August 2013. ISSN: 2161-4407.[51] Peter Blouw, Xuan Choo, Eric Hunsberger, and C. Eliasmith. Benchmarking Keyword Spotting Efficiency onNeuromorphic Hardware.