Control for Multifunctionality: Bioinspired Control Based on Feeding in Aplysia californica
Victoria A. Webster-Wood, Jeffrey P. Gill, Peter J. Thomas, Hillel J. Chiel
NNoname manuscript No. (will be inserted by the editor)
Control for Multifunctionality
Bioinspired Control Based on Feeding in
Aplysia californica
Victoria A. Webster-Wood · Jeffrey P. Gill · Peter J. Thomas · Hillel J. Chiel
Received: date / Accepted: date
Abstract
Animals exhibit remarkable feats of behavioral flexibility and mul-tifunctional control that remain challenging for robotic systems. The neuraland morphological basis of multifunctionality in animals can provide a sourceof bio-inspiration for robotic controllers. However, many existing approaches tomodeling biological neural networks rely on computationally expensive modelsand tend to focus solely on the nervous system, often neglecting the biome-chanics of the periphery. As a consequence, while these models are excellenttools for neuroscience, they fail to predict functional behavior in real time,which is a critical capability for robotic control. To meet the need for real-timemultifunctional control, we have developed a hybrid Boolean model frameworkcapable of modeling neural bursting activity and simple biomechanics at speedsfaster than real time. Using this approach, we present a multifunctional modelof
Aplysia californica feeding that qualitatively reproduces three key feedingbehaviors (biting, swallowing, and rejection), demonstrates behavioral switch-ing in response to external sensory cues, and incorporates both known neural
V.A. Webster-WoodDepartment of Mechanical Engineering; Department of Biomedical Engineering; McGowanInstitute for Regenerative MedicineCarnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA, 15213E-mail: [email protected]. GillDepartment of BiologyCase Western Reserve University, 2080 Adelbert Road, Cleveland, OH 44106-7080P.J. ThomasDepartment of Mathematics, Applied Mathematics and Statistics; Department of BiologyDepartment of Cognitive Science; Department of Electrical, Computer and Systems Engi-neeringCase Western Reserve University, 10900 Euclid Avenue, Cleveland, Ohio 44106-4901H.J. ChielDepartment of Biology; Department of Neurosciences; Department of Biomedical Engineer-ingCase Western Reserve University, 2080 Adelbert Road, Cleveland, OH 44106-7080 a r X i v : . [ q - b i o . N C ] A ug Victoria A. Webster-Wood et al. connectivity and a simple bioinspired mechanical model of the feeding appa-ratus. We demonstrate that the model can be used for formulating testablehypotheses and discuss the implications of this approach for robotic controland neuroscience.
Multifunctionality, a basis for behavioral flexibility, is critical for navigatingand adapting to a complex changing environment. In animals as well as hu-mans, multifunctionality is observed across a wide range of behaviors. Livingsystems must smoothly shift from one behavior to another while varying spe-cific behaviors to handle changing environmental conditions. Even relativelysimple organisms demonstrate multifunctional control. For example, grass-cutter ants use their mandibles to cut stalks of grass, carry them to the nestand manipulate them once in their nests [118]; frogs exhibit swimming, walk-ing, and hopping gaits [133,132]. The tremendous range and adaptability ofcontrol is observed even more strongly in human manipulation: humans can usetheir hands to lift barbells substantially heavier than their own body weight,but also use the very same hands to play complex piano concertos.Despite the obvious importance of multifunctionality for animal systems,truly multifunctional control remains a challenge for robotics [120]. To de-velop robotic controllers for multifunctional behavior, one possible approachwould be to develop a methodology that can map multifunctional biologicalsystems onto simulated devices or robots. Such a methodology would enableresearchers to develop control architectures through rapid prototyping andsimulation. The controllers could then be effectively improved by comparisonto the original biological system, and by assessing their effectiveness as a simu-lated controller for an artificial device. Including known neurons, connections,and biomechanics underlying multifunctional behavior allows the models toimmediately suggest testable experimental hypotheses, clarifying the biolog-ical mechanisms of multifunctionality. At the same time, to make the simu-lation useful for artificial or robotic devices, the modeling framework shouldrun faster than real-time. A computationally efficient, biologically relevantframework could then lead to direct real-time control of the original biologicalsystem and of an artificial robotic system, and thus provide a bridge fromneuroscience to robotics.What mediates multifunctional behavior in biological nervous systems, andwhat can we learn from them for robotic control? Three alternative neural ar-chitectures have been proposed for multifunctional control: dedicated controlcircuits, population-based control circuits, and re-organizing control circuits[100]. The first alternative dedicates a control circuit to each behavioral func-tion. For example, an escape circuit might suppress and override a swimmingcircuit [61]. Functionally decomposing behavior, and assigning dedicated con-trol to each function, has historically been the traditional approach to roboticcontrol, such as in traditional finite state machines [93,122,103,116]. The draw- ontrol for Multifunctionality 3 back is that controlling a wide repertoire of behaviors can lead to a combina-torial explosion, making this approach impractical in general, and it is clearlynot used for most animal behaviors. A second alternative is encoding solutionsthrough the activity of a neuronal population. For example, the direction of amotor response may be encoded by a broadly tuned population of neurons [44,45,123]. Population encoding is the basis of many machine learning approachesto robotic control [126]. A drawback of this solution is that it is difficult toisolate subnetworks with specific functionalities, so it can be difficult to under-stand how the system works. A third possibility, which appears to be a morecommon solution in biological systems [100], is that of reorganizing circuits: byvarying the timing and phasing of activity and incorporating feedback fromthe periphery (body), single circuits can be reconfigured to produce severalmultifunctional behaviors. For example, the same multifunctional circuit incrustacea can be reconfigured to generate qualitatively different ingestive be-haviors [124,125]. Despite increasing evidence that this third alternative maybe the most common for biological control, relatively few robotic control ar-chitectures are based on this solution.How can one effectively implement any of the three neural architecturesfor multifunctionality? One possibility is to use machine learning to allow thecontroller to “learn” a multi-functional network architecture. Machine learn-ing has led to many applications and predictive modeling approaches by re-lying on large training datasets and intense computational power [55,70,153,60,50,146,135,1]. Since the relationship between network form and functionis often very complex, it has not been easy to understand how the resultingnetworks actually function, or to use them to direct experimental analyses ofan actual biological system. Thus, another approach has been to develop de-tailed models of individual neurons and networks based on actual experimentalmeasurements; the detailed dynamics of individual neurons can be approxi-mated using multiconductance, multicompartment biophysical models [58,40,81]. The drawback of this approach is that large numbers of parameters mustbe set experimentally, and given the variability within nervous systems, theresulting network may not capture the original dynamics of the system [91,51,8,115]. A potential third way has been to use more phenomenological neuralmodels to capture aspects of neural architecture and dynamics with a greatlyreduced set of parameters, and these have been successfully used for biologicalmodeling and control [114] including those inspired by insects [141,142,14,10,9], lobsters [2,3,4],
Pleurobranchaea [13], lampreys [114,15,77] and fish [39],salamanders [11,52], and other tetrapods [62,63].Possible nominal model approaches to capture neural circuit dynamics in-clude the use of integrate-and-fire neurons [69,96,143], rate models [151], dis-crete asynchronous event-based models [6], and at the simplest level, Booleanmodels or finite automata [2,119,122]. Integrate-and-fire nodes have been suc-cessfully implemented in synthetic nervous systems using neuron pool circuitmodels for robotic control [141,62,63,142]. However, the complexity of theanimal models used for bio-inspiration precludes the possibility of capturingfull circuit connectivity or individual identifiable neurons. Population firing
Victoria A. Webster-Wood et al. rate models are often used to represent neural activity for therapeutic brain-machine interface technologies development [106,87,85]. Firing-rate models ofneural networks go back at least to the Wilson-Cowan equations [151,150,35],and have helped understand neural behaviors as diverse as spontaneous pat-tern formation [41], processing of sensory input signals [7,12,152], and motorcontrol [129,56,130,135,20,8]. Boolean network models, being closely relatedto finite state machines, originated with the seminal study by McCulloch andPitts [93] and have found application in robotics and as reduced models of bio-logical systems such as gene regulatory and signal transduction networks [108,34,110,121,46,37,80,109]. Of these nominal models, Boolean network modelslikely provide the lowest computational cost, while still capturing the overallon/off behavior of neuronal bursting. Such models have been used to describeneural activity recorded during multifunctional behaviors [2], and are the foun-dation of finite automata [93]. Boolean network models can be used to capturea wide range of biological phenomena [108,110,33,53,131] and can even be ex-tended to capture stochastic processes [117].In many neural models, the focus is on the network controller, withoutaccounting for the dynamics of the periphery, or body. For applications inbioinspired robotic control, a computational modeling approach is needed thatcaptures both the dynamics of the neural circuitry and the critical interactionsbetween the brain, the body, and the environment [19,23]. As with neuralcomponents, a variety of models have been developed to capture the nonlinearproperties of individual muscles, and their organization into muscular struc-tures [159]. The complexity of such muscle models can vary from capturingmuscle biochemical kinetics using a cross-bridge model [156,38,54,113,158] tospring-damper representations such as used in the linear Hill muscle model[57,127]. Such models can be fit to match muscle physiology observed in ani-mal systems [157], and used to model complex musculature such as muscularhydrostats [21]. Fundamentally, the role of these muscle models is to capturethe integration of muscle activation dynamics into a resulting tension. Onceagain, although these muscle models are important for modeling mechani-cal systems, complex structures involving both the kinematics and kinetics ofmultiple muscles, in general, will have high computational overhead [138,139,107]. Thus, if a simulation is to run faster than real time, one must use moresimplified models.To meet the need for computationally efficient, explainable, multifunctionalcontrollers, we have developed a hybrid Boolean network model framework,i.e. primarily using Boolean network elements but using continuous mechanicalmodels. This framework combines discrete Boolean logic calculations of neu-ral activity with simplified semi-continuous second-order muscle dynamics andperipheral mechanics. To our knowledge, mixed Boolean (neural) / continuous(biomechanical) models have not previously been used for motor control. Theuse of Boolean logic for capturing neural activity results in a computationallyefficient control algorithm that can run faster than real-time. The use of a sim-plified model of the peripheral mechanics provides sufficient sensory feedbackfor the controller to adjust to changing environmental conditions, and allows ontrol for Multifunctionality 5 key characteristics of each of the multifunctional behaviors to be observed. Todemonstrate mapping from a known multifunctional biological system, an an-imal model is needed with a relatively small neural network controlling a well-understood musculature. Therefore, we demonstrate this model framework formultifunctional control using the experimentally tractable model system offeeding in the marine mollusk
Aplysia californica . The resulting model con-trols a simplified mechanical model of the feeding apparatus and successfullydemonstrates ingestive behaviors, including biting and swallowing, as well asrejection of inedible materials. In this paper, we will first describe prior modelsof the
Aplysia feeding neural circuitry and periphery, then describe our novelBoolean model framework. We will demonstrate how the hybrid Boolean con-troller is developed based on observations from behavioral, biomechanical, andelectrophysiological experiments and the existing literature. Finally, we will usethe resulting model to show multifunctional control, and illustrate how it canbe used to make testable experimental predictions.
Aplysia
Feeding and Neural Circuitry
Feeding behavior in
Aplysia is multifunctional and has been well character-ized. Three key feeding behaviors are observed in the intact animal: biting,swallowing, and rejection. Animals flexibly switch between behaviors as sen-sory inputs vary, e.g., switching from biting to swallowing once food (seaweed)is successfully grasped. Moreover, as the animal encounters seaweeds that im-pose varying mechanical loads, the animal may robustly adjust the magnitudeand duration of force it exerts to ingest the seaweed [89,47]. These multifunc-tional behaviors provide a model system for intelligent robotic grasper control.Previous models of the neural circuitry and peripheral mechanics have beenreported that form the foundation for the hybrid Boolean model presentedhere, and we will briefly review the relevant aspects of these previous models.2.1 Prior Neural Circuit Analyses and ModelsThere is a wealth of information on the circuitry controlling feeding in theexperimentally tractable
Aplysia nervous system. The tractability is a resultof several factors: there are relatively few neurons responsible for feeding be-havior (on the order of 200 motor neurons and dozens of key interneurons[17,137]); neurons in
Aplysia are large, pigmented, and have similar synapticinputs, outputs, morphology and biophysical properties from one animal tothe next, and can thus be identified as unique individuals [78]; the somata,which are the largest parts of the neurons, are electrically excitable and elec-trically compact, so that stimulating or inhibiting the neuron at the somacontrols its outputs [27]. Together, these features make it possible to deter-mine detailed neural circuitry that applies across all animals. In particular,the neural circuitry involved in
Aplysia feeding has been extensively studied
Victoria A. Webster-Wood et al. [29]. Two ganglia contain the primary neurons responsible for generating therelevant multifunctional behavior: the cerebral and buccal ganglia. The buccalganglion contains many of the primary motor neurons that innervate the mus-culature of the feeding apparatus, as well as interneurons and sensory neuronsinvolved in feeding [29]. The cerebral ganglion is the primary locus for manykey interneurons responsible for guiding behavioral switching [29]. Many ofthe neurons of the feeding circuitry can be consistently identified between an-imals due to their location, size, and electrical characteristics [149,66,22,136,76,144,24]. In particular, many of the motor neurons innervating key mus-cles have been previously identified, including B3/B6/B9 innervation of the I3retractor muscle [26,25], B31/B32/B61/B62 innervation of the I2 protractormuscle [64], B7 innervation of the hinge retractor [154], B8a/b innervation ofthe grasper [102,25,42], and B38 activation of the anterior region of the I3retractor muscle [26]. The coordination of these motor outputs is mediatedvia many known interneurons both in the buccal and cerebral ganglia [29].The neural circuitry controlling feeding in isolation from the musculaturethat mediates feeding has been modeled in detail. Cataldo et al. [18] developeda network model with Hodgkin-Huxley-type neurons incorporating known dataon conductances and the roles of important second messengers in individualidentified neurons mediating feeding behavior (using the SNNAP modelingplatform [160]) which has been recently updated by Costa et al. [28]. Thismodel includes key motor neurons and interneurons in the buccal ganglia.Using this approach, they were able to generate ingestion-like and rejection-like neural activity. However, the model did not take into account the roleof cerebral-buccal interneurons (CBIs) in switching between the two ingestivebehaviors, and thus could not differentiate between bite-like and swallow-likepatterns, nor did it provide control of a simulated periphery, and thus couldnot incorporate sensory feedback during feeding, which we have argued mayplay a critical role in generating robust feeding behavior [89,128].2.2 Prior Mechanical ModelsWhile many studies have investigated the neural circuitry underlying
Aplysia feeding behavior, and some have developed detailed models of that circuitry,fewer have considered the critical role of the peripheral biomechanics on thecontrol architecture and behavior. However, the parallel evolution of the pe-ripheral musculature and control circuitry result in a tightly coupled system[19]. To understand and create multifunctional controllers, we must understandthe interactions of the complete system.Kinematic and kinetic models of the
Aplysia feeding apparatus have pre-viously been reported in the literature. Based on MRI images during feeding,kinematic models have been developed to capture the mechanics of feeding[104,36]. These models highlight the morphological computation inherent inthe feeding apparatus. In particular, the kinematic changes observed duringfeeding reveal how shape changes in the grasper can change the mechanical ontrol for Multifunctionality 7 advantage of key muscles [107,138]. In addition to kinematic models, basic ki-netic models have been proposed which capture the dynamics of key structuresthroughout feeding [138]. Such models can be extended through the inclusionof kinematic reconfiguration observed through MRI imaging [107]. However,the existing mechanical models do not yet include the neural circuitry neededfor controller development.2.3 Prior Neuromechanical ModelsAbstract neuromechanical models, which combine neural control and biome-chanics into a unified model of
Aplysia feeding, have also be developed, such asa stable heteroclinic channel model [128,89]. This model captures the CPG-like activity of the feeding circuitry using three mutually inhibitory nodesrepresenting pools of motor neurons. Though the nodes do not map preciselyto known neural connectivity, their dynamics can be simulated rapidly, con-nected to basic kinematic models of the periphery, and respond to changes insensory inputs, such as the load on the seaweed. Furthermore, stable hetero-clinic channel controllers have been successfully translated to robotic applica-tions [59,148]. However, such models do not provide insight into the detailedneural mechanisms underlying multifunctional control.
Our approach to modeling begins with experimental observations from intactanimals of both their feeding behavior and recordings of the major motor neu-ronal activity controlling feeding. These observations of the functional outputsof the system motivated an outside-to-inside modeling approach: first, a mini-mal set of peripheral structures and muscles are represented; second, the directcontrollers of those muscles (motor neurons) are added; finally, layers of localand ultimately global control mediated by interneurons are added.3.1 Experimental Methods and Data AnalysisThe activity patterns of identified neurons during distinct feeding behaviorswere obtained experimentally from intact animals via chronically implantedelectrodes. Materials and procedures are described in detail by [47] and aresummarized here.Adult
Aplysia californica (200–450 g) were anesthetized via injection ofisotonic magnesium chloride solution (333 mM) and immersion in chilled arti-ficial sea water (1-5 ◦ C) for at least 10 minutes. A small incision in the bodywall near the head was made which permitted access to the feeding appara-tus (buccal mass). Differential electrodes, comprised of twisted pairs of fine(25- µ m diameter), insulated stainless steel wires (see [30] for fabrication de-tails), were implanted on the protractor muscle I2, the radular nerve (RN), Victoria A. Webster-Wood et al. and buccal nerves 2 (BN2) and 3 (BN3). Together these recording sites per-mitted extracellular monitoring of nearly all of the major motor neurons of thecircuitry controlling feeding (I2: B31/B32/B61/B62; RN: B8a/b; BN2: B38,B6/B9, B3; BN3: B7), as well as an important pair of multiaction interneurons(BN3: B4/B5) [88]. The incision was closed with a suture. Animals recoveredafter 1-3 days.Instrumented animals were presented with different food stimuli to elicitdifferent feeding responses. To elicit bites, which are failed attempts to graspfood [84], dried nori (Deluxe Sushi-Nori, nagai roasted seaweed, Nagai Nori,USA INC, Torrance, Ca) was touched to the rhinophores, anterior tentacles,and perioral zone until protractions of the feeding grasper were visible. Toelicit swallows, animals were permitted to grasp and ingest the food. To elicitrejections, animals were first enticed to partially swallow a polyethylene tubeby simultaneously touching nori to the perioral zone; after several centimetersof tubing were swallowed, the nori was removed, and eventually the animalrejected the tube by pushing it out of the mouth using its grasper.For some swallows, an unbreakable food stimulus (double-sided tape be-tween two uniform strips of dried nori) was anchored to a force transducerand suspended vertically over the animal. The animal attempted to swallowthe strip, but because it was anchored and unbreakable it could only makeprogress until tension began to develop in the anchored strip. After this, theanimal continued to attempt to swallow despite the increase in load for up toseveral minutes.An electromyogram from the protractor muscle I2 and extracellular nervesignals from RN, BN2, and BN3 were digitally recorded, along with swallowingforce measured by the force transducer. Video was captured simultaneously sothat behaviors could be reviewed during analysis.Analysis of experimental data was aided by the Python package neurotic (NEURoscience Tool for Interactive Characterization) [48], and analysis pro-cedures were similar to those described by [47]. Briefly, spikes were detectedusing window discriminators. Units corresponding to identified neurons canbe identified from nerve recordings because axonal nerve projections and therelative amplitude and timing of spikes is consistent from animal to animal[88]. Amplitude thresholds were determined manually. Spikes were groupedinto bursts using firing frequency criteria (see [47] for details; for B7, the burstinitiation and termination frequencies were 20 Hz and 10 Hz, respectively,based on observations by [154,155]). Video was used to determine the tim-ing of inward movement of food during swallowing and outward movement oftubing during rejection.3.2 Simplified Model Framework for Multifunctional ControlTo develop a simplified model of multifunctional control, we employed a demand-driven complexity approach: rather than modeling the complex dynamics ofall possible units in the neural network, and the detailed biomechanics, we ontrol for Multifunctionality 9 identified key neuronal elements based on functional outputs during behav-ior, modeled minimal associated peripheral mechanics, and refined both mod-els to reproduce multifunctional behaviors. The result is a hybrid Booleanmodel consisting of the peripheral biomechanics and neural circuitry. Neuralactivity is represented using discrete Boolean units, whereas the biomechanicsare calculated continuously in space using a semi-implicit integration scheme(see Appendix A.1). In the following sections, we present the proposed hybridBoolean model framework applied to the multifunctional feeding behavior of
Aplysia .3.3 System Identification Based on Key Biomechanical and Neural Elements
Aplysia’s feeding behavior is multifunctional. As an animal attempts to ingestfood, it bites (a failed grasp); once it succeeds in grasping food, it pulls it intothe buccal cavity (i.e., it swallows ). If it encounters inedible material, it pushesit out of the buccal cavity (i.e., it rejects food). The animal must continuouslyshift flexibly among these different behaviors as it encounters the changingproperties of food (e.g., mechanical load, toughness and texture). Based onthe known neural circuitry in the buccal ganglia and our recordings of each ofthe three feeding behaviors in intact behaving animals, we identified criticalmotor neurons and musculature necessary to reproduce multifunctional feedingin simulation.
Biting: In Aplysia , biting is characterized by strong protraction of the grasper,which closes prior to peak protraction as it attempts to grasp food, followedby weak retraction when food is not grasped (Figure 1.A1). As grasping at-tempts are unsuccessful in this behavior, no force is applied to the seaweed.Key muscles and motor neurons involved in this behavior include the protrac-tor muscle I2 and its associated motor neurons B31/B32 and B61/B62 [64],the grasper closer muscle I4 and its motor neurons B8a/b [102,25,42], and to alesser extent the jaw closer muscle I3 and its motor neurons B6/B9 [25]. In theexperimental data shown in Figure 1.A2, the very limited B6/B9 activity isprobably insufficient to mediate the level of retraction observed in previouslyreported magnetic resonance imaging data [105]. It is therefore likely that ad-ditional muscle units are required for the onset of retraction. Indeed, previousbiomechanical models suggest that the hinge muscle, which is activated byneuron B7 [154], plays a critical role in retraction during biting [138]. As aresult of this analysis, the demand-driven model should incorporate four mus-cle groups (I3, I2, grasper closure, and hinge) and four neural groups (B6/B9,B31/B32, B8a/b, and B7) to produce biting. Swallowing:
If seaweed is successfully grasped at peak protraction during abite, swallowing is initiated (Figure 1.B1). During swallowing, the grasperstrongly retracts while closed on the seaweed. To re-position the grasper topull more seaweed inwards, it is then weakly protracted while open. If it were
Fig. 1
Biting, swallowing, and rejection have distinct functional, kinematic, and neuralcontrol characteristics. A1 Biting is illustrated schematically in a sequence of cross-sectionsof the feeding apparatus. Biting begins with strong protraction of the open grasper towardsthe jaws (to the right), mediated by the protractor muscle I2 and the motor neurons B31/B32and B61/B62; the grasper closes (indicated by a shape change from circular to elliptic)near the peak of protraction through the action of closure motor neurons B8a/b; havingfailed to grasp food, the closed grasper retracts weakly towards the esophagus (to the left),mediated by activation of the hinge muscle via B7 and the jaw muscle I3 via B6/B9. A2 Outputs of the motor control system (muscle and nerve activity) were recorded during biting,allowing timing of identified motor neuron activity to be determined. An understandingof the biomechanics (A1) and the neural control permits mapping the motor pattern tothe kinematic sequence (circled numbers). Colored boxes around spikes indicate bursts ofactivity sufficiently intense to elicit functional movements. Bars indicate the protraction (P)and retraction (R) phases. A3 A simplified, discrete representation of the bursts of motorneuronal activity in A2. In this column, the I2 motor pool is abbreviated to “B31/B32”for brevity. B1 Swallowing begins with pinching the anterior jaws, mediated by the motorneuron B38, to prevent loss of food while the open grasper protracts; protraction is weakerthan in biting; the grasper closes; the closed grasper retracts strongly to deliver food to theesophagus through recruitment of the jaw motor neuron B3, as well as intensified activationof B6/B9 and B7. B2 The motor pattern contains indications of each kinematic differencebetween biting and swallowing. Swallowing force and time of inward movement of foodare also indicated. The multi-action interneurons B4/B5 are also active at a moderate levelduring swallowing and may act to delay the jaw motor neurons. B3 A discrete representationof the motor neuronal activity in B2, with B4/B5 active at a moderate level (dashed line). C1 Rejection begins with closing of the grasper; the closed grasper strongly protracts, expellinginedible material; the jaws are delayed from closing, giving the grasper enough time to open(indicated by a shape change from elliptic to circular) so that food will not be pulled backin during retraction; the open grasper retracts. C2 An essential difference between rejectionand the ingestive behaviors is the timing of grasper closure, seen in the motor pattern asB8a/b activity during protraction. B4/B5 is very intensely activated during rejections andis responsible for the delay in jaw closure. Outward movement of the inedible material isalso indicated. C3 In the discrete representation of C2, B4/B5 intensity is elevated relativeto swallowing. Note that motor patterns (A2, B2, C2) are plotted on identical time scalesto emphasize differences in duration; discrete representations (A3, B3, C3) are rescaled fordirect comparison of burst phasing. B1 and B2 are modified from [47] with permission.ontrol for Multifunctionality 11 protracted too strongly, it might push seaweed out. Thus, during the retractionphase, the animal exerts strong forces on seaweed, whereas during protraction,it exerts minimal or even slightly negative forces. Similar muscle groups areactivated in swallowing as are in biting, but with changes in duration andintensity. In addition, to prevent seaweed from slipping out, the anterior regionof the I3 jaw muscle is pinched closed by activating the B38 motor neuron[95]. The changes in motor neuronal timing (Figure 1.B2) can be understoodfrom the biomechanics: First, to ensure that seaweed is not pushed out duringprotraction, the activation of the grasper closure motor neurons B8a/b occursnear the end of protraction (rather than overlapping the end of protraction, asis observed during biting). Second, to ensure that protraction is weaker, theprotractor muscle I2 is less strongly activated than in biting. Third, to ensurethat the grasper releases near the end of retraction, the grasper motor neuronsB8a/b and the jaw motor neurons B6/B9 cease activity at about the sametime. Fourth, the major jaw motor neuron B3 is recruited to generate greaterretraction force. Finally, to maintain a hold on seaweed after the grasper opens,the B38 motor neuron is activated during the protraction phase to pinch theanterior of the jaw muscle onto seaweed.
Rejection:
If an inedible object is detected as a result of the combined sen-sory cues in the esophagus (e.g., a noxious mechanical stimulus), grasper (alack of chemical stimulus), and at the lips (a lack of chemical stimulus), theinedible material will be rejected. This is a critical behavior for the animal, asit must be able to free the buccal cavity of inedible material in order to locateedible food.
Rejection is characterized by strong protraction with the grasperclosed, followed by retraction with the grasper open (Figure 1.C1). Similar toswallowing and biting, the I3 muscle, the I2 muscle, and the I4 muscle are allactivated. However, the timing changes (Figure 1.C2): First, the grasper closermotor neurons B8a/b are activated during the protraction phase (i.e., duringactivation of the I2 protractor muscle and the B31/B32/B61/B62 motor neu-rons), rather than during the retraction phase, ensuring that the grasper closesand pushes out inedible material [101,102]. Second, since the inedible mate-rial is not retained during the protraction phase, the B38 motor neuron isnot activated, and no pinch is observed. Finally, since it is critical to retractthe grasper with its halves open (so as not to pull inedible material back in),the jaw motor neurons (B6/B9/B3) are initially inhibited at the onset of re-traction by the B4/B5 multiaction neurons (since closure of the jaw muscleswould push the grasper halves shut); instead, initial retraction is mediated bythe hinge muscle (activated by motor neuron B7) [155].This analysis of the animal data allows us to identify the key muscles andmotor neurons necessary to produce the multifunctional behaviors of interest(see Table 1) and allows us to develop a simplified biomechanical model of theperiphery to integrate into our controller model.
Table 1
Key muscle and motor neurons included in the hybrid Boolean network model ofthe
Aplysia feeding apparatus.
Muscle Role Motor neurons References
I2 protraction B31/B32/B61/B62 [64]I3 retraction; pinch B3/B6/B9; B38 [26,25,95]I4 grasper closure B8a/b [102,25,42]hinge retraction B7 [154]
Table 2
Key interneurons in both the cerebral and buccal ganglia included in the hybridBoolean network model of the
Aplysia feeding apparatus.
Neuron Primary behaviors References
CBI-2 biting and rejection [71]CBI-3 biting and swallowing [73,98,72]CBI-4 swallowing and rejection [71]B64 protraction-to-retraction transition [66]B4/B5 rejection [73]B20 rejection [73,74]B40 biting [74,71]B30 swallowing [71]
Aplysia , there are several key musclegroups that contribute to feeding behavior. Protraction of the grasper is pri-marily mediated by the I2 muscle, innervated by neurons B31/B32, whichhave both interneuronal and motor neuronal functions, and by motor neuronsB61/B62 [64]. The motor neurons B8a/b activate the I4 muscle which resultsin closing of the grasper and pressure on the seaweed [102]; in strong swallows,in which the grasper is more protracted, grasper closure also induces a retrac-tion force at the onset of grasping [154]. Retraction is primarily mediated bythe activity of the I3 muscle; in addition, when the grasper is very strongly pro-tracted, the hinge contributes to retraction during biting and rejection [138].Additionally, during swallowing, the anterior region of the I3 muscle tightensdown on seaweed to prevent its release and expulsion during the protractionphase when the grasper is open [95]; we will refer to this action as a pinch.These key muscle groups provide the foundation for the biomechanical model(Figure 2.A-C).Using these muscle groups, we derived a simplified model of the feedingapparatus that captures the basic mechanics of the head, grasper, and seaweedalong a one-dimensional axis. In our model, mechanically tough seaweed isfirmly affixed to a force transducer as described in Section 3.1 (Figure 2). Inthis preparation, the mechanically tough seaweed is unable to move relative tothe force transducer during swallowing so long as the seaweed is unbroken. As aconsequence, rather than the animal being stationary and pulling the seaweedinto the esophagus, the animal grasps the seaweed and pulls its head forwardalong the seaweed, so that seaweed moves into the head during the retraction ontrol for Multifunctionality 13
Fig. 2
A novel biomechanical model of the feeding system. A A schematic showing theposition of the feeding apparatus (black) within the body of
Aplysia (red). B A sketch ofthe buccal mass, drawn by Dr. Richard F. Drushel, with a cut-away to reveal internal mus-culature, reproduced with permission from [139]. C A planar schematic of the buccal massshowing the I2, I3, I4, and hinge muscles, and the position of seaweed during ingestion. D A 2D schematic representation (top) of the 1D biomechanical model (bottom) implementedin the hybrid Boolean network framework. This model does not account for the masses orshapes of any of the components. All positions and forces are constrained to the x-direction.The effect of the body and neck on the head is represented by the spring constant K h wherethe reference position of the spring x corresponds to the ground plane (i.e. x = 0). Thepresence and fixation of seaweed to a force transducer varies based on the behavior beingsimulated. During swallowing, a force threshold determines whether the seaweed is fixed tothe force transducer, or breaks away. This allows the mechanical strength of the seaweed tobe varied. E All possible forces on the head (top) and grasper (bottom). To model interac-tions of the seaweed with the jaws (yellow circles, top) and with the grasper (magenta circle,bottom), the friction forces between the seaweed and the relevant jaws ( F f,h ) and grasper( F f,g ) are calculated, based on the pressure at the location and user-specified coefficients offriction (see Appendix A.5). phase but may then move out again as the animal releases the seaweed undertension (Figure 3.B). Thus, in this simplified biomechanical model, the forcesand motion of the grasper vary with the type of behavior (Figure 3 A-C) anddepend on the friction exerted by the grasper on the seaweed, the frictionexerted by the jaws (anterior portion of the I3 muscle) on the seaweed, andthe mechanical strength of the seaweed. For this simplified model the massesof the bodies are neglected due to the quasi-static nature of Aplysia feedingmuscle movements wherein the inertial forces are low relative to the viscousand elastic forces.Muscle forces are determined by a first-order relationship between normal-ized motor neuron activity, N , and normalized muscle activation, A , and afirst-order relationship between activation and normalized tension, T ; numeri- Fig. 3
Demonstration of one cycle for each of the multifunctional behaviors. Bold arrowsat the bottom of each schematic indicate changes in grasper position from one behavioralphase to the next, whereas bold arrows at the top indicate changes in head position. A A schematic representation of the simplified model during biting. No seaweed or tube ispresent and only the grasper moves throughout the cycle. B A schematic representation ofthe simplified model during swallowing. Seaweed is present and fixed to a force transducer.Of particular note is the motion of the head during swallowing. Because the seaweed is fixedto the rigid force transducer, the activation of the I3 muscle when the seaweed is being firmlygrasped results in the head being pulled forwards along the seaweed (B, Retraction panel),so long as the force on the seaweed does not exceed the force threshold. C A schematicrepresentation of the simplified model during rejection. Rather than seaweed, a tube issimulated to provide mechanical stimulation without any chemical cues. The tube is notfixed to an external object and is therefore free to be pushed forward during the rejection.Note the outward movement of the marks on the tube after the rejection cycle concludes (C,Rest panel). For both biting and rejection there is a slight forward motion of the grasperfrom the fully retracted position to the rest configuration shown here.ontrol for Multifunctionality 15 cally, this is implemented using a first-order accurate semi-implicit integrationscheme based on operator splitting (see Appendix A.1) which makes it rela-tively easy to add new components to the control network: dAdt = N ( t ) − A ( t ) τ m −→ A ( t + h ) = τ m A ( t ) + hN ( t ) τ m + h (1) dTdt = A ( t ) − T ( t ) τ m −→ T ( t + h ) = τ m T ( t ) + hA ( t ) τ m + h (2)where A is the muscle activation, τ m is the activation time constant of the givenmuscle, N is the activity of the innervating motor neuron, T is muscle tension,and h is the time step. Similar equations can be used to express the grasper andpinch pressures. Muscle tensions are combined with equations for normalizedmechanical advantage and a maximum force parameter in units of force tocalculate applied force on the grasper, head, and food objects. Activation-Tension, Activation-Pressure, Tension-Force, and Pressure-Force relationshipsfor each muscle as appropriate can be found in Appendices A.4 and A.5.The subsequent motion of the grasper and head are calculated based on thecontributions of individual muscles and the friction applied to the seaweed bythe grasper and jaws. In the absence of external forces, the motions of the headfall between x = 0 (i.e., rest) and 1 (i.e., full extension of the head, Figure2.D). Similarly, the grasper motion falls between 0 (i.e., full retraction), and1 (i.e., full protraction).The motions of the head and grasper are calculated based on quasi-staticequations of motion. In a system with inertial, viscous, elastic, and externalforces, the equations of motion can be expressed in the form: F − kx − c ˙ x = m ¨ x (3)In Aplysia feeding, accelerations and masses are small, so inertial forces arenegligible [139]. Therefore the equations of motion simplify to have the form: F − kx = c ˙ x (4)which can be written as: ˙ x = (cid:80) Fc (5)Therefore, the motions of the head and grasper are calculated as: ddt x g x h = F g c g F h c h (6)where x g and x h are the positions, F g and F h are the net forces, and c g and c h are viscous damping coefficients for the grasper and head, respectively. Forconvenience we set c g = c h = 1. For this model, the total force on the grasper ( F g ) includes the forces dueto contraction of the I2 muscle ( F I2 ), I3 muscle ( F I3 ), and hinge ( F hinge ), aspring connecting the grasper to the head ( F sp,g ) representing the surroundingmusculature and connective tissue, and friction due to the interaction of thegrasper with an object, e.g. seaweed or tube, ( F f,g ): F g = F I2 + F sp,g − F I3 − F hinge + F f,g (7)The total force on the head ( F h ) includes those listed above as well asfriction between the jaws and the object ( F f,h ) and a spring representing themusculature and connective tissue connecting the head to the rest of the body( F sp,h ). In Appendix A.5, we show how this simplifies to: F h = F sp,h + F f,g + F f,h (8)Details for calculating each of these component forces can be found inAppendix A.5, and a table of symbols can be found in Appendix A.2. Thismodel represents a substantial simplification of the continuum mechanics of thesoft bodied structures that make up the Aplysia feeding apparatus. However,the model captures the key muscle groups identified in the animal experiments,as well as some of the configuration-dependent mechanical advantages of thesemuscles. Additional muscle groups and kinematic effects can be easily addedinto the hybrid Boolean model expressions.3.5 A Boolean Network Model of Neural CircuitryRather than using a complex neural model, each neuron in the hybrid con-troller presented here is represented by a Boolean logic statement. In contrastto highly detailed models of neural activity which are relatively computa-tionally expensive, such as leaky-integrate-and-fire models [69,96,143], simplespiking models [68], or Hodgkin-Huxley neurons [58], the Boolean represen-tation of neurons approximates neural activity based on bursts of activityobserved in the animal data (Figure 1).As a first approximation, such bursts can be represented as on during firingactivity and off when quiescent. In the Boolean representation, the activityof the neuron (whether it is active or inactive) is determined by the combinedlogic of the inputs. For example, a simplified neural unit, N , with three inputsis shown in Figure 4. If these inputs are equally weighted, then node N canonly be active if S is active and if inputs S and S are not active. Therefore,the activity of the node N at the next discrete step, ( j + 1), can be calculatedusing Boolean logic based on the state of the synaptic inputs at the currentdiscrete step, ( j ), as follows: N ( j + 1) = S ( j ) ( ! S ( j )) ( ! S ( j )) (9)where the inputs S i and output N have numeric values 0 (off) or 1 (on),! represents Boolean negation, and the AND operator is implemented using ontrol for Multifunctionality 17 Fig. 4
An example node in the Boolean network model. This example neuron is innervatedby one excitatory input, S , and two inhibitory inputs, S and S . multiplication. To account for neurons with variable bursting intensities, weextend the Boolean framework to include three-state model neurons such thatquiescence is represented as 0, weak firing as 1, and strong firing as 2. If anormal logical AND were used, the difference between states 1 and 2 wouldbe lost, but this difference remains when variables are multiplied.To account for known variations in the strength of inputs to neurons, ad-ditional logical calculations can be added to refine the activation of a givenmodel neuron. For example, if N is active if S is activated or if S is notactivated but is still inhibited by any activation of S , the logic calculationcould be modified as follows: N ( j + 1) = ( S ( j ) (cid:107) ( ! S ( j ))) ( ! S ( j )) (10)where (cid:107) represents the OR operator. When modeling three-state neural inputsthat can fire strongly, ORs can be implemented using addition to preserve themagnitude of firing. In our simplified modeling framework, once the key musculature has beenidentified, a motor control layer is implemented. This layer consists of motorneurons known to innervate the key musculature (Table 1) and is built usingBoolean model neurons (Figure 5): B31/B32/B61/B62 for activating the I2protractor muscle; B8a/b for closing the grasper; B6/B9/B3 for activating theI3 retractor muscle; B38 for pinching the anterior jaws; and B7 for activatingthe hinge muscle.Though much is known about the
Aplysia feeding circuitry, there are stillopen areas of investigation, including exact sensory feedback pathways. There-fore, in the proposed model, we have implemented proprioceptive sensory feed-back based on the kinematics of the grasper. In some cases, these sensory path-ways directly innervate motor neurons in the current framework. However, itis likely that these are mediated through sensory neurons and interneurons inthe animal. Such additional units could be easily added to the framework asthey are identified. These sensory feedback inputs are implemented based ontunable thresholds of the grasper position and pressure of closing. Such thresh-olds may allow the model to be fit to individual animals or enable modulationof the network by tuning the values of thresholds. It should also be highlighted
Fig. 5
Schematic of the motor control layer based on the key motor neurons and muscula-ture identified previously (see Figure 1). See Appendix A.3 for logic formulations. Sensoryfeedback pathways (inputs to motor neurons) are hypothesized based on the relative timingof motor neuronal activity. These sensory pathways are implemented in the model as actingdirectly on the motor neurons, but are likely to act via sensory neurons and interneuronsin the actual neural circuitry. In this diagram, the Grasper Pressure inputs to B6/B9 andB3 and to B31/B32 are shown without end caps. These connections vary depending onthe behavioral state as indicated by the insets in the bottom left corner of the image. SeeAppendix A.3 for detailed circuit specifications. that in this model the larger motor pool B31/B32/B61/B62 is sometimes ab-breviated as B31/B32; the rationale for doing so is that B31/B32 have bothmotor neuronal and interneuronal properties [66].
Building on the first layer of the model, we can add local coordination throughthe inclusion of known interneurons (Figure 6). This layer coordinates the func-tional timing of the activity in the motor layer such that effective behaviorsare generated. For the
Aplysia case study presented here, this layer includeskey interneurons identified in prior literature including B30, B40, B64, andB20. In addition, we have included B4/B5 based on the existing literaturedocumenting its importance (e.g. [43,147,155]) and our own preliminary datathat it may play a role in mediating grasper release needed to switch rapidlyto rejection (unpublished observations). In this model, B30 and B40 are rep-resented as a single model neuron as they both serve primarily to provideinhibition to B8a/b during ingestive behaviors [71]. The differences in howthey are activated (B30 receives excitatory stimuli from CBI-4, and B40 re-ceives excitatory input from CBI-2 [71]) are included in the model neuron’slogic (see Appendix A.3). The connectivity of this layer with the motor layerwas established based on the previous literature. As with the motor layer, somesensory feedback pathways are proposed such that proprioceptive inputs can ontrol for Multifunctionality 19
Fig. 6
Schematic of the local control layer added to the motor control layer shown previ-ously. See Appendix A.3 for logic formulations. The local control layer consists of knowninterneurons in the buccal ganglia based on the previous literature (see Tables 1 and 2). Inthis diagram, the retraction-triggered proprioceptive feedback to B4/B5 varies with behav-ioral state as shown by the inset (3). This feedback is only present during rejection and isinhibitory during this behavior. See Appendix A.3 for detailed circuit specifications. excite or inhibit specific interneurons based on the grasper position relative tomodel thresholds.
This two-layer model, when properly stimulated, can independently producethe three behaviors of interest. However, it does not allow coordinated be-havioral switching based on external sensory cues. To add this capability, weadd a cerebral ganglion layer, again referring to the existing literature, whichresponds to three external stimuli: mechanical and chemical stimulation of thelips, and mechanical stimulation in the grasper (Figure 7). Cerebral-buccalinterneurons 2 and 4 (CBI-2 and CBI-4) play critical roles in rejection as wellas in biting and swallowing, respectively [71]. The transition from egestive toingestive behaviors appears to be handled, at least in part, by the inhibitionof key buccal interneurons by CBI-3 [73,98]. Although there are other CBIsthat have been shown to play some role in feeding behaviors, such as CBI-12modulating the timing of protraction and retraction in swallowing [75], CBIs2, 3, and 4 were selected as a minimum set to generate behavioral switchingamong the behaviors of interest. Using a demand-driven complexity frame-work, additional CBIs or CBI effects could be included in future iterations ifsuch variations in timing were deemed necessary.
Fig. 7
Schematic of the full Boolean network controller including the cerebral interneuronsfor behavioral switching and global control. See Appendix A.3 for logic formulations. Exter-nal sensory cues are implemented as acting directly on relevant cerebral-buccal interneurons.However, such proprioceptive and exteroceptive feedback may be mediated through addi-tional sensory neurons or interneurons in the actual neural circuitry of the animal. CBIsinteract primarily with the local control layer to control behavioral switching. Strong inhibi-tion from CBI-3 that overrides other inputs to B20 is shown with a bold connection. Dashedconnections from B4/B5 represent hypothetical inhibition and excitation that occurs onlyif the presynaptic node is strongly activated (represented as a 2 in the modeling framework,rather than the Boolean 0 or 1). https://github.com/CMU-BORG/Aplysia-Feeding-Boolean-Model . Archivedcode is available through Zenodo (doi:10.5281/zenodo.3978414). ontrol for Multifunctionality 21
Aplysia
Feeding Boolean ModelUsing the hybrid Boolean model approach, we developed a functional con-troller based on known neural circuitry while taking into consideration theeffect of the peripheral biomechanics. Using only 20 of the possible thousandsof neurons in the
Aplysia ganglia, the Boolean model is capable of producingmultifunctional behaviors (Figure 8). In the presence of mechanical and chem-ical stimulation at the lips, the controller generates rhythmic biting patternscharacterized by a strong protraction followed by a relatively weaker retrac-tion, with grasper closure in-phase with retraction. As no seaweed is in thegrasper, no force is experienced by the seaweed (Figure 8.A). If mechanicalstimulation is applied to the grasper while mechanical and chemical stimula-tion is present at the lips, indicating the presence of edible material in thegrasper, the model qualitatively reproduces swallowing with a weaker protrac-tion phase followed by a strong retraction (Figure 8.B). This results in highpositive (ingestive) force being applied to the seaweed during the retractionphase. In contrast, the presence of mechanical stimulation at the lips and inthe grasper without chemical stimulation at the lips indicates the presence ofinedible material in the grasper. Under such conditions, the model success-fully generates rejection-like behaviors (Figure 8.C). The inedible material isgrasped during the protraction phase, resulting in a negative (egestive) forcebeing applied to it during protraction, pushing it out of the buccal cavity.In addition to being multifunctional, the Boolean model framework ex-hibits robustness within a single behavior. During
Aplysia feeding, robustnessis observed when animals attempt to feed on seaweeds of varying mechanicalstrength or that are attached to the substrate by a holdfast. Increasing mechan-ical load increases the duration of swallows overall and the retraction phasein particular [65,128,47]. The Boolean model presented here reproduces thisphenomenon even though the behavior has not been explicitly programmed.The adjustment to seaweed strength is instead an emergent property of thecontrol network and biomechanics. By including a force threshold in our biome-chanical model, we can vary the force at which the seaweed “breaks”, therebyallowing the grasper to move again as it is no longer anchored to the rigid forcetransducer (Figure 2). Increasing the strength of the seaweed by increasing thevalue of this threshold results in a longer period for each swallow due to theincreased retraction duration (Figure 9), as is observed in behaving animals[47].4.2 Behavioral Switching Based on Sensory CuesTruly multifunctional controllers need to be able to appropriately switch be-tween behaviors. In addition to being able to reproduce distinct behavior
Fig. 8
The hybrid Boolean network model and simplified periphery is capable of producingthe functional characteristics of the three targeted behaviors: biting ( A ), swallowing ( B ),and rejection ( C ). A In biting, chemical stimuli are present at the lips while mechanicalstimuli are absent at the lips and at the grasper. This results in motion of the peripherywhich includes a strong protraction (open bars above grasper motion) followed by weakerretraction (closed bars), and grasper closure coincides with retraction. No force is appliedto the seaweed as it is not yet grasped. Thickening of the grasper motion trace representsthe position of the grasper when closing pressure would be great enough to hold an objectfirmly. In biting this has minimal effect as no material is present in the grasper. B Inswallowing, both mechanical and chemical stimuli are present at the lips and mechanicalstimuli are present in the grasper. Protraction of the grasper results in near zero forceon the seaweed, whereas retraction of the grasper results in strong positive force on theseaweed. The arrow indicates the recoil of the grasper at the time it first releases the seaweed.Thickening of the grasper motion trace represents the position of the grasper when staticfriction between the grasper and object is present, indicating that the seaweed is beingfirmly grasped. C In rejection, a mechanical stimulus (inedible material) is present at boththe lips and in the grasper, but chemical stimuli are absent. Grasper closure coincides withprotraction. Protraction of the grasper results in increasingly negative forces (pushing theinedible material out) while retraction results in forces approaching zero. Thickening of thegrasper motion trace represents the position of the grasper when static friction between thegrasper and object is present, indicating that the tube is firmly grasped. through coordinated variation of motor neuron activation, the model can alsoswitch between behaviors in response to changing sensory inputs.In the animal, a change from biting to swallowing motor patterns is ob-served when seaweed is present at the lips and the grasper successfully grabsthe seaweed, so the grasper now senses a mechanical stimulus. We assessedthe controller’s ability to reproduce this transition by applying a step changeto the mechanical stimulus in the grasper near the peak of protraction duringthe biting cycle (Figure 10.A). As a result, the model successfully transitionedfrom biting-like to swallowing-like neural and behavioral patterns.Similarly, a transition from swallowing to rejection is observed when ined-ible food is detected in the grasper. This transition can be seen in the animalby inducing it to bite and swallow a polyethylene tube while simultaneouslytouching the lips with food, and removing the food stimulus after some length ontrol for Multifunctionality 23
Fig. 9
Characteristic examples of grasper motion and measured force on the force trans-ducer during swallowing with varying seaweed strength thresholds, z S . Thickening of thegrasper motion trace represents the position of the grasper when closing pressure would begreat enough to hold an object firmly. In the top and bottom panels, protraction is indicatedby the open bars above grasper motion and retraction by closed bars. Seaweed thresholdsincrease from z S = 0 . z S = 0 . of tubing has been ingested. This behavioral transition is observed in the modelwhen chemical stimuli are removed during swallowing. This results in a sen-sory state in which only mechanical stimuli are present both at the lips and inthe grasper. As a consequence, the model successfully transitions from swal-lowing to rejection (Figure 10.B). A sudden drop in force on the seaweed isobserved as the grasper briefly releases the seaweed and transitions to graspingthe seaweed during protraction in order to push the seaweed out of the feedingapparatus.4.3 Using the Model to Propose Testable HypothesesThe modeling framework provides a significant advantage over population-based neural control schemes such as machine learning because the network isexplainable and grounded in an animal’s neurobiology. As a consequence, themodel is a tool not only for robotic control, but also for generating and testingpotential neurobiological hypotheses. To mimic electrophysiology experiments,“electrodes” can be added to the Boolean logic statements for a given modelneuron as excitatory or inhibitory inputs, and the Boolean architecture can be Fig. 10
By changing the combination of external stimuli, the hybrid Boolean networkcontroller can appropriately switch between behaviors. A When mechanical and chemicalstimuli are present at the lips, but no mechanical stimuli are in the grasper, the networkproduces biting-like behavior. A step change, halfway through the simulation, in the me-chanical stimuli in the grasper representing a successful grasp attempt switches the networkto producing swallow-like behavior. B When mechanical stimuli are present at the lips andin the grasper, and chemical stimuli are present at the lips, the model produces swallowing-like behavior. Loss of chemical stimuli at the lips halfway through the simulation triggersthe model to initiate rejection-like behavior. extended to include a strongly excited state wherein activity is set to 2 ratherthan 1, as was implemented for B4/B5.One such testable hypothesis is the role of the B4/B5 multi-action neuronsin behavioral switching. Gardner has previously shown that these multi-actionneurons have widespread outputs to many neurons within the buccal ganglia[43]. Furthermore, B4/B5 have been observed to be intensely activated duringrejection and less so in biting and swallowing [147], an observation that is alsoseen in the animal data shown above (Figure 1). B4/B5 have also been observedto fire strongly in response to sudden increases in load on seaweed duringswallowing [49]. The intense firing in B4/B5 may be critical for delaying theonset of activity in the jaw muscles during rejection. As the grasper protractsclosed and retracts open during rejection, it passes through the lumen of thejaws as it rejects the inedible material. If the jaws closed prematurely, thegrasper could be forced shut and food could be pulled back into the buccalcavity [155]. These observations led us to hypothesize that strong activationof B4/B5 could be used to trigger transient rejection behavior. ontrol for Multifunctionality 25
Fig. 11
In a hypothetical experiment in which postulated network connections are added,strong B4/B5 stimulation leads to transient egestive behavior. The modeling frameworkallows the network to be easily modified to accommodate the addition of “electrodes” tostimulate individual neurons, as well as individual timing properties, such as refractoryperiods. Here, strongly stimulating B4/B5 (red rectangular overlay) with the postulatedconnections for the B4/B5 neuron shown as dashed lines in Figure 7, and with a refractoryperiod affecting CBI-3, results in temporarily switching from swallowing-like to rejection-likebehavior (blue rectangular overlay).
To test this hypothesis, we have postulated connections from B4/B5 toCBI-2 (excitatory) and CBI-3 (inhibitory) and added them to the Booleanmodel, as well as an electrode to strongly excite B4/B5 transiently. In theactual animal, the postulated connections may be indirect. Additionally, themodel makes it possible to easily include a refractory period associated witha connection. We have included one such refractory period for CBI-3, whichagain may be indirect, during which it remains inhibited after strong inhibitionfrom B4/B5. To test the hypothesis that B4/B5 stimulation can temporarilyswitch behavior from ingestion to rejection in the model, we strongly excitedB4/B5 as the grasper approached the peak of retraction. This strong excita-tion resulted in inhibition of B8a/b and therefore the pressure on the seaweedwas released, causing an abrupt drop in force. The model then transitionedto rejection-like behavior for the duration of the CBI-3 refractory period, af-ter which the model returned to swallowing-like behavior (Figure 11). In theabsence of these postulated connections, the model does not transition torejection-like behavior when B4/B5 is strongly excited (data not shown). Themodel thus makes specific testable predictions.
The hybrid Boolean model framework applied to
Aplysia feeding results ina multilayer controller based on the known neural circuitry and peripheralbiomechanics of the animal. This model is capable of reproducing three keybehaviors observed in feeding (Figure 8), reproduces robustness within a be-havior by adjusting to varying mechanical load during swallowing (Figure 9),captures the ability to switch between behaviors in response to sensory cues(Figure 10), and provides a straightforward means of using the model to sug-gest testable hypotheses about circuit function (Figure 11). The model is easilyextensible as additional neural units or sensory feedback pathways are identi-fied or if additional features of the biomechanics need to be captured.5.1 LimitationsThere is a long-standing debate in the neurobiology community on the relativeroles of central pattern generator-like circuitry, where a rhythmic pattern canbe generated in the absence of sensory feedback, versus chain reflexes, whereeach phase of the pattern initiates sensory feedback critical for generating thenext part of the behavior [83]. It is likely that both modalities contribute tobehavior. For example, central pattern generators are heavily influenced bysensory feedback [111] and chain reflexes have central components [15,5,112].Since our model focuses on behavioral forces and movements, we constructedit to depend heavily on sensory feedback and less on the intrinsic internaldynamics of the neural circuit. As a consequence, removing all sensory inputswill stop the model from oscillating. This is an application of demand-drivencomplexity to reduce computational cost by focusing on a reduced set of in-ternal connections, cells, and dynamics sufficient to qualitatively reproducemultifunctionality. Such intrinsic mechanisms, however, could be added to themodel in future iterations. A testable hypothesis from these observations wouldbe that in the intact behaving animal, sensory feedback may play as significanta role as intrinsic mechanisms during feeding. This question could be investi-gated in intact animal models as well as in suspended buccal mass preparations[94] to determine the relative importance of these factors.Though the model provides an accessible framework for capturing knownnetworks for multifunctional control, it still has many parameters that must beset. The model architecture can easily be implemented based on known circuitconnectivity. Thresholds, time constants, and maximum forces can be approxi-mated through measuring the relevant strength of synaptic inputs and relevantforce and movement outputs. However, if it is important for a modeling appli-cation to capture the detailed time series of neural and muscle activity, suchas individual spikes or bursting, more details may be required (see Section 2).On the other hand, if the focus is on overall behavior and the system has slowmuscles, fast details may not be as important for obtaining appropriate be- ontrol for Multifunctionality 27 havioral outputs. In future work, parameters could be found using automatedapproaches such as optimization or machine learning.Additionally, the model implementation does not capture the cycle-to-cyclevariability observed in actual animal behavior [31]. Although deterministiccontrol is useful in many robotic applications, variability plays a critical rolein behavioral flexibility. Variability is observed not only between individuals,but also within a given individual as it repeats a behavior. Indeed, variabilityin biological control contributes to the overall success of behaviors and species[31,91]. For example, by using different techniques to pull on seaweed, theanimal may be able to effectively fatigue the material and cause it to break[134,82]. As a consequence, animals may vary a behavior even if the mechanicalload is identical, which is not yet captured by our current model. Althoughmany robots can be programmed to handle a variety of situations, fewer robotsare capable of autonomous multifunctional behaviors [120]. A biologically-inspired approach may allow variability to be effectively harnessed to improveautonomous robot performance. Moreover, harnessing variability might allowclosed loop controllers of the nervous system to be fit to individual animals.On the other hand, some variability observed in animal behavior may notbe true stochasticity, but rather results from changing internal states dueto neuromodulation (e.g. [32]). As the neural locus of such internal statesand effects is identified, these variables could be included in our modelingframework in the future to better capture these effects. The model could beextended to include variability through the inclusion of stochastic processes foractivity switching in each node. Furthermore, the faster-than-real-time speed(2-3 orders of magnitude on standard CPU hardware) of the model allowsmany instantiations with small parameter variations to be run in parallel,thereby capturing the variability observed both within a given behavior andbetween individual animals, or for finding optimal solutions.A final limitation of the current model is that, due to the Boolean natureof our external sensory cues, intermediate behaviors [101], such as repeatedlymoving the seaweed back and forth, are not captured. Previous literature hasdemonstrated the importance of such intermediate behaviors. For example,before rejections, the animal may attempt to reposition food and retry swal-lowing [101,79]. This ability to selectively reposition, reject, and swallow alonga continuum of behaviors is important for feeding efficiency [79]. Althoughthe current model does not capture these behaviors, modifications to the in-terneuronal circuitry motivated by experimental findings and by hypothesizedconnections may better capture these behaviors.5.2 Conclusions and Future DirectionsThe hybrid Boolean network control framework presented here leads to a bioin-spired, computationally efficient controller capable of producing key multifunc-tional behaviors observed in the animal. Its computational efficiency stems,in part, from using a demand-driven complexity approach which minimizes the number of neurons and connections used to reproduce the desired be-havior. This framework and the use of the semi-implicit integration schemein Appendix A.1 allow new nodes and connections to be added with relativeease through modification of the Boolean logic statements. As a consequence,additional neurons, temporal effects, connections, and sensory pathways caneasily be added based on the existing literature [29] and future experimen-tal results. Although additional neurons and connections were not necessaryto produce multifunctionality, including them may improve controller robust-ness. Our modeling framework will make it possible to clarify how sensoryfeedback affects behavior as additional connections are added. Moreover, oursimple biomechanical model could readily be incorporated into a much morerealistic neural circuit model [28,18] to assess the role of sensory feedback onmore detailed neural mechanisms, even though this may reduce computationalefficiency.More complex biomechanical models can also be interfaced with the hy-brid Boolean controller to capture morphological intelligence , i.e., how thestructural biomechanics itself contributes to the control of the system. In the
Aplysia feeding system, morphological intelligence is demonstrated by the ef-fect of changes in the grasper shape on the mechanical advantage of the I2and I3 muscles [107]. The phase in which shape changes occur can either in-crease the mechanical advantage of a muscle, allowing it to generate higherforces, or diminish the muscles effectiveness, creating regions where the timingof the control signal is less critical. The simplicity of the modeling frameworkallows morphology and more detailed biomechanical models [104,107,139] tobe integrated in future iterations.Although we have applied the model framework to
Aplysia feeding, theframework can be extended to many other multifunctional systems. In
Aplysia feeding, differences between the key behaviors are largely the result of shiftsin phasing between muscle activity in the grasper relative to protraction andretraction (Figure 1). Similarly, in locomotion, changes in relative timing ofswing and stance are observed as animals transition from walking to running[90,16,86]. Another multifunctional behavior observed in legged systems, hop-ping, uses the same periphery as walking and running but synchronizes thephase of muscle activity between legs [145,99]. The framework can be adaptedto such multifunctional behaviors through application of the multilayered con-troller design. Similar approaches have been previously reported in mammalianneural circuit controllers using more physiological neuron models [97,92,67].The modeling framework has several advantages over other control andmodeling approaches. Similar to the discrete event-based neural network modelrecently reported by Bazenkov et al. [6], this modeling approach allows rapidsimulation of multifunctional behavior. However, unlike such prior discretemodels, the Boolean model framework presented here includes the known neu-ral circuitry and simplified biomechanics of the periphery. The direct relation-ship to the underlying circuitry makes it possible to both generate and testspecific neurobiological hypotheses; at the same time, the relative simplicityof the network makes it attractive as a basis for robot control. Furthermore, ontrol for Multifunctionality 29 unlike current artificial neural network architectures, synthetic nervous sys-tems including the hybrid Boolean model are explainable: the structure ofthe networks directly informs the functional outputs of the systems. Althoughthe connections and trained weights of artificial neural networks may providesimilar control capabilities, these networks must be trained on large datasets.Part of the strength of synthetic nervous systems is that they use a basis set ofdynamics derived from biological neurons and thus can generate robust controleven without additional training [140,142,141,62,63].
Acknowledgements
HJC and JPG were supported by NSF grant IOS1754869. VWWwas supported by startup funding from the Carnegie Mellon University Department of Me-chanical Engineering.
Conflict of interest
The authors declare that they have no conflict of interest.
References
1. Aggarwal, S., Chugh, N.: Signal processing techniques for motor imagerybrain computer interface: A review. Array (January), 100003 (2019).doi:10.1016/j.array.2019.100003. URL https://doi.org/10.1016/j.array.2019.100003
Homarusamericanus . Journal of Comparative Physiology A (1), 29–46 (1977).doi:10.1007/bf006677835. B¨assler, U.: Functional principles of pattern generation for walking movements of stickinsect forelegs: The role of the femoral chordotonal organ afferences. Journal of Experi-mental Biology (1), 125–147 (1988). URL https://jeb.biologists.org/content/136/1/125
6. Bazenkov, N.I., Boldyshev, B.A., Dyakonova, V., Kuznetsov, O.P.: Simulatingsmall neural circuits with a discrete computational model. Biological Cyber-netics (2020). doi:10.1007/s00422-020-00826-w. URL https://doi.org/10.1007/s00422-020-00826-w
7. Beck, J.M., Pouget, A.: Exact inferences in a neural implementation of a hidden Markovmodel. Neural Computation (5), 1344–1361 (2007). doi:10.1162/neco.2007.19.5.13448. Beer, R.D., Chiel, H.J., Gallagher, J.C.: Evolution and analysis of model cpgs forwalking: II. General principles and individual variability. Journal of ComputationalNeuroscience (2), 119–147 (1999). doi:10.1023/A:1008920021246. URL https://doi.org/10.1023/A:1008920021246
9. Beer, R.D., Chiel, H.J., Quinn, R.D., Espenschied, K.S.: A distributed neural networkarchitecture for hexapod robot locomotion. Neural Computation (3), 356–365 (1992)10. Beer, R.D., Chiel, H.J., Sterling, L.S.: A biological perspective on au-tonomous agent design. Robotics and Autonomous Systems (1), 169 –186 (1990). doi:https://doi.org/10.1016/S0921-8890(05)80034-X. URL . Designing Au-tonomous Agents0 Victoria A. Webster-Wood et al.11. Bicanski, A., Ryczko, D., Knuesel, J., Harischandra, N., Charrier, V., Ekeberg, ¨O.,Cabelguen, J.M., Ijspeert, A.J.: Decoding the mechanisms of gait generation in sala-manders by combining neurobiology, modeling and robotics. Biological Cybernetics (5), 545–564 (2013). doi:10.1007/s00422-012-0543-112. Brosch, T., Neumann, H.: Interaction of feedforward and feedback streams in visualcortex in a firing-rate model of columnar computations. Neural Networks , 11–16 (2014). doi:10.1016/j.neunet.2014.02.005. URL http://dx.doi.org/10.1016/j.neunet.2014.02.005
13. Brown, J.W., Caetano-Anoll´es, D., Catanho, M., Gribkova, E., Ryckman, N., Tian, K.,Voloshin, M., Gillette, R.: Implementing goal-directed foraging decisions of a simplernervous system in simulation. eNeuro (1), 1–10 (2018). doi:10.1523/ENEURO.0400-17.201814. B¨uschges, A.: Sensory control and organization of neural networks mediating coordi-nation of multisegmental organs for locomotion. Journal of Neurophysiology (3),1127–1135 (2005). doi:10.1152/jn.00615.200415. B¨uschges, A., Akay, T., Gabriel, J.P., Schmidt, J.: Organizing network action for loco-motion: insights from studying insect walking. Brain Research Reviews (1), 162–171(2008). URL
16. Cappellini, G., Ivanenko, Y.P., Poppele, R.E., Lacquaniti, F.: Motor patterns in hu-man walking and running. Journal of Neurophysiology (6), 3426–3437 (2006).doi:10.1152/jn.00081.200617. Cash, D., Carew, T.J.: A quantitative analysis of the development of the central ner-vous system in juvenile aplysia californica. Journal of Neurobiology (1), 25–47(1989). doi:10.1002/neu.480200104. URL https://onlinelibrary.wiley.com/doi/abs/10.1002/neu.480200104
18. Cataldo, E., Byrne, J.H., Baxter, D.A.: Computational model of a central patterngenerator. Lecture Notes in Computer Science (including subseries Lecture Notesin Artificial Intelligence and Lecture Notes in Bioinformatics) , 242–256(2006). doi:10.1007/11885191 1719. Chiel, H.J., Beer, R.D.: The brain has a body: Adaptive behavior emerges from inter-actions of nervous system, body and environment. Trends in Neurosciences (12),553–557 (1997). doi:10.1016/S0166-2236(97)01149-120. Chiel, H.J., Beer, R.D., Gallagher, J.C.: Evolution and analysis of model CPGs forwalking: I. Dynamical Modules. Journal of Computational Neuroscience (2), 99–118(1999). doi:10.1023/A:100892002124621. Chiel, H.J., Crago, P., Mansour, J.M., Hathi, K.: Biomechanics of a muscular hydrostat:a model of lapping by a reptilian tongue. Biological Cybernetics (5), 403–415 (1992).doi:10.1007/BF0020098422. Chiel, H.J., Kupfermann, I., Weiss, K.R.: An identified histaminergic neuron can mod-ulate the outputs of buccal-cerebral interneurons in Aplysia via presynaptic inhibition.The Journal of Neuroscience (January), 49–63 (1988)23. Chiel, H.J., Ting, L.H., Ekeberg, ¨O., Hartmann, M.J.: The brain in its body: Motorcontrol and sensing in a biomechanical context. Journal of Neuroscience (41), 12807–12814 (2009). doi:10.1523/JNEUROSCI.3338-09.200924. Chiel, H.J., Weiss, K.R., Kupfermann, I.: An identified histaminergic neuron modulatesfeeding motor circuitry in Aplysia . Journal of Neuroscience (8), 2427–2450 (1986).doi:10.1523/jneurosci.06-08-02427.1986. URL
25. Church, P.J., Lloyd, P.E.: Activity of multiple identified motor neurons recorded in-tracellularly during evoked feeding-like motor programs in
Aplysia . Journal of Neuro-physiology (4), 1794–1809 (1994). doi:10.1152/jn.1994.72.4.179426. Church, P.J., Whim, M.D., Lloyd, P.E.: Modulation of neuromuscular transmis-sion by conventional and peptide transmitters released from excitatory and in-hibitory motor neurons in Aplysia . Journal of Neuroscience (7), 2790–2800 (1993).doi:10.1523/jneurosci.13-07-02790.199327. Connor, J.A., Kretz, R., Shapiro, E.: Calcium levels measured in a presynaptic neu-rone of Aplysia under conditions that modulate transmitter release. The Journal ofPhysiology (1), 625–642 (1986). doi:10.1113/jphysiol.1986.sp016137. URL https://physoc.onlinelibrary.wiley.com/doi/abs/10.1113/jphysiol.1986.sp016137 ontrol for Multifunctionality 3128. Costa, R.M., Baxter, D.A., Byrne, J.H.: Computational model of the distributedrepresentation of operant reward memory : combinatoric engagement of intrinsicand synaptic plasticity mechanisms. Learning & Memory , 236–249 (2020).doi:10.1101/lm.051367.12029. Cropper, E.C., Jing, J., Weiss, K.R.: The feeding network of Aplysia . In: The OxfordHandbook of Invertebrate Neurobiology, December, pp. 400–422. Oxford UniversityPress (2019). doi:10.1093/oxfordhb/9780190456757.013.1930. Cullins, M.J., Chiel, H.J.: Electrode fabrication and implantation in
Aplysia californica for multi-channel neural and muscular recordings in in-tact, freely behaving animals. Journal of Visualized Experiments (40),e1791 (2010). doi:10.3791/1791. URL
31. Cullins, M.J., Gill, J.P., McManus, J.M., Lu, H., Shaw, K.M., Chiel, H.J.: Sen-sory feedback reduces individuality by increasing variability within subjects. Cur-rent Biology (20), 2672–2676 (2015). doi:10.1016/j.cub.2015.08.044. URL http://dx.doi.org/10.1016/j.cub.2015.08.044
32. Cullins, M.J., Shaw, K.M., Gill, J.P., Chiel, H.J.: Motor neuronal activity varies leastamong individuals when it matters most for behavior. Journal of Neurophysiology (3), 981–1000 (2015). doi:10.1152/jn.00729.2014. URL http://jn.physiology.org/lookup/doi/10.1152/jn.00729.2014
33. Dallidis, S.E., Karafyllidis, I.G.: Boolean network model of the
Pseudomonas aerugi-nosa quorum sensing circuits. IEEE Transactions on Nanobioscience (3), 343–349(2014). doi:10.1109/TNB.2014.234543934. De Jong, H.: Modeling and simulation of genetic regulatory systems: A lit-erature review. Journal of Computational Biology (1), 67–103 (2002).doi:10.1089/1066527025283320835. Destexhe, A., Sejnowski, T.J.: The Wilson-Cowan model, 36 years later. BiologicalCybernetics (1), 1–2 (2009). doi:10.1007/s00422-009-0328-336. Drushel, R.F., Neustadter, D.M., Hurwitz, I., Crago, P.E., Chiel, H.J.: Kinematic mod-els of the buccal mass of Aplysia californica . The Journal of Experimental Biology (Pt 10), 1563–83 (1998). URL
37. Edwards, R., Siegelmann, H.T., Aziza, K., Glass, L.: Symbolic dynamics and compu-tation in model gene networks. Chaos (1), 160–169 (2001). doi:10.1063/1.133649838. Eisenberg, E., Hill, T.L., Chen, Y.: Cross-bridge model of muscle contraction. quan-titative analysis. Biophysical Journal (2), 195–227 (1980). doi:10.1016/S0006-3495(80)85126-5. URL http://dx.doi.org/10.1016/S0006-3495(80)85126-5
39. Ekeberg, ¨O.: A combined neuronal and mechanical model of fish swimming. BiologicalCybernetics (5-6), 363–374 (1993). doi:10.1007/bf0019943640. Ekeberg, ¨O., Wall´en, P., Lansner, A., Tr˚av´en, H., Brodin, L., Grillner, S.: A computerbased model for realistic simulations of neural networks. Biological Cybernetics (2),81–90 (1991). doi:10.1007/bf0020238241. Ermentrout, B.: Neural networks as spatio-temporal pattern-forming systems. Reportson Progress in Physics (1998), 353–430 (2010)42. Evans, C.G., Cropper, E.C.: Proprioceptive input to feeding motor programs inAplysia. Journal of Neuroscience (19), 8016–8031 (1998). doi:10.1523/jneurosci.18-19-08016.199843. Gardner, D.: Interconnections of identified multiaction interneurons in buc-cal ganglia of Aplysia . Journal of Neurophysiology (2), 349–361 (1977).doi:10.1152/jn.1977.40.2.34944. Georgopoulos, A.P., Ashe, J., Smyrnis, N., Taira, M.: The motor cortex and the codingof force. Science (5064), 1692–1695 (1992). doi:10.1126/science.256.5064.1692.URL https://science.sciencemag.org/content/256/5064/1692
45. Georgopoulos, A.P., Kettner, R.E., Schwartz, A.B.: Primate motor cortex and free armmovements to visual targets in three-dimensional space. II. Coding of the direction ofmovement by a neuronal population. Journal of Neuroscience (8), 2928–2937 (1988).doi:10.1523/jneurosci.08-08-02928.198846. Giacomantonio, C.E., Goodhill, G.J.: A Boolean model of the gene regulatory networkunderlying mammalian cortical area development. PLoS Computational Biology (9)(2010). doi:10.1371/journal.pcbi.10009362 Victoria A. Webster-Wood et al.47. Gill, J.P., Chiel, H.J.: Rapid adaptation to changing mechanical load byordered recruitment of identified motor neurons. eNeuro (3) (2020).doi:10.1523/ENEURO.0016-20.2020. URL
48. Gill, J.P., Garcia, S., Ting, L.H., Wu, M., Chiel, H.J.: neurotic : Neuroscience tool forinteractive characterization. eNeuro (3) (2020). doi:10.1523/ENEURO.0085-20.2020.URL
49. Gill, J.P., Vorster, A.P.A., Lyttle, D.N., Keller, T.A., Stork, S.C., Chiel, H.J.: Neuralcorrelates of adaptive responses to changing load in feeding
Aplysia (2018). URL . Poster pre-sented at Society for Neuroscience 48th Annual Meeting, San Diego, CA50. Glaser, J.I., Chowdhury, R.H., Perich, M.G., Miller, L.E., Kording, K.P.: Machinelearning for neural decoding (2017). Http://arxiv.org/abs/1708.0090951. Golowasch, J., Goldman, M.S., Abbott, L.F., Marder, E.: Failure of averaging in theconstruction of a conductance-based neuron model. Journal of Neurophysiology (2),1129–1131 (2002). doi:10.1152/jn.00412.200152. Harischandra, N., Cabelguen, J.M., Ekeberg, ¨O.: A 3D musculo-mechanical model ofthe salamander for the study of different gaits and modes of locomotion. Frontiers inNeurorobotics (DEC), 1–10 (2010). doi:10.3389/fnbot.2010.0011253. Harris, S.E., Sawhill, B.K., Wuensche, A., Kauffman, S.: A model of transcriptionalregulatory networks based on biases in the observed regulation rules. Complexity (4),23–40 (2002). doi:10.1002/cplx.1002254. Haselgrove, J.C., Huxley, H.E.: X-ray evidence for radial cross-bridge movement and forthe sliding filament model in actively contracting skeletal muscle. Journal of MolecularBiology (4) (1973). doi:10.1016/0022-2836(73)90222-255. Hausknecht, M., Stone, P.: Deep recurrent q-learning for partially observable mdps.AAAI Fall Symposium - Technical Report FS-15-06 , 29–37 (2015)56. Heuer, H., Schmidt, R.A., Ghodsian, D.: Generalized motor programs for rapid biman-ual tasks: a two-level multiplicative-rate model. Biological Cybernetics (4), 343–356(1995). doi:10.1007/BF0019947057. Hill, A.: The heat of shortening and the dynamic constants of muscle. Proceedings ofthe Royal Society of London. Series B - Biological Sciences (843), 136–195 (1938).doi:10.1098/rspb.1938.005058. Hodgkin, A., Huxley, A.: A quantitative description of membrane current and its ap-plication to conduction and excitation in nerve. The Journal of Physiology (4)(1952)59. Horchler, A.D., Daltorio, K.A., Chiel, H.J., Quinn, R.D.: Designing responsive patterngenerators: stable heteroclinic channel cycles for modeling and control. Bioinspiration& biomimetics (2), 026001 (2015)60. Hosman, T., Vilela, M., Milstein, D., Kelemen, J.N., Brandman, D.M., Hochberg,L.R., Simeral, J.D.: BCI decoder performance comparison of an LSTM recurrentneural network and a Kalman filter in retrospective simulation. InternationalIEEE/EMBS Conference on Neural Engineering, NER , 1066–1071(2019). doi:10.1109/NER.2019.871714061. Huang, Z., Satterlie, R.A.: Neuronal mechanisms underlying behavioral switching ina pteropod mollusc. Journal of Comparative Physiology A (6), 875–887 (1990).doi:10.1007/BF0018733562. Hunt, A., Schmidt, M., Fischer, M., Quinn, R.: A biologically based neural systemcoordinates the joints and legs of a tetrapod. Bioinspiration and Biomimetics (5)(2015). doi:10.1088/1748-3190/10/5/05500463. Hunt, A., Szczecinski, N., Quinn, R.: Development and training of a neural controllerfor hind leg walking in a dog robot. Frontiers in Neurorobotics (APR), 1–16 (2017).doi:10.3389/fnbot.2017.0001864. Hurwitz, I., Goldstein, R.S., Susswein, A.J.: Compartmentalization of pattern-initiation and motor functions in the b31 and b32 neurons of the buccal gan-glia of Aplysia californica . Journal of Neurophysiology (4), 1514–27 (1994).doi:10.1152/jn.1994.71.4.1514. URL ontrol for Multifunctionality 3365. Hurwitz, I., Susswein, A.J.: Adaptation of feeding sequences in Aplysia oculifera tochanges in the load and width of food. Journal of experimental biology (1), 215–235(1992)66. Hurwitz, I., Susswein, A.J.: B64, a newly identified central pattern generator ele-ment producing a phase switch from protraction to retraction in buccal motor pro-grams of
Aplysia californica . Journal of Neurophysiology (4), 1327–1344 (1996).doi:10.1152/jn.1996.75.4.132767. Ivashko, D.G., Prilutsky, B.I., Markin, S.N., Chapin, J.K., Rybak, I.A.: Modeling thespinal cord neural circuitry controlling cat hindlimb movement during locomotion.Neurocomputing , 621–629 (2003)68. Izhikevich, E.: Simple model of spiking neurons. IEEE Transactions on Neural Net-works (6), 1569–1572 (2003). doi:10.1109/TNN.2003.820440. URL
69. Izhikevich, E.M.: Neural excitability, spiking and bursting. International Journal ofBifurcation and Chaos (06), 1171–1266 (2000). doi:10.1142/S021812740000084070. Jaques, N., Gu, S., Turner, R.E., Eck, D.: Workshop track -iclr 2017 tuning recurrentneural networks with re- inforcement learning. In: ICLR 2017, pp. 1–13 (2017)71. Jing, J., Cropper, E.C., Hurwitz, I., Weiss, K.R.: The construction of movement withbehavior-specific and behavior-independent modules. Journal of Neuroscience (28),6315–6325 (2004). doi:10.1523/JNEUROSCI.0965-04.200472. Jing, J., Cropper, E.C., Weiss, K.R.: Network functions of electrical couplingpresent in multiple and specific sites in behavior-generating circuits. Elsevier Inc.(2017). doi:10.1016/B978-0-12-803471-2.00005-9. URL http://dx.doi.org/10.1016/B978-0-12-803471-2.00005-9
73. Jing, J., Weiss, K.R.: Neural mechanisms of motor program switching in
Aplysia . Jour-nal of Neuroscience (18), 7349–7362 (2001). doi:10.1523/jneurosci.21-18-07349.200174. Jing, J., Weiss, K.R.: Interneuronal basis of the generation of related but dis-tinct motor programs in Aplysia : Implications for current neuronal models of ver-tebrate intralimb coordination. Journal of Neuroscience (14), 6228–6238 (2002).doi:10.1523/jneurosci.22-14-06228.200275. Jing, J., Weiss, K.R.: Generation of variants of a motor act in a modularand hierarchical motor network. Current Biology (19), 1712–1721 (2005).doi:10.1016/j.cub.2005.08.05176. Kabotyanski, E.A., Baxter, D.A., Byrne, J.H.: Identification and characterizationof catecholaminergic neuron B65, which initiates and modifies patterned activity inthe buccal ganglia of Aplysia . Journal of Neurophysiology (2), 605–21 (1998).doi:10.1152/jn.1998.79.2.605. URL
77. Kamali Sarvestani, I., Kozlov, A., Harischandra, N., Grillner, S., Ekeberg, ¨O.: A com-putational model of visually guided locomotion in lamprey. Biological Cybernetics (5), 497–512 (2013). doi:10.1007/s00422-012-0524-478. Kandel, E.: Cellular Basis of Behavior: An Introduction to Behavioral Neurobiology.Books in psychology. W. H. Freeman (1976). URL https://books.google.com/books?id=gHZ1QgAACAAJ
79. Katzoff, A., Ben-Gedalya, T., Hurwitz, I., Miller, N., Susswein, Y.Z., Susswein, A.J.:Nitric oxide signals that
Aplysia have attempted to eat, a necessary component ofmemory formation after learning that food is inedible. Journal of Neurophysiology (3), 1247–1257 (2006). doi:10.1152/jn.00056.2006. URL https://doi.org/10.1152/jn.00056.2006 . PMID: 1673822180. Kauffman, S.A.: The origins of order: Self-organization and selection in evolution.Oxford University Press, USA (1993)81. Koch, C., Segev, I., et al.: Methods in neuronal modeling: From ions to networks. MITpress (1998)82. Koehl, M.A.: Wave-swept shore: the rigors of life on a rocky coast. Univ of CaliforniaPress (2006)83. Kuo, A.D.: The relative roles of feedforward and feedback in the control of rhythmicmovements. Motor control (2), 129–145 (2002)84. Kupfermann, I.: Feeding behavior in Aplysia : a simple system for the study of moti-vation. Behavioral Biology (1), 1–26 (1974). doi:10.1016/S0091-6773(74)91644-7.URL (3), 1999 (1999)86. Li, L., Van Den Bogert, E.C., Caldwell, G.E., Van Emmerik, R.E., Hamill, J.: Coordi-nation patterns of walking and running at similar speed and stride frequency. HumanMovement Science (1), 67–85 (1999). doi:10.1016/S0167-9457(98)00034-787. Lu, C.W., Patil, P.G., Chestek, C.A.: Chapter seven - current challenges to theclinical translation of brain machine interface technology. In: C. Hamani, E. Moro(eds.) Emerging Horizons in Neuromodulation, International Review of Neurobiol-ogy , vol. 107, pp. 137–160. Academic Press (2012). doi:https://doi.org/10.1016/B978-0-12-404706-8.00008-5. URL
88. Lu, H., McManus, J.M., Chiel, H.J.: Extracellularly identifying motor neurons fora muscle motor pool in
Aplysia californica . Journal of Visualized Experiments(73), e50189 (2013). doi:10.3791/50189. URL
89. Lyttle, D.N., Gill, J.P., Shaw, K.M., Thomas, P.J., Chiel, H.J.: Robustness, flexibil-ity, and sensitivity in a multifunctional motor control model. Biological Cybernetics (1), 25–47 (2017). doi:10.1007/s00422-016-0704-890. Mann, R.A., Hagy, J.: Biomechanics of walking, running, and sprinting. The AmericanJournal of Sports Medicine (5), 345–350 (1980). doi:10.1177/03635465800080051091. Marder, E., Taylor, A.L.: Multiple models to capture the variability in biological neu-rons and networks. Nature Neuroscience (2), 133–138 (2011)92. Markin, S.N., Klishko, A.N., Shevtsova, N.A., Lemay, M.A., Prilutsky, B.I., Rybak,I.A.: A neuromechanical model of spinal control of locomotion. In: Neuromechanicalmodeling of posture and locomotion, pp. 21–65. Springer (2016)93. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in ner-vous activity. The Bulletin of Mathematical Biophysics (4), 115–133 (1943).doi:10.1007/BF0247825994. McManus, J.M., Lu, H., Chiel, H.J.: An in vitro preparation for eliciting and recordingfeeding motor programs with physiological movements in Aplysia californica . JoVE(Journal of Visualized Experiments) (70), e4320 (2012)95. McManus, J.M., Lu, H., Cullins, M.J., Chiel, H.J.: Differential activation of an iden-tified motor neuron and neuromodulation provide
Aplysia ’s retractor muscle an addi-tional function. Journal Neurophysiology (4), 778–791 (2014)96. Mihala, S., Niebur, E.: A generalized linear integrate-and-fire neural model pro-duces diverse spiking behaviors. Neural Computation (3), 704–718 (2009).doi:10.1162/neco.2008.12-07-680. URL
97. Molkov, Y.I., Bacak, B.J., Talpalar, A.E., Rybak, I.A.: Mechanisms of left-right coordi-nation in mammalian locomotor pattern generation circuits: A mathematical modelingview. PLoS Comput Biol (5), e1004270 (2015)98. Morgan, P.T., Jing, J., Vilim, F.S., Weiss, K.R.: Interneuronal and peptidergic controlof motor pattern switching in Aplysia . Journal of Neurophysiology (1), 49–61 (2002).doi:10.1152/jn.00438.200199. Moritani, T., Oddsson, L., Thorstensson, A.: Phase-dependent preferential activationof the soleus and gastrocnemius muscles during hopping in humans. Journal of Elec-tromyography and Kinesiology (1), 34–40 (1991). doi:https://doi.org/10.1016/1050-6411(91)90024-Y. URL (10), 413 – 420 (1994). doi:https://doi.org/10.1016/0166-2236(94)90015-9.URL In vivo buccal nerve activity that distinguishes ingestionfrom rejection can be used to predict behavioral transitions in
Aplysia . Journal ofComparative Physiology. A, Sensory, Neural, and Behavioral Physiology (1), 17–32 (1993). doi:10.1007/bf00214712. URL https://doi.org/10.1007/bf00214712
Aplysia . Journal of ComparativePhysiology A (5), 519–536 (1993). doi:10.1007/BF00197761ontrol for Multifunctionality 35103. Mulgaonkar, Y., Araki, B., Koh, J.S., Guerrero-Bonilla, L., Aukes, D.M., Makineni, A.,Tolley, M.T., Rus, D., Wood, R.J., Kumar, V.: The flying monkey: A mesoscale robotthat can run, fly, and grasp. Proceedings - IEEE International Conference on Roboticsand Automation , 4672–4679 (2016). doi:10.1109/ICRA.2016.7487667104. Neustadter, D.M., Drushel, R.F., Crago, P.E., Adams, B.W., Chiel, H.J.: A kinematicmodel of swallowing in
Aplysia californica based on radula/odontophore kinematicsand in vivo magnetic resonance images. Journal of Experimental Biology (20),3177–3206 (2002)105. Neustadter, D.M., Herman, R.L., Drushel, R.F., Chestek, D.W., Chiel, H.J.: The kine-matics of multifunctionality: comparisons of biting and swallowing in
Aplysia califor-nica . Journal of Experimental Biology (2), 238–260 (2007)106. Nicolelis, M.A., Lebedev, M.A.: Principles of neural ensemble physiology underlyingthe operation of brain-machine interfaces. Nature Reviews Neuroscience (7), 530–540 (2009). doi:10.1038/nrn2653107. Novakovic, V.A., Sutton, G.P., Neustadter, D.M., Beer, R.D., Chiel, H.J.: Mechanicalreconfiguration mediates swallowing and rejection in Aplysia californica . Journal ofComparative Physiology A: Neuroethology, Sensory, Neural, and Behavioral Physiol-ogy (8), 857–870 (2006). doi:10.1007/s00359-006-0124-7108. Oishi, K., Klavins, E.: Framework for engineering finite state machines in gene regu-latory networks. ACS Synthetic Biology (9), 652–665 (2014). doi:10.1021/sb4001799109. Packard, N., Wolfram, S.: Two-dimensional cellular automata. Journal of StatisticalPhysics (March), 901–946 (1985). doi:10.1201/9780429494093-6110. Payne, J.L., Wagner, A.: Constraint and contingency in multifunctional gene regulatorycircuits. PLoS Computational Biology (6) (2013). doi:10.1371/journal.pcbi.1003071111. Pearson, K.: Common principles of motor control in vertebrates and invertebrates.Annual review of neuroscience (1), 265–297 (1993)112. Pearson, K.G.: Central Pattern Generation: A Concept Under Scrutiny, pp. 167–185.Springer US, Boston, MA (1987). doi:10.1007/978-1-4615-9492-5 10. URL https://doi.org/10.1007/978-1-4615-9492-5_10 (5), 1966–1979(1995). doi:10.1016/S0006-3495(95)80374-7114. Prescott, T.J., Ayers, J.L., Grasso, F., Verschure, P.F.M.J.: Chapter 17. Embodiedmodels and neurorobotics. In: From Neuron to Cognition via Computational Neuro-science, chap. Embodied M, pp. 483–512. MIT Press (2016)115. Prinz, A.A., Bucher, D., Marder, E.: Similar network activity from disparate circuitparameters. Nature neuroscience (12), 1345–1352 (2004)116. Ravn, A.P., Rischel, H., Holdgaard, M., Eriksen, T.J., Conrad, F., Andersen, T.O.:Hybrid control of a robot a case study. Hybrid Systems II pp. 391–404 (1995).doi:10.1007/3-540-60472-3 20117. Rivera Torres, P.J., Serrano Mercado, E.I., Anido Rif´on, L.: Probabilistic Booleannetwork modeling of an industrial machine. Journal of Intelligent Manufacturing (4),875–890 (2018). doi:10.1007/s10845-015-1143-4118. R¨oschard, J., Roces, F.: Cutters, carriers and transport chains: Distance-dependentforaging strategies in the grass-cutting ant Atta vollenweideri . Insectes Sociaux (3),237–244 (2003). doi:10.1007/s00040-003-0663-7119. Rosin, D.P., Rontani, D., Gauthier, D.J., Sch¨oll, E.: Experiments on autonomousboolean networks. Chaos (2) (2013). doi:10.1063/1.4807481120. Royakkers, L., van Est, R.: A literature review on new robotics: Automa-tion from love to war. International Journal of Social Robotics (5), 549–570 (2015). doi:10.1007/s12369-015-0295-x. URL http://dx.doi.org/10.1007/s12369-015-0295-x (4), 641–656 (2010). doi:10.1016/j.jtbi.2010.07.022122. S.C. Kleene: Representation of events in nerve nets and finite automata. Tech. rep.,U.S. Air Force Project RAND (1951)6 Victoria A. Webster-Wood et al.123. Schwartz, A.B., Kettner, R.E., Georgopoulos, A.P.: Primate motor cortex and free armmovements to visual targets in three-dimensional space. I. Relations between single celldischarge and direction of movement. Journal of Neuroscience (8), 2913–2927 (1988).doi:10.1523/jneurosci.08-08-02913.1988124. Selverston, A.I.: Dynamic biological networks: the stomatogastric nervous system. MITpress (1992)125. Selverston, A.I., Russell, D.F., Miller, J.P., King, D.G.: The stomatogastric nervoussystem: Structure and function of a small neural network. Progress in Neurobiology , 215 – 289 (1976). doi:https://doi.org/10.1016/0301-0082(76)90008-3. URL , 25–51(2015). doi:10.1007/s10827-014-0519-3129. Shea-Brown, E., Rinzel, J., Rakitin, B.C., Malapani, C.: A firing rate model ofParkinsonian deficits in interval timing. Brain Research (1), 189–201 (2006).doi:10.1016/j.brainres.2005.10.070130. Shoham, S., Paninski, L.M., Fellows, M.R., Hatsopoulos, N.G., Donoghue, J.P., Nor-mann, R.A.: Statistical encoding model for a primary motor cortical brain-machineinterface. IEEE Transactions on Biomedical Engineering (7), 1312–1322 (2005).doi:10.1109/TBME.2005.847542131. Siegle, L., Schwab, J.D., K¨uhlwein, S.D., Lausser, L., T¨umpel, S., Pfister, A.S., K¨uhl,M., Kestler, H.A.: A boolean network of the crosstalk between IGF and wnt signaling inaging satellite cells. PLoS ONE (3), 1–24 (2018). doi:10.1371/journal.pone.0195126132. Stamhuis, E., Aerts, P., Nauwelaerts, S.: Swimming and jumping in a semi-aquaticfrog. Animal Biology (1) (2005)133. Stehouwer, D.J.: Development of anuran locomotion: Ethological and neurophys-iological considerations. Journal of Neurobiology (10), 1467–1485 (1992).doi:10.1002/neu.480231008134. Stewart, H.L.: Consequences of flexural stiffness and buoyancy for hydrodynamicforces, light interception and dispersal of a tropical alga. University of California,Berkeley (2004)135. Sussillo, D., Nuyujukian, P., Fan, J.M., Kao, J.C., Stavisky, S.D., Ryu, S., Shenoy, K.: Arecurrent neural network for closed-loop intracortical brain-machine interface decoders.Journal of Neural Engineering (2) (2012). doi:10.1088/1741-2560/9/2/026027136. Susswein, A.J., Byrne, J.H.: Identification and characterization of neurons initiatingpatterned neural activity in the buccal ganglia of Aplysia . The Journal of Neuroscience (6), 2049–61 (1988). URL Aplysia feeding. Progress in Neurobiology (3), 304 – 317(2012). doi:https://doi.org/10.1016/j.pneurobio.2012.03.004. URL Aplysia aid retraction during biting but not during swallowing. Journal of Comparative Phys-iology A: Neuroethology, Sensory, Neural, and Behavioral Physiology (6), 501–514(2004). doi:10.1007/s00359-004-0517-4139. Sutton, G.P., Mangan, E.V., Neustadter, D.M., Beer, R.D., Crago, P.E., Chiel, H.J.:Neural control exploits changing mechanical advantage and context dependence togenerate different feeding responses in
Aplysia . Biological Cybernetics (5), 333–345(2004). doi:10.1007/s00422-004-0517-z140. Szczecinski, N.S., Chrzanowski, D.M., Cofer, D.W., Terrasi, A.S., Moore, D.R., Mar-tin, J.P., Ritzmann, R.E., Quinn, R.D.: Introducing mantisbot: Hexapod robot con-trolled by a high-fidelity, real-time neural simulation. IEEE International Conferenceon Intelligent Robots and Systems (September), 3875–3881 (2015).doi:10.1109/IROS.2015.7353922ontrol for Multifunctionality 37141. Szczecinski, N.S., Hunt, A.J., Quinn, R.D.: A functional subnetwork approach todesigning synthetic nervous systems that control legged robot locomotion. Fron-tiers in Neurorobotics (August) (2017). doi:10.3389/fnbot.2017.00037. URL http://journal.frontiersin.org/article/10.3389/fnbot.2017.00037/full (1-2), 99–112 (2018).doi:10.1007/s00422-017-0726-x. URL https://doi.org/10.1007/s00422-017-0726-x (2), 305–18 (1997). URL Aplysia . Neuroscience Letters (2), 307–310 (1991). doi:https://doi.org/10.1016/0304-3940(91)90595-K. URL (14), 2159–2170 (2000)146. Wang, Y., Truccolo, W., Borton, D.A.: Decoding hindlimb kinematics from primatemotor cortex using long short-term memory recurrent neural networks. Proceedings ofthe Annual International Conference of the IEEE Engineering in Medicine and BiologySociety, EMBS , 1944–1947 (2018). doi:10.1109/EMBC.2018.8512609147. Warman, E.N., Chiel, H.J.: A new technique for chronic single-unit extracellular record-ing in freely behaving animals using pipette electrodes. Journal of Neuroscience Meth-ods (2), 161–169 (1995). doi:10.1016/0165-0270(94)00144-6148. Webster, V.A., Lonsberry, A.J., Horchler, A.D., Shaw, K.M., Chiel, H.J., Quinn, R.D.:A segmental mobile robot with active tensegrity bending and noise-driven oscillators.In: 2013 IEEE/ASME International Conference on Advanced Intelligent Mechatronics:Mechatronics for Human Wellbeing, AIM 2013, pp. 1373–1380. Wollongong, Australia(2013). doi:10.1109/AIM.2013.6584286149. Weiss, K.R., Chiel, H.J., Koch, U., Kupfermann, I.: Activity of an identified histamin-ergic neuron, and its possible role in arousal of feeding behavior in semi-intact Aplysia .The Journal of Neuroscience (August), 2403–2415 (1986). doi:20026578150. Wilson, H.R., Cowan, J.D.: Excitatory and inhibitory interactions in localized popu-lations of model neurons. Biophysical Journal (1), 1–24 (1972). doi:10.1016/S0006-3495(72)86068-5151. Wilson, H.R., Cowan, J.D.: A mathematical theory of the functional dynam-ics of cortical and thalamic nervous tissue. Kybernetik (2), 55–80 (1973).doi:10.1007/BF00288786152. Wood, K.C., Blackwell, J.M., Geffen, M.N.: Cortical inhibitory interneurons con-trol sensory processing. Current Opinion in Neurobiology , 200–207 (2017).doi:10.1016/j.conb.2017.08.018153. Xie, Z., Schwartz, O., Prasad, A.: Decoding of finger trajectory from ECoG using deeplearning. Journal of Neural Engineering (3) (2018). doi:10.1088/1741-2552/aa9dbe154. Ye, H., Morton, D.W., Chiel, H.J.: Neuromechanics of coordination during swal-lowing in Aplysia californica . Journal of Neuroscience (5), 1470–1485 (2006).doi:10.1523/JNEUROSCI.3691-05.2006. URL Aplysia californica . Journal of Neuroscience (42), 10743–10755 (2006)156. Yu, S.N., Crago, P., Chiel, H.: A nonisometric kinetic model for smooth muscle. Amer-ican Journal of Physiology-Cell Physiology (3), C1025–C1039 (1997)157. Yu, S.N., Crago, P.E., Chiel, H.J.: Biomechanical properties and a kinetic simulationmodel of the smooth muscle I2 in the buccal mass of Aplysia . Biological Cybernetics , 505–513 (1999). doi:10.1007/s004220050579158. Zahalak, G.I., Ma, S.P.: Muscle activation and contraction: Constitutive relations baseddirectly on cross-bridge kinetics. Journal of Biomechanical Engineering (1), 52–62(1990). doi:10.1115/1.2891126. URL https://doi.org/10.1115/1.2891126 (4), 359–411(1989). URL http://europepmc.org/abstract/MED/2676342 (1), 294–308 (1994). URL http://jn.physiology.org/content/71/1/294 ontrol for Multifunctionality 39 A Appendices
A.1 Semi-Implicit Integration Scheme
Suppose a continuously varying quantity x satisfies the initial value problem dx ( t ) dt = − ( x ( t ) − x ∞ ( y ( t ) , . . . , y n ( t ))) τ , x ( t ) = x (11)where x ∞ ( t ) is set by the other variables in our system, say { y i } ni =2 , generally followingsome nonlinear dependencies, and τ is a fixed time constant. We would like to implement anumerical approximation to the exact solution for x , namely x ( t ) = e − ( t − t ) /τ x + 1 τ (cid:90) tt e − ( t − s ) /τ x ∞ ( y ( s ) , . . . , y n ( s )) ds, (12)along with the remaining variables that satisfy their own differential equations. Euler’s for-ward method is convenient to implement but prone to numerical instability. Euler’s backwardor implicit method is numerically stable but computationally expensive, as it requires solv-ing an implicit equation at each step. Both methods proceed from a discrete approximationof the derivative, namely x ( t + h ) − x ( t ) h ≈ dxdt . (13)In both cases we create an update rule x ( t ) → x ( t + h ), by evaluating the right hand sideof (13) at either time t or time t + h , and solving for x ( t + h ). Forward: x ( t + h ) − x ( t ) h = − ( x ( t ) − x ∞ ( y ( t ) , . . . , y n ( t )) τ (14) x ( t + h ) = x ( t ) − x ( t ) − x ∞ ( y ( t ) , . . . , y n ( t ) τ h (15) Backward: x ( t + h ) − x ( t ) h = − ( x ( t + h ) − x ∞ ( y ( t + h ) , . . . , y n ( t + h )) τ (16) x ( t + h ) = τx ( t ) + hx ∞ ( y ( t + h ) , . . . , y n ( t + h )) τ + h . (17)Since the variables y , . . . , y n appear on the right hand side of (17) evaluated at the latertime point, t + h , (17) is part of a system of n nonlinear equations that must be solvedsimultaneously to determine the system state at t + h . Both numerical schemes (15) and(17) are first-order accurate, meaning that the truncation error between the true solution(12) and the numerical approximation scales as O ( h ) on each time step, with a global error(after T/h time steps for a simulation of total runtime T ) that is O ( h ). Semi-implicit:
In our model implementation, we use a semi-implicit method based on theapproximation x ( t + h ) − x ( t ) h ≈ − ( x ( t + h ) − x ∞ ( y ( t ) , . . . , y n ( t )) τ , (18)namely x ( t + h ) = τx ( t ) + hx ∞ ( y ( t ) , . . . , y n ( t )) τ + h . (19)0 Victoria A. Webster-Wood et al.At each time step we update x using a weighted average of its past value x ( t ) and itstarget value x ∞ ( t ), with the (short) timestep h and the intrinsic time constant τ providingthe relative weight of past and future. We expect an accurate approximation to (12) provided h (cid:28) τ . As we show below, the method is first-order accurate, and numerically stable, but itdoes not require solving an implicit equation at each time step. Thus this method combinesthe advantages of both the forward and backward methods. The method may be seen as anexample of operator splitting. To see that (19) is first-order accurate, we assume that x ( t ) is smooth enough to haveTaylor expansions through the 2nd order. Thus, for h (cid:28) x ( t + h ) = x ( t ) + h dxdt ( t ) + O ( h ) , as h → x ( t ) + h x ∞ ( y ( t )) − x ( t ) τ + O ( h ) (21)= x ( t ) τ + hτ + h + h x ∞ ( y ( t )) − x ( t ) τ + h (cid:18) τ + hτ (cid:19) + O ( h ) (22)= τx ( t ) τ + h + hx ( t ) τ + h + (cid:18) τ + hτ (cid:19) hx ∞ ( y ( t )) τ + h − (cid:18) τ + hτ (cid:19) hx ( t ) τ + h + O ( h ) (23)= τx ( t ) τ + h + hx ∞ ( y ( t )) τ + h + hx ( t ) τ + h − hx ( t ) τ + h + O ( h ) (24)= τx ( t ) τ + h + hx ∞ ( y ( t )) τ + h + O ( h ) , as h → . (25)Thus, the semi-implicit scheme (19) is first-order accurate in the time step h .To see that (19) is numerically stable, suppose that we fix y so that x ∞ ( y ) = c, aconstant. Clearly if x ( t ) = c then x ( t + h ) = c as well, so x = c is a fixed point of theiteration (19), under this assumption. Numerical stability follows if we can show that x = c is a stable fixed point for all h >
0, as we now establish. Let x ( t + nh ) = c + a n , with a arbitrary. Then a n +1 = x ( t + nh + h ) − c (26)= τx ( t + nh ) + hcτ + h − c (27)= τ ( c + a n ) + hcτ + h − c (28)= ττ + h a n → , as n → ∞ (29)no matter the size of the timestep h >
0. Thus the scheme (19) is both (first-order) accurateand numerically stable.The head and grasper position variables x h , x g form a linearly coupled pair, for whichwe can extend the semi-implicit algorithm given in one-dimensional form above. In general,consider a nonhomogeneous linear system expressed in terms of a vector x , a matrix A , anda forcing vector b : d x dt = A ( t ) x ( t ) + b ( t ) . (30)To set up a semi-implicit first-order iteration scheme, observe that x ( t + h ) − x ( t ) h = A ( t ) x ( t + h ) + b ( t ) + O ( h ) , so (31) x ( t + h ) − hA ( t ) x ( t + h ) = x ( t ) + h b ( t ) + O ( h ) , therefore (32) x ( t + h ) = ( I − hA ( t )) − ( x ( t ) + h b ( t )) + O ( h ) . (33) Citation: MacNamara, Shev, and Gilbert Strang. “Operator splitting.” Splitting Meth-ods in Communication, Imaging, Science, and Engineering. Springer, 2016. 95-114.ontrol for Multifunctionality 41Dropping the O ( h ) term and writing the update scheme in MATLAB style notation gives x ( t + h ) = ( I − h ∗ A ( t )) \ ( x ( t ) + h ∗ b ( t )) . (34)(Here the backslash notation M \ u stands for M − u , i.e., the least-squares solution y to the linear system M y = u .) Comparing this update scheme with (19) it is easy to checkthat they are consistent for a single variable by setting A = − /τ and b = x ∞ /τ .In our case the head and grasper positions x h and x g comprise a linearly coupled system, x , and the coupling matrix A is 2 ×
2, so I − hA can be inverted explicitly, provided h issmaller than the reciprocal of the largest positive eigenvalue of A . (If no eigenvalues of A are positive real numbers, then I − hA can always be inverted.) For a general 2 × x ( t + h ) x ( t + h ) = 11 − h Tr A + h det A (1 − hA )( x + hb ) + hA ( x + hb ) hA ( x + hb ) + (1 − hA )( x + hb ) , (35)with all time-varying elements of the right hand side evaluated at time t . In (35) Tr A anddet A denote the trace and determinant of A , respectively. Thus, truncating terms of order O ( h ) and higher gives a first-order semi-implicit update scheme for two-component stateand forcing vectors x and b : x ( t + h ) = 11 − h Tr A ( t ) I + h − A A A − A x ( t ) + h b ( t ) . (36)In (36) I denotes the 2 × A.2 Table of Symbols
Symbol Meaning x x h head position relative to reference (83) x g grasper position relative to reference (83) x g/h = x g − x h grasper position relative to head Ni Boolean state of neuron iAi muscle activation associated with muscle iTi muscle tension associated with muscle iτm muscle activation time constant ch damping coefficient for head movements cg damping coefficient for grasper movements T I2 I2 protractor muscle tension (79) T I3 I3 retractor muscle tension (76) T hinge hinge muscle tension (81) P I4 grasper pressure (72) P I3,ant. anterior I3 pinch pressure (74) F I2 I2 protractor muscle force (85) F I3 I3 retractor muscle force (87) F hinge hinge muscle force (88) F I4 grasper closing force (90) F I3,ant. anterior I3 pinch force (100) F I2,max scaling parameter for I2 protractor muscle force F I3,max scaling parameter for I3 retractor muscle force F hinge,max scaling parameter for hinge muscle force F I4,max scaling parameter for grasper closing force F I3,ant.,max scaling parameter for anterior I3 pinch force Fh net force on head (97) Fg net force on grasper (84) Fo net force on object, force on transducer during swallowing (104)[hinge stretched] = [ x g/h > .
5] Boolean state of hinge stretch[unbroken] = [ Fo ≤ zs ] Boolean state of seaweed unbroken[lipschem] Boolean state of chemical stimulus at lips[lipsmech] Boolean state of mechanical stimulus at lips[graspermech] Boolean state of mechanical stimulus in grasper Kg grasper spring constant Kh head spring constant x g/h rest length of grasper spring x h rest length of the head spring µs,g coefficient of static friction between grasper and seaweed µk,g coefficient of kinetic friction between grasper and seaweed µs,h coefficient of static friction between head (jaws) and seaweed µk,h coefficient of kinetic friction between head (jaws) and seaweed ontrol for Multifunctionality 43 A.3 Boolean Logic of
Aplysia
Feeding Control
The following sections detail the logic implementations for the activity of each neuron inthe controller network. These interactions include known direct connections between neu-rons based on previous literature as well as hypothesized connections that may be director indirect, and indirect sensory feedback pathways. Sensory feedback pathways involvingproprioception of the grasper position and pressure exerted by the closed grasper are gatedby logic tests of position and pressure relative to user specified thresholds. Allowing thesethresholds to vary in response to activity levels of interneurons and external sensory cuesprovides an approximation of neuromodulation. All time varying elements on the right sideof the logic equations are at time ( j ). Cerebral Interneurons
1. Metacerebral Cell N MCC ( j + 1) = [arousal] (37)2. CBI-2CBI-2 is activated by sensory inputs present in biting and rejection, but not in swallow-ing. N CBI-2 ( j + 1) = N MCC ( ! N B64 ) (cid:0) ([lips mech ] [lips chem ] ! [grasper mech ]) (cid:107) ([grasper mech ] ! [lips chem ]) (cid:1) (38)With the hypothesized connections in Section 4.3, these equation changes to: N CBI-2 ( j + 1) = N MCC ( ! N B64 ) (cid:0) ([lips mech ] [lips chem ] ! [grasper mech ]) (cid:107) ([grasper mech ] ! [lips chem ]) (cid:107) (cid:0) N B4/B5 ≥ (cid:1) (cid:1) (39)3. CBI-3CBI-3 is activated by sensory inputs present in biting and swallowing, but not in rejec-tion. N CBI-3 ( j + 1) = N MCC [lips mech ] [lips chem ] (40)With the equations and refractory period proposed in Section 4.3, the logic implemen-tation for CBI-3 changes to include a gating state variable based on whether or not theneuron is in a refractory state following strong inhibition. Similar logic could be addedto other nodes in the network as needed based on animal experiments. This period wasincluded here as part of the hypothesis that strong activation of B4/B5 triggers rejec-tion in animals that are swallowing. This hypothesis and an assessment of whether thisrefractory period occurs in CBI-3 in animal preparations or whether this effect is dueto another mechanism could be tested experimentally. The equation becomes: N CBI-3 ( j + 1) = N MCC [lips mech ] [lips chem ] (cid:0) N B4/B5 < (cid:1) ( ! [refractory CBI-3 ]) (41)4. CBI-4CBI-4 is activated by sensory inputs present in swallowing and rejection, but not inbiting. N CBI-4 ( j + 1) = N MCC ([lips mech ] (cid:107) [lips chem ]) [grasper mech ] (42) Buccal Interneurons N B64 is influenced by the activity of the N MCC and N B31/B32 . It is alsoexcited by protraction and inhibited by retraction. The proprioceptive feedback is im-plemented as:B64 proprioception =( N CBI-3 (([grasper mech ] [protracted N B64 , swallow ]) (cid:107) (( ! [grasper mech ]) [protracted N B64 , bite ])) ) (cid:107) (( ! N CBI-3 ) [protracted N B64 , reject ]) (43)where, [protracted N B64 , swallow ] = [ x g/h > z B64 , swallow ] (44)[protracted N B64 , bite ] = [ x g/h > z B64 , bite ] (45)[protracted N B64 , reject ] = [ x g/h > z B64 , reject ] (46)This amounts to the threshold being depend on the behavior with different thresholdvalues for bites, swallows, and rejections. N B64 ( j + 1) = N MCC ( ! N B31/B32 ) B64 proprioception (47)2. B4/B5 N B4/B5 has been shown to have varying effects when firing strongly vs. weakly. Torepresent this in the modeling framework, quiescence is represented as 0, weak firing as1 and strong firing as 2. The neurons are quiescent during biting, and they fire weaklyduring the retraction phase of swallowing. The neurons fire strongly when stimulatedwith the external electrode and during the retraction phase of rejection. During rejection,B4/B5 is observed to cease firing, allowing B3/B6/B9 to fire briefly at the end of thebehavior. To implement this, we have used a proprioceptive feedback pathway whichinhibits the activity of B4/B5 once the grasper has reached a user-specified level ofretraction. N B4/B5 ( j + 1) = N MCC (cid:18) ( ! [electrode
B4/B5 ]) (cid:0)
2( ! N CBI-3 ) N B64 [protracted N B4/B5 ]+ N CBI-3 [grasper mech ] N B64 (cid:1) +2 [electrode
B4/B5 ] (cid:19) (48)where, [protracted N B4/B5 ] = [ x g/h > z B4/B5 ] (49)3. B20 N B20 ( j + 1) = N MCC (cid:0) N CBI-2 (cid:107) N CBI-4 (cid:107) N B31/B32 (cid:1) ! N CBI-3 ! N B64 (50)4. B40/B30 N B40/B30 has fast inhibitory and slow excitatory connections to N B8 . To capture this,we record the time (j) at which N B40/B30 transitions between states for later use in the N B8 activity calculations (see below). First, the activity of N B40/B30 in the next timestep is determined: N B40/B30 ( j + 1) = N MCC (cid:0) N CBI-2 (cid:107) N CBI-4 (cid:107) N B31/B32 (cid:1) ! N B64 (51)ontrol for Multifunctionality 45After calculating the new activity, we assess transitions as defined by the followingpseudocode:if ( N B40/B30 ( j ) == 0 AND N B40/B30 ( j + 1) == 1), then set t N B40/B30 ,on = j;if ( N B40/B30 ( j ) == 1 AND N B40/B30 ( j + 1) == 0), then set t N B40/B30 ,off = j;
Buccal Motor Neurons
1. B31/B32 N B31/B32 receives input from interneurons and proprioceptive feedback. To capturepossible modulation of N B31/B32 and generate multifunctional behavior under differentsensory cues, behavior-dependent proprioceptive inputs are implemented. Though theresulting full equation for N B31/B32 activity is large, it can be broken down to threesections: (1) if N CBI-3 is active and there is sensory stimuli in the grasper (swallowing),(2) if N CBI-3 is active and there is NOT sensory stimuli in the grasper (biting), and (3)if N CBI-3 is NOT active (rejection). N B31/B32 ( j + 1) = N MCC (cid:18) N CBI-3 [grasper mech ](( ! N B64 ) ((![pressure N B31/B32 , ingestion ]) (cid:107) N CBI-2 )(( ! N B31/B32 ) [retracted N B31/B32 , swallow , off ]+ N B31/B32 [retracted N B31/B32 , swallow , on ]))( ! [grasper mech ])(( ! N B64 ) ((![pressure N B31/B32 , ingestion ]) (cid:107) N CBI-2 )(( ! N B31/B32 ) [retracted N B31/B32 , bite , off ]+ N B31/B32 [retracted N B31/B32 , bite , on ]))+( ! N CBI-3 ) (cid:0) ( ! N B64 ) [pressure N B31/B32 , rejection ]( N CBI-2 (cid:107) N CBI-4 )(( ! N B31/B32 ) [retracted N B31/B32 , reject , off ]+ N B31/B32 [retracted N B31/B32 , reject , on ] (cid:1)(cid:19) (52)where, [pressure N B31/B32 , ingestion ] = [ P g > . p max ] (53)[pressure N B31/B32 , rejection ] = [ P g > . p max ] (54)[retracted N B31/B32 , swallow , off ] = [ x g/h < z N B31/B32 , swallow , off ] (55)[retracted N B31/B32 , swallow , on ] = [ x g/h < z N B31/B32 , swallow , on ] (56)[retracted N B31/B32 , bite , off ] = [ x g/h < z N B31/B32 , bite , off ] (57)[retracted N B31/B32 , bite , on ] = [ x g/h < z N B31/B32 , bite , on ] (58)[retracted N B31/B32 , reject , off ] = [ x g/h < z N B31/B32 , reject , off ] (59)[retracted N B31/B32 , reject , on ] = [ x g/h < z N B31/B32 , reject , on ] (60)6 Victoria A. Webster-Wood et al.2. B6/B9/B3 N B6/B9/B3 ( j + 1) = N MCC N B64 ( ! ( N B4/B5 ≥ (cid:18)(cid:0) ( N CBI-3 ( ! [grasper mech ])) [pressure N B6/B9/B3 , bite ] (cid:1) + (cid:0) ( N CBI-3 [grasper mech ]) [pressure N B6/B9/B3 , swallow ] (cid:1) +( ! N CBI-3 ) ( ! [pressure N B6/B9/B3 , reject ]) (cid:1)(cid:19) (61)where, [pressure N B6/B9/B3 , bite ] = [ P g > z N B6/B9/B3 , bite , pressure )] (62)[pressure N B6/B9/B3 , swallow ] = [ P g > z N B6/B9/B3 , swallow , pressure )] (63)[pressure N B6/B9/B3 , reject ] = [ P g > z N B6/B9/B3 , reject , pressure ] (64)3. B8a/b N B8 receives fast inhibitory and slow excitatory input from N B40/B30 [71,29]. In theBoolean framework here we implement this as an excitatory input immediately followingcessation of N B40/B30 activity for a user specified duration (duration N B40/B30 , excite ).Prior to calculating a new value for N B8 we first check whether the synaptic connectionfrom N B40/B30 is excitatory with the following statements:if ( N B40/B30 ( j ) == 0 AND j < ( t N B40/B30 ,off + duration N B40/B30 , excite )), then set N B40/B30 , excite = 1else set N B40/B30 , excite = 0 N B8 ( j + 1) = N MCC ( ! ( N B4/B5 ≥ N CBI-3 ( N B20 (cid:107) ( N B40/B30 , excite))( ! N B31/B32 ))+(( ! N CBI-3 ) N B20 )) (65)4. B7 N B7 ( j + 1) = N MCC (cid:0)(cid:0) ( ! N CBI-3 (cid:107) [grasper mech ]) ([protracted N B7 , reject ] (cid:107) [pressure N B7 ]) (cid:1) + (cid:0) ( N CBI-3 ! [grasper mech ]) ([protracted N B7 , bite ] (cid:107) [pressure N B7 ]) (cid:1)(cid:1) (66)where, [protracted N B7 , reject ] = [ x g/h > z B7,reject ] (67)[protracted N B7 , bite ] = [ x g/h > z B7,bite ] (68)[pressure N B7 ] = [ P g > z N B7 , pressure ] (69)5. B38 N B38 ( j + 1) = N MCC [grasper mech ] (cid:18) N CBI-3 [retracted N B38 ] (cid:19) (70)where, [retracted N B38 ] = [ x g/h < z B38 ] (71)ontrol for Multifunctionality 47
A.4 Muscle Forces
Contact forces, such as the pressure resulting from grasper closure and force due to theanterior pinch, are implemented as second-order responses to neural activation using thesemi-implicit integration scheme, Eq. (19), as shown in the following equations.1. Grasper Pressure P I4 ( t + h ) = τ I4 P I4 ( t ) + hA I4 ( t ) τ I4 + h (72) A I4 ( t + h ) = τ I4 A I4 ( t ) + hN B8 ( t ) τ I4 + h (73)2. Pinch Pressure P I3,ant. ( t + h ) = τ I3,ant. P I3,ant. ( t ) + hA I3,ant. ( t ) τ I3,ant. + h (74) A I3,ant. ( t + h ) = τ I3,ant. A I3,ant. ( t ) + h ( N B38 ( t ) + N B6/B9/B3 ( t )) τ I3,ant. + h (75)Muscle tensions for the remaining musculature were calculated using a second-order responseto the neural activity as outlined in the following equations.1. I3 Tension T I3 ( t + h ) = τ I3 T I3 ( t ) + hA I3 ( t ) τ I3 + h (76) A I3 ( t + h ) = τ I3 A I3 ( t ) + hN B6/B9/B3 ( t ) τ I3 + h (77)2. I2 TensionTime constants for I2 were tuned independently for ingestion and egestion to account forthe experimental observations that egestions have a longer period than ingestions. Suchvariation in responsiveness of the animal may exist due to differences in neuromodulationbetween the behaviors. Therefore the time constant for I2 is calculated as: τ I2 = N CBI-3 τ I2,ingestion + ( ! N CBI-3 ) τ I2,egestion (78) T I2 ( t + h ) = τ I2 T I2 + hA I2 τ I2 + h (79) A I2 ( t + h ) = τ I2 A I2 ( t ) + hN B31/B32 ( t ) τ I2 + h (80)3. Hinge Tension T hinge ( t + h ) = τ hinge T hinge ( t ) + hA hinge ( t ) τ hinge + h (81) A hinge ( t + h ) = τ hinge A hinge ( t ) + hN B7 ( t ) τ hinge + h (82)8 Victoria A. Webster-Wood et al. A.5 Biomechanical Model
The motions of the head and grasper are calculated based on the quasi-static equations ofmotion: ddt x g x h = F g c g F h c h (83)where x h is the position of the head relative to the ground frame, x g is the position of thegrasper relative to the ground frame, and c h and c g are the damping coefficients for themotion of the head and grasper, respectively. The forces on the grasper and head can becalculated as outlined in the following sections. A.5.1 Forces on the grasper
The positive direction for x g corresponds to protraction (Figure 2). The sum of forces onthe grasper is F g = F I2 + F sp,g − F I3 − F hinge + F f,g (84)where the component forces are defined and calculated as follows: F I2 : The force due to the I2 muscle. This value is dependent on the tension of the muscle aswell as the mechanical advantage. It is scaled by a tunable maximum parameter, F I2,max ,and is calculated as follows: F I2 = F I2,max T I2 ( t )(1 − x g/h ) (85)where x g/h = x g − x h is the position of the grasper relative to the head. F sp,g : The force in the spring connecting the grasper to the head. This spring representsthe surrounding musculature of the esophagus, buccal mass, and extrinsic muscles whichare not explicitly modeled. This is calculated as: F sp,g = K g ( x g/h − x g/h ) (86)where K g is the spring constant and x g/h is the rest length of the spring. F I3 : The force due to the I3 muscle which pushes the grasper backwards during retraction.This force is due to tension in I3 closing the muscular toroids. This value is dependent onthe tension of the muscle as well as the mechanical advantage. It is scaled by a tunablemaximum parameter, F I3,max , and is calculated as follows: F I3 = F I3,max T I3 ( t )( x g/h −
0) (87) F hinge : The force due to the hinge. This value is dependent on the tension of the muscle aswell as the mechanical advantage. It is scaled by a tunable maximum parameter, F hinge,max ,and is calculated as follows: F hinge = [hinge stretched] F hinge,max T hinge ( t )( x g/h − .
5) (88)where [hinge stretched] = [ x g/h > .
5] determines whether the hinge is sufficiently stretchedto produce any force [138]. F f,g : Friction resulting from the grasper closing on an object. To determine F f,g it is nec-essary to check if the grasper is slipping against the object by checking the inequality: | F I2 + F sp,g − F I3 − F hinge | ≤ | µ s,g F I4 | (89)ontrol for Multifunctionality 49where µ s,g is the coefficient for static friction between the grasper and the object. F I4 is thenormal force due to the grasper muscle I4 closing on the object. This is calculated directlyas the grasper pressure defined in the previous appendix applied to a unit area scaled by aparameter. F I4 = F I , max P I4 ( t ) (90)If the condition in Eq. (89) is true, then the contact is in a state of static friction and F f,g is calculated as: | F f,g | = F I2 + F sp,g − F I3 − F hinge (91)If the condition in Eq. (89) is not true, the contact is sliding and is in a state of kineticfriction, and F f,g is calculated as: | F f,g | = µ k,g F I4 (92)where µ k,g is the coefficient for kinetic friction between the grasper and the seaweed.The sign of the friction force is dependent on which direction the grasper would bemoving without the friction present, and F f,g can be calculated as: F f,g = − sgn( F I2 + F sp,g − F I3 − F hinge ) | F f,g | (93) A.5.2 Forces on head
The forces on the head are calculated as: F h = F sp,h − F sp,g − F I2 + F I3 + F hinge + F f,h (94)The muscles and grasper spring exert forces on the head equal and opposite to those onthe grasper. As the muscles contract and apply forces to move the grasper forward this alsostretches the spring between the grasper and head proportionally to the muscle force. Forthe quasi-static model, acceleration is assumed to be negligible and therefore the forces onthe grasper must equal zero.0 = F sp,g + F I2 − F I3 − F hinge + F f,g (95)Solving for the spring forces, F sp,g , and substituting into Eq. (94) yields: F h = F sp,h + (cid:8)(cid:8) F I2 − (cid:8)(cid:8) F I3 − (cid:24)(cid:24)(cid:24) F hinge + F f,g − (cid:8)(cid:8) F I2 + (cid:8)(cid:8) F I3 + (cid:24)(cid:24)(cid:24) F hinge + F f,h (96)which simplifies to: F h = F sp,h + F f,g + F f,h (97)where F sp,h is the spring force between the head and neck of the animal, F f,g is the previ-ously calculated friction force between the grasper and the object, and F f,h is the frictionforce resulting from the jaws pinching on the object. These components are calculated asfollows. F sp,h = K h ( x h − x h ) (98)where K h is the spring constant and x h is the rest length of the spring.To determine the value of F f,h it is necessary to check if the jaws are slipping relativeto the seaweed by checking the following inequality: | F sp,h + F f,g | ≤ | µ s,h F I3,ant. | (99)where µ s,h is the coefficient for static friction between the jaws and the seaweed. F I3,ant. isthe normal force due to the anterior portion of the I3 jaw muscle closing on the seaweed.0 Victoria A. Webster-Wood et al.This is calculated directly as the pinch pressure defined in the previous appendix applied toa unit area, scaled by a parameter, and multiplied by a mechanical advantage term: F I3,ant. = F I3,ant.,max P I3,ant. ( t )(1 − x g/h ) . (100)If the condition in Eq. (99) is true, the jaws are in static friction and F f,h is calculatedas: | F f,h | = F sp,h + F f,g (101)If the condition in Eq. (99) is not true, the jaws are slipping and F f,h is calculated as: | F f,h | = µ k,h F I3,ant. (102)where µ k,h is the coefficient for kinetic friction between the jaws and the seaweed.The sign of the friction force is dependent on which direction the head would be movingwithout the friction present and F f,h can be calculated as: F f,h = − sgn( F sp,h + F f,g ) | F f,h | (103) A.5.3 Force on objects
The force on the object if unbroken is equal to the sum of the friction forces where we usethe conventions that positive force indicates tension on the force transducer: F o = F f,g + F f,h (104)If F o ≤ z s , where z s is the user defined seaweed strength, the seaweed is not brokenand the motion of the bodies is calculated based on the forces calculated in the previoussections. If F o > z s , the seaweed is broken and can no longer transmit forces to the head orgrasper. Therefore the forces on the head and grasper are recalculated as: F h = F sp,h (105) F g = F I2 + F sp,g − F I3 − F hinge (106)A Boolean tracking variable [unbroken] is used to track whether the seaweed is intact (1)or broken (0). Once the seaweed breaks, it is not restored until the grasper has completed anew protraction and grasp motion. For this model, we have implemented this by resetting[unbroken] = 1 if at the current timestep [unbroken] == 0 AND x g/h < . x g/h ( j +1) > x g/h ( j ). These thresholds were tuned manually for this implementation. A.5.4 Updating Grasper and Head Positions
All of the forces in this biomechanical model are linearly dependent on the position of thehead, x h , and grasper, x g . Therefore they can each be rewritten in the form: F = A F x g x h + b F (107)As a consequence, the equations of motion can be rewritten in the form ddt x g x h = A c h A c h A c g A c g x g x h + b b (108)ontrol for Multifunctionality 51This can then be integrated with the semi-implicit integration scheme in Appendix A.1as: x ( t + h ) = 11 − h Tr A ( t ) I + h − A A A − A x ( t ) + h b ( t ) ..