Reconfiguring motor circuits for a joint manual and BCI task
Benjamin Lansdell, Ivana Milovanovic, Cooper Mellema, Eberhard E Fetz, Adrienne L Fairhall, Chet T Moritz
RReconfiguring motor circuits for a joint manual and BCI task
Benjamin Lansdell , Ivana Milovanovic , Cooper Mellema , Eberhard E Fetz , Adrienne LFairhall , Chet T Moritz , Applied Mathematics, University of Washington, Seattle, WA, USA Rehabilitation Medicine, University of Washington, Seattle, WA, USA Physiology and Biophysics, University of Washington, Seattle, WA, USA Center for Neurotechnology, University of Washington, Seattle, WA, USA UW Institute for Neural Engineering, University of Washington, Seattle, WA, USA Electrical & Computer Engineering, University of Washington, Seattle, WA, USA* [email protected]+ These authors contributed equally.
Abstract
Designing brain-computer interfaces (BCIs) that can be used in conjunction with ongoing motorbehavior requires an understanding of how neural activity co-opted for brain control interacts withexisting neural circuits. For example, BCIs may be used to regain lost motor function after stroke.This requires that neural activity controlling unaffected limbs is dissociated from activity controllingthe BCI. In this study we investigated how primary motor cortex accomplishes simultaneous BCIcontrol and motor control in a task that explicitly required both activities to be driven from thesame brain region (i.e. a dual-control task). Single-unit activity was recorded from intracortical,multi-electrode arrays while a non-human primate performed this dual-control task. We observedbroad changes in tuning of both units used to drive the BCI directly (control units) and units thatdid not directly control the BCI (non-control units). Using a measure of effective connectivity weobserved control-unit-specific dissociation from other units. Through an analysis of variance wefound that the intrinsic variability of the control units has a significant effect on task proficiency.Thus, provided this is accounted for, motor cortical activity is flexible enough to perform novel BCItasks that require active decoupling of natural associations to wrist motion.
Introduction
Broad application of brain-computer interfaces (BCIs) for control of neural prostheses requiresunderstanding how brain circuits can simultaneously engage in competing tasks. In the case ofpartial paralysis resulting from stroke, restoring function with a BCI-controlled stimulator wouldrequire coordination between control of the BCI and residual movement. Further, allowing directbrain control of external devices by healthy subjects has many potential industrial applications.Current state-of-the-art BCI control is achieved via intra-cortical brain signals. Typical BCIs inhumans and animals operate using population decoding based on real or imagined movement [1–4].BCI studies find that brain-control mappings that make use of activity observed during the naturalmotor repertoire are most effective [5, 6]. However, co-opting natural mappings between populationactivity and movement may pose a challenge when BCI control is required in parallel with ongoingmotor activity. We refer to BCIs designed to be used simultaneously with natural motor output asdual-control BCIs. In this article we investigate representation and performance in a dual-controlBCI. 1 a r X i v : . [ q - b i o . N C ] J a n ne strategy for designing dual-control BCIs is to take advantage of biofeedback and conditioningparadigms that allow for volitional control of neural signals [1, 3, 7–11]. These previous studies showthat effective control can be achieved even when selected control units show little prior tuning tomotor output, providing a broad candidate population of control signals [12]. Such neural interfacesmay thus be adapted to novel simultaneous-control tasks which require the coordination betweennetworks of neurons responsible for movement and brain control [13]. While subpopulations ofneurons encoding ipsilateral motion are a candidate control source for the BCI following stroke [14],a robust interface may require recording from the larger population of neurons that are involvedin natural, contralateral control. In such cases, networks engaged for brain control and motorcontrol may overlap. Proficient dual-control BCI usage thus requires that the networks coordinateor dissociate in a way that supports independent control of both natural movement and the BCI.Previous studies do demonstrate flexibility in motor cortex in operating a dual-control BCI. Inour previous study [13], single units that are wrist flexion/extension-tuned in a motor task couldbe used to control an independent axis in a task that requires concurrent wrist flexion/extension.A related study demonstrates similar concurrent-use BCIs are possible in human subjects usingECoG signals [15]. Another primate study shows robustness to interference from native motornetworks [16]. In this study, monkeys were required to perform an isometric force generation task andsimultaneously perform a BCI center-out task. Subjects showed gradual adaptation in performance,with preferred directional tuning of units adapting over the course of learning. These studies suggestthat dissociation of BCI and motor control networks is indeed possible for the purposes of controllinga dual-control BCI.While previous studies demonstrate that it is possible to dissociate motor control from BCIcontrol, the mechanisms of dissociation in the supporting neural networks has not been investigated.How is activity responsible for BCI control coordinated with the activity responsible for ongoing wristmovement during use of a dual-control BCI? We address this question here using a dual-control BCItask, where both BCI control and ongoing motor output are driven by the same cortical region (Fig.1). By recording activity of both control and non-control units, we study how their activity differsbetween dual-control, manual control, and brain-control tasks. Specifically, we seek to understandhow population activity observed during dual control relates to activity observed during manual-and brain-control tasks.Using both a linear-tuning analysis and an effective-connectivity analysis, we find that dual-control neural activity is more similar to manual-control neural activity than brain-control neuralactivity. By design, proficiency at the dual-control task requires units to fire independently of wristmotion. Therefore, we hypothesized that during the task the activity of control units would dissociatefrom previously co-tuned units. We find evidence that during dual control, control units dissociatespecifically from co-tuned units in a different way to how they dissociate in the brain control task,suggesting that dissociation required by the dual-control task can occur with single-unit specificity.Finally, we searched for factors that predict performance in the dual-control task. Building onprevious studies [5, 6], we find that measures of single unit intrinsic variability are predictive ofdual-control BCI task performance. Our analysis is based on experiments performed in [13]. Briefly, one male Macaque nemestrinamonkey was trained to perform a random target-pursuit motor task. The monkey was implantedwith two 96-channel multi-electrode arrays bilaterally in primary motor cortex. In this study onlydata from the left hemisphere were used. All procedures were in accordance with National Institutesof Health ‘Guide for the Care and Use of Laboratory Animals’. Further details of surgery andelectrode implantation, and behavioral training are detailed in [13].The monkey began each daily session by controlling the cursor with isometric wrist torque intwo dimensions (manual control, MC), then progressed to using the aggregate neural activity oftwo single units to control a cursor in one dimension (brain-control, BC). Subsequently, he used2 nit activity WristBrainPre-manual Brain control Dual control Post-manuala)
Untuned BrainWrist W r i s t W r i s t W r i s t c) Tuned decrease cell activity+flex wrist increase cell activity+flex wristWrist flexWrist ext Cell activity incCell activity decOne experimental sessionTuning in 2D wrist task targetcursorb)
Fig 1.
Dual-control BCI experimental setup. a) Isolated primary motor cortex unit activitycontrols a brain-control axis, while contralateral wrist torque determines a manual-control axis. Ineach experimental session, the monkey first performed a 2D manual wrist task. The monkey thenperformed a 1D brain-control task, in which cursor velocity was determined by neural activity oftwo units. The monkey then performed a dual-control task, where one axis is determined by wristtorque and the other by neural activity. Finally, the monkey performed a second 2D manual task.b) Linear encoding models were used to identify preferred direction and modulation depth (tuningangle and tuning strength) of each unit. The two untuned units shown have firing rate independentof wrist velocity direction, while two directionally tuned units shown respond preferentially to wristvelocity in particular directions. Radius indicates mean firing rate (Hz). Red line indicates tuningangle and strength. c) Random target pursuit task for dual-control BCI. Targets (grey disk)randomly appear, subject has fixed time to acquire and hold cursor within disk. Thus in thedual-control task, subjects are required to modulate activity of control units independently of wristmotion.the same neural activity to control the BCI in one dimension, while simultaneously using isometricwrist torque of the contralateral forelimb to control the cursor in a second orthogonal dimension(dual-control, DC; Fig 1).The random target pursuit task involved the monkey moving the cursor to the target andmaintaining the cursor within the target for at least 1s to receive a reward. Targets appearedrandomly, and a 0.5s break was provided between trials. For each trial there was a time-out periodof 40s. Besides the dimensionality, the task for the manual-control, brain-control and dual-controlsettings was the same. Each day a new pair of units was chosen to control the BCI. If directionallytuned units were used that day, units with approximately opposite preferred directions were selectedand paired to control the BCI.In order to ensure decoupling of unit activity and hand control in the dual control task, visualfeedback of one of these modalities had to be rotated to achieve independent degrees of freedom onthe monitor. Since the monkey was overtrained on the manual control task, we made a deliberatedecision to preserve the relation between wrist torque and cursor movement. Instead, we rotatedthe visual feedback of the units’ firing rate by 90 degrees relative to their preferred direction, evenfor units with little to no directional preference. This had minimal effect on the changes in tuningangles observed between conditions (Appendix B).All units with a mean firing rate above 5 Hz recorded from the hemisphere contralateral to armmovement were analyzed. The torque signals were smoothed with a Gaussian kernel of width 300ms3nd a linear tuning model was fit to each unit’s firing rate as a function of wrist torque. Overallperformance was quantified as the number of targets per minute acquired throughout the entirerecording.
The cursor position along the BCI axis, x t , was determined as a linear function of N control neurons’smoothed activity, y nt , N = 2, with gain α n and a running estimate of baseline firing b nt : x t = N (cid:88) n =1 α n ( y nt − b nt ) (1)where the baseline firing rate is defined as b nt = b nt − (1 − γ ) + γy nt (2)The baseline update rate is given by γ = 0 . The kinematic encoding model to wrist motion is defined for each unit y i simply through the linearrelation y it = α x t + τ + α x t + τ + c + (cid:15) (3)for velocity along task axes x and x , for predetermined time lag τ , and for Gaussian noise (cid:15) . Herean offset of τ = 75ms was used (Appendix D). Spikes are binned at 25 Hz. This time is consistentwith previous studies [17] and was, on average, the time lag of the maximum cross-correlationbetween cursor and neural activity (Appendix D). For most units, a velocity encoding model wasfound to yield higher cross-correlation between torque and neural activity than a position encodingmodel, so the results presented here are based on a velocity encoding model. Velocity is computedfrom position information using a cubic spline [17].The model is equivalent to a cosine encoding model through a simple transformation: y it = αr cos( θ − θ pref ) + c + (cid:15) (4)for α = (cid:112) ( α ) + ( α ) , θ pref = tan − ( α /α ), r = (cid:112) ( x ) + ( x ) and θ = tan − ( x /x ). Fromthis we can interpret α/c as a measure of modulation depth. We will refer to θ pref as tuning angleand α as tuning strength.For the linear tuning analysis, units that were significantly tuned in manual-control task ( R > .
01) throughout 99 sessions were selected. This resulted in 411 units being analyzed, 57 of whichwere control units. The average duration of the manual control sessions used was 6.20 minutes,comprising an average of 124 trials. The average duration of the brain control sessions used was10.1 minutes, comprising an average of 60.2 trials. The average duration of the dual-control sessionsused was 10.3 minutes, comprising an average of 54.5 trials.
Transfer entropy [18] is defined as the conditional mutual information between an observed timeseries Y and the history of a candidate related series X , conditional on the history of YI X → Y = H ( Y t | Y t − , . . . , Y t − T ) − H ( Y t | Y t − , . . . , Y t − T , X t − , . . . , X t − T ) (5)As with many information theoretic quantities, I X → Y is expensive to compute. An approximationtailored for spiking data is used, in particular the transfer entropy toolbox [19].4he method was validated using a synthetically generated dataset and compared with othercommon effective or functional connectivity measures. Specifically it was compared to linear Grangercausality, Poisson-process Granger causality, and correlation. On the synthetic data transfer entropywas most accurate. More details are provided in Appendix A.Transfer entropy is computed with spikes binned at 5ms intervals. Six minutes of recording datain each condition is used. Connections extend up to 30 time-bins into the past (150ms).The connectivity analysis is performed as follows. Within each session, between recordingconditions (manual to brain control and manual to dual control), changes in connectivity arecompared for different populations of pairs of units. Recalling transfer entropy is a directed measureof connectivity, the populations of unit pairs are: to control units and to non-control units; co-tunedunits to control units and non-co-tuned units to control units; and co-tuned units to non-controlunits and non-co-tuned units to non-control units.For a given session, generally no more than two pairs of units could be identified within 45degrees of the unit of interest (e.g. the control unit). Thus in comparisons involving co-tuned units,for each session only the top two co-tuned units were selected. These are defined as the two unitpairs having the closest tuning angle. All other pairs of units in the session were classified as notco-tuned (defined this way, most cotuned units are within 45 degrees of each other, while mostnon-cotuned units are more than 45 degrees apart, Fig 4a). In non-co-tuned populations generallymany more than two pairs of units can be identified within a session. Thus to ensure these largerpopulations are better sampled, but still balanced with populations of co-tuned units, the selection oftwo units is bootstrap sampled 50 times for each session. Such generated populations are combinedover all sessions. All co-tuned populations are selected on the basis of tuning during manual controlrecordings. This resulted in approximately 4400 connections being analyzed over the 99 sessions. Given the form of the linear decoding model (Eq 1), it is reasonable to model the cursor trajectoryas a moving average time series: x t = N (cid:88) n =1 P (cid:88) p =1 k p y nt − p + c for some model order P to be determined. Spikes are binned at 25Hz, and cursor data is smoothedwith a cubic spline, as per the linear tuning model [17].From the linear model we compute the maximum likelihood estimate given both neurons, ˆ β iMLE ,and the maximum likelihood estimate not given neuron j , ˆ β i \ jMLE . The difference between the twolog-likelihoods of the MLE models provides a Granger-causality type metric [20] G j → i = 2 L ( ˆ β iMLE ; x , y ) − L ( ˆ β i \ jMLE ; x , y )that quantifies the influence that one neuron has on the cursor trajectory. The likelihood ratio isoften used as the basis for hypothesis testing, though the quantity itself can be interpreted as aform of transfer entropy, provided the data is Gaussian distributed [21].The Granger causality toolbox MVGC [20] was used to perform this analysis. The approach iswell suited since the cursor position is, by definition, a moving average model of the neural data. Themethod assumes a covariance stationary time series. By construction (Eq 1) the cursor trajectoryhas this property, provided that the neural data is second-order stationary. This was confirmed bychecking the spectral radius of the vector autoregressive model was less than one.To verify the notion that G j → i has quantitative meaning we generated synthetic data fromneural recordings. A cursor trajectory was generated as the weighted sum of the two units’ decodedtrajectories, according to (Eq 1). Following this we computed G j → i for different weightings. Astrong monotonic relationship exists between G j → i and the weight that determines how much eachunit contributed to the cursor trajectories (Appendix C), demonstrating that the control metric canrecover which unit contributes more to cursor control.For the variability analysis, 198 control units are chosen from the 99 sessions.5 Results
Given that the dual-control task requires both brain control and manual control, we first soughtto understand how the population activity under dual control relates to that observed during themanual-control and brain-control tasks. To do this, a monkey performed the following trials. Eachsession began with of a block of manual-control trials, then brain-control, dual-control and concludedwith a second block of manual-control trials. During each session a fixed population of neurons wasidentified, allowing changes in their tuning and effective connectivity properties between tasks to becalculated. We selected neurons that were active ( > Next we focused on changes among populations of co-tuned units during the dual-control task. Wedefine co-tuned units as pairs of units that share preferred direction to wrist motion. Efficientlycompleting the dual-control task requires the networks associated with wrist control to operateindependently from networks associated with BCI control. Thus, for a control unit that is tuned towrist flexion in a manual control task, any trial that requires wrist flexion and a decrease in controlunit activity would decorrelate the control unit’s activity from other units that are also tuned towrist flexion. The dual-control task does not require that cotuned non-control units decorrelatefrom one another. Nor should the dual-control task encourage the control unit to decorrelate fromnon co-tuned units, since those other units are not activated during wrist flexion. On the basis of6 ontrolnon-control0 P o s t m anua l c on t r o l ( o ) -0.5 0 0.5 -1 0 1-0.5 0 0.5 -1 0 1-0.5 0 0.5 -1 0 1 -2006004002000 D ua l c on t r o l ( o ) B r a i n c on t r o l ( o ) Manual control (o) Manual control (o)Manual control (o)a) Manual controlDual controlBrain controlPosition Velocityb)c) d) c on t r o l non - c on t r o l c on t r o l non - c on t r o l M anua l c on t r o l B r a i n c on t r o l D ua l c on t r o l | ang l e | ( o ) | s t r eng t h | Dual controlBrain control*** * * *18012060
Fig 2. a) Changes in tuning angle for control and non-control units between tasks. b) Meanabsolute differences in tuning angle between manual control task and both brain control and dualcontrol tasks are significant. (two sided t-test; all p (cid:28) . p = 0 . p (cid:28) . ) P o s t m anua l c on t r o l Manual control D ua l c on t r o l B r a i n c on t r o l Manual control Manual controlr2=0.42 r2=0.21r2=0.220 4 8 12 x10-3 0 4 8 12 x10-30 4 8 12 x10-30x10-31284 x10-4 b) c on t r o l non - c on t r o l c on t r o l non - c on t r o l Dual controlBrain control* *0-1.6-1.2-0.8-0.4 t r an s f e r en t r op y control unitnon-control unit Fig 3. a) Changes in effective connectivity between tasks. Transfer entropy between the same pairof units within a session is computed for each task and plotted against one another. Data arepooled over all available sessions. b) For each pair of units, summary of effective connectivitychanges between manual control and brain-control (left) and manual control and dual-control (right)tasks. Comparisons are made for connections to a control unit (unfilled square) versus to anon-control unit (unfilled circle) from all other recorded units (filled circles). Changes aresignificantly larger for connections to control units than non-control units (Wilcoxon rank sum; p (cid:28) .
001 ). A negative value means lower connectivity in the brain-control or dual-controlcondition compared to manual control.
Given these insights into the neural activity supporting control of a dual-control BCI, we nextsought to understand which features of the recorded primary motor activity, if any, may relate tohow well the dual-control task is performed. In our previous study we did not identify differences intuning that were predictive of performance [13] – using both tuned or untuned control units lead tosimilar performance. However, related BCI studies do show that decoders that utilize the naturalmotor repertoire are most effective (e.g. [5, 6]). Thus we hypothesized that similar factors may affectperformance in a dual-control BCI.As our BCI is based on the activity of pairs of units, we performed an analysis of the variability ofonly these control units. We term the variance of the control unit activity during the manual-controltask the intrinsic variability, by analogy with the intrinsic manifold computed in larger neuralpopulations [6]. We sought to know if the intrinsic variability of the control units is predictive ofhow those units will be used for cursor control in the dual-control task. To quantify how much aunit is used to move the cursor, we used a Granger-causality based metric, G , that measures howmuch the activity of each control unit can predict the cursor trajectory (Fig 5a, see Methods). Weobserved that a high G in each unit is related to high performance (Fig 5b). Further, the unitthat contributes more to cursor movement is the unit with the higher intrinsic variability (Fig 5c).Consistent with these results, we observed that high performance in the dual-control task onlyoccurs when the intrinsic variability of the control units is high (Fig 5d). Another factor, controlunit firing rate, was not found to significantly relate to dual control performance (data not shown).To investigate if variance increased throughout a block of dual control trials, as might lead to greatercontrol and performance, we split the set of trials into early and late halves. No significant differencebetween early and late trials was observed (Fig 5e). Variability is thus a strong factor that predictsperformance and informs how the dual-control task is performed.8 )c) Cotuned populationd) Random population -3 -3 -3 -3 Brain control Dual controlNon-control unitControl unit Non-control unitControl unitNon-control unitControl unit Non-control unitControl unitBrain control Dual control** *** ** angle
Non-control unitControl unit3000100200 RandomCotunedControl unitNon-control unitRandom populationCotuned population angle b) Cotuned population statistics
Control unitNon-control unitControl unitNon-control unit
Fig 4.
Effective connectivity analysis of pairs of cotuned units. a) Effective connectivity iscomputed for pairs of units for all tasks in a session and then data are pooled over all sessions.Transfer entropy is compared between manual control and brain- or dual-control tasks. Pairs areclassified as either co-tuned (indicated by same color) or randomly selected (different colors).Recorded units are compared to either a control or non-control unit. Color wheel indicates tuningangle for example co-tuned and randomly selected units. b) Distribution of tuning angle amongstcotuned population and a non-cotuned population, as identified in the first manual-control task. c)Change in transfer entropy for pairs of units between manual-control and brain- or dual-controltasks, for pairs of units that are cotuned in the manual-control task. The Orange line indicates themean change, along with significance of change from zero (black line; two sided t-test, * = p < . In this study we investigate how neural activity supports a dual-control BCI where natural movementcontrols movement in one dimension while brain activity controls the task in a second dimension.We compared tuning, effective connectivity and variability observed during a dual-control task to abrain-control and manual-control task. 9 G ( c on t r o l un i t ) -1 0 1 Relative intrinsic variance(unit 1 - unit 2) -0.8-0.400.40.8 ∆ G Neural activity D ua l c on t r o l pe r f o r m an c e ( s u cc e ss e s / m i nu t e ) Intrinsic variance (unit 1) I n t r i n s i c v a r i an c e ( un i t ) D ua l c on t r o l pe r f o r m an c e ( s u cc e ss e s / m i nu t e ) n n n x t y t e) D ua l c on t r o l v a r i an c e Intrinsic variance high Glow G early late
Fig 5. a) Control units chosen based on their tuning and activity in manual control recording (leftpanel) exert different levels of control over cursor in dual control recording (right panel). Dashedlines represent cursor motion contribution from individual control units, while solid purple linerepresents their aggregate. How much each contributes to cursor motion is measured with aGranger-causality metric, G, which reflects how much the cursor can be predicted on the basis ofthe activity of each control unit. b) High levels of G are positively correlated with high performanceindicated by warmer colors clustering to the right (F-test, p (cid:28) . p (cid:28) . R = 0 . p = 0 . We observed that overall tuning angles are significantly different between manual-control and dual-control. This finding is similar to observations made in a previous primate dual-control study [16],in which the authors found that producing static force with the arm during the BCI task perturbsthe neural map formed during the standard BCI task. Here we found that, tuning angles were moresimilar to tuning observed during natural motion during dual-control trials than brain-control trials.This suggests that the dual-control task engages native motor networks more than the brain-controltask does. It is also possible that natural movement networks are constrained to retain their naturaltuning during the dual control task.In the brain-control task, we observed control-unit specific changes in tuning angle away fromthose observed in the manual-control task. This effect, however, was absent in the dual-control task.Greater modification of control units tuning angle in the brain-control task appears to be consistentwith results from other studies in which visuomotor rotation was applied to a subset of unitscontrolling the cursor [22, 23]. Applied visuomotor rotation resulted in the rotated subpopulationshifting their tuning angles more than the non-rotated population. Conversely, between the dual-control and manual-control task, tuning angles changed similarly for both control and non-control10nits, suggesting that the dual-control task requires population-wide changes in its relation to wristmotion.Previous BCI studies show differential tuning strength of control versus non-control units [24, 25],while here we report that both control and non-control units undergo similar changes in tuningstrength in the brain control and dual control task. Interestingly, Ganguly et al. observed nodifferential tuning strength of control and non-control groups of units during the initial learningprocess, but a difference appeared after 2-3 days of performance with the same units. Similarly,Law et al [25] saw a stronger effect in late compared to early trials. This suggests that the shortamount of practice time in the present study for each pair of control units may be insufficient toaffect modulation depth.
We sought a better understanding of the changes between cotuned units underlying the dual-control task compared with the brain control task. We observed a selective decrease in effectiveconnectivity between co-tuned units and the control unit in dual control, and a non-selective decreasein connectivity between co-tuned units in brain control. This suggests that the dissociation requiredin the dual control task does affect interactions between individual units in the network. Our resultsshow that co-tuned networks do not change their relation with one another unless needed, when oneunit is chosen as a control unit. This is similar to the study of Hwang et al 2013 [5], which showsthat neurons maintain their relation to each other when a task is perturbed. Changes in effectiveconnectivity to support motor control tasks have been observed in other studies [26]. Thus effectiveconnectivity provides insight into changes that occur while performing a dual-control BCI task.Effective connectivity, sometimes referred to as functional connectivity, has been used to studyproperties of motor cortical networks also using directed information [27, 28] and Granger causality[29]. In this study we chose to use transfer entropy as a non-parametric measure of effectiveconnectivity, as it performed best on a synthetic validation dataset (Appendix A). A drawbackof all these methods is that they are unable to distinguish between direct interactions and theeffect of hidden, or latent inputs to recorded units. Given the 400 µ m electrode separation of theBlackrock Utah array used here, direct synaptic connections between units recorded on differentelectrodes are unlikely [30] and all relations between units may be mediated through unobservedconnections. A more appropriate model then may be one that separates latent and direct connectivitycomponents [31–33]. Finally, in the dual-control task we found that performance is related to each unit’s variability. Wefound that units with low intrinsic variability do not make a good choice for BCI control units. Thisis not a surprising result, as the BCI decoder will weigh units with higher variance more heavily.Similar to previous studies [6], the units with low variability in the manual control task do notincrease their variability in the dual control task, even if our results suggest that this would benefitperformance.Previous studies have reported that factors relating to variability are of primary importancewhen selecting BCI control units. For instance, Yu et al 2009 showed that BCI performance wasbetter when low-dimensional latent dynamics were inferred and utilized, independent of any kindof external movement or task parameter [34]. A related study showed that trial-by-trial spikepredictions based only on the firing rates of simultaneously recorded motor cortex neurons tend tooutperform predictions based on external parameters [35].Constraints imposed by how well the BCI mapping aligns with the population’s ‘intrinsic manifold’have also been identified [5, 6]. In this study the ‘control space’ is only two dimensional, whereasprevious studies have explored the concept in larger control spaces consisting of 10s to 100s ofunits. The low-dimensionality of the present study may facilitate insight into these larger controlspaces. In Sadtler et al 2014 [6], for instance, within-manifold perturbations were less detrimental11o performance than ‘outside manifold’ perturbations. In the present study the ‘control space’is deliberately limited to two dimensions to facilitate insight into how a focused population ofneurons must change their activity to accomplish the dual control task. It is possible that therole of variability of control units also explains these previous results. That is, it is possible thatthe intrinsic manifolds are predominantly defined by axes parallel to highly varying/firing units,and it is these units that dominate the manifold, not linear combinations of units. In other words,correlations between units may not be significant; within/outside manifold perturbations becomesimply the inclusion or otherwise of varying units that are able to contribute to the control of theBCI.Along these lines, Athalye et al 2017 [36] focus on individual (‘private’) and shared variability.They report that private variability decreases throughout learning while shared variability increases.Our sessions are likely too short to observe this transition. However the relevance of individualunits’ variability in dual-control performance, rather than their coordination, is consistent withthese findings. But these ideas warrant further study.
Our results demonstrate that dissociation of motor cortical units required by a dual-control taskcan occur with individual neuron specificity, through task-related changes in both the control andnon-control populations. This and previous work [5, 6] suggests that, provided specific constraintsare taken into account, the motor cortex can flexibly adapt to a variety of challenging controltasks. Tasks that require dissociations from established correlations between neural activity andmovement can be learned even over the course of a short recording session. Alongside their clinicalvalue as a potential treatment for stroke or other brain injury, BCI paradigms provide insight intothe physiological principles that guide motor control [37] by allowing the relevant populations formotor control and learning to be directly observed. As learning to coordinate separate tasks is animportant and general motor control skill, we believe the dual-control BCI promises to yield furtherinsight into the function of primary motor cortex.
Supporting informationA Validating effectivity connectivity analysis
There are a number of candidate measures of effective functional connectivity between units, includingGranger causality, transfer entropy and correlation [Nigam 2016, Sporns 2007]. Before applyingthese to our data we sought to understand each method’s success in identifying effective connectivityin a synthetic dataset with known connectivity. For this, we generated a dataset using a pointprocess model [38]. This model was designed to simulate an ensemble of cortical neurons in motorcortex, and has been used previously for making predictions about brain-BCI interactions. Theconnectivity methods we validated were covariance, a Granger causality metric based on a Poissonprocess GLM [29], a Granger causality metric based on a linear firing rate model, and transferentropy [19]. Details for each method are below. A simulated spiking network of 30 Izhikevichneurons is divided into 3 densely interconnected groups with some inter-group connections [38]. Ascommon input, the neurons are stimulated with a rectified sinusoidal input. The input was a 2 Hzsinusoidal envelope. The simulation was run for 3 min, with a baseline firing rate of 1 Hz (producingon average 17 Hz activity in simulation). For our transfer entropy analysis, we used the TransferEntropy Toolbox [19] with a 1ms bin size, a max delay of 20 ms, 1-3 time bins and 1-3 past timebins to calculate the transfer entropy between neurons. We then normalized the transfer entropymetric by z-scoring. For our linear Granger causality analysis, using MLE we fit a linear model ofthe activity of all neurons 1-20 ms in the past (including the past history of the unit being predicted)in order to predict the activity of unit i . We then removed unit j from the model and re-estimatedthe model. The change in deviance (normalized by z-scoring) between the old and new model was12 ROC - 3 min sin: 1 Hz min rate1-Sp S n ROC - 3 min sin: 5 Hz min rate1-Sp S n Method Area Method Area
Fig 6.
Validation of effective connectivity models. ROC curve comparing performance of effectiveconnectivity methods on simulated spike trains. The transfer entropy has the largest ROC area forin simulated data. Refer to Appendix A for more information. | ∆ θ | MC-MC MC-BC MC-DC 020406080100 control non-controlnon-control control120 | ∆ θ | ** * MC - BC MC - DCA)B) 00.10.20.30.40.5 ∆ s t r eng t h control non-controlnon-control controlMC - BC MC - DC00.10.20.30.40.5 ∆ s t r eng t h MC-MC MC-BC MC-DC ** * | | | | Fig 7.
Change in tuning angle between conditions for control units that were displayed in theirpreferred direction on the screen, rather than rotating by 90 degrees as in the remainder of thestudy. Changes are similar to those observed in the 90 degree rotated dataset (compare with Fig.2). Overall larger changes are observed between manual control and brain control tasks thanmanual control and dual control tasks. A) Tuning angle. Left: all p (cid:28) .
001 (two sided t-tests).Right: non-significant. B) Tuning strength. Left: MC-BC p (cid:28) . p = 0 . p = 0 . B Unrotated sessions tuning analysis
Refer to Fig. 7.
C Validation of Granger causality cursor control metric
Refer to Fig. 8. 13 P r opo r t i on o f t o t a l G C sc o r e o f un i t A BCI weight of unit A
Fig 8.
Validation of Granger-causality cursor control metric. Synthetic cursor trajectories aregenerated using neural data from two control units and the decoder (Eq 1). Contribution weightsfor each of the units are varied according to (Eq 1). 108 trajectories are generated for eachweighting and the cursor control metric, G j → i , is computed. Blue curve indicates mean proportionof control score as a function of weight; dashed red curves indicates standard deviation; dottedblack line indicates identity line. The Granger causality metric thus reliably measures thecontribution of each unit to the cursor motion. − − |score| l ag ( s ) FERU0 2 4 6 8 10 − − |score| l ag ( s ) FERU
A) B)C) D) U n i t U n i t offset (s) offset (s) Position encoding Velocity encoding
Fig 9.
Optimal time lags for simple tuning models. Peak of cross-correlation between filtered firingand flexion-extension (FE) and radial-ulnar (RU) torque output gives an optimal encoding lag foreach channel. The absolute value of the z-scored peak provides a simple encoding ‘strength’. a) Therelation between score and optimal lag for direct FE and RU output for cursor positions model. b)The same relation for time-differenced torque output – related to cursor velocity. Scores below 4are wildly varying in optimal lag, while scores above 4 are more tightly constrained to near zero.The ‘velocity encoding’ responses is optimal near a lag of zero. Similarly, fitting a simple GLMPoisson model with a stimulus filter of length 1 (only one time point) for different time lags queriesat what time delay the neuron’s firing is most informative about the stimulus. The change in eachmodel’s likelihood when the stimulus filter is included gives an indication of the strength of theencoding at this time lag. The optimal time lag can be computed by taking the maximum in thischange in likelihood. c) The optimal time lag for a set of units under a position encoding model. d)The optimal time lag for a set of units under a velocity encoding model. The mean optimal time lagwas approximately -100ms – indicating spiking occurs before the motor output. As a compromisebetween these two sets of analyses, a fixed time lag of -75ms was used.
D Optimal time lags for linear tuning models
Refer to Fig. 9. 14 cknowledgments
The authors thank Paul House for expert array implantations, Rob Robinson for animal handling anddata collection, Charlie Matlack for task design, and Larry Shupe for technical support. This workwas supported by an American Heart & Stroke Association Scientist Development Grant [NCRP09SDG2230091]; The Paul G. Allen Family Foundation (Allen Distinguished Investigator Award);National Institutes of Health [NS12542, RR00166]; the Center for Neurotechnology, a NationalScience Foundation Engineering Research Center [EEC-1028725]; and the WRF UW Institute forNeuroengineering (ALF).
References
1. J. L. Collinger, B. Wodlinger, J. E. Downey, W. Wang, E. C. Tyler-Kabara, D. J. Weber,A. J. C. McMorland, M. Velliste, M. L. Boninger, and A. B. Schwartz, “High-performanceneuroprosthetic control by an individual with tetraplegia,”
The Lancet , vol. 381, no. 9866, pp.557–564, 2013. [Online]. Available: http://dx.doi.org/10.1016/S0140-6736(12)61816-92. V. Gilja, C. Pandarinath, C. H. Blabe, P. Nuyujukian, J. D. Simeral, A. a. Sarma, B. L.Sorice, J. a. Perge, B. Jarosiewicz, L. R. Hochberg, K. V. Shenoy, and J. M. Henderson,“Clinical translation of a high-performance neural prosthesis.”
Nature medicine
Science , vol. 348, no. 6237, pp. 906–910, 2015.[Online]. Available: http://dx.doi.org/10.7910/DVN/GJDUTV4. L. R. Hochberg, D. Bacher, B. Jarosiewicz, N. Y. Masse, J. D. Simeral, J. Vogel, S. Haddadin,J. Liu, S. S. Cash, P. van der Smagt, and J. P. Donoghue, “Reach and grasp by people withtetraplegia using a neurally controlled robotic arm,”
Nature
Current Biology , vol. 23, no. 5, pp. 353–361, 2013. [Online].Available: http://dx.doi.org/10.1016/j.cub.2013.01.0276. P. T. Sadtler, K. M. Quick, M. D. Golub, S. M. Chase, S. I. Ryu, E. C. Tyler-Kabara, B. M.Yu, and A. P. Batista, “Neural constraints on learning,”
Nature , vol. 512, no. 7515, pp.423–426, 2014.7. E. E. Fetz, “Operant conditioning of cortical unit activity.”
Science
Journal of neurophysiology ,vol. 36, no. 2, pp. 179–204, 1973.9. E. E. Fetz, “Volitional control of neural activity: implications for brain–computer interfaces,”
The Journal of Physiology , vol. 579, no. 3, pp. 571–579, 2007.10. C. T. Moritz, S. I. Perlmutter, and E. E. Fetz, “Direct control of paralysed musclesby cortical neurons,”
Nature
Journal of neural engineering , vol. 8, no. 2, p. 25017, 2011.152. E. Fetz and D. Finocchio, “Correlations between activity of motor cortex cells and arm musclesduring operantly conditioned response patterns,”
Experimental Brain Research , vol. 23, no. 3,pp. 217–240, sep 1975. [Online]. Available: http://link.springer.com/10.1007/BF0023973613. I. Milovanovic, R. Robinson, E. E. Fetz, and C. T. Moritz, “Simultaneous andindependent control of a brain-computer interface and contralateral limb movement.”
Brain computer interfaces (Abingdon, England) . IEEE, sep 2009, pp. 5502–5505. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=533455915. L. Bashford, J. Wu, D. Sarma, K. Collins, J. Ojemann, and C. Mehring, “Natural movementwith concurrent brain-computer interface control induces persistent dissociation of neuralactivity,” in
BCI Society Meeting , CA, USA, 2016.16. A. L. Orsborn, H. G. Moorman, S. A. Overduin, M. M. Shanechi, D. F. Dimitrov, andJ. M. Carmena, “Closed-Loop Decoder Adaptation Shapes Neural Plasticity for SkillfulNeuroprosthetic Control,”
Neuron , vol. 82, no. 6, pp. 1380–1393, 2014. [Online]. Available:http://linkinghub.elsevier.com/retrieve/pii/S089662731400363817. S. Shoham, L. Paninski, M. Fellows, N. Hatsopoulos, J. Donoghue, and R. Normann,“Statistical Encoding Model for a Primary Motor Cortical Brain-Machine Interface,”
IEEETransactions on Biomedical Engineering , vol. 52, no. 7, pp. 1312–1322, jul 2005. [Online].Available: http://ieeexplore.ieee.org/document/1440610/18. T. Schreiber, “Measuring information transfer,”
Physical review letters
PLoS ONE , vol. 6, no. 11, p. e27431, nov 2011. [Online]. Available:http://dx.plos.org/10.1371/journal.pone.002743120. L. Barnett and A. K. Seth, “The MVGC multivariate Granger causality toolbox: A newapproach to Granger-causal inference,”
Journal of Neuroscience Methods , vol. 223, pp. 50–68,2014. [Online]. Available: http://linkinghub.elsevier.com/retrieve/pii/S016502701300370121. L. Barnett, A. B. Barrett, and A. K. Seth, “Granger Causality and Transfer EntropyAre Equivalent for Gaussian Variables,”
Physical Review Letters , vol. 103, no. 23, pp.238 701–238 701, dec 2009. [Online]. Available: http://link.aps.org/doi/10.1103/PhysRevLett.103.23870122. B. Jarosiewicz, S. M. Chase, G. W. Fraser, M. Velliste, R. E. Kass, and A. B. Schwartz,“Functional network reorganization during learning in a brain-computer interface paradigm.”
Proceedings of the National Academy of Sciences of the United States of America , vol. 105,no. 49, pp. 19 486–19 491, 2008.23. S. M. Chase, R. E. Kass, and A. B. Schwartz, “Behavioral and neural correlates ofvisuomotor adaptation observed through a brain-computer interface in primary motorcortex.”
Journal of neurophysiology
NatureNeuroscience
Journal of neurophysiology
Science , vol. 318, no. 5858, pp. 1934–1937,2007.27. C. J. Quinn, T. P. Coleman, N. Kiyavash, and N. G. Hatsopoulos, “Estimating thedirected information to infer causal relationships in ensemble neural spike train recordings,”
Journal of Computational Neuroscience , vol. 30, no. 1, pp. 17–44, 2011. [Online]. Available:http://link.springer.com/10.1007/s10827-010-0247-228. Z. Zhe Chen, D. Putrino, D. Ba, S. Ghosh, R. Barbieri, and E. Brown, “A regularizedpoint process generalized linear model for assessing the functional connectivity in thecat motor cortex,” in
PLoS Computational Biology , vol. 7,no. 3, p. e1001110, 2011. [Online]. Available: http://dx.plos.org/10.1371/journal.pcbi.100111030. W. S. Smith and E. E. Fetz, “Synaptic Linkages Between Corticomotoneuronal Cells AffectingForelimb Muscles in Behaving Primates,”
Journal of Neurophysiology , vol. 102, no. 2, 2009.31. D. Pfau, E. A. Pnevmatikakis, and L. Paninski, “Robust learning of low-dimensional dynamicsfrom large neural ensembles,” pp. 2391–2399, 2013.32. J. Semedo, A. Zandvakili, A. Kohn, C. K. Machens, and B. M. Yu, “Extracting LatentStructure From Multiple Interacting Neural Populations,” pp. 2942–2950, 2014.33. J. H. Macke, L. Buesing, J. P. Cunningham, B. M. Yu, K. V. Shenoy, and M. Sahani,“Empirical models of spiking in neural populations,” pp. 1350–1358, 2011.34. B. M. Yu, J. P. Cunningham, G. Santhanam, S. I. Ryu, K. V. Shenoy, and M. Sahani,“Gaussian-process factor analysis for low-dimensional single-trial analysis of neural populationactivity,”
Journal of Neurophysiology , vol. 102, no. 1, p. 614, 2009. [Online]. Available:papers2://publication/uuid/68746D64-7B64-42F9-95B8-F2BBCC41D80135. I. H. Stevenson and K. P. Kording, “How advances in neural recording affect dataanalysis,”
Nature Neuroscience
Neuron , vol. 93, no. 4, pp. 955–970.e5, 2017. [Online]. Available:http://dx.doi.org/10.1016/j.neuron.2017.01.016177. M. D. Golub, S. M. Chase, A. P. Batista, and B. M. Yu, “Brain–computerinterfaces for dissecting cognitive processes underlying sensorimotor control,”
CurrentOpinion in Neurobiology , vol. 37, pp. 53–58, 2016. [Online]. Available: http://dx.doi.org/10.1016/j.conb.2015.12.00538. G. Lajoie, N. Krouchev, J. F. Kalaska, A. Fairhall, and E. E. Fetz, “Correlation-based modelof artificially induced plasticity in motor cortex by a bilateral Brain-Machine Interface,”