Discrimination Among Multiple Cutaneous and Proprioceptive Hand Percepts Evoked by Nerve Stimulation with Utah Slanted Electrode Arrays in Human Amputees
David M. Page, Suzanne M. Wendelken, Tyler S. Davis, David T. Kluger, Douglas T. Hutchinson, Jacob A. George, Gregory A. Clark
1 Abstract — Objective:
This paper aims to demonstrate functional discriminability among restored hand sensations with different locations, qualities, and intensities that are evoked by microelectrode stimulation of residual afferent fibers in human amputees.
Methods:
We implanted a Utah Slanted Electrode Array (USEA) in the median and ulnar residual arm nerves of three transradial amputees and delivered stimulation via different electrodes and at different frequencies to produce various locations, qualities, and intensities of sensation on the missing hand. Blind discrimination trials were performed to determine how well subjects could discriminate among these restored sensations.
Results:
Subjects discriminated among restored sensory percepts with varying cutaneous and proprioceptive locations, qualities, and intensities in blind trials, including discrimination among up to 10 different location-intensity combinations (15/30 successes, p < 0.0005 ). Variations in the site of stimulation within the nerve, via electrode selection, enabled discrimination among up to 5 locations and qualities (35/35 successes, p < 0.0001 ). Variations in the stimulation frequency enabled discrimination among 4 different intensities at the same location (13/20 successes, p < 0.005 ). One subject discriminated among simultaneous, alternating, and isolated stimulation of two different USEA electrodes, as may be desired during multi-sensor closed-loop prosthesis use (20/25 successes, p < 0.001 ). Conclusion:
USEA stimulation enables encoding of a diversity of functionally discriminable sensations with different locations, qualities, and intensities.
Significance:
These percepts provide a potentially rich source of sensory feedback that may enhance performance and embodiment during multi-sensor, closed-loop prosthesis use.
Index Terms —Amputee, haptics, microelectrode array, nerve stimulation, neural interface, neural prosthesis, peripheral nerve interface, proprioception, prosthesis, prosthetic limb, sensory feedback, tactile feedback, touch I. I NTRODUCTION
LINICALLY-available arm prostheses do not currently provide amputees with naturalistic and detailed tactile sensory feedback. Sensation from a prosthesis has been shown to be important for performance of functional tasks and for prosthesis embodiment [1]–[4], and amputees indicate interest in having sensory feedback from their prosthesis [5]–[10]. Peripheral-nerve interface approaches, such as Utah Slanted Electrode Arrays (USEAs), cuff electrodes, transverse intrafascicular multichannel electrodes (TIMEs), flat interface nerve electrodes (FINEs), and longitudinal intrafascicular electrodes (LIFEs) have demonstrated the ability to evoke sensory percepts at different locations, and of different qualities (e.g., submodalities) and intensities on the missing hand of amputees [11]–[18]. However, few of these have formally assessed functional discriminability among sensory percepts at different locations, and of different intensities and qualities [16], [18]. Basic functional discrimination has been shown for objects of different shapes/sizes and compliances during closed-loop prosthesis control [1], [4], [19], [20]. The USEA provides intrafascicular access to nerve fibers spanning the cross-section of a peripheral nerve via 100 penetrating microelectrodes. In contrast to other peripheral nerve interfaces, USEAs offer cross-sectional nerve access via many channels, enabling activation of numerous sensory percepts spanning the hand [15]–[17]. The selection of different stimulation electrodes enables activation of different axons or subsets of axons with different projected field locations on the hand, and potentially with different sensory qualities. The stimulus intensity at each location can be encoded based on the frequency of stimulation [15], [21]. Despite this understanding, prior publications using USEAs have not fully tested the extent to which human subjects can discriminate among multiple proprioceptive and cutaneous sensory percepts at different locations, and of different qualities and intensities, such as would be desirable during multisensor closed-loop prosthesis control. The extent of restored sensory feedback needed to reproduce a naturalistic sense of touch via a prosthetic limb is uncertain. However, discrimination tasks in intact human hands suggest that encoding of many locations, qualities and intensities, potentially via different receptor subtypes, is likely needed. For example, cutaneous location discrimination in the intact hand has been performed previously via a 2-point discrimination task, in which functional discriminability was achievable for stimuli as close as 0.55 mm apart [22]. This high level of discriminability is likely attributable to intensity encoding via a population of afferents both close to, and distant from, the site of applied tactile pressure (receptor density is on the order of 1 per square millimeter on the palmar hand [23], [24]). Natural activation patterns in the human hand include activation of several different cutaneous mechanoreceptor subtypes innervating many different locations on the hand. The number of discriminable locations on an intact human hand has not been formally quantified, but, based on these prior publications, is likely on the order of hundreds of sensory locations. In microneurography studies, intact subjects have also discriminated among tactile percepts with the same location, but with different intensities. A roughly linear, nearly 3-fold increase in perceived intensity was noted both for normal
David M. Page, Suzanne M. Wendelken, Tyler S. Davis, David T. Kluger, Douglas T. Hutchinson, Jacob A. George and Gregory A. Clark
Discrimination Among Multiple Cutaneous and Proprioceptive Hand Percepts Evoked by Nerve Stimulation with Utah Slanted Electrode Arrays in Human Amputees C M ATERIAL AND M ETHODS A. Volunteers
Three transradial amputees participated in this study, referred to as S3, S4, and S5. Subject S3 was a 50-year-old male with a left-arm amputation that had occurred 21 years prior. Subject S4 was a 36-year-old male with bilateral amputations that had occurred 16 years prior. Subject S5 was a 43-year-old male with bilateral amputations that had occurred 24 years prior. Each subject underwent psychological and medical assessments prior to participating in the study. Subjects were provided with training materials prior to implantation of the electrodes to allow them to learn the concepts and methods of the experiments in advance and thus reduce post-implant training time. These included mirror-box or prosthesis-video training materials [35] as reported with previous subjects [15]–[17]. The subjects were monitored for medical risks both during and after the implant period, and subjects S4 and S5 were treated for implant-related infections that resolved without issue. The consenting process and experimental procedures were approved by the University of Utah Institutional Review Board, and the Department of the Navy Human Research Protection Program. B. Device
Two USEAs (Blackrock Microsystems, Salt Lake City, UT, USA) were implanted in each subject: one in the median nerve and one in the ulnar nerve. The implant location for subject S3 was in the left forearm distal to the elbow, whereas the implants for subject S4 and S5 were placed midway along the left upper arm, proximal to the medial epicondyle and to many motor branch points. USEAs consisted of 100 silicon microelectrodes (sputtered iridium oxide) spaced 400 m apart in a 10x10 grid across a 4x4 mm square base. The electrodes varied from ~0.75 to 1.5 mm in length to allow cross-sectional access to fibers at different depths within the peripheral arm nerves [36]. Separate looped platinum wires were also implanted as stimulation return leads and for use as recording reference and ground leads. These looped platinum wires were placed close to (within ~5 mm of) the USEAS at the time of implantation, and were generally sutured to the epineurium along with the USEA lead wires within a centimeter of each USEA [37]. Electrical connection to each USEA electrode was available via an external printed circuit board that was coupled percutaneously to USEAs via a bundle of gold lead wires. Connection of the external circuit board to stimulation and recording hardware was made via a ZIF-Clip-96 connector cable (Tucker-Davis Technologies Inc., Alachua, FL, USA) for S3 and S4, or a 96-channel Gator connector cable (Ripple LLC, Salt Lake City, UT, USA) for S5. The slanted nature of the USEAs enables cross-sectional nerve access to fibers at different depths, thereby increasing the possibility of activation of different axons or subsets of axons with each electrode [36]. An effort was made during the implant surgery to implant USEAs into the nerves so that the electrodes were positioned squarely perpendicular to the length of the nerve, which maximizes the cross-sectional nerve coverage of the USEA electrodes. The two-dimensional distance between two electrodes on the cross-sectional projection plane is likely the most influential factor on their ability to activate different axons or subsets of axons (Fig. 1). The stimulation amplitude on a given electrode influences which axons near the tip of the electrode are activated, whereas the stimulation frequency 3 influences their firing rate. The stimulation amplitude may also influence firing rate when modulated at peri-threshold levels, for example, when only a subset of stimulation pulses in a pulse train result in generation of an action potential [18]. C. Surgical and Experimental Procedures
Subjects were given prophylactic antibiotics the day before, the day of, and for several days following the implant surgery (100 mg minocycline, 7 days, twice per day). USEAs were implanted in each subject under general anesthesia, via similar methods to those described in past publications [37]. For subject S5, electromyography leads were also placed in the muscles of the forearm for recording purposes, as described in [20], [37]. After exposure of each nerve implant site, the epineurium was dissected away, and USEAs were inserted into the nerve using a pneumatic insertion tool (Blackrock Microsystems, Salt Lake City, UT, USA) [38]. USEA lead wires and reference and ground wires were sutured to the epineurium, and a collagen wrap (AxoGen Inc., Alachua, FL, USA) was secured around the USEA, nerve, and reference and ground wires using vascular clips (Fig. 2a). For subject S5, the epineurium was sutured around the USEAs and reference and ground wires prior to placement of the collagen wrap. Dexamethasone (0.1 mg/kg) was delivered intravenously to the subjects during surgical closure as a potential means for decreasing the foreign body response [39], [40]. The site of percutaneous wire passage (Fig. 2b) was redressed roughly once per week using an antibiotic wound patch (Biopatch, Ethicon US LLC, Somerville, NJ, USA). Subjects S4 and S5 both experienced infections at the USEA percutaneous wire passage site with subsequent full recoveries after USEA extraction and antibiotic treatment. Implants were removed after 4 weeks, 5 weeks, and 13 weeks, for S3, S4, and S5, respectively. The USEAs from subject S3 were removed along with the section of implanted neural tissue for histological analysis [41]. Experimental sessions were typically carried out 1-3 days per week, for 1-5 hours each. In addition to the stimulation-evoked sensory percepts reported here, experiments consisted of impedance testing, decoding of neuronal and myoelectric signals for prosthesis movement control, and closed-loop control of a prosthetic hand. D. Microstimulation
Electrical stimulation was delivered using the IZ2-128 System (Tucker-Davis Technologies Inc., Alachua, FL, USA) for S3 and S4, or the Grapevine System (Ripple LLC, Salt Lake City, UT, USA) for S4 and S5. Stimulation pulses were biphasic (cathodic first) with each phase typically having a duration of 200 s (with a 100- s interphase interval). Stimulation amplitudes and frequencies were kept below 120 µA and 500 Hz. These stimulation parameters were approved by the University of Utah IRB and have been used with USEAs in multiple subjects before without any considerable neural damage [41]. Subjects used either custom software to indicate the location, quality, and intensity or size of each USEA-evoked sensory percept on the image of a hand, or verbal descriptions. Subjects selected percept qualities from a list or created their own descriptors as necessary. Representations of percept locations and sizes, such as those shown in Figs. 3-7, were created based on the subjects’ software markings as well as their verbal descriptions as appropriate. Full-USEA threshold maps were collected periodically for each subject, as described previously [15]–[17]. These full-USEA maps provided a basis for selection of the electrodes used in the discrimination trials reported here. The electrodes chosen for a given discrimination task were typically selected based on outputs of a threshold mapping session performed within a week prior. Electrodes were chosen which provided distinct percepts with the desired location and/or quality. The temporal stability of sensory percepts or impedance was not generally used as a criteria for selection. During discrimination trials for subjects S3 and S4, a 200-ms train of stimulation was delivered at 200 Hz each time the subject or an experimenter pressed a button. For subject S5, three or four 500-ms trains of 100-Hz stimulation (unless noted otherwise, such as during intensity-encoding sessions) were delivered at a 50% duty cycle after the subject or the experimenters pressed a button. Subject S5 was typically instructed to determine the final percept intensity, quality, and location classification on the basis of the percept evoked by either the initial train in a trial or the final train in a trial for a given session, although practices varied depending on the session. Prior to discrimination trials, the activation threshold amplitude for each electrode (in A) was determined by incrementally increasing the amplitude until the subject perceived a sensation. E. Discrimination Trials and Data Analysis
Discrimination trials were performed by all three subjects during different stimulation sessions. A stimulation session typically included mapping the percept locations, qualities, and intensities associated with several different USEA electrodes, and then down selecting to electrodes and stimulation frequencies which represented a subset of locations, qualities, intensities, or combinations for formal discrimination trials. For location discrimination trials, electrodes were preferentially down-selected to represent sensation on different gross anatomical hand regions, such as different digits and the palm. When available, electrodes that evoked sensations at the same location but with different qualities were used for quality discrimination trials. Intensity discrimination trials were performed using different frequency encoding patterns on the same electrode or subset of electrodes. The number of electrodes selected for each discrimination study was based on the time remaining in a session and the desire to compare multiple trials of stimulation on each electrode in a blinded, randomized fashion. No sensory discrimination tasks were attempted with more functional levels (e.g., location, quality, intensity, or combinations) than those reported herein. Discrimination trial results reported here were not pooled across sessions or subjects; however, we have included results from similar discrimination trial configurations for different subjects. 4 Discrimination experiments were performed by delivering randomly-ordered stimulation trials in which the subject was required to classify the location, quality, and/or intensity of the evoked percept for each trial. Stimulation conditions varied across trials, including stimulation via different USEA electrodes or combinations of electrodes, and/or use of different stimulation frequencies. Formal discrimination trials were preceded by informal practice trials in which the subject experienced each different stimulation condition and formulated category labels for the percept associated with the condition. Once the subject felt comfortable identifying the location, quality, and/or intensity of the different stimulation conditions, formal blind trials commenced in which the subject was required to select one of his predetermined percept categories in response to each stimulation trial. For subject S3 and S4, discrimination trial stimulation conditions included different electrodes and combinations of electrodes. For subject S5, stimulation conditions included different electrodes and/or stimulation frequencies. Importantly, catch trials (no stimulation) were added as a stimulation condition for subjects S4 and S5 to test the hypothesis that sensory percepts were indeed evoked by USEA stimulation (in contrast to pseudesthesia). A summary of discrimination tasks and the number of USEA electrodes used can be found in Table 1. Data analysis for discrimination trials was performed using the binomial test, where the probability of guessing the correct classification on a given trial was determined as the inverse of the number of predetermined classification categories. Hypothesis testing was performed with a critical value of α = 0.05. A Bonferroni adjustment was made to the critical value for post-hoc tests by dividing the critical value by the number of post-hoc tests performed. III. R ESULTS
All subjects successfully performed functional discrimination trials for percepts at different locations and of different qualities; percepts with the same location but different qualities; and percepts with the same location and quality but with different intensities. Additionally, subject S5 performed combined location/quality/intensity discrimination trials, including trial sets with cutaneous percepts and trial sets with proprioceptive percepts. Functional discrimination among percepts of different locations, qualities, and intensities will be important for future use of sensory feedback from multiple prosthesis-coupled sensors during closed-loop prosthesis control. A. Location Discrimination
Subject S3 successfully discriminated among 5 stimulation conditions that evoked sensation at five different hand locations: ring finger tip, little finger tip, little finger base, wrist, and combined perception at all four of these locations. These percepts were evoked by individual stimulation of four ulnar-nerve-USEA electrodes and combined simultaneous stimulation of all four of these electrodes, respectively. Stimulation amplitudes for the four electrodes ranged from 14- 30 A. The subject discriminated among these stimulation conditions by classifying the percept evoked into one of the 5 predetermined classification categories in 35/35 successful trials (20% chance per trial, p < m within the nerve, yet they each evoked consistently unique sensory percepts, suggesting selectivity in axon activation. Additionally, the combined stimulation of all four electrodes did not result in emergent sensory percepts (a percept with different quality or location from the four individual percepts), suggesting that current summation during simultaneous stimulation was limited. To better study current summation during simultaneous stimulation of multiple electrodes in subject S3, we selected two ulnar-nerve-USEA electrodes with tips placed less than ~899 m apart within the nerve (~805 m cross-sectional projection separation assuming USEAs were implanted squarely perpendicular to the nerve) and delivered four stimulation conditions: individual stimulation of each of the two electrodes in isolation, simultaneous stimulation of both electrodes with no time shift, and simultaneous stimulation of both electrodes with a 3-ms time shift relative to each other to produce an interleaved stimulation pattern between the two electrodes. Stimulation amplitudes for the two electrodes were 23 A and 20 A, and stimulation was delivered continuously for 4 s during each trial. The individual stimulation via two different electrodes consistently produced sensations of little-finger-tip sting and lateral-palm tingle, respectively, whereas interleaved stimulation of these electrodes (3 ms time shift difference, 200 Hz) consistently reproduced both of these percepts concurrently with no emergent sensations, and simultaneous stimulation (no time shift difference, 200 Hz) consistently produced both of these percepts concurrently accompanied by an emergent ‘massage’ feeling bridging between them. The participant successfully identified among these four sensations in 20/25 trials (25% chance per trial, p < 0.001, binomial test; Fig. 3b). One plausible explanation for the emergent massage feeling during simultaneous stimulation with no time shift difference is that multiple additional axons may have been activated due to spatiotemporal current summation from the two electrodes [36]. Subject S4 also performed location-discrimination trials, including discrimination among eight different cutaneous stimulation configurations: individual stimulation of each of 3 ulnar-nerve-USEA electrodes, simultaneous combined stimulation using each combination of subsets of 2 of these 3 electrodes, simultaneous combined stimulation using all 3 electrodes, and no stimulation. The participant successfully identified between these sensations in 11/24 trials (12.5% chance per trial, p < A depending on the electrode. Single-electrode percepts included sensation of tingle on the ring finger, touch on the little finger and palm (sometimes with an associated sense of little-finger movement), and tingle on the outer edge of the little finger. The 5 precise nature of combination percepts was not fully documented prior to beginning the formal trials; but informally, the subject indicated that they consisted of a combined sensation of the percepts evoked by the individual electrodes, potentially with fused projected fields or emergent sensations. Importantly, these trials also included a condition of “no stimulation,” which was not included in testing with subject S3. Subject S4 successfully identified when stimulation was delivered compared with when no stimulation was delivered in 24/24 trials (50% chance per trial, p < B. Quality Discrimination
Subject S3 successfully discriminated between two evoked percepts with the same location, but with two distinct qualities, produced via stimulation of two different ulnar-nerve-USEA electrodes (Fig. 4). The tips of these electrodes were separated by ~2.1 mm within the nerve (~578 m cross-sectional projection separation assuming USEAs were implanted squarely perpendicular to the nerve). Stimulation amplitudes for the two electrodes were 11 A and 12 A. Prior to formal discrimination trials, the subject identified the percepts evoked by these two different electrodes as having identical intensities and locations near the ring-fingertip (“Right on, exact same space”), but differing qualities of vibration and tingle, respectively. In subsequent formal trials, the subject consistently discriminated between the percepts evoked by the two electrodes in 30/30 trials (50% chance per trial, p < C. Intensity Discrimination
Subject S5 successfully discriminated among 4 different cutaneous-percept intensities, encoded via stimulation with different frequencies on a single median-nerve-USEA electrode that evoked a sensation of tingle on all four fingertips, although the percept seemed to isolate to the middle-finger alone during later stimulation trials. The stimulation amplitude used during trials was 25 A. During informal practice trials, the subject designated four intensity levels as “high,” “medium,” “light,” or “nothing,” corresponding to stimulation at 100 Hz, 70 Hz, 35 Hz or no stimulation, respectively. During subsequent formal trials, the subject correctly classified these percept intensities in 13/20 trials (25% chance per trial, p < D. Combined Location and Intensity Discrimination
Subject S5 performed combined location- and intensity-discrimination trials, similar to what may be used as part of a multisensor closed-loop prosthesis. Trials were performed for both cutaneous and proprioceptive percepts, each with multiple intensity levels encoding either cutaneous pressure/touch, or joint position. Three cutaneous percepts were encoded at distinct hand locations via median-nerve-USEA stimulation on three different electrodes, with associated percept descriptions of index-fingertip pressure, middle-fingertip touch, and palm pressure (stimulation amplitudes ranged from 17-64 A). For each of these electrodes, stimulation was delivered at stimulation frequencies of 30 Hz, 70 Hz, or 100 Hz, corresponding to “light,” “medium,” and “heavy” touch or pressure for each location. Sham stimulation was also used (i.e., no stimulation), making a total of 10 classification categories (three intensities at each of three percept locations, plus sham). Subject S5 successfully discriminated among these 10 stimulation conditions in 15/30 trials (10% chance per trial, p < p < p =
Subject S5 also successfully performed combined location and quality discrimination for two proprioceptive percepts that encoded index-finger and middle-finger flexion positions, respectively, via median-nerve USEA stimulation (Fig. 7). Specifically, 17- A stimulation was delivered at 30 Hz, 80 Hz, or 150 Hz on one median-nerve USEA electrode to encode 10 , 90 , or ~180 /fully-closed flexion on the middle finger (compared to rest position). On a different median-nerve USEA electrode (~1.6 mm away; ~409 m separation in nerve cross-sectional projection assuming USEAs were implanted squarely perpendicular to the nerve), 40- A stimulation was delivered at 100 Hz, 50 Hz, or 150 Hz to encode 20 , 50 , or ~180 /fully-closed flexion on the index finger. During practice trials, the subject felt strongly that the nonmonotonic frequency-intensity encoding for the index finger joint position was accurate. However, during formal trials, confusion among the 20 , 50 , and 180 conditions on the index finger was common. A sham condition was also included, creating a total of 7 classification categories (three intensities on each of two digits, plus sham). Subject S5 successfully discriminated among these proprioceptive digit and joint-position combinations in 21/40 trials (14.3% chance per trial, p < p < p < E. Qualitative Descriptions of Sensory Percepts
Our subjects reported enjoyment of the variety of sensations evoked by USEA stimulation, including both proprioceptive 6 and cutaneous sensations. After his first stimulation session, subject S3 stated, “My hand is starting to stimulate like it’s starting to wake up or something. It really feels good. […] It’s good to know that there’s something still there.” In response to the proprioceptive percept of middle-finger flexion delivered during proprioception discrimination trials, subject S5 stated that the sensation felt “exactly like movement of the middle finger.” When asked to describe one of the sensory percepts evoked during cutaneous location-intensity discrimination trials, subject S5 stated, “It feels like touch. It feels like if I touched that door.” Subject S5 also indicated that depth perception was limited in the virtual environment. IV. D ISCUSSION
We have demonstrated that USEA stimulation can be used to encode sensory percepts with functionally-discriminable locations, qualities, and intensities. Further, discrimination was possible for both cutaneous and proprioceptive percepts. Encoding of cutaneous sensory percepts with different locations and qualities was achieved by stimulation of different USEA electrodes or combinations of electrodes, presumably resulting in activation of different axons or subsets of axons within the nerve. Encoding of sensory percepts with different intensities was achieved by modulation of the stimulation frequency, presumably resulting in an increased firing rate in activated axons. We have also demonstrated that subjects can discriminate among multiple location-intensity combined percepts such as would be desired during closed-loop prosthesis control. Additionally, we have shown that multi-electrode stimulation in an interleaved pattern allows for simultaneous activation of multiple sensory percepts without emergent sensations (in comparison with simultaneous stimulation patterns, which did evoke emergent sensations). Future use of simultaneous and interleaved multielectrode stimulation may allow for improvements in the number and nature of restored percepts. This result also provides an important proof-of-concept for a method of interleaving stimulation via different USEA electrodes when current-summation effects are not desired, for example, during closed-loop prosthesis control with simultaneous USEA-evoked sensory feedback from multiple prosthesis sensors. Although we have shown that USEA electrodes as close as 800 m within the nerve cross-section can evoke distinct sensory percepts, simultaneous stimulation via these electrodes often results in current summation and potentially undesired activation of additional axons that evoke additional sensation. Use of interleaved stimulation allows for simultaneous generation of the individual sensory percepts without current-summation effects. During closed-loop prosthesis control, interaction with the external environment may result in simultaneous activation of multiple prosthesis sensors, potentially generating simultaneous stimulation via multiple USEA electrodes. Algorithms may be developed and incorporated to interleave stimulation on different USEA electrodes to prevent current-summation effects. One tradeoff of interleaving stimulation is that a more frequent occurrence of stimulation artifact will likely be produced in USEA electrode recordings, possibly interrupting the ability to perform neural recording decodes for prosthesis movement control. In this case, it may be desirable to develop stimulation artifact blanking approaches or to implant separate recording electrodes in a distant location where stimulation artifact will be minimized (e.g., the residual limb muscles or a distant nerve location). It is uncertain in some cases whether proprioceptive percepts were elicited by activating proprioceptive fibers directly or by generating secondary proprioceptive signaling after activating motor fibers. In the majority of cases, we did not observe visible muscle twitching in the residual limb during USEA stimulation; however, there is a possibility that single motor fibers were activated without producing a visible muscle twitch. However, proprioceptive percepts associated with movements of missing hand muscles do not suffer from this potential confound and hence are likely due to direct activation of proprioceptive fibers. Sensory feedback from the hand has been shown to be important for identifying when contact events between the hand and the environment occur and for identifying object properties such as curvature, texture, and weight. These complex properties are interpreted using sensory integration across various proprioceptive and cutaneous channels with many receptive fields. Cutaneous information, encoded via multiple different receptors (e.g., slowly-adapting I, slowly-adapting II, rapidly-adapting I, and rapidly-adapting II), provides information regarding contact locations, object texture, object slippage, and gross shape [26], [28], [42]–[45]. Proprioceptive channels provide information regarding hand conformation and position, which, in conjunction with cutaneous information, provides information regarding object shape, weight, and counterforce [46]. Many of these object properties are challenging to deduce using visual feedback alone, particularly when feedback is needed rapidly during motor tasks [47] or when handling opaque objects. The goal of functional discrimination among a variety of sensory channels is ultimately to provide the brain with sufficient information to deduce useful information regarding interactions with the external environment. Our gross encoding of 3 stimulus locations, each with 3 different intensities, may be sufficient to assist subjects in identifying gross object properties such as size and compliance [4]. However, more complex properties such as curvature and skin indentation direction and force gradations will likely require encoding via sensory percepts of different submodalities (e.g., RAI and SAI) that have nearby projected fields [48]. Restored sensation via multiple axons with adjacent projected fields may be critical for naturalistic sensorimotor hand control because real-time neural encoding of object properties likely involves cortical comparison of spike timings from neurons with adjacent receptive fields [49]. We anticipate that functional prosthesis control will improve with increasing numbers and variety of discriminable sensory feedback channels. Importantly, the data reported in this paper represents the most complex sensory discrimination tasks attempted with subjects S4 and S5; future studies should be designed to attempt 7 more sophisticated discrimination tasks. In addition to functional performance benefits of discriminable, multi-sensor prosthesis feedback, we anticipate that there will be substantial psychological benefits to restoring sensory feedback to amputees, such as prosthesis embodiment and reduction in phantom pain [2], [3]. We hypothesize that the sense of prosthesis embodiment will increase as a function of the number of discriminable sensory percepts provided for feedback. The ultimate goal of restored prosthesis sensation is not just to provide subjects with a useful tool, but also to provide subjects with a prosthesis that is perceived by subjects as a replacement hand. Although the results of this report do not begin to approximate the sophistication of an intact hand (hundreds of discriminable cutaneous locations, and ~10 discriminable force levels), this work represents a substantial improvement from commercially available prostheses with no sensory feedback. Specifically, we have demonstrated that different USEA-evoked percepts are discriminable from each other, including up to 3 gross-level hand regions such as different digits and the palm, each with 3 different intensities. Although this does not recreating the performance of intact human hands, this level of discriminability has been shown to result in substantial functional improvements in fragile object manipulation and haptic perception [1], [4], and performance may improve with long-term use [50]. We have also shown that this discriminability can be achieved across multiple trials in a discrimination task during a single session, demonstrating that percepts are not pseudesthesias. Ongoing work should focus on discrimination among successively closer projected fields to identify minimum discriminable distances. Additionally, interleaved, multielectrode stimulation strategies may produce surround inhibition effects that could improve functional discrimination. Although USEAs offer the highest channel count of any peripheral nerve interface, the 100 channels likely will not provide the incredibly fine level of resolution that would be required to completely restore sensory hand function. Development of a neural interface that may provide such resolution remains as a substantial challenge to the field. One minor limitation of this report is the lack of control of the intensity and/or quality of sensory percepts during location discrimination trials. Specifically, subjects described the different sensory percepts by indicating both their location and their quality, raising the possibility that the discrimination may not have been performed based on location alone. The same argument can be made for the intensity of the percept, as the intensity of the sensation at each location was not carefully noted and was not controlled. Future studies regarding location discrimination should attempt to control for the quality and intensity of the percepts. We have also demonstrated in this report that selective activation of distinct axons or subsets of axons is possible using USEA electrodes as close as ~800 m within the nerve. Stimulation amplitudes were between 7-64 A for the trials reported here, which apparently allowed for focal activation of axons within the local area of an electrode tip without activating axons associated with electrodes ~800 m away. Future testing should be performed using closer electrodes, such as neighboring electrodes that are ~400 m apart, to see if selectivity is achievable. Additionally, we anticipate that selectivity will decrease primarily as a function of cross-sectional projection distance, suggesting that electrodes that are directly distal/proximal to each other are less likely to evoke selective sensory percepts due to the possibility that the same axon(s) will pass near each electrode tip. More data from electrodes with a variety of different cross-sectional projection distances is needed to perform such an assessment. Future USEA designs may use a steeper slant to allow for improved selectivity along distal-proximal rows. The results reported here used generally frequencies above 100 Hz for location discrimination trials, as these frequencies tended to produce consistently distinct percepts. Prior work has shown that both frequency and amplitude can be used to modulate the perceived intensity of percepts [18], suggesting that location discrimination would be comparable across stimulation frequency and amplitude as long as the perceived intensity was similar. Nevertheless, alternative methods for encoding intensity in sensory percepts should also be investigated including the use of stimulation amplitude or activation of multiple neighboring electrodes. Subject S5 often indicated that perithreshold stimulation amplitudes evoked weak percepts compared with the stronger percepts evoked by suprathreshold amplitudes at the same stimulation frequency (comfort-level amplitude was typically 5-10 uA above threshold amplitude). We hypothesize that this intensity change is not due to recruitment of additional nerve fibers at these increasing amplitudes, but rather is due to the increased probability of evoking an action potential with each stimulation pulse at suprathreshold levels (compared with perithreshold levels), effectively increasing the firing frequency of the axon(s). Inherent in this hypothesis is the prediction that an increase in stimulation amplitude may encode increasing intensities until a saturation-point is reached (i.e., when each stimulation pulse produces a single action potential in the nerve fiber). Future intensity-encoding experiments using frequency modulation should use suprathreshold stimulation amplitudes rather than perithreshold amplitudes to decrease stochastic variability in frequency encoding at the axon level. Although functional discrimination among sensory percepts provides an important metric for demonstrating that percepts are distinct, other assessments may provide additional information. For example, the results presented here do not provide an indication of the theoretical resolution of percept intensities or locations. Future experiments should include mapping of the just-noticeable-difference (JND) between percepts at different locations or of different intensities [18], [34]. JNDs can also be quantified for percepts of different intensities to indicate, for example, the minimum discriminable frequency differences for stimulation on an electrode. JNDs should be mapped at multiple frequency levels to provide a test of Weber’s law, which predicts that the JND will scale linearly with stimulation frequency [51]. Informally, we observed habituation of some sensory 8 percepts during intensity discrimination trials in subject S5. We did not explicitly study the habituation phenomenon extensively. However prior research has documented this phenomenon for neural stimulation [52], and habituation of responses evoked by natural sensory stimuli is ubiquitous [53], [54]. Informally, we found that in some cases there seemed to be less nominal habituation if the stimulation was delivered at an amplitude at least 150% of threshold, and if we allowed for ~30 s of rest between each trial. Despite this, the subject’s performance discriminating among intensities typically declined slightly as trials continued, and the subject tended to underestimate the percept intensity in later trials compared with earlier trials in a session. The habituation of USEA-evoked sensory percepts should be studied and understood explicitly in future studies, particularly for longer-duration sensory prosthesis use. Intact subjects exhibit habituation in response to tactile stimulation of the skin [55]–[59]. Past studies indicate that the rate of nominal habituation varies as a function of stimulus frequency and inversely as a function of stimulus strength for neural interface stimulation [14], [52], suggesting that algorithms that use either frequency or charge per pulse to encode stimulus intensity may have different consequences across time, even though the two parameters may be functionally interchangeable for a given time point. We hypothesize that use of suprathreshold stimulation intensities, or addition of interpulse variability into stimulation trains (in contrast to constant-frequency stimulation), to produce more biomimetic stimulation patterns [4], may help reduce the effects of habituation. Although the sensory percepts restored via USEA stimulation are generally stable within a 2-3 h session, the projected field location, quality, and intensity associated with each electrode often varies across sessions [17]. Due to this limitation, we did not attempt to repeat identical functional discrimination tasks in different sessions. This instability may be due to a number of factors, including micromechanical shifts of the USEA relative to nerve fibers, the developing foreign body response to implanted USEAs, or degradation or failure of USEA electrodes and/or wire bundles [60]. Ongoing improvements to USEAs should continue, with reliability and longevity as a high priority. Longer-duration implants may also result in improved stability. Additionally, novel stimulation strategies, such as multielectrode stimulation, may decrease the variability in population encoding due to microshifts of USEAs. Also, multielectrode population encoding using biomimetic, receptor-type-specific stimulation patterns may decrease between-session variability in which axons are recruited, while also potentially improving the discriminability and naturalism of some USEA-evoked sensory percepts. USEAs and intraneural electrodes, in contrast to epineural cuff electrodes, are capable of communicating with the peripheral nervous system on its own terms by independently activating subsets of different populations of specific receptor types with known projected fields in naturalistic, custom-tailored, tunable patterns. Ultimately, we foresee development of a closed-loop prosthesis system with multiple discriminable sensory percepts coupled to sensors that span a physical prosthetic hand for use in activities of daily living. We anticipate that discriminable sensory feedback via a prosthesis will enhance motor control, particularly in scenarios where visual feedback is limited or undesired. Also, we anticipate that discriminable, multisensor feedback with variable intensity and tunable quality will enhance the level of embodiment of a prosthetic limb, helping amputees to feel as though their prosthesis is a replacement hand, in addition to being a useful tool. Sensory feedback during closed-loop control, and any associated limb embodiment, may also alleviate phantom pain and many of the psychological difficulties associated with losing a hand. V. C ONCLUSION
We have shown that human amputees implanted with Utah Slanted Electrode Arrays in their residual peripheral arm nerves can discriminate among a variety of restored hand sensations in blind trials, including: a) percepts with different hand locations, b) percepts with different qualities, and c) percepts with different intensities. Additionally, we have demonstrated that one subject was able to discriminate among cutaneous or proprioceptive percepts with different combinations of location and intensity, such as may occur during functional prosthesis use with multiple graded sensors for feedback. Furthermore, we have presented a multielectrode stimulation strategy using interleaved stimulation, which may be useful for evoking multiple sensory percepts concurrently without the effects of current summation during closed-loop prosthesis control. Our subjects enjoyed most of the sensory percepts and appreciated feeling controlled sensation from their amputated hand. Future work should include investigation of functional discriminability using multielectrode biomimetic stimulation patterns, as well exploration of the limit of functional discriminability resolution with USEAs. We hypothesize that functionally-discriminable sensory percepts with different locations, qualities, and intensities, used during closed-loop prosthesis control, will enable enhanced embodiment and improvements in motor performance for prosthesis users. A
CKNOWLEDGMENT
This work was sponsored by the Defense Advanced Research Projects Agency (DARPA) BTO Hand Proprioception and Touch Interfaces (HAPTIX) program under the auspices of Dr. Doug Weber through the Space and Naval Warfare Systems Center, Pacific Grant/Contract No. N66001-15-C-4017, and the DARPA MTO under the auspices of Dr. Jack Judy through the Space and Naval Warfare Systems Center, Pacific Grant/Contract No. N66001-12-C-4042. Additional funding was also provided by the National Institutes of Health (NIH NCATS Award No. 1ULTR001067). This paper constitutes a chapter of the doctoral dissertation of David M. Page, of the Department of Biomedical Engineering, University of Utah.
The authors further acknowledge the contributions of Richard A. Normann, Heather A. C. Wark, and Bradley Greger in prior studies related to this research. We also acknowledge Sliman J. Bensmaia and Hannes Saal for general consulting and input regarding the encoding of cutaneous sensation, and David 9 J. Warren for assistance in final revision of the manuscript and for general contributions to the design of hardware for these experiments. Conflict of interest disclosure: Dr. Clark is an inventor on a US patent on the electrode arrays and methods for protecting recordings from Utah Electrode Arrays from noise contaminations when used for recording (US Patent Nos. 8,359,083, 8,639,312, and 8,855,737). Dr. Clark, Dr. Hutchinson, Dr. Page, and Dr. Kluger are inventors of an implant carrier device used during surgical implantation of USEAs (patent pending, International Application no. PCT/US2017/044427). Dr. Clark, Dr. Davis, Dr. Page, and Dr. Kluger are inventors of the concept of using implanted electrode arrays to block transmission of neural signals (patent pending, US Application No. 15/583,568). R
EFERENCES [1] S. Raspopovic et al. , “Restoring Natural Sensory Feedback in Real-Time Bidirectional Hand Prostheses,”
Sci. Transl. Med. , vol. 6, no. 222, pp. 222ra19-222ra19, Feb. 2014, doi: 10.1126/scitranslmed.3006820. [2] P. D. Marasco, K. Kim, J. E. Colgate, M. A. Peshkin, and T. A. Kuiken, “Robotic touch shifts perception of embodiment to a prosthesis in targeted reinnervation amputees,”
Brain , vol. 134, no. 3, pp. 747–758, Mar. 2011, doi: 10.1093/brain/awq361. [3] D. M. Page et al. , “Motor Control and Sensory Feedback Enhance Prosthesis Embodiment and Reduce Phantom Pain After Long-Term Hand Amputation,”
Front. Hum. Neurosci. , vol. 12, 2018, doi: 10.3389/fnhum.2018.00352. [4] J. A. George et al. , “Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand,”
Sci. Robot. , vol. 4, no. 32, p. eaax2352, Jul. 2019, doi: 10.1126/scirobotics.aax2352. [5] S. M. Engdahl, B. P. Christie, B. Kelly, A. Davis, C. A. Chestek, and D. H. Gates, “Surveying the interest of individuals with upper limb loss in novel prosthetic control techniques,”
J. NeuroEngineering Rehabil. , vol. 12, Jun. 2015, doi: 10.1186/s12984-015-0044-2. [6] E. Biddiss, D. Beaton, and T. Chau, “Consumer design priorities for upper limb prosthetics,”
Disabil. Rehabil. Assist. Technol. , vol. 2, no. 6, pp. 346–357, Nov. 2007, doi: 10.1080/17483100701714733. [7] D. J. O. Atkins, D. C. Y. M. Heard, and W. H. Donovan, “Epidemiologic Overview of Individuals with Upper-Limb Loss and Their Reported Research Priorities,”
J. Prosthet. , vol. 8, no. 1, pp. 2–11, 1996. [8] P. J. Kyberd and W. Hill, “Survey of upper limb prosthesis users in Sweden, the United Kingdom and Canada,”
Prosthet. Orthot. Int. , vol. 35, no. 2, pp. 234–241, Jun. 2011, doi: 10.1177/0309364611409099. [9] E. A. Biddiss and T. T. Chau, “Upper limb prosthesis use and abandonment: A survey of the last 25 years,”
Prosthet. Orthot. Int. , vol. 31, no. 3, pp. 236–257, Sep. 2007, doi: 10.1080/03093640600994581. [10] S. M. Engdahl, C. A. Chestek, B. Kelly, A. Davis, and D. H. Gates, “Factors associated with interest in novel interfaces for upper limb prosthesis control,”
PloS One , vol. 12, no. 8, p. e0182482, 2017, doi: 10.1371/journal.pone.0182482. [11] G. S. Dhillon, T. B. Krüger, J. S. Sandhu, and K. W. Horch, “Effects of short-term training on sensory and motor function in severed nerves of long-term human amputees,”
J. Neurophysiol. , vol. 93, no. 5, pp. 2625–2633, May 2005, doi: 10.1152/jn.00937.2004. [12] G. S. Dhillon, S. M. Lawrence, D. T. Hutchinson, and K. W. Horch, “Residual function in peripheral nerve stumps of amputees: implications for neural control of artificial limbs1,”
J. Hand Surg. , vol. 29, no. 4, pp. 605–615, Jul. 2004, doi: 10.1016/j.jhsa.2004.02.006. [13] G. S. Dhillon and K. W. Horch, “Direct neural sensory feedback and control of a prosthetic arm,”
IEEE Trans. Neural Syst. Rehabil. Eng. , vol. 13, no. 4, pp. 468–472, Dec. 2005, doi: 10.1109/TNSRE.2005.856072. [14] D. W. Tan, M. A. Schiefer, M. W. Keith, J. R. Anderson, J. Tyler, and D. J. Tyler, “A neural interface provides long-term stable natural touch perception,”
Sci. Transl. Med. , vol. 6, no. 257, pp. 257ra138-257ra138, Oct. 2014, doi: 10.1126/scitranslmed.3008669. [15] T. S. Davis et al. , “Restoring motor control and sensory feedback in people with upper extremity amputations using arrays of 96 microelectrodes implanted in the median and ulnar nerves,”
J. Neural Eng. , vol. 13, no. 3, p. 036001, Jun. 2016, doi: 10.1088/1741-2560/13/3/036001. [16] G. A. Clark et al. , “Using multiple high-count electrode arrays in human median and ulnar nerves to restore sensorimotor function after previous transradial amputation of the hand,” in
Engineering in Medicine and Biology Society (EMBC), 2014 36th Annual International Conference of the IEEE , 2014, pp. 1977–1980. [17] S. Wendelken et al. , “Restoration of motor control and proprioceptive and cutaneous sensation in humans with prior upper-limb amputation via multiple Utah Slanted Electrode Arrays (USEAs) implanted in residual peripheral arm nerves,”
J. NeuroEngineering Rehabil. , vol. 14, p. 121, Nov. 2017, doi: 10.1186/s12984-017-0320-4. [18] E. L. Graczyk, M. A. Schiefer, H. P. Saal, B. P. Delhaye, S. J. Bensmaia, and D. J. Tyler, “The neural basis of perceived intensity in natural and artificial touch,”
Sci. Transl. Med. , vol. 8, no. 362, pp. 362ra142-362ra142, Oct. 2016, doi: 10.1126/scitranslmed.aaf5187. [19] K. Horch, S. Meek, T. G. Taylor, and D. T. Hutchinson, “Object discrimination with an artificial hand using electrical stimulation of peripheral tactile and proprioceptive pathways with intrafascicular electrodes,”
IEEE Trans. Neural Syst. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. , vol. 19, no. 5, pp. 483–489, Oct. 2011, doi: 10.1109/TNSRE.2011.2162635. [20] D. T. Kluger et al. , “Virtual Reality Provides an Effective Platform for Functional Evaluations of 10 Closed-Loop Neuromyoelectric Control,”
IEEE Trans. Neural Syst. Rehabil. Eng. , vol. 27, no. 5, pp. 876–886, May 2019, doi: 10.1109/TNSRE.2019.2908817. [21] M. A. Muniak, S. Ray, S. S. Hsiao, J. F. Dammann, and S. J. Bensmaia, “The Neural Coding of Stimulus Intensity: Linking the Population Response of Mechanoreceptive Afferents with Psychophysical Behavior,”
J. Neurosci. , vol. 27, no. 43, pp. 11687–11699, Oct. 2007, doi: 10.1523/JNEUROSCI.1486-07.2007. [22] H. E. Wheat, A. W. Goodwin, and A. S. Browning, “Tactile resolution: peripheral neural mechanisms underlying the human capacity to determine positions of objects contacting the fingerpad,”
J. Neurosci. , vol. 15, no. 8, pp. 5582–5595, Aug. 1995, doi: 10.1523/JNEUROSCI.15-08-05582.1995. [23] R. S. Johansson and A. B. Vallbo, “Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin.,”
J. Physiol. , vol. 286, pp. 283–300, Jan. 1979. [24] M. Paré, A. M. Smith, and F. L. Rice, “Distribution and terminal arborizations of cutaneous mechanoreceptors in the glabrous finger pads of the monkey,”
J. Comp. Neurol. , vol. 445, no. 4, pp. 347–359, Apr. 2002, doi: 10.1002/cne.10196. [25] H. E. Wheat, L. M. Salo, and A. W. Goodwin, “Human ability to scale and discriminate forces typical of those occurring during grasp and manipulation,”
J. Neurosci. Off. J. Soc. Neurosci. , vol. 24, no. 13, pp. 3394–3401, Mar. 2004, doi: 10.1523/JNEUROSCI.4822-03.2004. [26] M. Hollins and S. J. Bensmaïa, “The coding of roughness,”
Can. J. Exp. Psychol. Rev. Can. Psychol. Expérimentale , vol. 61, no. 3, pp. 184–195, Sep. 2007. [27] H. P. Saal and S. J. Bensmaia, “Touch is a team effort: interplay of submodalities in cutaneous sensibility,”
Trends Neurosci. , vol. 37, no. 12, pp. 689–697, Dec. 2014, doi: 10.1016/j.tins.2014.08.012. [28] S. Bensmaia, “Tactile intensity and population codes,”
Behav. Brain Res. , vol. 190, no. 2, pp. 165–173, Jul. 2008, doi: 10.1016/j.bbr.2008.02.044. [29] S. J. Bensmaïa, J. C. Craig, T. Yoshioka, and K. O. Johnson, “SA1 and RA afferent responses to static and vibrating gratings,”
J. Neurophysiol. , vol. 95, no. 3, pp. 1771–1782, Mar. 2006, doi: 10.1152/jn.00877.2005. [30] M. Hollins and E. A. Roy, “Perceived intensity of vibrotactile stimuli: the role of mechanoreceptive channels,”
Somatosens. Mot. Res. , vol. 13, no. 3–4, pp. 273–286, 1996, doi: 10.3109/08990229609052583. [31] S. N. Flesher et al. , “Intracortical microstimulation of human somatosensory cortex,”
Sci. Transl. Med. , vol. 8, no. 361, pp. 361ra141-361ra141, Oct. 2016, doi: 10.1126/scitranslmed.aaf8083. [32] G. Chai, D. Zhang, and X. Zhu, “Developing non-somatotopic phantom finger sensation to comparable levels of somatotopic sensation through user training with electrotactile stimulation,”
IEEE Trans. Neural Syst. Rehabil. Eng. , vol. 25, no. 5, pp. 469–480, May 2017, doi: 10.1109/TNSRE.2016.2580905. [33] M. D’ Alonzo, L. F. Engels, M. Controzzi, and C. Cipriani, “Electro-cutaneous stimulation on the palm elicits referred sensations on intact but not on amputated digits,”
J. Neural Eng. , vol. 15, no. 1, p. 016003, 2018, doi: 10.1088/1741-2552/aa81e2. [34] J. A. George, M. R. Brinton, P. C. Colgan, G. K. Colvin, S. J. Bensmaia, and G. A. Clark, “Intensity Discriminability of Electrocutaneous and Intraneural Stimulation Pulse Frequency in Intact Individuals and Amputees,”
ArXiv200108808 Cs Q-Bio , Jan. 2020. [35] V. S. Ramachandran and D. Rogers-Ramachandran, “Synaesthesia in phantom limbs induced with mirrors,”
Proc. Biol. Sci. , vol. 263, no. 1369, pp. 377–386, Apr. 1996. [36] A. Branner, R. B. Stein, and R. A. Normann, “Selective stimulation of cat sciatic nerve using an array of varying-length microelectrodes,”
J. Neurophysiol. , vol. 85, no. 4, pp. 1585–1594, Apr. 2001. [37] J. A. George, T. S. Davis, M. R. Brinton, and G. A. Clark, “Intuitive neuromyoelectric control of a dexterous bionic arm using a modified Kalman filter,”
J. Neurosci. Methods , vol. 330, p. 108462, Nov. 2019, doi: 10.1016/j.jneumeth.2019.108462. [38] P. J. Rousche and R. A. Normann, “A method for pneumatically inserting an array of penetrating electrodes into cortical tissue,”
Ann. Biomed. Eng. , vol. 20, no. 4, pp. 413–422, Jul. 1992, doi: 10.1007/BF02368133. [39] L. Spataro et al. , “Dexamethasone treatment reduces astroglia responses to inserted neuroprosthetic devices in rat neocortex,”
Exp. Neurol. , vol. 194, no. 2, pp. 289–300, Aug. 2005, doi: 10.1016/j.expneurol.2004.08.037. [40] Y. Zhong and R. V. Bellamkonda, “Dexamethasone coated neural probes elicit attenuated inflammatory response and neuronal loss compared to uncoated neural probes,”
Brain Res. , vol. 1148, pp. 15–27, May 2007, doi: 10.1016/j.brainres.2007.02.024. [41] M. B. Christensen, H. A. C. Wark, and D. T. Hutchinson, “A histological analysis of human median and ulnar nerves following implantation of Utah slanted electrode arrays,”
Biomaterials , vol. 77, pp. 235–242, Jan. 2016, doi: 10.1016/j.biomaterials.2015.11.012. [42] K. O. Johnson and S. S. Hsiao, “Neural Mechanisms of Tactual form and Texture Perception,”
Annu. Rev. Neurosci. , vol. 15, no. 1, pp. 227–250, Mar. 1992, doi: 10.1146/annurev.ne.15.030192.001303. [43] R. M. Friedman, P. S. Khalsa, K. W. Greenquist, and R. H. LaMotte, “Neural coding of the location and direction of a moving object by a spatially distributed population of mechanoreceptors,”
J. Neurosci. Off. J. Soc. Neurosci. , vol. 22, no. 21, pp. 9556–9566, Nov. 2002. [44] V. G. Macefield, C. Häger-Ross, and R. S. Johansson, “Control of grip force during restraint of an object held between finger and thumb: responses of cutaneous afferents from the digits,”
Exp. Brain Res. , vol. 108, no. 1, pp. 155–171, Feb. 1996, doi: 10.1007/BF00242913. [45] T. Yoshioka, S. J. Bensamaïa, J. C. Craig, and S. S. Hsiao, “Texture perception through direct and indirect touch: An analysis of perceptual space for tactile textures in two modes of exploration,”
Somatosens. Mot. Res. , vol. 24, no. 1–2, pp. 53–70, 2007, doi: 10.1080/08990220701318163. [46] J. M. Yau, S. S. Kim, P. H. Thakur, and S. J. Bensmaia, “Feeling form: the neural basis of haptic shape perception,”
J. Neurophysiol. , vol. 115, no. 2, pp. 631–642, Feb. 2016, doi: 10.1152/jn.00598.2015. [47] R. S. Johansson and J. R. Flanagan, “Tactile sensory control of object manipulation in human,” in
The Senses: A Comprehensive Reference , New York: Academic Press, 2007, pp. 67–86. [48] H. P. Saal, S. Vijayakumar, and R. S. Johansson, “Information about Complex Fingertip Parameters in Individual Human Tactile Afferent Neurons,”
J. Neurosci. , vol. 29, no. 25, pp. 8022–8031, Jun. 2009, doi: 10.1523/JNEUROSCI.0665-09.2009. [49] T. Gollisch and M. Meister, “Rapid Neural Coding in the Retina with Relative Spike Latencies,”
Science , vol. 319, no. 5866, pp. 1108–1111, Feb. 2008, doi: 10.1126/science.1149639. [50] E. L. Graczyk, L. Resnik, M. A. Schiefer, M. S. Schmitt, and D. J. Tyler, “Home Use of a Neural-connected Sensory Prosthesis Provides the Functional and Psychosocial Experience of Having a Hand Again,”
Sci. Rep. , vol. 8, no. 1, p. 9866, Jun. 2018, doi: 10.1038/s41598-018-26952-x. [51] Gös. Ekman, “Weber’s Law and Related Functions,”
J. Psychol. , vol. 47, no. 2, pp. 343–352, Apr. 1959, doi: 10.1080/00223980.1959.9916336. [52] E. L. Graczyk, B. P. Delhaye, M. A. Schiefer, S. J. Bensmaia, and D. J. Tyler, “Sensory adaptation to electrical stimulation of the somatosensory nerves,”
J. Neural Eng. , vol. 15, no. 4, p. 046002, 2018, doi: 10.1088/1741-2552/aab790. [53] R. F. Thompson and W. A. Spencer, “Habituation: A model phenomenon for the study of neuronal substrates of behavior,”
Psychol. Rev. , vol. 73, no. 1, pp. 16–43, 1966, doi: 10.1037/h0022681. [54] C. H. Rankin et al. , “Habituation revisited: an updated and revised description of the behavioral characteristics of habituation,”
Neurobiol. Learn. Mem. , vol. 92, no. 2, pp. 135–138, Sep. 2009, doi: 10.1016/j.nlm.2008.09.012. [55] M. Hollins, K. A. Delemos, and A. K. Goble, “Vibrotactile adaptation on the face,”
Percept. Psychophys. , vol. 49, no. 1, pp. 21–30, Jan. 1991, doi: 10.3758/bf03211612. [56] M. Hollins, A. K. Goble, B. L. Whitsel, and M. Tommerdahl, “Time course and action spectrum of vibrotactile adaptation,”
Somatosens. Mot. Res. , vol. 7, no. 2, pp. 205–221, 1990, doi: 10.3109/08990229009144707. [57] U. Berglund and B. Berglund, “Adaptation and Recovery in Vibrotactile Perception:,”
Percept. Mot. Skills , Aug. 2016, doi: 10.2466/pms.1970.30.3.843. [58] J. Mei, R. P. Tuckett, D. A. Poulos, K. W. Horch, J. Y. Wei, and P. R. Burgess, “The neural signal for skin indentation depth. II. Steady indentations,”
J. Neurosci. Off. J. Soc. Neurosci. , vol. 3, no. 12, pp. 2652–2659, Dec. 1983. [59] P. R. Burgess, J. Mei, R. P. Tuckett, K. W. Horch, C. M. Ballinger, and D. A. Poulos, “The neural signal for skin indentation depth. I. Changing indentations,”
J. Neurosci. Off. J. Soc. Neurosci. , vol. 3, no. 8, pp. 1572–1585, Aug. 1983. [60] B. Baker et al. , “Improved Long-term Performance of Utah Slanted Arrays in Clinical Studies,” presented at the Society for Neuroscience, San Diego, CA, 03-Nov-2018. Table 1. Summary of Discrimination Tasks Participant Discrimination Task Total Conditions Number of Electrodes
S3 Location 5 4 S3 Location 4 2 S4 Location 8 3 S4 On-off 2 1 S3 Quality 2 2 S5 Intensity 4 1 S5 Location-Intensity 10 3 S5 Location-Intensity 7 2 Fig. 1. Absolute electrode distance versus cross-sectional projection distance. The 10x10 USEA provides cross-sectional coverage of peripheral nerves, increasing the possibility of activating different axons or subsets of axons with stimulation of each different electrode. Activation of different populations of axons is important for evoking sensory percepts with different locations or qualities. This diagram depicts a USEA implanted in a section of nerve, with an example axon that passes nearby two neighboring electrodes. Although the absolute distance between USEA electrodes is important for assessing stimulation selectivity limits, the cross-sectional distance between electrode tips more precisely indicates the likelihood that electrode tips are close to the same axon(s). For example, there is a ~409 m absolute distance compared with a ~400 m horizontal distance in and a ~83 m vertical distance, not counting the exposure length of the electrode tip itself. Fig. 2. USEA implant methods. (a) Photograph of a USEA in the median nerve of subject S4 taken shortly after pneumatic insertion. The bundle of gold lead wires as well as the separate ground and reference wires were later bundled to the nerve using a collagen nerve wrap. The USEAs were implanted with the long electrodes distally, to avoid damaging axons that may be recruited via stimulation of other USEA electrodes. (b) The USEA lead wires and ground and reference wires for each USEA (one in the median nerve; one in the ulnar nerve) remained attached to external connector boards via percutaneous incisions on either the lower or upper arm (subject S3 lower arm, subjects S4 and S5 upper arm). Stimulation hardware was attached to one or more of these external connectors during experimental sessions. Fig. 3. Location discrimination trials. Each sub-figure depicts the location of electrodes which evoked different hand sensations, as well as a table showing the number of trials of the specified electrode or combination that evoked sensation in each hand location. For trials evoking multiple hand locations, the trial count in the cell for each designated location was incremented. Bold borders in tables indicate the ‘correct’ answer as designated in pre-trial programming. (a) Subject S3 successfully discriminated among percepts evoked via individual stimulation of 4 different ulnar-nerve-USEA electrodes (electrode locations on the USEA are shown as colored circles in figure), as well as simultaneous stimulation of all 4 electrodes (4 categories shown, 5th category was concurrent perception at all four locations; 35/35 successful trials, p < 0.0001, binomial test). (b) Subject S3 also discriminated successfully between simultaneous (converging solid arrows) versus interleaved stimulation (converging dashed arrows) of two ulnar-nerve-USEA electrodes, as well as individual stimulation of the two electrodes (separate solid arrows). Interleaved stimulation (3-ms time shift difference, 200 Hz) reproduced the original percepts simultaneously with no bridging sensation, whereas simultaneous stimulation (no time shift difference, 200 Hz) produced both of these percepts accompanied by an emergent “massage” bridging between them (20/25 successful trials, p < 0.001, binomial test). (c) Subject S4 discriminated among eight different stimulation configurations: individual stimulation of each of 3 ulnar-nerve-USEA electrodes, simultaneous combined stimulation using different subsets of 2 of these 3 electrodes, simultaneous combined stimulation using all 3 electrodes, and no stimulation (11/24 correct trials, p < 0.006, binomial test). Importantly, these trials with S4 also included a condition of “no stimulation,” which was identified with 100% accuracy, indicating that percepts were indeed evoked by USEA stimulation (in contrast to pseudesthesia). These three experiments also demonstrate the selectivity of USEA-electrode stimulation, with unique percepts being generated by electrodes as close as 800 m. Fig. 4. Quality discrimination trials. Subject S3 successfully discriminated between stimulation of two different USEA electrodes that evoked sensation at the same location, but with different qualities (vibration versus tingle). Regarding the locations of the two percepts, the subject said they were “Right on, exact same space.” He also indicated that these sensory percepts were the same intensity level. The subject successfully performed the classification in 30/30 trials (p < 0.0001, binomial test). Electrode locations on the USEA are shown as colored circles in the figure. In the table, cell number/color indicates number of trials of the specified electrode that were assigned a perceived percept quality. Bold borders indicate the ‘correct’ answer as designated in pre-trial programming. Fig. 5. Intensity discrimination trials. Subject S5 discriminated between four percept intensities, evoked by stimulation of a single median-nerve-USEA electrode at three different frequencies (35 Hz, 70 Hz, 100 Hz) or sham (no stimulation). The evoked sensory percept was described as ‘tingle’ on all four fingertips, although in a later session this percept seemed to consolidate to the middle finger only. The subject successfully classified these different intensities in 13/20 trials (p < 0.005, binomial test). Electrode location on the USEA is shown as a colored circle in the figure. In the table, cell number/color indicates number of trials of the specified frequency that were assigned a perceived pressure level. Bold borders indicate the ‘correct’ answer as designated in pre-trial programming. Fig. 6. Combined cutaneous location and intensity discrimination. Subject S5 discriminated among combinations of different cutaneous percept locations and intensities. Three median-nerve-USEA electrodes evoked cutaneous “pressure” or “touch” percepts on the index finger, middle finger, and palm, respectively. Three frequencies (35 Hz, 70 Hz, and 100 Hz) were used to encode three different intensities via each electrode. Sham trials were also included (no stimulation) for a total of ten classification categories. The subject correctly classified the combination in 15/30 trials (p < 0.0005, binomial test). In post-hoc analysis, we found that most of the subject’s success was attributed to accurate location discrimination (26/30 correct trials, p < 0.0005, binomial test for location classification independent of intensity classification, using a corrected critical value of α = 0.005). Electrode locations on the USEA are shown as colored circles in the figure. In the table the cell number/color indicates the number of trials of the specified electrode (designated as e8, e569, and e66) and frequency that were assigned a perceived finger location and sensory intensity. Bold borders indicate the ‘correct’ answer as designated in pre-trial calibration programming. Fig. 7. Combined proprioceptive location and quality discrimination. Subject S5 discriminated between combinations of different proprioceptive percept locations and intensities. Two median-nerve-USEA electrodes evoked perception of proprioceptive flexion of the index finger and the middle finger, respectively. Three frequencies were used on each electrode to encode three different joint positions. Sham trials were included (no stimulation) representing a fully-open rest position for a total of seven classification categories. The subject correctly classified 21/40 trials (p < 0.0001, binomial test). Electrode locations on the USEA are shown as colored circles in the figure. In the table, cell number/color indicates the number of trials of the specified electrode (designated as e24 and e18) and frequency that were assigned a perceived finger and flexion angle. Note that the subject felt strongly during practice trials that the non-monotonic frequency-position encoding for the index finger was accurate, however we found that confusion between the 20 , 50 , and 180