Brock A. Wester

Demonstration of a Semi-Autonomous Hybrid Brain–Machine Interface Using Human Intracranial EEG, Eye Tracking, and Computer Vision to Control a Robotic Upper Limb Prosthetic

To increase the ability of brain-machine interfaces (BMIs) to control advanced prostheses such as the modular prosthetic limb (MPL), we are developing a novel system: the Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE). This system utilizes hybrid input, supervisory control, and intelligent robotics to allow users to identify an object (via eye tracking and computer vision) and initiate (via brain-control) a semi-autonomous reach-grasp-and-drop of the object by the MPL. Sequential iterations of HARMONIE were tested in two pilot subjects implanted with electrocortico-graphic (ECoG) and depth electrodes within motor areas. The subjects performed the complex task in 71.4% (20/28) and 67.7% (21/31) of trials after minimal training. Balanced accuracy for detecting movements was 91.1% and 92.9%, significantly greater than chance accuracies (p <; 0.05). After BMI-based initiation, the MPL completed the entire task 100% (one object) and 70% (three objects) of the time. The MPL took approximately 12.2 s for task completion after system improvements implemented for the second subject. Our hybrid-BMI design prevented all but one baseline false positive from initiating the system. The novel approach demonstrated in this proof-of-principle study, using hybrid input, supervisory control, and intelligent robotics, addresses limitations of current BMIs.

Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject

OBJECTIVE We used native sensorimotor representations of fingers in a brain-machine interface (BMI) to achieve immediate online control of individual prosthetic fingers. APPROACH Using high gamma responses recorded with a high-density electrocorticography (ECoG) array, we rapidly mapped the functional anatomy of cued finger movements. We used these cortical maps to select ECoG electrodes for a hierarchical linear discriminant analysis classification scheme to predict: (1) if any finger was moving, and, if so, (2) which digit was moving. To account for sensory feedback, we also mapped the spatiotemporal activation elicited by vibrotactile stimulation. Finally, we used this prediction framework to provide immediate online control over individual fingers of the Johns Hopkins University Applied Physics Laboratory modular prosthetic limb. MAIN RESULTS The balanced classification accuracy for detection of movements during the online control session was 92% (chance: 50%). At the onset of movement, finger classification was 76% (chance: 20%), and 88% (chance: 25%) if the pinky and ring finger movements were coupled. Balanced accuracy of fully flexing the cued finger was 64%, and 77% had we combined pinky and ring commands. Offline decoding yielded a peak finger decoding accuracy of 96.5% (chance: 20%) when using an optimized selection of electrodes. Offline analysis demonstrated significant finger-specific activations throughout sensorimotor cortex. Activations either prior to movement onset or during sensory feedback led to discriminable finger control. SIGNIFICANCE Our results demonstrate the ability of ECoG-based BMIs to leverage the native functional anatomy of sensorimotor cortical populations to immediately control individual finger movements in real time.

Development and characterization of in vivo flexible electrodes compatible with large tissue displacements

Electrical activity is the ultimate functional measure of neuronal tissue and recording that activity remains a key technical challenge in neuroscience. The mechanical mismatch between rigid electrodes and compliant brain tissue is a critical limitation in applications where movement is an inherent component. An electrode that permits recording of neural activity, while minimizing tissue disruption, is beneficial for applications that encompass both normal physiological movements and those which require consistent recording during large tissue displacements. In order to test the extreme of this range of movement, flexible electrodes were developed to record activity during and immediately following cortical impact in the rat. Photolithography techniques were used to fabricate flexible electrodes that were readily insertable into the brain using a parylene C base and gold conduction lines and contact pads, permitting custom geometry. We found that this electrode configuration retained mechanical and electrical integrity following both durability studies and large movements within the cortex. This novel flexible electrode configuration provides a novel platform for experimentally examining neuronal activity during a range of brain movements.

Simultaneous Neural Control of Simple Reaching and Grasping With the Modular Prosthetic Limb Using Intracranial EEG

Intracranial electroencephalographic (iEEG) signals from two human subjects were used to achieve simultaneous neural control of reaching and grasping movements with the Johns Hopkins University Applied Physics Lab (JHU/APL) Modular Prosthetic Limb (MPL), a dexterous robotic prosthetic arm. We performed functional mapping of high gamma activity while the subject made reaching and grasping movements to identify task-selective electrodes. Independent, online control of reaching and grasping was then achieved using high gamma activity from a small subset of electrodes with a model trained on short blocks of reaching and grasping with no further adaptation. Classification accuracy did not decline (p <; 0.05, one-way ANOVA) over three blocks of testing in either subject. Mean classification accuracy during independently executed overt reach and grasp movements for (Subject 1, Subject 2) were (0.85, 0.81) and (0.80, 0.96), respectively, and during simultaneous execution they were (0.83, 0.88) and (0.58, 0.88), respectively. Our models leveraged knowledge of the subjects individual functional neuroanatomy for reaching and grasping movements, allowing rapid acquisition of control in a time-sensitive clinical setting. We demonstrate the potential feasibility of verifying functionally meaningful iEEG-based control of the MPL prior to chronic implantation, during which additional capabilities of the MPL might be exploited with further training.

Hollow polymer microneedle array fabricated by photolithography process combined with micromolding technique

Transdermal drug delivery through microneedles is a minimally invasive procedure causing little or no pain, and is a potentially attractive alternative to intramuscular and subdermal drug delivery methods. This paper demonstrates the fabrication of a hollow microneedle array using a polymer-based process combining UV photolithography and replica molding techniques. The key characteristic of the proposed fabrication process is to define a hollow lumen for microfluidic access via photopatterning, allowing a batch process as well as high throughput. A hollow SU-8 microneedle array, consisting of 825μm tall and 400 μm wide microneedles with 15-25 μm tip diameters and 120 μm diameter hollow lumens was designed, fabricated and characterized.

Mechanically driven microtweezers with integrated microelectrodes

This paper presents a method for fabricating a fundamental MEMS tool—microtweezers. Microtweezers offer an attractive option to meet the increasing need to grasp, manipulate and excise microstructures or biological components. The microtweezers presented here augment a standard micromanipulator, allowing precise positioning in three dimensions. An additional micro-drive control knob, which is affixed to the micromanipulator, allows actuation of the tweezer tips through the use of a tether-cable drive system. This drive actuates the tweezer tips by the reciprocating motion of two microfabricated parts: the tweezers and tweezer box. A simple three-layer planar fabrication scheme allows for a broad range of tweezer styles (straight and serrated tips) and sizes (microns to millimeters). For these studies, 20 µ mw ide and 10 µm thick nickel beams were developed for the tweezer tips, which could endure 20 mN of force. To demonstrate the concept of microassembly, pick and place operations were performed on 10 µm thick film structures. Additional functionality was achieved by integrating platinum-black microelectrodes into parylene-coated tweezers to allow electrophysiological functions such as cellular stimulation and recording. Ultimately, this unique and simple design affords extraordinarily delicate control that is potentially beneficial for applications in microassembly, electrophysiology and microsurgery. (Some figures in this article are in colour only in the electronic version)

A collaborative BCI approach to autonomous control of a prosthetic limb system

Existing brain-computer interface (BCI) control of highly dexterous robotic manipulators and prosthetic devices typically rely solely on neural decode algorithms to determine the users intended motion. Although these approaches have made significant progress in the ability to control high degree of freedom (DOF) manipulators, the ability to perform activities of daily living (ADL) is still an ongoing research endeavor. In this paper, we describe a hybrid system that combines elements of autonomous robotic manipulation with neural decode algorithms to maneuver a highly dexterous robotic manipulator for a reach and grasp task. This system was demonstrated using a human patient with cortical micro-electrode arrays allowing the user to manipulate an object on a table and place it at a desired location. The preliminary results for this system are promising in that it demonstrates the potential to blend robotic control to perform lower level manipulation tasks with neural control that allows the user to focus on higher level tasks thereby reducing the cognitive load and increasing the success rate of performing ADL type activities.

The effects of chronic intracortical microstimulation on neural tissue and fine motor behavior.

OBJECTIVE One approach to conveying sensory feedback in neuroprostheses is to electrically stimulate sensory neurons in the cortex. For this approach to be viable, it is critical that intracortical microstimulation (ICMS) causes minimal damage to the brain. Here, we investigate the effects of chronic ICMS on the neuronal tissue across a variety of stimulation regimes in non-human primates. We also examine each animals ability to use their hand--the cortical representation of which is targeted by the ICMS--as a further assay of possible neuronal damage. APPROACH We implanted electrode arrays in the primary somatosensory cortex of three Rhesus macaques and delivered ICMS four hours per day, five days per week, for six months. Multiple regimes of ICMS were delivered to investigate the effects of stimulation parameters on the tissue and behavior. Parameters included current amplitude (10-100 μA), pulse train duration (1, 5 s), and duty cycle (1/1, 1/3). We then performed a range of histopathological assays on tissue near the tips of both stimulated and unstimulated electrodes to assess the effects of chronic ICMS on the tissue and their dependence on stimulation parameters. MAIN RESULTS While the implantation and residence of the arrays in the cortical tissue did cause significant damage, chronic ICMS had no detectable additional effect; furthermore, the animals exhibited no impairments in fine motor control. SIGNIFICANCE Chronic ICMS may be a viable means to convey sensory feedback in neuroprostheses as it does not cause significant damage to the stimulated tissue.

HARMONIE: A multimodal control framework for human assistive robotics

Effective user control of highly dexterous and robotic assistive devices requires intuitive and natural modalities. Although surgically implanted brain-computer interfaces (BCIs) strive to achieve this, a number of non-invasive engineering solutions may provide a quicker path to patient use by eliminating surgical implantation. We present the development of a semi-autonomous control system that utilizes computer vision, prosthesis feedback, effector centric device control, smooth movement trajectories, and appropriate hand conformations to interact with objects of interest. Users can direct a prosthetic limb through an intuitive graphical user interface to complete multi-stage tasks using patient appropriate combinations of control inputs such as eye tracking, conventional prosthetic controls/joysticks, surface electromyography (sEMG) signals, and neural interfaces (ECoG, EEG). Aligned with activities of daily living (ADL), these tasks include directing the prosthetic to specific locations or objects, grasping of objects by modulating hand conformation, and action upon grasped objects such as self-feeding. This Hybrid Augmented Reality Multimodal Operation Neural Integration Environment (HARMONIE) semi-autonomous control system lowers the users cognitive load, leaving the bulk of command and control of the device to the computer. This flexible and intuitive control system could serve patient populations ranging from wheelchair-bound quadriplegics to upper-limb amputees.

Flight simulation using a Brain-Computer Interface: A pilot, pilot study.

&NA; As Brain‐Computer Interface (BCI) systems advance for uses such as robotic arm control it is postulated that the control paradigms could apply to other scenarios, such as control of video games, wheelchair movement or even flight. The purpose of this pilot study was to determine whether our BCI system, which involves decoding the signals of two 96‐microelectrode arrays implanted into the motor cortex of a subject, could also be used to control an aircraft in a flight simulator environment. The study involved six sessions in which various parameters were modified in order to achieve the best flight control, including plane type, view, control paradigm, gains, and limits. Successful flight was determined qualitatively by evaluating the subjects ability to perform requested maneuvers, maintain flight paths, and avoid control losses such as dives, spins and crashes. By the end of the study, it was found that the subject could successfully control an aircraft. The subject could use both the jet and propeller plane with different views, adopting an intuitive control paradigm. From the subjects perspective, this was one of the most exciting and entertaining experiments she had performed in two years of research. In conclusion, this study provides a proof‐of‐concept that traditional motor cortex signals combined with a decoding paradigm can be used to control systems besides a robotic arm for which the decoder was developed. Aside from possible functional benefits, it also shows the potential for a new recreational activity for individuals with disabilities who are able to master BCI control. HighlightsA Brain‐Computer Interface controlled flight simulator is tested in a pilot study.The subject successfully controlled aircrafts using signals from her motor cortex.She learned to control the system with different planes, views, and locations.This system could potentially be used for transport control or recreation.