Sunil L. Kukreja

A kilohertz kilotaxel tactile sensor array for investigating spatiotemporal features in neuromorphic touch

The spatiotemporal structure of mechanoreceptor responses is known to facilitate rapid tactile processing in the brain. To investigate whether such mechanisms can be replicated by artificial systems, tactile sensors with a large number of elements and sub-millisecond response time are required. In this work, we present a 4096 element tactile sensor array that can be sampled at over 5kHz. Experimental results demonstrate that high spatiotemporal resolution aids to resolve force magnitude and direction of tactile events. Such a platform enables the development of time dependent neuromorphic algorithms for tactile learning and signal processing.

Stable force-myographic control of a prosthetic hand using incremental learning

Force myography has been proposed as an appealing alternative to electromyography for control of upper limb prosthesis. A limitation of this technique is the non-stationary nature of the recorded force data. Force patterns vary under influence of various factors such as change in orientation and position of the prosthesis. We hereby propose an incremental learning method to overcome this limitation. We use an online sequential extreme learning machine where occasional updates allow continual adaptation to signal changes. The applicability and effectiveness of this approach is demonstrated for predicting the hand status from forearm muscle forces at various arm positions. The results show that incremental updates are indeed effective to maintain a stable level of performance, achieving an average classification accuracy of 98.75% for two subjects.

Detection of compensatory balance responses using wearable electromyography sensors for fall-risk assessment

Loss of balance is prevalent in older adults and populations with gait and balance impairments. The present paper aims to develop a method to automatically distinguish compensatory balance responses (CBRs) from normal gait, based on activity patterns of muscles involved in maintaining balance. In this study, subjects were perturbed by lateral pushes while walking and surface electromyography (sEMG) signals were recorded from four muscles in their right leg. To extract sEMG time domain features, several filtering characteristics and segmentation approaches are examined. The performance of three classification methods, i.e., k-nearest neighbor, support vector machines, and random forests, were investigated for accurate detection of CBRs. Our results show that features extracted in the 50-200Hz band, segmented using peak sEMG amplitudes, and a random forest classifier detected CBRs with an accuracy of 92.35%. Moreover, our results support the important role of biceps femoris and rectus femoris muscles in stabilization and consequently discerning CBRs. This study contributes towards the development of wearable sensor systems to accurately and reliably monitor gait and balance control behavior in at-home settings (unsupervised conditions), over long periods of time, towards personalized fall risk assessment tools.Loss of balance is prevalent in older adults and populations with gait and balance impairments. The present paper aims to develop a method to automatically distinguish compensatory balance responses (CBRs) from normal gait, based on activity patterns of muscles involved in maintaining balance. In this study, subjects were perturbed by lateral pushes while walking and surface electromyography (sEMG) signals were recorded from four muscles in their right leg. To extract sEMG time domain features, several filtering characteristics and segmentation approaches are examined. The performance of three classification methods, i.e., k-nearest neighbor, support vector machines, and random forests, were investigated for accurate detection of CBRs. Our results show that features extracted in the 50-200Hz band, segmented using peak sEMG amplitudes, and a random forest classifier detected CBRs with an accuracy of 92.35%. Moreover, our results support the important role of biceps femoris and rectus femoris muscles in stabilization and consequently discerning CBRs. This study contributes towards the development of wearable sensor systems to accurately and reliably monitor gait and balance control behavior in at-home settings (unsupervised conditions), over long periods of time, towards personalized fall risk assessment tools.

Spike-based tactile pattern recognition using an extreme learning machine

We present a biologically-inspired approach for tactile pattern recognition. Our aim is to develop a low-cost tactile module that can be applied to large areas by integrating sensors with processing circuits. To accomplish this goal a flexible tactile sensor array was developed using piezoresistive fabric material. The output of the tactile array was represented as a spatiotemporal spike pattern to emulate neural signals from mechanoreceptors in the skin. A hardware implemented Extreme Learning Machine (ELM) was used to process the tactile information. The ELM chip is an event-driven system that is massively parallel and energy-efficient. For these reasons, our proposed architecture offers a fast and energy-efficient alternative for processing spatiotemporal tactile patterns. The performance of the system was evaluated during a real-time object classification task, where it achieved 90% accuracy for binary classification.

Encapsulation of Piezoelectric Transducers for Sensory Augmentation and Substitution with Wearable Haptic Devices

The integration of polymeric actuators in haptic displays is widespread nowadays, especially in virtual reality and rehabilitation applications. However, we are still far from optimizing the transducer ability in conveying sensory information. Here, we present a vibrotactile actuator characterized by a piezoelectric disk embedded in a polydimethylsiloxane (PDMS) shell. An original encapsulation technique was performed to provide the stiff active element with a compliant cover as an interface towards the soft human skin. The interface stiffness, together with the new geometry, generated an effective transmission of vibrotactile stimulation and made the encapsulated transducer a performant component for the development of wearable tactile displays. The mechanical behavior of the developed transducer was numerically modeled as a function of the driving voltage and frequency, and the exerted normal forces were experimentally measured with a load cell. The actuator was then tested for the integration in a haptic glove in single-finger and bi-finger condition, in a 2-AFC tactile stimulus recognition test. Psychophysical results across all the tested sensory conditions confirmed that the developed integrated haptic system was effective in delivering vibrotactile information when the frequency applied to the skin is within the 200–700 Hz range and the stimulus variation is larger than 100 Hz.

CONE: Convex-Optimized-Synaptic Efficacies for Temporally Precise Spike Mapping

Spiking neural networks are well suited to perform time-dependent pattern recognition problems by encoding the temporal dimension in precise spike times. With an appropriate set of weights, a spiking neuron can emit precisely timed action potentials in response to spatiotemporal input spikes. However, deriving supervised learning rules for spike mapping is nontrivial due to the increased complexity. Existing methods rely on heuristic approaches that do not guarantee a convex objective function and, therefore, may not converge to a global minimum. In this paper, we present a novel technique to obtain the weights of spiking neurons by formulating the problem in a convex optimization framework, rendering it be compatible with the established methods. We introduce techniques to influence the weight distribution and membrane trajectory, and then study how these factors affect robustness in the presence of noise. In addition, we show how the existence of a solution can be determined and assess memory capacity limits of a neuron model using synthetic examples. The practical utility of our technique is further assessed by its application to gait-event detection using the experimental data.

A compliant modular robotic hand with fabric force sensor for multiple versatile grasping modes

This paper presents the development of a compliant, modular, reconfigurable, and sensorized robotic hand with multiple grasping capabilities. Each finger consists of a soft pneumatic actuator with embedded fabric force sensor and a detachable casing. The casing has a through hole for housing the actuator and special connectors for attachment to other casings. One casing each from the thumb and finger parts has a protrusion for connecting both parts together via a screw tightening system. The through-hole design allows different grasping length to be achieved and the inflated pneumatic channel of the actuator locks it in place. The modular robotic hand is capable of various versatile grasping tasks by simply changing the bending direction of the actuator, the distance between the thumb and finger parts, the grasping length of the actuator, or attaching/detaching additional fingers to the hand. (1) Hook grasping with single finger, (2) pinching with pad opposition, (3) reverse grasping for holding a pipe-like object, (4) wrap grasping with palm opposition, as well as (5) picking up an object through its handle with one thumb and two or more fingers were illustrated. These studies show the capability of the compliant modular robotic hand in performing various types of grasping by simply using different configurations of the casings. The excellent payload-to-weight ratio of the robotic hand was demonstrated. Also, the fabric force sensor that was embedded in the soft actuator indicated the difference in grasping forces that were applied to different objects during hook grasping. The modular robotic hand has the potential to broaden or substitute the usage of existing robotic hands, especially in applications where soft versatile configurable grasping is desired.

Design and preliminary evaluation of haptic devices for upper limb stimulation and integration within a virtual reality cave

During the last decade significant advances have been made in vibrotactile actuator design that are leading to the development of novel haptic technologies. Similarly, important innovations have been made in the area of virtual reality for scene rendering and user tracking. However, the integration of these technologies has not been well explored. In this paper, we outline a broad design philosophy and integration plan of these tools. In addition, we give an overview of applications for such a cohesive set of technologies. Preliminary results are provided to demonstrate their critical importance and future widespread use.

Eye tracking and EEG synchronization to analyze microsaccades during a workload task

Electroencephalography (EEG) and eye tracking are two fields that have evolved independently to study topics such as mental workload, attention and fatigue in cognitive neuroscience. However, little research has been devoted to integrating data from these two fields. In this paper, we investigate the utility of a specific type of eye movement, microsaccades, to analyze cognitive activity. We assess mental workload using event related potentials (ERPs) correlated with microsaccades during experiments where task complexity is designed to be greater with an increase in visual degradation. We also develop a modified eye movement algorithm to identify microsaccades during tasks that permit regular saccades and blinks. We compare ERPs at microsaccade onset locked epochs to those of stimulus onset locked epochs. Our results show a clear correlation of ERP activations to both latency and activation areas. These findings provide important insights for analyzing sophisticated tasks in a non-invasive fashion.

Discrimination of Dynamic Tactile Contact by Temporally Precise Event Sensing in Spiking Neuromorphic Networks

This paper presents a neuromorphic tactile encoding methodology that utilizes a temporally precise event-based representation of sensory signals. We introduce a novel concept where touch signals are characterized as patterns of millisecond precise binary events to denote pressure changes. This approach is amenable to a sparse signal representation and enables the extraction of relevant features from thousands of sensing elements with sub-millisecond temporal precision. We also proposed measures adopted from computational neuroscience to study the information content within the spiking representations of artificial tactile signals. Implemented on a state-of-the-art 4096 element tactile sensor array with 5.2 kHz sampling frequency, we demonstrate the classification of transient impact events while utilizing 20 times less communication bandwidth compared to frame based representations. Spiking sensor responses to a large library of contact conditions were also synthesized using finite element simulations, illustrating an 8-fold improvement in information content and a 4-fold reduction in classification latency when millisecond-precise temporal structures are available. Our research represents a significant advance, demonstrating that a neuromorphic spatiotemporal representation of touch is well suited to rapid identification of critical contact events, making it suitable for dynamic tactile sensing in robotic and prosthetic applications.