Wang Wei Lee

Bio-mimetic strategies for tactile sensing

In this work, a tactile sensing system is built for pattern recognition using spiking neurons. Tactile information is acquired using a fabric based binary tactile sensor array and converted into spatiotemporal spiking patterns that mimic mechanoreceptors in the skin. Through physical experiments, we show that the spike patterns efficiently represent information such as local curvature of objects in contact, which are easily distinguished using a supervised spike-timing based learning algorithm. High classification accuracy (>99%) and fast convergence rate (tens of epochs) of the classifier indicates good separation between different stimuli using the spatiotemporal spike representation.

A kilohertz kilotaxel tactile sensor array for investigating spatiotemporal features in neuromorphic touch

The spatiotemporal structure of mechanoreceptor responses is known to facilitate rapid tactile processing in the brain. To investigate whether such mechanisms can be replicated by artificial systems, tactile sensors with a large number of elements and sub-millisecond response time are required. In this work, we present a 4096 element tactile sensor array that can be sampled at over 5kHz. Experimental results demonstrate that high spatiotemporal resolution aids to resolve force magnitude and direction of tactile events. Such a platform enables the development of time dependent neuromorphic algorithms for tactile learning and signal processing.

Gait event detection through neuromorphic spike sequence learning

We present a novel sampling and processing method for detecting gait events from an insole pressure sensor. Inspired by how tactile data is processed in the brain, we propose the use of timing, instead of intensity, as our event detection feature. By sacrificing the need for accurate intensity measurements, it is possible to achieve superior temporal resolution, which is arguably more important given the need for timely feedback. In this paper, we demonstrate temporally accurate gait-event detection of 1.2±7ms (mean and standard deviation) for heel-strike and 0.2± 14ms for toe-off events compared to the reference system, and a success rate of above 97% in most trials, using only 1 bit of pressure information per channel. Our method thus has the potential to achieve much lower computational complexity and bandwidth, both of which are key to low-cost, portable solutions for prosthetics, exoskeletons or long-term gait monitoring applications.

Stable force-myographic control of a prosthetic hand using incremental learning

Force myography has been proposed as an appealing alternative to electromyography for control of upper limb prosthesis. A limitation of this technique is the non-stationary nature of the recorded force data. Force patterns vary under influence of various factors such as change in orientation and position of the prosthesis. We hereby propose an incremental learning method to overcome this limitation. We use an online sequential extreme learning machine where occasional updates allow continual adaptation to signal changes. The applicability and effectiveness of this approach is demonstrated for predicting the hand status from forearm muscle forces at various arm positions. The results show that incremental updates are indeed effective to maintain a stable level of performance, achieving an average classification accuracy of 98.75% for two subjects.

Tactile feedback in upper limb prosthetic devices using flexible textile force sensors

Many upper limb amputees are faced with the difficult challenge of using a prosthesis that lacks tactile sensing. State of the art research caliber prosthetic hands are often equipped with sophisticated sensors that provide valuable information regarding the prosthesis and its surrounding environment. Unfortunately, most commercial prosthetic hands do not contain any tactile sensing capabilities. In this paper, a textile based tactile sensor system was designed, built, and evaluated for use with upper limb prosthetic devices. Despite its simplicity, we demonstrate the ability of the sensors to determine object contact and perturbations due to slip during a grasping task with a prosthetic hand. This suggests the use of low-cost, customizable, textile sensors as part of a closed-loop tactile feedback system for monitoring grasping forces specifically in an upper limb prosthetic device.

CONE: Convex-Optimized-Synaptic Efficacies for Temporally Precise Spike Mapping

Spiking neural networks are well suited to perform time-dependent pattern recognition problems by encoding the temporal dimension in precise spike times. With an appropriate set of weights, a spiking neuron can emit precisely timed action potentials in response to spatiotemporal input spikes. However, deriving supervised learning rules for spike mapping is nontrivial due to the increased complexity. Existing methods rely on heuristic approaches that do not guarantee a convex objective function and, therefore, may not converge to a global minimum. In this paper, we present a novel technique to obtain the weights of spiking neurons by formulating the problem in a convex optimization framework, rendering it be compatible with the established methods. We introduce techniques to influence the weight distribution and membrane trajectory, and then study how these factors affect robustness in the presence of noise. In addition, we show how the existence of a solution can be determined and assess memory capacity limits of a neuron model using synthetic examples. The practical utility of our technique is further assessed by its application to gait-event detection using the experimental data.

A compliant modular robotic hand with fabric force sensor for multiple versatile grasping modes

This paper presents the development of a compliant, modular, reconfigurable, and sensorized robotic hand with multiple grasping capabilities. Each finger consists of a soft pneumatic actuator with embedded fabric force sensor and a detachable casing. The casing has a through hole for housing the actuator and special connectors for attachment to other casings. One casing each from the thumb and finger parts has a protrusion for connecting both parts together via a screw tightening system. The through-hole design allows different grasping length to be achieved and the inflated pneumatic channel of the actuator locks it in place. The modular robotic hand is capable of various versatile grasping tasks by simply changing the bending direction of the actuator, the distance between the thumb and finger parts, the grasping length of the actuator, or attaching/detaching additional fingers to the hand. (1) Hook grasping with single finger, (2) pinching with pad opposition, (3) reverse grasping for holding a pipe-like object, (4) wrap grasping with palm opposition, as well as (5) picking up an object through its handle with one thumb and two or more fingers were illustrated. These studies show the capability of the compliant modular robotic hand in performing various types of grasping by simply using different configurations of the casings. The excellent payload-to-weight ratio of the robotic hand was demonstrated. Also, the fabric force sensor that was embedded in the soft actuator indicated the difference in grasping forces that were applied to different objects during hook grasping. The modular robotic hand has the potential to broaden or substitute the usage of existing robotic hands, especially in applications where soft versatile configurable grasping is desired.

Discrimination of Dynamic Tactile Contact by Temporally Precise Event Sensing in Spiking Neuromorphic Networks

This paper presents a neuromorphic tactile encoding methodology that utilizes a temporally precise event-based representation of sensory signals. We introduce a novel concept where touch signals are characterized as patterns of millisecond precise binary events to denote pressure changes. This approach is amenable to a sparse signal representation and enables the extraction of relevant features from thousands of sensing elements with sub-millisecond temporal precision. We also proposed measures adopted from computational neuroscience to study the information content within the spiking representations of artificial tactile signals. Implemented on a state-of-the-art 4096 element tactile sensor array with 5.2 kHz sampling frequency, we demonstrate the classification of transient impact events while utilizing 20 times less communication bandwidth compared to frame based representations. Spiking sensor responses to a large library of contact conditions were also synthesized using finite element simulations, illustrating an 8-fold improvement in information content and a 4-fold reduction in classification latency when millisecond-precise temporal structures are available. Our research represents a significant advance, demonstrating that a neuromorphic spatiotemporal representation of touch is well suited to rapid identification of critical contact events, making it suitable for dynamic tactile sensing in robotic and prosthetic applications.

Soft haptics using soft actuator and soft sensor

In this paper, we presented fabric-based soft tactile actuator and soft sensor. The force characterization result indicates that the actuator is able to produce force up to about 2.20(±0.017)N, when it is supplied with 80kPa of pressurized air. Hence it is capable of producing sufficient amount of force, which surpasses the humans haptic perception threshold. The thin, sheet nature of the material creates lightweight actuator, which improves the payload-to-weight ratio. The pneumatic-based operation principle creates a safer human-machine interface. Thus, it eliminates possible occurrence of safety issues such as the danger of applying high voltages to users skin in case of malfunction. Direct force coupling of the soft actuator with the sensor is established to enable transmission of force information from the sensor to the actuator. The test profile indicates that the actuator is able to produce similar force profile as that of the sensor. This opens up possibility of developing soft tactile sensors and actuators based gloves, which can be paired and applied in virtual-reality based training and rehabilitation programs. Superimposition of multiple soft actuators would create an array that provides shape and size specific haptic feedback.

FPGA implementation of a FA-1 mechanoreceptor model for efficient representation of tactile features

Spatiotemporal spike patterns from a population of mechanoreceptors provide a concise representation of tactile stimuli that facilitates rapid sensory processing in the brain. Efficient models of mechanoreceptors are needed for the adoption of spike-based processing for robotic tactile sensing applications. This paper presents a biomimetic model of the fast-adapting type 1 (FA-1) mechanoreceptor, implemented on a field-programmable-gate-array (FPGA). The simplicity of this model enables its realization on large arrays of sensing elements while operating with sub-millisecond temporal precision required to capture deformation patterns. We illustrate this capability by interfacing with a 4096 element tactile sensor array with a 5.2 kHz sampling rate. Through physical experiments, we demonstrate the discrimination of force magnitude and local curvature during transient mechanical contact, using spike patterns obtained from the model. The approach has the potential to deliver responsive full-body tactile sensing in robotic and prosthetic applications.