Masashi Sekine

Psycho-physiological assessment of a prosthetic hand sensory feedback system based on an auditory display: a preliminary study

BackgroundProsthetic hand users have to rely extensively on visual feedback, which seems to lead to a high conscious burden for the users, in order to manipulate their prosthetic devices. Indirect methods (electro-cutaneous, vibrotactile, auditory cues) have been used to convey information from the artificial limb to the amputee, but the usability and advantages of these feedback methods were explored mainly by looking at the performance results, not taking into account measurements of the user’s mental effort, attention, and emotions. The main objective of this study was to explore the feasibility of using psycho-physiological measurements to assess cognitive effort when manipulating a robot hand with and without the usage of a sensory substitution system based on auditory feedback, and how these psycho-physiological recordings relate to temporal and grasping performance in a static setting.Methods10 male subjects (26+/-years old), participated in this study and were asked to come for 2 consecutive days. On the first day the experiment objective, tasks, and experiment setting was explained. Then, they completed a 30 minutes guided training. On the second day each subject was tested in 3 different modalities: Auditory Feedback only control (AF), Visual Feedback only control (VF), and Audiovisual Feedback control (AVF). For each modality they were asked to perform 10 trials. At the end of each test, the subject had to answer the NASA TLX questionnaire. Also, during the test the subject’s EEG, ECG, electro-dermal activity (EDA), and respiration rate were measured.ResultsThe results show that a higher mental effort is needed when the subjects rely only on their vision, and that this effort seems to be reduced when auditory feedback is added to the human-machine interaction (multimodal feedback). Furthermore, better temporal performance and better grasping performance was obtained in the audiovisual modality.ConclusionsThe performance improvements when using auditory cues, along with vision (multimodal feedback), can be attributed to a reduced attentional demand during the task, which can be attributed to a visual “pop-out” or enhance effect. Also, the NASA TLX, the EEG’s Alpha and Beta band, and the Heart Rate could be used to further evaluate sensory feedback systems in prosthetic applications.

Comparison of reflected green light and infrared photoplethysmography

We evaluated the accuracy of pulse rate measurements obtained by reflected green light photoplethysmography (PPG) compared to reflected infrared light photoplethysmography and ECG. The wavelengths of the green and infrared light were 525 and 880 nm, respectively, and experiments were performed at 25°C and at a skin temperature below 15°C. The pulse rate obtained from reflected green light PPG was compared with the ECG RR interval and the pulse rate from reflected infrared PPG. The results indicated a stronger correlation between green PPG and ECG results at both temperatures. These results suggested that reflected green light PPG had an advantage over reflected infrared PPG, especially at temperatures below 15°C.

To Design a Small Pneumatic Actuator Driven Parallel Link Mechanism for Shoulder Prostheses for Daily Living Use

Only in Japan, there are about 82,000 upper limb amputees (Ministry of Health, Labour and Welfare, 2005). Using upper limb prostheses could restore the function for them, thus improve significantly the quality of their activities of daily living [ADL]. Compared with below-elbow prostheses, shoulder prostheses are left behind in their development, due to high degrees of freedom [DOF] required, which demands a large number of actuators, thus denotes a large size and a heavy weight, and complicated control mechanism. Recently, there is a certain body of research on developing robotic devices that could be used as prostheses for shoulder amputees (Jacobson et al., 1982; Motion Control, Inc., 20062011; The Johns Hopkins University Applied Physics Laboratory [APL], 2011; Troncossi et al., 2005, 2009a, 2009b). These research efforts have led to artificial prostheses with high functionality and performance. For example, the prosthetic arm of Defense Advanced Research Projects Agency and APL, has 25 DOFs, individual finger movements, dexterity that approaches that of the human limb, natural control, sensory feedback, and a number of small wireless devices that can be surgically implanted (or injected) to allow access to intramuscular signals(APL, 2011). The Utah Arm 3, a modification of the previous Utah Arm that has been the premier myoelectric arm for above elbow amputees, has two microcontrollers that are programmed for the hand and elbow, accordingly, allowing separate inputs and hence simultaneous control of both, and that is, the wearer can operate the hand and elbow concurrently for natural function (Jacobson et al., 1982; Motion Control, Inc., 2006-2011). The hybrid electric prosthesis for single arm amputee of Tokyo Denki University possesses a ball joint of 3 DOFs in humeral articulation. Patient operates the prosthesis to optional point by pressing a switch with the other healthy limb to free the joint, and releases to fix and hold the prosthetic arm stably (Nasu et al., 2001). Moreover, the electromechanical shoulder articulation with 2 DOFs for upper-limb prosthesis that has two actuated joints embedded harmonic drives, an inverted slider crank mechanism, and ball screw, has been developed (Troncossi et al., 2005, 2009a, 2009b). These prostheses have the following characteristics: they are more or less anthropomorphic, basically supported by metal frames or parts, driven by electric motors, therefore, many of them seem to be not suitable for the daily living use: they are not light weight, not convenient, with a bad portability, and lack of backdrivability which could contribute to the safety use in daily living.

New Computer-based Cognitive Function Test for the Elderly

We developed a modified trail-making test using a PC and touch panel and compared it with the mini mental state examination (MMSE). The test consisted of a series of numbers from 1 to 36, randomly arranged across the display. The object of the test was for the subject to touch the numbers in order, beginning with 1 and ending with 36, in as little time as possible. The system consisted of a PC and a liquid crystal display (LCD) touch-panel screen. One hundred and thirty-four patients with dementia performed the test. Sixty of the 134 patients (15 male, 45 female; average age, 81.1plusmn7 years) were diagnosed as having Alzheimers disease and the others had cerebrovascular dementia. Sixty-two of 134 patients (23 male, 39 female; average age, 77.6plusmn8 years; MMSE score, 21.5plusmn5.6 points) completed the test. The correlation coefficient between test performance time and MMSE score was -0.534. This test may also be a useful indicator of focal frontal lesions and can be used as an early screening test for Alzheimers disease

Monitoring rehabilitation training for hemiplegic patients by using a tri-axial accelerometer

In rehabilitation training for hemiplegic patients, bed-to-wheelchair transfer is most important and allows a patients early independence. We developed a monitoring system for transfer training that is quantitative and uses accelerometry. Two tri-axial accelerometers were attached to the subjects, at the head and waist. Subjects were trained in moving from a sitting position to standing, then turning through, about 90 degrees, and then sitting on the bed. The acceleration signals at the two sites were recorded via a multi-telemeter system, converted to a digital signal, and stored in a computer. Data were analysed by LabView and displayed on the computer screen as real time motion. The system can be operated by one staff member, and the patients did not feel restricted. We were able to evaluate the time course of the signals, and phase-plane locus of the vertical, lateral and horizontal directions of the signals. The transfer of hemiplegic patients occurred in two motions: a standing-up motion, and a sitting-down motion. Accordingly, we observed twin peaks in the plot of the original signal. In contrast, healthy volunteers moved smoothly and twin peaks were not present.

Evaluation by accelerometry of walking pattern before falls in hemiplegic patients

Hemiplegic patients often fall because of a lack of balance-during walking. They can become bed-ridden, or suffer failing syndrome after falling. The aim of this study was to evaluate the walking pattern just prior to falls among high-risk patients in a rehabilitation setting. A triaxial accelerometer was fixed to the subjects waist, and the triaxial acceleration signals were recorded. Thirty-one subjects walked down a corridor under supervision. The data were digitized at a sampling rate of 200 Hz, and analysed using a discrete wavelet transform. The variables required to evaluate falls were related to the reconstructed signal at level -3. We classified falls into three types. In Type 1 falls, the impact acceleration signals in the vertical direction were larger in walking just prior to a fall. In Type 2 falls, small impact acceleration signals in the vertical direction were observed. In Type 3 falls, the walking cycle changed irregularly just prior to a fall and the impact acceleration was larger and smaller before the fall. From these results, falls were evaluated. The next step will be predicting falls.

Variable Impedance Control Based on Impedance Estimation Model with EMG Signals during Extension and Flexion Tasks for a Lower LimbRehabilitation Robotic System

Rehabilitation robotic devices could be used as an effective tool to restore impaired motion functionality. Due to the human-robot cooperative nature of rehabilitation, these systems are expected to be user-oriented i.e., they should be controlled considering users dynamic characteristics. In this article, we proposed a variable impedance control, in which desired impedance of a system was setup to match human joint stiffness estimated from Electromyogram (EMG) signals recorded. Two experiments were performed in this work. The objective of the first experiment (Experiment-1) was to study the relationship between EMG and changing impedance in knee joint extension and flexion tasks. Based on the recorded data, a nonlinear model was proposed to express the relationship between EMG and changing impedance. The results show that Root Mean Square of the EMG signals (RMS-EMG) of target muscles increases, as specified elastic modulus increases for both tasks, but there is a significant difference (p<0.01, t-test) between the extension and flexion task. The second experiment (Experiment-2) was to confirm the effectiveness of the variable impedance control with the motion-dependent models acquired in Experiment-1. Four different control policies were tested, i.e., NA: No Assist; FO: using EMG-impedance model from the Flexion Task; EO: using EMG-impedance graph from the Extension Task; FE: using two EMG-impedance models from Flexion and Extension task correspondently. Results indicate that the proposed control model (FE) achieved a smaller discrepancy (p<0.01, t-test) between desired angle and the reached angle than the control with EO or NA cases. Moreover, a small sum of RMS_EMG from the variable impedance control with motion- dependent models denoted less effort required than the NA (p<0.01, t-test) case or the control with FO (p<0.01, t-test). Results also indicate that the proposed nonlinear and motion-dependent variable impedance control method achieved a smaller angular discrepancy (P<0.05, t-test) than linear variable impedance control.

Evaluation of gait parameters by the knee accelerations

A portable micro-processor based data acquisition system has been developed to measure the acceleration of both legs during gait for ambulatory subjects and to evaluate temporal and spatial gait parameters. Experiments were performed on ten hemiplegic patients with different Brunnstrom stage (Br stage) grading. The orthogonal acceleration at the knee joint was measured from walking at a normal speed. The results according to various gait parameters such as gait cycle and stance were significantly different among patients with a different Br stage. This monitor may promise to allow quantitative evaluation of hemiplegic patients both in a hospital environment and in their home.

Development of a small sucker manipulator for underwater surgery support.

WaFLES (Water-Filled LaparoEndoscopic Surgery) is an operative method suggested by Igarashi et al., which has several advantages, such as, preventing the drying of inner organs, and being able to use ultrasound devices for real time monitoring. However, grasping of inner organs with usual forceps for move and incision purpose is difficult. Therefore our ultimate goal is to develop a small sucker manipulator for WaFLES support. Experiments were conducted to explore suitable suction cups for underwater application, and suitable structure (cup-to-cup distance, elasticity of binding material, layouts of multiple cups) for a multiple-cup assembly, in terms of adsorption force and tolerance to sideslip. Experiment results showed that 1) the shape of each single suction cup for the underwater application was identified; 2) the structure of the multiple-cup assembly affects the adsorption force and tolerance to sideslip.

Development of Robust Behaviour Recognition for an at-Home Biomonitoring Robot with Assistance of Subject Localization and Enhanced Visual Tracking

Our research is focused on the development of an at-home health care biomonitoring mobile robot for the people in demand. Main task of the robot is to detect and track a designated subject while recognizing his/her activity for analysis and to provide warning in an emergency. In order to push forward the system towards its real application, in this study, we tested the robustness of the robot system with several major environment changes, control parameter changes, and subject variation. First, an improved color tracker was analyzed to find out the limitations and constraints of the robot visual tracking considering the suitable illumination values and tracking distance intervals. Then, regarding subject safety and continuous robot based subject tracking, various control parameters were tested on different layouts in a room. Finally, the main objective of the system is to find out walking activities for different patterns for further analysis. Therefore, we proposed a fast, simple, and person specific new activity recognition model by making full use of localization information, which is robust to partial occlusion. The proposed activity recognition algorithm was tested on different walking patterns with different subjects, and the results showed high recognition accuracy.