Stephen A. Mascaro

A reconfigurable holonomic omnidirectional mobile bed with unified seating (RHOMBUS) for bedridden patients

A hybrid wheelchair/bed system for bedridden persons is developed and tested. A powered wheelchair can be docked to a bed portion and reconfigured to a flat stationary bed so that the bedridden person does not have to change seating when transferring between the chair and bed. Moreover the wheelchair can also be docked to a toilet directly and automatically. A holonomic omnidirectional vehicle with a ball wheel mechanism is used for the wheelchair. The high maneuverability and holonomic nature of the vehicle allow the chair to be docked easily and precisely against a fixture. The wheelchair is equipped with teleconferencing facility so that the bedridden patient may communicate face-to-face with a distal caregiver, friends and relatives. This paper describes the basic concept of the reconfigurable holonomic omnidirectional mobile bed with unified seating (RHOMBUS). Issues on the mechanical design of the vehicle, chair, and bed are addressed, followed by control issues. A force-guided docking control method using force sensors embedded in the bumper of the vehicle is developed. A prototype system is designed and tested.

Measuring Fingertip Forces by Imaging the Fingernail

This paper presents an external camera method for measuring fingertip forces by imaging the fingernail and surrounding skin. This method is an alternative to the photoplethysmograph sensor originally developed by one of the authors. A 3D model of the fingernail surface and skin is obtained with a stereo camera and laser striping system. Subsequent images from a single camera are registered to the 3D model by adding fiducial markings to the fingernail. Calibration results with a force sensor show that the measurement range depends on the region of the fingernail and skin. A Bayesian method is developed to predict fingertip force given coloration changes. Preliminary accuracy results for normal and shear force measurement are presented. In comparison to the results using the photoplethysmograph fingernail sensor, our results are more accurate and double the range of forces that can be transduced, all the way up to the saturation level.

Predicting Fingertip Forces by Imaging Coloration Changes in the Fingernail and Surrounding Skin

This paper presents an external camera method for measuring fingertip forces by imaging the fingernail and surrounding skin. A 3-D model of the fingernail surface and skin is obtained with a stereo camera and laser striping system. Subsequent images from a single camera are registered to the 3-D model by adding fiducial markings to the fingernail. Calibration results with a force sensor show that the measurement range depends on the region of the fingernail and skin. A generalized least squares model is developed to predict fingertip force given coloration changes, and results for normal and shear force measurement are presented.

Estimation of Fingertip Force Direction With Computer Vision

This paper presents a method of imaging the coloration pattern in the fingernail and surrounding skin to infer fingertip force direction (which includes four major shear-force directions plus normal force) during planar contact. Nail images from 15 subjects were registered to reference images with random sample consensus (RANSAC) and then warped to an atlas with elastic registration. With linear discriminant analysis, common linear features corresponding to force directions, but irrelevant to subjects, are automatically extracted. The common feature regions in the fingernail and surrounding skin are consistent with observation and previous studies. Without any individual calibration, the overall recognition accuracy on test images of 15 subjects was 90%. With individual training, the overall recognition accuracy on test images of 15 subjects was 94%. The lowest imaging resolution, without sacrificing classification accuracy, was found to be between 10-by-10 and 20-by-20 pixels.

Finger posture and shear force measurement using fingernail sensors: initial experimentation

A new method for measuring both the posture of human fingers and shear force at human fingertips is presented. Instead of using a traditional electronic glove with bending sensors embedded along the finger and shear sensors embedded beneath the fingertip, a wearable fingernail sensor is used to measure resulting changes in coloration of the fingernail. Since the sensor is mounted on the fingernail, finger posture can be measured without wearing a glove that hampers the motion of the fingers. Similarly, shear forces can be measured without covering the finger pad and obstructing the humans natural haptic sense. In the past, fingernail sensors with a one-dimensional array of photodetectors have been used to measure normal touching forces at the fingertip. A new fingernail sensor with a two-dimensional spatial array of photodetectors is constructed in order to measure finger posture and shear forces. Experiments are performed in order to measure the outputs of the photodetectors in response to changes in finger posture and applied normal and shear forces. Analysis is then performed to correlate the outputs to the inputs and provide a means of estimating normal forces, shear forces, and changes in finger posture.

Fingernail touch sensors: spatially distributed measurement and hemodynamic modeling

Advancements on a new type of touch sensor for detecting contact pressure at human fingertips are presented. A fingernail is instrumented with miniature LEDs and photodetectors in order to measure changes in the nail color when the fingertip is pressed against a surface. The fingernail sensor allows the fingers to directly contact the environment without obstructing the humans natural haptic senses. Reflectance photoplethysmography is used to measure the pattern of nail color, i.e., the blood content under the fingernail. Hemodynamic modeling, based on actual fingertip anatomy and physiology, is used to investigate the dynamics of the change in blood volume at multiple locations under the fingernail. An optical model is created to relate the blood volume to the light intensity measured by the photodetectors. The theoretical analysis is verified through model simulation and experimentation using a prototype fingernail sensor.

A Biologically Inspired Wet Shape Memory Alloy Actuated Robotic Pump

This paper presents the concept, design, optimization, and experimental analysis of a biologically inspired wet shape memory alloy (SMA) actuated pump for robotic and mechatronic systems. Just as the human heart provides energy to the muscles in the body, the robotic SMA pump distributes thermofluidic energy to arrays of SMA actuators that function as robotic muscles. Furthermore, the robotic pump draws from its own fluidic output to assist in the actuation of its own internal SMA actuators, just as a portion of the blood pumped by the human heart supplies energy to its own muscles. A feasibility analysis provides insight to limits of configuration parameters. Through dynamic modeling and simulation of the system, various configurations can be analyzed for optimization of the output. The effects of changing configuration parameters were explored via simulation and validated with experimentation. The current prototype of the SMA heart is capable of pumping 2.1 times more fluid than is required to sustain its own actuation. This is the first successful implementation of such a robotic pump, such that it has a net positive thermofluidic output to provide to other actuators while sustaining its own actuation via thermofluidic feedback. Furthermore, this SMA pump is capable of pumping a net output of 66 mL/min, which is two orders of magnitude larger than the output of any other SMA pump.

Docking control of holonomic omnidirectional vehicles with applications to a hybrid wheelchair/bed system

A method for docking a vehicle with a fixture is developed and applied to a hybrid wheelchair/bed system. A powered wheelchair is docked to a bed portion and reconfigured to a flat stationary bed so that the bedridden person does not have to change seating when transferring between the chair and bed. A holonomic omnidirectional vehicle with a ball wheel mechanism is used for the wheelchair. The high maneuverability and holonomic nature of the vehicle allow the wheelchair to be docked precisely with a fixture. An instrumented bumper is developed to detect contacts with the fixture and ensure rider comfort. Using sensor information from the bumpers, the contact state of the vehicle is recognized at all times. Moreover, the orientation of the fixture is estimated from the contact information, and using a framework of discrete event control, the vehicle is aligned and guided towards the fixture despite large misalignments. A prototype system was designed and tested. The vehicle was successfully docked to the bed despite a small clearance ratio as well as large lateral and angular initial misalignments.

Dynamic Thermomechanical Modeling of a Wet Shape Memory Alloy Actuator

This paper presents combined thermal and mechanical models of a wet shape memory alloy (SMA) wire actuator. The actuator consists of a SMA wire suspended concentrically in a compliant tube. Actuation occurs as hot and cold water that are alternately pumped through the tube to contract and extend the wire, respectively. The thermomechanical model presented in this paper accounts for the nonuniform temperature change of the SMA wire due to alternating the temperature of the flow along the wire. The thermal portion of the model consists of analysis of the heat transfer between the fluid and the SMA wire. Heat loss to the environment and the temperature change of the fluid through the actuator are taken into account. Based on this analysis, the temperature of the wire at segments along its length can be determined as a function of time. The mechanical portion of the model approximates the strain-martensite fraction and martensite fraction-temperature relationships. By combining the thermal and mechanical models, the displacement of the wire can be determined as a function of time. The combined thermomechanical model will be useful for predicting the performance of wet SMA actuators in a variety of applications.

3-D force control on the human fingerpad using a magnetic levitation device for fingernail imaging calibration

This paper demonstrates fast, accurate, and stable force control in three axes simultaneously when a flat surface is pressed against the human fingerpad. The primary application of this force control is for the automated calibration of a fingernail imaging system, where video images of the human fingernail are used to predict the normal and shear forces that occur when the fingerpad is pressed against a flat surface. The system consists of a six degree-of-freedom magnetic levitation device (MLD), whose flotor has been modified to apply forces to the human fingerpad, which is resting in a passive restraint. The system is capable of taking simultaneous steps in normal force and two axes of shear forces with a settling time of less than 0.2 seconds, and achieves a steady-state error as small as 0.05 N in all three axes. The system is also capable of tracking error of less than 0.2 N when the shear force vector rotates with a frequency of 1 rad/s. This paper also demonstrates the successful tracking of a desired force trajectory in three dimensions for calibrating a fingernail imaging system.