Gareth J. Monkman

Palpation imaging using a haptic system for virtual reality applications in medicine.

In the field of medical diagnosis, there is a strong need to determine mechanical properties of biological tissue, which are of histological and pathological relevance. Malignant tumors are significantly stiffer than surrounding healthy tissue. One of the established diagnosis procedures is the palpation of body organs and tissue. Palpation is used to measure swelling, detect bone fracture, find and measure pulse, or to locate changes in the pathological state of tissue and organs. Current medical practice routinely uses sophisticated diagnostic tests through magnetic resonance imaging (MRI), computed tomography (CT) and ultrasound (US) imaging. However, they cannot provide direct measure of tissue elasticity. Last year we presented the concept of the first haptic sensor actuator system to visualize and reconstruct mechanical properties of tissue using ultrasonic elastography and a haptic display with electrorheological fluids. We developed a real time strain imaging system for tumor diagnosis. It allows biopsies simultaneously to conventional ultrasound B-Mode and strain imaging investigations. We deduce the relative mechanical properties by using finite element simulations and numerical solution models solving the inverse problem. Various modifications on the haptic sensor actuator system have been investigated. This haptic system has the potential of inducing real time substantial forces, using a compact lightweight mechanism which can be applied to numerous areas including intraoperative navigation, telemedicine, teaching and telecommunication.

Ultra-soft PDMS-based magnetoactive elastomers as dynamic cell culture substrata.

Mechanical cues such as extracellular matrix stiffness and movement have a major impact on cell differentiation and function. To replicate these biological features in vitro, soft substrata with tunable elasticity and the possibility for controlled surface translocation are desirable. Here we report on the use of ultra-soft (Young’s modulus <100 kPa) PDMS-based magnetoactive elastomers (MAE) as suitable cell culture substrata. Soft non-viscous PDMS (<18 kPa) is produced using a modified extended crosslinker. MAEs are generated by embedding magnetic microparticles into a soft PDMS matrix. Both substrata yield an elasticity-dependent (14 vs. 100 kPa) modulation of α-smooth muscle actin expression in primary human fibroblasts. To allow for static or dynamic control of MAE material properties, we devise low magnetic field (≈40 mT) stimulation systems compatible with cell-culture environments. Magnetic field-instigated stiffening (14 to 200 kPa) of soft MAE enhances the spreading of primary human fibroblasts and decreases PAX-7 transcription in human mesenchymal stem cells. Pulsatile MAE movements are generated using oscillating magnetic fields and are well tolerated by adherent human fibroblasts. This MAE system provides spatial and temporal control of substratum material characteristics and permits novel designs when used as dynamic cell culture substrata or cell culture-coated actuator in tissue engineering applications or biomedical devices.

A NEW ER FLUID BASED HAPTIC ACTUATOR SYSTEM FOR VIRTUAL REALITY

The concept and some steps in the development of a new actuator system which enables the haptic perception of mechanically inhomogeneous virtual objects are introduced. The system consists of a two-dimensional planar array of actuator elements containing an electrorheological (ER) fluid. When a user presses his fingers onto the surface of the actuator array, he perceives locally variable resistance forces generated by vertical pistons which slide in the ER fluid through the gaps between electrode pairs. The voltage in each actuator element can be individually controlled by a novel sophisticated switching technology based on optoelectric gallium arsenide elements. The haptic information which is represented at the actuator array can be transferred from a corresponding sensor system based on ultrasonic elastography. The combined sensor-actuator system may serve as a technology platform for various applications in virtual reality, like telemedicine where the information on the consistency of tissue of a real patient is detected by the sensor part and recorded by the actuator part at a remote location.

Technologies for haptic displays in teleoperation

Since the 1960s many alphanumeric to tactile data conversion methods have been investigated, mainly with the ultimate aim of assisting the blind. More recently, interest has been directed toward the display of pictures on haptically explorable surfaces – tactile imaging – for a range of medical, remote sensing and entertainment purposes. This paper examines the technologies which have been utilised for haptically explorable tactile displays over the past three decades, focussing on those which appear commercially viable in the immediate future.

A New Haptic Sensor Actuator System for Virtual Reality Applications in Medicine

The pathological state of soft tissues is often correlated with changes in stiffness. Malignant tumors are significantly stiffer and more immobile than surrounding healthy tissue. (hard lesions, “nodes” in organs: tumors; calcifications in vessels: arteriosclerosis). The main problem is, that such information is usually not available or can only be obtained by manual palpation, which is subjective and limited in sensitivity. It requires intuitive assessment and does not allow quantitative documentation. On the one hand a suitable sensor is required for quantitative measurement of mechanical tissue properties. On the other hand, there is also a need for a realistic mechanical display of such tissue properties. Suitable actuator arrays with high spatial resolution acting in real time are required. A haptic sensor actuator system is presented in this paper including a sensitive sensor part and an actuator array for different applications. The mechanical consistency of an object is to be locally specified using a sensor system and represented perceptibly in a remote position on a tactile display (actuator system) for the user. The sensor system uses ultrasound (US) elastography, whereas the actuator array is based on electrorheological (ER) fluids.

3D-Shape Recognition Based Tactile Sensor

a surface recognition algorithm capable of determining contact surfaces types by means of tactile sensor fusion is proposed. The authors present a recognition processes for 3-dimensional deformations in a 2-dimensional parametric domain. Tactile information is extracted by physical contact with a grasped object through a sensing medium. Information is obtained directly at the interface between the object and the sensing device and relates to three-dimensional position and orientation of the object in the presence of noise. The technique called “eigenvalue trajectory analysis”, is introduced and adopted for specifying the margin of classification and classification thresholds. The authors demonstrate mathematically that this approach, which complements existing work, offers significant computational advantages when applied to challenging contact scenarios such as dynamic recognition of contact deformations.

Optimisation of prehension force through tactile sensing

Purpose – The purpose of this paper is to analyze surface deformations caused by shear and moment forces on tactile materials and present a method to detect and reduce the risk of slippage by controlling the normal force as measured by tactile sensor arrays.Design/methodology/approach – A predictive model has been proposed which uses a basic method adapted to real applications in grasp optimization. Prevention of premature release with minimum prehension force is addressed without the need to measure the coefficient of friction between object and robot gripper. Predictive models have been used to develop a set of rules which predict the pre‐slip based on fluctuations in tactile signal data.Findings – The tactile sensors can be used in a “nonlinear” manner during manipulation tasks. When the gripper finger first makes contact with an object, the stress distribution under the finger skin varies rapidly. Predictive models have been used to develop a set of rules which predict the pre‐slip based on fluctuatio...

Heavy duty robotic precision fracture repositioning

The application of robotics in manufacturing industry is increasingly spreading to other fields such as service, security and medical, and more recently into orthopedic surgery. Most research projects to date have concentrated on the lighter side of non‐invasive surgery, camera, laser guidance, light cutting and milling through bone. Just as in industrial production and processing applications, the choice of robot and its accompanying control and programming system is absolutely paramount. This simple fact has been justified in recent research dealing with the heavier forms of fracture repositioning robotics in accident surgery. This paper discusses the development of the complete system including robot, end‐effector and sensors.

Entwicklung eines haptischen Sensor-Aktor-Systems für Anwendungen in der virtuellen Realität

Die mechanischen Eigenschaften des biologischen Gewebes stellen wichtige Diagnoseinformationen dar und sind von groser histologischer und pathologischer Bedeutung. Insbesondere Tumoren stel- len sich oft als Gewebeverhartungen dar. Das Problem ist, dass konven- tionelle bildgebende Verfahren (Ultraschall, Computertomographie, Ma- gnetresonanztomographie) oftmals keine pathologische Verhartungen im Gewebe (wie z. B. in Brust- oder Prostatakrebs) erkennen konnen. In diesem Projekt wird ein neuartiges System fur die Erfassung und Darstellung haptischer Informationen in der virtuellen Realitat entwickelt. Mit einem haptischen Sensor-Aktor-System soll die Konsistenz eines Objektes ortsaufgelost erfasst und an anderer Stelle fur den Benutzer tastbar dargestellt werden. Das haptische System wird als eine Technologieplattform angesehen, auf der anschliesend verschiedene Produkte fur die Medizintechnik, die Unter-haltungsindustrie oder den Ausbildungssektor entwickelt werden konnen.

A Haptic System for Virtual Reality Applications Based on Ultrasound Elastography and Electro-Rheological Fluids

Mechanical properties of biological tissue represent important diagnostic information and are of histological and pathological relevance. Malignant tumors are significantly stiffer and more immobile than surrounding healthy tissue. Hard calcifications in vessels occur due to arteriosclerosis. The problem is, that such information is usually not available or can only be obtained by manual palpation, which is subjective and limited in sensitivity. It requires intuitive assessment and does not allow quantitative documentation. Unfortunately, none of the established medical imaging equipment such as magnetic resonance imaging (MRI) or X-ray computed tomography (CT) can provide direct measure of tissue elasticity. On the one hand a suitable sensor is required for quantitative measurement of mechanical tissue properties. On the other hand there is also some need for a realistic haptic display of such tissue properties. Suitable actuator arrays with high spatial resolution acting in real time are required. A haptic sensor actuator system is presented in this paper including a sensitive sensor part and an actuator array for different applications. The mechanical consistency of an object is to be locally specified using a sensor system and represented perceptibly in a remote position on an actuator system for the user. The sensor system uses ultrasound (US) elastography, whereas the actuator array is based on electrorheological (ER) fluids.