James T. Hing

A biplanar fluoroscopic approach for the measurement, modeling, and simulation of needle and soft-tissue interaction.

A methodology for modeling the needle and soft-tissue interaction during needle insertion is presented. The approach consists of the measurement of needle and tissue motion using a dual C-arm fluoroscopy system. Our dual C-arm fluoroscopy setup allows real time 3-D extraction of the displacement of implanted fiducials in the soft tissue during needle insertion to obtain the necessary parameters for accurate modeling of needle and soft-tissue interactions. The needle and implanted markers in the tissue are tracked during the insertion and withdrawal of the needle at speeds of 1.016 mm/s, 12.7 mm/s and 25.4 mm/s. Both image and force data are utilized to determine important parameters such as the approximate cutting force, puncture force, the local effective modulus (LEM) during puncture, and the relaxation of tissue. We have also validated the LEM computed from our finite element model with arbitrary needle puncture tasks. Based on these measurements, we developed a model for needle insertion and withdrawal that can be used to generate a 1-DOF force versus position profile that can be experienced by a user operating a haptic device. This profile was implemented on a 7-DOf haptic device designed in our laboratory.

Reality-based needle insertion simulation for haptic feedback in prostate brachytherapy

There is a strong need to improve the tools clinicians use for training in procedures such as prostate brachytherapy where the success rate is directly related to the clinicians level of experience. Accurate haptic feedback is needed for developing improved surgical simulators and trainers for such procedures. In prostate brachytherapy, accurate needle placement of radioactive seeds in the prostate is crucial to the success of the surgery and to the quality of life of the patient. Therefore, a trainer or simulator for this and other types of needle insertion tasks require an accurate reality-based quantification and model of the needle and soft tissue interaction. To achieve this, we utilize the X-ray images produced by a dual C-arm fluoroscope setup during a needle insertion task to obtain parameters needed for accurate modeling of soft tissue and needle interactions. The needle and implanted markers in the tissue are tracked during the insertion and withdrawal of the needle at speeds of 1.016 mm/sec, 12.7 mm/sec and 25.4 mm/sec. Both image and force data are utilized to determine important parameters such as the local effective modulus during puncture and the approximate cutting force for soft tissue samples. A finite element model was built using the data to model needle puncture of tissue

Development of an Unmanned Aerial Vehicle Piloting System with Integrated Motion Cueing for Training and Pilot Evaluation

UAV accidents have been steadily rising as demand and use of these vehicles increases. A critical examination of UAV accidents reveals that human error is a major cause. Advanced autonomous systems capable of eliminating the need for human piloting are still many years from implementation. There are also many potential applications of UAVs in near Earth environments that would require a human pilot’s awareness and ability to adapt. This suggests a need to improve the remote piloting of UAVs. This paper explores the use of motion platforms to augment pilot performance and the use of a simulator system to asses UAV pilot skill. The approach follows studies on human factors performance and cognitive loading. The resulting design serves as a test bed to study UAV pilot performance, create training programs, and ultimately a platform to decrease UAV accidents.

Reality-Based Estimation of Needle and Soft-Tissue Interaction for Accurate Haptic Feedback in Prostate Brachytherapy Simulation

Prostate Brachytherapy is the implantation of radioactive seeds into the prostate as a treatment for prostate cancer. The success rate of the procedure is directly related to the physician’s level of experience. In addition, minor deviations in seed alignment caused by gland compression/retraction, gland edema (swelling) and needle deflections can create significant areas of over or under dosage to the gland and/or injury to surrounding nerves and organs, leading to increased morbidity. Therefore, reductions in brachytherapy complication rates will be dependent on improving the tools physicians use for training to improve the accuracy of needle guidance and deployment of ‘seeds’ within the prostate gland. Through our novel approach of using two C-ARM fluoroscopes, we propose a reality-based approach for estimating needle and soft tissue interaction for the purpose of eventually developing an accurate seed placement training simulator with haptic feedback for prostate brachytherapy. By recording implanted fiducial movement and needle-soft tissue interaction forces, we can: extract the local effective modulus during puncture events, quantify tissue deformation, obtain an approximate cutting force, and build a finite element model to provide accurate haptic feedback in the training simulator for needle insertion tasks.

Development and Evaluation of a Chase View for UAV Operations in Cluttered Environments

Civilian applications for UAVs will bring these vehicles into low flying areas cluttered with obstacles such as building, trees, power lines, and more importantly civilians. The high accident rate of UAVs means that civilian use will come at a huge risk unless we design systems and protocols that can prevent UAV accidents, better train operators and augment pilot performance. This paper presents two methods for generating a chase view to the pilot for UAV operations in cluttered environments. The chase view gives the operator a virtual view from behind the UAV during flight. This is done by generating a virtual representation of the vehicle and surrounding environment while integrating it with the real-time onboard camera images. Method I presents a real-time mapping approach toward generating the surrounding environment and Method II uses a prior model of the operating environment. Experimental results are presented from tests where subjects flew in a H0 scale environment using a 6 DOF gantry system. Results showed that the chase view improved UAV operator performance over using the traditional onboard camera view.

Improving unmanned aerial vehicle pilot training and operation for flying in cluttered environments

Future applications will bring unmanned aerial vehicles (UAVs) to new environments such as urban areas, causing a change in the way that UAVs are currently operated. However, UAV accidents still occur at a much higher rate than the accident rate for commercial airliners. Therefore, there is a need to better train UAV pilots and augment their performance to minimize accidents. In this paper, the authors present two methods for generating a chase view point (similar to the view of being towed behind the aircraft). Through use of these viewpoints, the authors propose to increase the situational awareness of UAV operators when flying in cluttered environments. The chase view combines a rotated onboard camera view with a virtual representation of the vehicle and the surrounding operating environment. Experiments were conducted evaluating a chase view versus a traditional onboard camera view during UAV flights using a 6 DOF gantry system. Results showed that the chase view improved UAV operator performance.

A Motion Platform Integrated UAV Pilot Training and Evaluation System for Future Civilian Applications

The potential for UAVs to benefit the civilian consumer is driving the demand for the integration of these vehicles into the national airspace. With UAV accidents occurring at a significantly higher rate than commercial airlines, the urgent issue becomes designing systems and protocols that can prevent UAV accidents, better train UAV operators and augment pilot performance. This paper presents three directions of research stemming from the goal of a UAV piloting and training system. Research direction one is the development of a research platform to assess UAV pilot skills and recreate the sensation of shared fate for UAV pilots. The second research direction looks at utilizing flight simulation packages to create virtual tools for training UAV pilots. The third direction covers the investigation of UAV’s in near earth environments as future applications will place UAVs in these areas.© 2008 ASME

Integrating Motion Platforms With Unmanned Aerial Vehicles to Improve Control, Train Pilots and Minimize Accidents

As the number of UAVs increase, the risk of accidental crashes also grows. A critical examination of these accidents reveals that human error is a major cause. This suggests a need to improve the remote piloting of UAVs. This paper explores the use of motion platforms to augment pilot performance. This approach follows studies on human factors performance and cognitive loading. The resulting design serves as a test bed to study UAV pilot performance, create training programs, and ultimately a platform to avoid UAV accidents.© 2008 ASME

Mixed reality for unmanned aerial vehicle operations in near Earth environments

Future applications will bring unmanned aerial vehicles (UAVs) to near Earth environments such as urban areas, causing a change in the way UAVs are currently operated. Of concern is that UAV accidents still occur at a much higher rate than the accident rate for commercial airliners. A number of these accidents can be attributed to a UAV pilots low situation awareness (SA) due to the limitations of UAV operating interfaces. The main limitation is the physical separation between the vehicle and the pilot. This eliminates any motion and exteroceptive sensory feedback to the pilot. These limitation on top of a small field of view from the onboard camera results in low SA, making near Earth operations difficult and dangerous. Autonomy has been proposed as a solution for near Earth tasks but state of the art artificial intelligence still requires very structured and well defined goals to allow safe autonomous operations. Therefore, there is a need to better train pilots to operate UAVs in near Earth environments and to augment their performance for increased safety and minimization of accidents.

Reality-based Haptic Feedback for Needle Puncture - Modeling, Validation, and Simulation

Reality-based models and accurate simulations of needle insertions into soft tissue require the extraction of internal tissue deformation and needle forces during insertion. It also requires the development of devices that can accurately display these forces to the user. Our dual C-Arm fluoroscopy setup allows for the real time 3-D extraction of the displacement of implanted fiducials in the soft tissue during needle insertion. Using this information and collected force data, we have, validated the LEM computed from our finite element model with arbitrary needle puncture tasks. Additionally, we have simulated a realistic needle puncture task using the 7 degree-of-freedom haptic feedback device developed in our laboratory.