Momen Abayazid

Needle path planning and steering in a three-dimensional non-static environment using two-dimensional ultrasound images

Needle insertion is commonly performed in minimally invasive medical procedures such as biopsy and radiation cancer treatment. During such procedures, accurate needle tip placement is critical for correct diagnosis or successful treatment. Accurate placement of the needle tip inside tissue is challenging, especially when the target moves and anatomical obstacles must be avoided. We develop a needle steering system capable of autonomously and accurately guiding a steerable needle using two-dimensional (2D) ultrasound images. The needle is steered to a moving target while avoiding moving obstacles in a three-dimensional (3D) non-static environment. Using a 2D ultrasound imaging device, our system accurately tracks the needle tip motion in 3D space in order to estimate the tip pose. The needle tip pose is used by a rapidly exploring random tree-based motion planner to compute a feasible needle path to the target. The motion planner is sufficiently fast such that replanning can be performed repeatedly in a closed-loop manner. This enables the system to correct for perturbations in needle motion, and movement in obstacle and target locations. Our needle steering experiments in a soft-tissue phantom achieves maximum targeting errors of 0.86 ± 0.35 mm (without obstacles) and 2.16 ± 0.88 mm (with a moving obstacle).

3D flexible needle steering in soft-tissue phantoms using Fiber Bragg Grating sensors

Needle insertion procedures are commonly used for surgical interventions. In this paper, we develop a three-dimensional (3D) closed-loop control algorithm to robotically steer flexible needles with an asymmetric tip towards a target in a soft-tissue phantom. Twelve Fiber Bragg Grating (FBG) sensors are embedded on the needle shaft. FBG sensors measure the strain applied on the needle during insertion. A method is developed to reconstruct the needle shape using the strain data obtained from the FBG sensors. Four experimental cases are conducted to validate the reconstruction method (single-bend, double-bend, 3D double-bend and drilling insertions). In the experiments, the needle is inserted 120 mm into a soft-tissue phantom. Camera images are used as a reference for the reconstruction experiments. The results show that the mean needle tip accuracy of the reconstruction method is 1.8 mm. The reconstructed needle shape is used as feedback for the steering algorithm. The steering algorithm estimates the region that the needle can reach during insertion, and controls the needle to keep the target in this region. Steering experiments are performed for 110 mm insertion, and the mean targeting accuracy is 1.3 mm. The results demonstrate the capability of using FBG sensors to robotically steer needles.

Real-time three-dimensional flexible needle tracking using two-dimensional ultrasound

Needle insertion is one of the most commonly performed minimally invasive procedures. Visualization of the needle during insertion is key for either successful diagnosis or therapy. This work presents the real-time three-dimensional tracking of flexible needles during insertion into a soft-tissue simulant using a two-dimensional (2D) ultrasound transducer. The transducer is placed perpendicular to the needle tip to measure its position. During insertion the transducer is robotically repositioned to track the needle tip. Positioning of the transducer is accomplished by a compensator, that uses the needle insertion velocity corrected by needle tip velocities to determine out-of-plane motion. Experiments are performed to validate the needle tip pose during tracking. The maximum mean errors in needle tip position along the x-, y- and z-axes are 0.64 mm, 0.25 mm and 0.27 mm, respectively. The error in tip orientations (θ-about the y-axis and φ-about the z-axis) are 2.68° and 2.83°, respectively. This study demonstrates the ability to compute the needle tip pose using a 2D ultrasound transducer. The tip pose can be used to robotically steer needles, and thereby improve accuracy of medical procedures.

Teleoperation of Steerable Flexible Needles by Combining Kinesthetic and Vibratory Feedback

Needle insertion in soft-tissue is a minimally invasive surgical procedure that demands high accuracy. In this respect, robotic systems with autonomous control algorithms have been exploited as the main tool to achieve high accuracy and reliability. However, for reasons of safety and responsibility, autonomous robotic control is often not desirable. Therefore, it is necessary to focus also on techniques enabling clinicians to directly control the motion of the surgical tools. In this work, we address that challenge and present a novel teleoperated robotic system able to steer flexible needles. The proposed system tracks the position of the needle using an ultrasound imaging system and computes needles ideal position and orientation to reach a given target. The master haptic interface then provides the clinician with mixed kinesthetic-vibratory navigation cues to guide the needle toward the computed ideal position and orientation. Twenty participants carried out an experiment of teleoperated needle insertion into a soft-tissue phantom, considering four different experimental conditions. Participants were provided with either mixed kinesthetic vibratory feedback or mixed kinesthetic-visual feedback. Moreover, we considered two different ways of computing ideal position and orientation of the needle: with or without set-points. Vibratory feedback was found more effective than visual feedback in conveying navigation cues, with a mean targeting error of 0.72 mm when using set-points, and of 1.10 mm without set-points.

Ultrasound-guided three-dimensional needle steering in biological tissue with curved surfaces

In this paper, we present a system capable of automatically steering a bevel-tipped flexible needle under ultrasound guidance toward a physical target while avoiding a physical obstacle embedded in gelatin phantoms and biological tissue with curved surfaces. An ultrasound pre-operative scan is performed for three-dimensional (3D) target localization and shape reconstruction. A controller based on implicit force control is developed to align the transducer with curved surfaces to assure the maximum contact area, and thus obtain an image of sufficient quality. We experimentally investigate the effect of needle insertion system parameters such as insertion speed, needle diameter and bevel angle on target motion to adjust the parameters that minimize the target motion during insertion. A fast sampling-based path planner is used to compute and periodically update a feasible path to the target that avoids obstacles. We present experimental results for target reconstruction and needle insertion procedures in gelatin-based phantoms and biological tissue. Mean targeting errors of 1.46±0.37 mm, 1.29±0.29 mm and 1.82±0.58 mm are obtained for phantoms with inclined, curved and combined (inclined and curved) surfaces, respectively, for insertion distance of 86-103 mm. The achieved targeting errors suggest that our approach is sufficient for targeting lesions of 3mm radius that can be detected using clinical ultrasound imaging systems.

Target motion predictions for pre-operative planning during needle-based interventions

During biopsies, breast tissue is subjected to displacement upon needle indentation, puncture, and penetration. Thus, accurate needle placement requires pre-operative predictions of the target motions. In this paper, we used ultrasound elastography measurements to non-invasively predict elastic properties of breast tissue phantoms. These properties were used in finite element (FE) models of indentation of breast soft tissue phantoms. To validate the model predictions of target motion, experimental measurements were carried out. Breast tissue phantoms with cubic and hemispherical geometries were manufactured and included materials with different elastic properties to represent skin, adipose tissue, and lesions. Ultrasound was used to track the displacement of the target (i.e., the simulated lesion) during indentation. The FE model predictions were compared with ultrasound measurements for cases with different boundary conditions and phantom geometry. Maximum errors between measured and predicted target motions were 12% and 3% for the fully supported and partially supported cubic phantoms at 6.0 mm indentation, respectively. Further, FE-based parameter sensitivity analysis indicated that increasing skin elastic modulus and reducing the target depth location increased the target motion. Our results indicate that with a priori knowledge about the geometry, boundary conditions, and linear elastic properties, indentation of breast tissue phantoms can be accurately predicted with FE models. FE models for pre-operative planning in combination with robotic needle insertions, could play a key role in improving lesion targeting for breast biopsies.

Control of untethered soft grippers for pick-and-place tasks

In order to handle complex tasks in hard-to-reach environments, small-scale robots have to possess suitable dexterous and untethered control capabilities. The fabrication and manipulation of soft, small-scale grippers complying to these requirements is now made possible by advances in material science and robotics. In this paper, we use soft, small-scale grippers to demonstrate pick-and-place tasks. The precise remote control is obtained by altering both the magnetic field gradient and the temperature in the workspace. This allows us to regulate the position and grasping configuration of the soft thermally-responsive hydrogel-nanoparticle composite magnetic grippers. The magnetic closed-loop control achieves precise localization with an average region-of-convergence of the gripper of 0.12±0.05 mm. The micro-sized payload can be placed with a positioning error of 0.57±0.33 mm. The soft grippers move with an average velocity of 0.72±0.13 mm/s without a micro-sized payload, and at 1.09±0.07 mm/s with a micro-sized payload.

Experimental evaluation of co-manipulated ultrasound-guided flexible needle steering

A teleoperation system for bevel‐tipped flexible needle steering has been evaluated. Robotic systems have been exploited as the main tool to achieve high accuracy and reliability. However, for reasons of safety and acceptance by the surgical community, keeping the physician tightly in the loop is preferable.

Steering of Flexible Needles Combining Kinesthetic and Vibratory Force Feedback

Needle insertion in soft-tissue is a minimally invasive surgical procedure which demands high accuracy. In this respect, robotic systems with autonomous control algorithms have been exploited as the main tool to achieve high accuracy and reliability. However, for reasons of safety and acceptance by the surgical community, autonomous robotic control is not desirable. Thus, it is necessary to focus more on techniques enabling clinicians to directly control the motion of surgical tools. In this work we address that challenge and present a novel teleoperated robotic system able to steer flexible needles. The proposed system tracks the position of the needle using an ultrasound imaging system, and, from that, it computes needles ideal position and orientation to reach a given target. The master haptic interface then provides mixed kinesthetic-vibratory navigation cues about this ideal position and orientation to the clinician as she steers the needle. Six subjects carried out an experiment of teleoperated needle insertion into a soft-tissue phantom. They showed a mean targeting error of 1.36 mm. An additional experiment of remote teleoperation has been carried out to highlight the passivity-based stability of the proposed system.

Segmentation and three-dimensional reconstruction of lesions using the automated breast volume scanner (ABVS)

Ultrasound is an effective tool for breast cancer diagnosis. However, its relatively low image quality makes small lesion analysis challenging. This promotes the development of tools to help clinicians in the diagnosis.