Omid Mohareri

Indirect adaptive tracking control of a nonholonomic mobile robot via neural networks

This paper presents the design and implementation of a novel adaptive trajectory tracking controller for a nonholonomic wheeled mobile robot (WMR) with unknown parameters and uncertain dynamics. The learning ability of neural networks is used to design a robust adaptive backstepping controller that does not require the knowledge of the robot dynamics. The kinematic controller gains are tuned on-line to minimize the velocity error and improve the trajectory tracking characteristics. The performance of the proposed control algorithm is verified and compared with the classical backstepping controller through simulation and experiments on a commercial mobile robot platform.

Automatic Localization of the da Vinci Surgical Instrument Tips in 3-D Transrectal Ultrasound

Robot-assisted laparoscopic radical prostatectomy (RALRP) using the da Vinci surgical system is the current state-of-the-art treatment option for clinically confined prostate cancer. Given the limited field of view of the surgical site in RALRP, several groups have proposed the integration of transrectal ultrasound (TRUS) imaging in the surgical workflow to assist with accurate resection of the prostate and the sparing of the neurovascular bundles (NVBs). We previously introduced a robotic TRUS manipulator and a method for automatically tracking da Vinci surgical instruments with the TRUS imaging plane, in order to facilitate the integration of intraoperative TRUS in RALRP. Rapid and automatic registration of the kinematic frames of the da Vinci surgical system and the robotic TRUS probe manipulator is a critical component of the instrument tracking system. In this paper, we propose a fully automatic registration technique based on automatic 3-D TRUS localization of robot instrument tips pressed against the air-tissue boundary anterior to the prostate. The detection approach uses a multiscale filtering technique to identify and localize surgical instrument tips in the TRUS volume, and could also be used to detect other surface fiducials in 3-D ultrasound. Experiments have been performed using a tissue phantom and two ex vivo tissue samples to show the feasibility of the proposed methods. Also, an initial in vivo evaluation of the system has been carried out on a live anaesthetized dog with a da Vinci Si surgical system and a target registration error (defined as the root mean square distance of corresponding points after registration) of 2.68 mm has been achieved. Results show this methods accuracy and consistency for automatic registration of TRUS images to the da Vinci surgical system.

Intraoperative Registered Transrectal Ultrasound Guidance for Robot-Assisted Laparoscopic Radical Prostatectomy

PURPOSE To provide unencumbered real-time ultrasound image guidance during robot-assisted laparoscopic radical prostatectomy, we developed a robotic transrectal ultrasound system that tracks the da Vinci® Surgical System instruments. We describe our initial clinical experience with this system. MATERIALS AND METHODS After an evaluation in a canine model, 20 patients were enrolled in the study. During each procedure the transrectal ultrasound transducer was manually positioned using a brachytherapy stabilizer to provide good imaging of the prostate. Then the transrectal ultrasound was registered to the da Vinci robot by a previously validated procedure. Finally, automatic rotation of the transrectal ultrasound was enabled such that the transrectal ultrasound imaging plane safely tracked the tip of the da Vinci instrument controlled by the surgeon, while real-time transrectal ultrasound images were relayed to the surgeon at the da Vinci console. Tracking was activated during all critical stages of the surgery. RESULTS The transrectal ultrasound robot was easy to set up and use, adding 7 minutes (range 5 to 14) to the procedure. It did not require an assistant or additional control devices. Qualitative feedback was acquired from the surgeons, who found transrectal ultrasound useful in identifying the urethra while passing the dorsal venous complex suture, defining the prostate-bladder interface during bladder neck dissection, identifying the seminal vesicles and their location with respect to the rectal wall, and identifying the distal prostate boundary at the apex. CONCLUSIONS Real-time, registered robotic transrectal ultrasound guidance with automatic instrument tracking during robot-assisted laparoscopic radical prostatectomy is feasible and potentially useful. The results justify further studies to establish whether the approach can improve procedure outcomes.

Direct vibro-elastography FEM inversion in Cartesian and cylindrical coordinate systems without the local homogeneity assumption.

To produce images of tissue elasticity, the vibro-elastography technique involves applying a steady-state multi-frequency vibration to tissue, estimating displacements from ultrasound echo data, and using the estimated displacements in an inverse elasticity problem with the shear modulus spatial distribution as the unknown. In order to fully solve the inverse problem, all three displacement components are required. However, using ultrasound, the axial component of the displacement is measured much more accurately than the other directions. Therefore, simplifying assumptions must be used in this case. Usually, the equations of motion are transformed into a Helmholtz equation by assuming tissue incompressibility and local homogeneity. The local homogeneity assumption causes significant imaging artifacts in areas of varying elasticity. In this paper, we remove the local homogeneity assumption. In particular we introduce a new finite element based direct inversion technique in which only the coupling terms in the equation of motion are ignored, so it can be used with only one component of the displacement. Both Cartesian and cylindrical coordinate systems are considered. The use of multi-frequency excitation also allows us to obtain multiple measurements and reduce artifacts in areas where the displacement of one frequency is close to zero. The proposed method was tested in simulations and experiments against a conventional approach in which the local homogeneity is used. The results show significant improvements in elasticity imaging with the new method compared to previous methods that assumes local homogeneity. For example in simulations, the contrast to noise ratio (CNR) for the region with spherical inclusion increases from an average value of 1.5-17 after using the proposed method instead of the local inversion with homogeneity assumption, and similarly in the prostate phantom experiment, the CNR improved from an average value of 1.6 to about 20.

Asymmetric force feedback control framework for teleoperated robot-assisted surgery

Lack of haptic feedback in teleoperated robot-assisted surgery (RAS) is known to be detrimental in many surgical tasks. While performing a class of force sensitive tasks, surgeons commonly use both hands. Oftentimes, one hand is used to exert tension/compression forces, and the other to hold the suture knot or tissue. A novel control framework to accomplish haptic force feedback for two-handed tasks in teleoperated RAS is presented in this paper. The force applied on the surgical environment by the action hand is not transferred back to the same hand, but rather to the other hand. In two-handed tasks that involve an action and a reaction force, this provides an intuitive way of feeling the action. Because the loop in not closed from the slave back to the master of the same hand, it does not have a destabilizing effect. The technique can be easily implemented using a variable-structure controller that combines two PD controllers and a switch. It has been evaluated with an experimental setup consisting of four haptic devices with promising results.

A VISION-BASED LOCATION POSITIONING SYSTEM VIA AUGMENTED REALITY: AN APPLICATION IN HUMANOID ROBOT NAVIGATION

In this paper, we present a vision-based localization system using mobile augmented reality (MAR) and mobile audio augmented reality (MAAR) techniques, applicable to both humans and humanoid robots navigation in indoor environments. In the first stage, we propose a system that recognizes the location of a user from the image sequence of an indoor environment using its onboard camera. The location information is added to the users view in the form of 3D objects and audio sounds with location information and navigation instruction content via augmented reality (AR). The location is recognized by using the prior knowledge about the layout of the environment and the location of the AR markers. The image sequence can be obtained using a smart phones camera and the marker detection, 3D object placement and audio augmentation will be performed by the phones operating processor and graphical/audio modules. Using this system will majorly reduce the hardware complexity of such navigation systems, as it replaces a system consisting of a mobile PC, wireless camera, head-mounted displays (HMD) and a remote PC with a smart phone with camera. In the second stage, the same algorithm is employed as a novel vision-based autonomous humanoid robot localization and navigation approach. The proposed technique is implemented on a humanoid robot NAO and improves the robots navigation and localization performance previously done using an extended Kalman filter (EKF) by presenting location-based information to the robot through different AR markers placed in the robot environment.

Multi-parametric 3D quantitative ultrasound vibro-elastography imaging for detecting palpable prostate tumors.

In this article, we describe a system for detecting dominant prostate tumors, based on a combination of features extracted from a novel multi-parametric quantitative ultrasound elastography technique. The performance of the system was validated on a data-set acquired from n = 10 patients undergoing radical prostatectomy. Multi-frequency steady-state mechanical excitations were applied to each patients prostate through the perineum and prostate tissue displacements were captured by a transrectal ultrasound system. 3D volumetric data including absolute value of tissue elasticity, strain and frequency-response were computed for each patient. Based on the combination of all extracted features, a random forest classification algorithm was used to separate cancerous regions from normal tissue, and to compute a measure of cancer probability. Registered whole mount histopathology images of the excised prostate gland were used as a ground truth of cancer distribution for classifier training. An area under receiver operating characteristic curve of 0.82 +/- 0.01 was achieved in a leave-one-patient-out cross validation. Our results show the potential of multi-parametric quantitative elastography for prostate cancer detection for the first time in a clinical setting, and justify further studies to establish whether the approach can have clinical use.

Automatic detection and localization of da vinci tool tips in 3d ultrasound

Radical prostatectomy (RP) is viewed by many as the gold standard treatment for clinically localized prostate cancer. State of the art radical prostatectomy involves the da Vinci surgical system, a laparoscopic robot which provides the surgeon with excellent 3D visualization of the surgical site and improved dexterity over standard laparoscopic instruments. Given the limited field of view of the surgical site in Robot-Assisted Laparoscopic Radical Prostatectomy (RALRP), several groups have proposed the integration of Transrectal Ultrasound (TRUS) imaging in the surgical work flow to assist with the resection of prostate and sparing the Neuro-Vascular Bundle (NVB). Rapid and automatic registration of TRUS imaging coordinates to the da Vinci tools or camera is a critical component of this integration. We propose a fully automatic registration technique based on accurate and automatic localization of robot tool tips pressed against the air-tissue boundary of the prostate, in 3D TRUS. The detection approach uses a multi-scale filtering technique to uniquely identify and localize the tool tip in the ultrasound volume and could also be used to detect other surface fiducials in 3D ultrasound. Feasibility experiments using a phantom and two ex vivo tissue samples yield promising results with target registration error (defined as the root mean square distance of corresponding points after registration) of (

Bimanual telerobotic surgery with asymmetric force feedback: A daVinci ® surgical system implementation

1.80\ mm

A retrofit eye gaze tracker for the da Vinci and its integration in task execution using the da Vinci Research Kit

) that proves the systems accuracy for registering 3D TRUS to the da Vinci surgical system.