Oliver Weede

Proactive Robot Task Selection Given a Human Intention Estimate

Intuitive human-robot cooperation presents a challenge to robots since it demands of them a high level of understanding of the human user. Our approach is to estimate the human intention and select and proactively execute an appropriate robot task without requiring an explicit user command. This paper describes the concept and details of our proactive execution module and how it forms an integral part of our intuitive human-robot cooperation system. We present implementation details and the results of an evaluation scenario.

An intelligent and autonomous endoscopic guidance system for minimally invasive surgery

The endoscopic guidance system for minimally invasive surgery presented here autonomously aligns the laparoscopic camera with the end-effectors of the surgeons instruments. It collects information on the movements of the instruments from former interventions and can therefore predict them for autonomous guidance of the endoscopic camera. Knowledge is extracted by trajectory clustering, maximum likelihood classification and a Markov model to predict states. Alternative movements in an ongoing intervention are modeled. A first prototype of a robotic platform for minimally invasive surgery is described, which has two instrument arms, an autonomous robotic camera assistant and two haptic devices to control the instrument arms. The approach of long-term prediction and optimal camera positioning was tested in a phantom experiment with a hit rate of over 89% for predicting the movement of the end-effectors. Including this prediction for computing the camera position, leads to 29.2% less movements and to an improved visibility of the instruments.

Simulation tool for 3D shape sensors based on Fiber Bragg gratings and optimization of measurement points

3D shape sensing using Fiber Bragg gratings has attracted the interest of several research groups in recent years, but so far no possibility has been presented to optimize the sensing system by simulation. Almost always, the gratings (the points of strain measurement) are distributed equidistantly, and the reconstruction algorithm has not been verified by simulation. In this paper we present our simulation tool that works on a mathematical basis and includes two parts: the first part describes how to determine the strains of Fiber Bragg gratings mounted along a given shape. The second part describes our algorithm to reconstruct the shape from this strain data using theory of differential geometry. This reconstruction algorithm is evaluated within the simulation environment and can be adapted to different system behaviors like circular bending of the instrument, cubic bending or bending according to the Euler beam theory. Furthermore, the gratings can be optimized with respect to their position and designated Bragg wavelength. To demonstrate the effectiveness of the simulation tool, an optimization has been conducted for the grating positions along a shape with two independent bendable segments. For each configuration of the shape, the reconstruction results using the optimized grating positions were significantly better than when using equidistantly distributed gratings.

Towards Cognitive Medical Robotics in Minimal Invasive Surgery

Up to date, medical robots for minimal invasive surgery do not provide assistance appropriate to the workflow of the intervention. A simple concept of a cognitive system is presented, which is derived from a classic closed-loop control. As implementation, we present a cognitive medical robot system using lightweight robots with redundant kinematics. The robot system includes several control modes and human-machine interfaces. We focus on describing knowledge acquisition about the workflow of an intervention and present two example applications utilizing the acquired knowledge: autonomous camera guidance and planning of minimal invasive port (trocar) positions in combination with an initial robot setup. Port planning is described as optimization problem. The autonomous camera system includes a mid-term movement prediction of the ongoing intervention. The cognitive approach to a medical robot system includes taking the environment into account. The goal is to create a system that acts as a human assistant, who perceives the situation, understands the context based on his knowledge and acts appropriate.

Workflow analysis and surgical phase recognition in minimally invasive surgery

In this paper, a new approach is described to recognize the phases of a single-port sigma resection intraoperatively, based on the position signal of the surgical instruments, the endoscopic video and an audio signal, signaling coagulations. Approaches for detecting the coagulation sounds, as well as the instruments visible in the endoscopic video using a bag of words model are detailed. The intervention phases are regarded as classes of a naive Bayes classifier. Features that differentiate intervention phases are examined. The naive Bayes classifier is extended by a dynamic feature, which includes the order of the intervention phases and their duration. First results show that in 93.2% the recognized phases are classified as true positive.

Combining shape sensor and haptic sensors for highly flexible single port system using Fiber Bragg sensor technology

Single port surgery is a promising approach to further optimize the benefits of robot assisted minimally invasive surgery. Advantages include further minimization of trauma and reduction of scars, but it also increases the need for flexible, high-end instruments and miniaturized sensors. This paper describes a new concept of a miniaturized sensor system which combines a kinesthetic sensor, a tactile sensor and a position sensor. Latter is realized as a shape sensor. The combination of the sensors is possible because they are all based on Fiber Bragg Grating technology. A concept for each sensor is also presented. They are improved compared to current research and adapted to our needs. The miniaturized sensor system is integrated into an innovative, highly flexible single port system.

A cognitive path-guidance-system for minimally invasive surgery

The presented path-guidance system is able to learn movements and to predict motion. It shall enhance safe navigation for surgeons in minimally invasive surgery by creating a virtual fixture which holds the end-effectors motion to a desired path and warning the surgeon in a dangerous situation. Surgeons can demonstrate interventions and best practices. The system collects information from surgeon demonstrated trajectories, defined as best practices, and extracts knowledge to provide guidance for other users to carry out the same intervention. Knowledge extraction is achieved through trajectory clustering, maximum likelihood classification and a Markov model to predict states. The fundamental task is to guide a surgeon along a desired trajectory (navigated path) and prevent them entering into zones of risk. The path is not sequential, furcations are permitted and modeled showing alternatives in the ongoing intervention. An evaluation with a pelvitrainer showed good results with over 89% hit rate in predicting the motion.

The “Iceberg Phenomenon” As Soon as One Technological Problem in NOTES Is Solved, the Next One Appears!

Purpose. Though already proclaimed about 7 years ago, natural orifice transluminal endoscopic surgery (NOTES) is still in its early stages. A multidisciplinary working team tried to analyze the technical obstacles and identify potential solutions. Methods. After a comprehensive review of the literature, a group of 3 surgeons, 1 gastroenterologist, 10 engineers, and 1 representative of biomedical industry defined the most important deficiencies within the system and then compiled as well as evaluated innovative technologies that could be used to help overcome these problems. These technologies were classified with regard to the time needed for their implementation and associated hindrances, where priority is based on the level of impact and significance that it would make. Results. Both visualization and actuation require significant improvement. Advanced illumination, mist elimination, image stabilization, view extension, 3-dimensional stereoscopy, and augmented reality are feasible options and could optimize visual information. Advanced mechatronic platforms with miniaturized, powerful actuators, and intuitive human–machine interfaces could optimize dexterity, as long as enabling technologies are used. The latter include depth maps in real time, precise navigation, fast pattern recognition, partial autonomy, and cognition systems. Conclusion. The majority of functional deficiencies that still exist in NOTES platforms could be overcome by a broad range of already existing or emerging enabling technologies. To combine them in an optimal manner, a permanent dialogue between researchers and clinicians is mandatory.

Knowledge-based planning of port positions for minimally invasive surgery

The success of a minimally invasive intervention strongly depends on the position of the incision in the abdominal wall for inserting the instruments and the endoscopic camera. A new knowledge-based system for planning these ports is presented. First time, the positioning of the surgeon and the camera assistant in respect to the patient is included. Modeling of the patient, the intervention and the optimization criteria are described. The intervention model is inferred from recorded trajectories of the surgical instruments. As an example application rectal resection is chosen. The optimization is performed with the Seed Throwing Optimization meta-heuristic. A new set of criteria for optimal setup configuration is presented, including ergonomic working directions, collision avoidance and reachability of target areas. The combination of these criteria using a t-norm is described. The optimized port positions are projected on the patients abdomen with an augmented reality system. The time needed for the planning algorithm was measured and is sufficient. The results show that the computed configuration leads to maximum access in the operation field in the narrow pelvis, maximum separation dexterity for the surgeon and the camera assistant and allows an ergonomic completion of the intervention.

Knowledge-Based System for Port Placement and Robot Setup Optimization in Minimally Invasive Surgery

Abstract A new knowledge-based planning system is presented, that optimizes the positions of ports for the instruments and the endoscopic camera in manual and robot assisted laparoscopic surgery. For robot assisted interventions, the initial robot configuration of the da Vinci ® Surgical System is included in the optimization. Modeling of the patient, the intervention and the telemanipulator are described. The patient model is including the pneumoperitoneum. The intervention model is inferred from recorded trajectories of the surgical instruments. The optimization is performed with the Seed Throwing Optimization meta-heuristic. Criteria for optimization are reachability of target areas, collision avoidance and dexterity as well as ergonomic factors. These criteria are modeled as fuzzy sets. The optimized port positions are projected on the patients abdomen with a new laser-based projection system. The error of the laser-based projection was measured. The advantages and disadvantages of the system are discussed and compared to an existing augmented reality system. First results show that the computed configuration leads to maximum access in the operation field in the narrow pelvis, maximum separation of the telemanipulators arms and allows an ergonomic completion of the intervention.