Caspar Gruijthuijsen

Real-Time Segmentation of Non-rigid Surgical Tools Based on Deep Learning and Tracking

Real-time tool segmentation is an essential component in computer-assisted surgical systems. We propose a novel real-time automatic method based on Fully Convolutional Networks (FCN) and optical flow tracking. Our method exploits the ability of deep neural networks to produce accurate segmentations of highly deformable parts along with the high speed of optical flow. Furthermore, the pre-trained FCN can be fine-tuned on a small amount of medical images without the need to hand-craft features. We validated our method using existing and new benchmark datasets, covering both ex vivo and in vivo real clinical cases where different surgical instruments are employed. Two versions of the method are presented, non-real-time and real-time. The former, using only deep learning, achieves a balanced accuracy of 89.6% on a real clinical dataset, outperforming the (non-real-time) state of the art by 3.8% points. The latter, a combination of deep learning with optical flow tracking, yields an average balanced accuracy of 78.2% across all the validated datasets.

Intuitive teleoperation of active catheters for endovascular surgery

Advances in miniature surgical instrumentation are key to less invasive and safer medical interventions. In cardiovascular procedures interventionalists turn towards catheter-based interventions, treating patients considered unfit for classical more invasive approaches. Improvements in design and steerability of catheters could further reduce the invasiveness of these interventions. For example, by improving controllability and interaction forces with the vessels, tissue damage could be limited. Through improved steerability and coordinated control, operation times and exposure to radiation might also be reduced. Latter argument formed the original motivation for the development of teleoperated robotic catheters. Despite the large kinematic dissimilarity and thus non-trivial mapping between joystick input and catheter output motion, few investigations have been conducted to find intuitive mappings that allow straightforward catheter steering. This paper presents some recent work in this direction. Three promising mappings are proposed. The mappings were implemented and validated upon a robotic catheter moving inside an artificial aorta model. Experimental results show good steerability of the robotic catheter for all the mappings. Although superiority of one mapping with respect to the others was observed, further investigation and validation is planned. In the future, additional visual cues that increase the situational awareness of the user are expected to further simplify the steering.

Body wall force sensor for simulated minimally invasive surgery: Application to fetal surgery

Surgical interventions are increasingly executed minimal invasively. Surgeons insert instruments through tiny incisions in the body and pivot slender instruments to treat organs or tissue below the surface. While a blessing for patients, surgeons need to pay extra attention to overcome the fulcrum effect, reduced haptic feedback and deal with lost hand-eye coordination. The mental load makes it difficult to pay sufficient attention to the forces that are exerted on the body wall. In delicate procedures such as fetal surgery, this might be problematic as irreparable damage could cause premature delivery. As a first attempt to quantify the interaction forces applied on the patients body wall, a novel 6 degrees of freedom force sensor was developed for an ex-vivo set up. The performance of the sensor was characterised. User experiments were conducted by 3 clinicians on a set up simulating a fetal surgical intervention. During these simulated interventions, the interaction forces were recorded and analysed when a normal instrument was employed. These results were compared with a session where a flexible instrument under haptic guidance was used. The conducted experiments resulted in interesting insights in the interaction forces and stresses that develop during such difficult surgical intervention. The results also implicated that haptic guidance schemes and the use of flexible instruments rather than rigid ones could have a significant impact on the stresses that occur at the body wall.

Position control of robotic catheters inside the vasculature based on a predictive minimum energy model

Accurate and precise control of catheters inside a vasculature is a difficult yet important task. Current manual approaches require significant surgical skill. Over the years, surgeons build up a sort of mental kinematic map telling them how to handle the catheter in order to steer the catheter tip safely through the vessel system. The input-output behaviour of the catheter is complex and depends heavily on its configuration within and contacts with the vasculature. This paper introduces an alternative approach to control robotic catheters. The input-output behaviour or so-called differential kinematics are derived from a patient-specific vasculature model, following a minimum-energy argumentation. The validity of the proposed approach is demonstrated experimentally. Whereas the performance of model-based approaches is obviously greatly influenced by the correctness of estimated parameters, within this work we show experimentally how reasonable performance can already be achieved within the setup that was constructed. We expect that there is still ample room for improvement by, e.g., putting more sophisticated identification, modeling and collision detection schemes into place.

An Automatic Registration Method for Radiation-Free Catheter Navigation Guidance

Catheter navigation is typically based on fluoroscopy. This implies exposure to harmful radiation, lack of depth perception and limited soft-tissue contrast. Catheter navigation would benefit from guidance that makes better use of detailed pre-operatively acquired MR/CT images, while reducing radiation exposure and improving spatial awareness of the catheter pose and shape. A prerequisite for such guidance is an accurate registration between the catheter tracking system and the MR/CT scans. Existing registration methods are lengthy and cumbersome as they require a lot of user interaction. This forms a major obstacle for their adoption into clinical practice. This paper proposes a radiation-free registration method that minimizes the impact on the surgical workflow and avoids most user interaction. The method relies on catheters with embedded sensors that provide intra-operative data that can either belong to the vessel wall or to the lumen of the vessel. Based on the acquired surface and lumen points an accurate registration is computed automatically, with minimal user interaction. Validation of the proposed method is performed on a synthetic yet realistic aorta phantom. Input from electromagnetic tracking, force sensing, and intra-vascular ultrasound are used as intra-operative sensory data.

A mixed-reality surgical trainer with comprehensive sensing for fetal laser minimally invasive surgery

PurposeSmaller incisions and reduced surgical trauma made minimally invasive surgery (MIS) grow in popularity even though long training is required to master the instrument manipulation constraints. While numerous training systems have been developed in the past, very few of them tackled fetal surgery and more specifically the treatment of twin-twin transfusion syndrome (TTTS). To address this lack of training resources, this paper presents a novel mixed-reality surgical trainer equipped with comprehensive sensing for TTTS procedures. The proposed trainer combines the benefits of box trainer technology and virtual reality systems. Face and content validation studies are presented and a use-case highlights the benefits of having embedded sensors.MethodsFace and content validity of the developed setup was assessed by asking surgeons from the field of fetal MIS to accomplish specific tasks on the trainer. A small use-case investigates whether the trainer sensors are able to distinguish between an easy and difficult scenario.ResultsThe trainer was deemed sufficiently realistic and its proposed tasks relevant for practicing the required motor skills. The use-case demonstrated that the motion and force sensing capabilities of the trainer were able to analyze surgical skill.ConclusionThe developed trainer for fetal laser surgery was validated by surgeons from a specialized center in fetal medicine. Further similar investigations in other centers are of interest, as well as quality improvements which will allow to increase the difficulty of the trainer. The comprehensive sensing appeared to be capable of objectively assessing skill.

Cognitive Autonomous Catheters Operating in Dynamic Environments

Advances in miniaturized surgical instrumentation are key to less demanding and safer medical interventions. In cardiovascular procedures interventionalists turn towards catheter-based interventions, treating patients considered unfit for more invasive approaches. A positive outcome is not guaranteed. The risk for calcium dislodgement, tissue damage or even vessel rupture cannot be eliminated when instruments are maneuvered through fragile and diseased vessels. This paper reports on the progress made in terms of catheter design, vessel reconstruction, catheter shape modeling, surgical skill analysis, decision making and control. These efforts are geared towards the development of the necessary technology to autonomously steer catheters through the vasculature, a target of the EU-funded project Cognitive AutonomouS CAtheters operating in Dynamic Environments (CASCADE). Whereas autonomous placement of an aortic valve implant forms the ultimate and concrete goal, the technology of individual building blocks to reach such ambitious goal is expected to be much sooner impacting and assisting interventionalists in their daily clinical practice.