Rob Reilink

Evaluation of flexible endoscope steering using haptic guidance

Steering the tip of a flexible endoscope relies on the physicians dexterity and experience. For complex flexible endoscopes, conventional controls may be inadequate.

Image-based flexible endoscope steering

Manually steering the tip of a flexible endoscope to navigate through an endoluminal path relies on the physicians dexterity and experience. In this paper we present the realization of a robotic flexible endoscope steering system that uses the endoscopic images to control the tip orientation towards the direction of the lumen. Two image-based control algorithms are investigated, one is based on the optical flow and the other is based on the image intensity. Both are evaluated using simulations in which the endoscope was steered through the lumen. The RMS distance to the lumen center was less than 25% of the lumen width. An experimental setup was built using a standard flexible endoscope, and the image-based control algorithms were used to actuate the wheels of the endoscope for tip steering. Experiments were conducted in an anatomical model to simulate gastroscopy. The image intensity-based algorithm was capable of steering the endoscope tip through an endoluminal path from the mouth to the duodenum accurately. Compared to manual control, the robotically steered endoscope performed 68% better in terms of keeping the lumen centered in the image.

A feasibility study of optical flow-based navigation during colonoscopy

Introduction In endoscopic surgical procedure, surgeons need to manipulate endoscopes and surgical instruments inside of the body cavity in order to observe a large field-of-view and estimate the distance between the surgical instruments and anatomical area of interest. This is a difficult task while watching 2-D endoscopic images on a monitor. So, there is a risk of the surgical instruments or the endoscope physically damaging the internal organs. Wasol Co., Ltd has developed a 3-D endoscope that has a single optical channel and CCD camera. This 3-D endoscope can acquire left and right endoscopic images at different viewpoints by using a rapidly-rotated transparent plate in front of the CCD camera. However, the diameter of the single optical channel must be large so that the endoscope can acquire endoscopic images of a sufficient quality for stereovision. Shinko Optical Co., Ltd. has developed a 3-D endoscope that has two 1/10-inch CCD cameras at the distal end of its sleeve. This 3-D endoscope enables surgeons to easily estimate the distance between the instruments and anatomical area of interest. Although this method is a simple solution for stereovision, its diameter is difficult to reduce because two CCD cameras are placed in parallel at its distal end. We have previously developed a u5-mm 3-D endoscope with a single CCD camera and vibration mechanism using a cam mechanism. This endoscope can acquire left and right endoscopic images for stereoscopic vision in a way which is synchronized with the periodical motion generated by the vibration mechanism. The image quality is high enough for stereoscopic vision and more than 65 % of volunteers can perceive depth correctly, however the cam-vibration mechanism is large and unsterilizable; and for this reason, this endoscope cannot be used for real surgery. To overcome these problems, we have developed a high-imagequality and thin 3-D endoscope with a single CCD camera and sterilizable pneumatic vibration mechanism. Methods The developed 3-D endoscope system consists of a 1/10-inch CCD camera (SONY), an outer sleeve (7 mm in outer diameter, 100 mm in length, stainless), an inner sleeve (5 mm in outer diameter, 200 mm in length, stainless), a pneumatic vibration mechanism consisting of a case with two ports (Fluororesin) and a slider with o-rings (light curing resin), an air compressor (TFP04B-10C, ANEST IWATA Corporation), an air dryer (RDG-22C, ANEST IWATA Corporation), an auxiliary air tank (SAT-33H-100, ANEST IWATA Corporation), two air solenoid valves (MHA2-MS1H-3/2G-2-K, Festo Corporation), a digital output board (PCI-2426C, Interface Corporation), two image capture boards (IP7000BD, Hitachi Information & Control Solutions Co., Ltd.) and a monitor for stereovision (LCD-SK3, Shinko Optical Co., Ltd.) (Fig. 1a). The 1/10-inch CCD camera is attached to the distal end of the sleeve and has an 80 horizontal field-of-view and a 66 vertical field-of-view. The pneumatic vibration mechanism is attached to the proximal end of the sleeve. A fulcrum point is set at middle of the inner sleeve as shown in Fig. 1a. The distance between the fulcrum point and the 1/10-inch CCD camera is 100 mm. The pneumatic vibration mechanism, which is made of sterilizable plastic, generates a triangular wave as follows: When compressed air is pumped into Port A, the slider moves to the right. In the inverse condition, when compressed air is pumped into Port B, the slider moves to the left (Fig. 1b). The sequence is repeated for the proximal end of the sleeve, which is attached to the slider, to vibrate and thus the vibration is transmitted to its distal end with the 1/10-inch CCD camera. The peak-to-peak amplitude and frequency of the vibration are 0.5 mm and 10 Hz respectively (Fig. 2a). The 1/10-inch CCD camera alternately acquires left and right images when its displacement is at maximum or minimum of the amplitude (at this point, the velocity of the 1/10-inch CCD camera is Fig. 1 a System configuration. b Pneumatic vibration mechanism Left Compressed air is pumped into Port A, Right Compressed air is pumped into PortB

Design of a user interface for intuitive colonoscope control

The goal of this study is to improve the efficiency and efficacy of the standard colonoscopy procedure. This is done by addressing the intuitiveness of colonoscope control. For this purpose an interface in the form of a grip was designed that allows the user to intuitively steer and drive the colonoscope. The Grip controls the orientation of the tip as if the colonoscope were a stiff instrument that pivots at the anus of a patient. To test the principle, experiments were conducted on a simulator operated by novice subjects. Initial experiments show a significant decrease in introduction time of 156 seconds (p<0.005). This technology will enhance current colonoscopy practice and open up possibilities for future applications of colonoscopy.

Mechatronic design of the Twente humanoid head

This paper describes the mechatronic design of the Twente humanoid head, which has been realized in the purpose of having a research platform for human-machine interaction. The design features a fast, four degree of freedom neck, with long range of motion, and a vision system with three degrees of freedom, mimicking the eyes. To achieve fast target tracking, two degrees of freedom in the neck are combined in a differential drive, resulting in a low moving mass and the possibility to use powerful actuators. The performance of the neck has been optimized by minimizing backlash in the mechanisms, and using gravity compensation. The vision system is based on a saliency algorithm that uses the camera images to determine where the humanoid head should look at, i.e. the focus of attention computed according to biological studies. The motion control algorithm receives, as input, the output of the vision algorithm and controls the humanoid head to focus on and follow the target point. The control architecture exploits the redundancy of the system to show human-like motions while looking at a target. The head has a translucent plastic cover, onto which an internal LED system projects the mouth and the eyebrows, realizing human-like facial expressions.

Endoscopic camera control by head movements for thoracic surgery

In current video-assisted thoracic surgery, the endoscopic camera is operated by an assistant of the surgeon, which has several disadvantages. This paper describes a system which enables the surgeon to control the endoscopic camera without the help of an assistant. The system is controlled using head movements, so the surgeon can use his/her hands to operate the instruments. The system is based on a flexible endoscope, which leaves more space for the surgeon to operate his/her instruments compared to a rigid endoscope. The endoscopic image is shown either on a monitor or by means of a head-mounted display. Several trial sessions were performed with an anatomical model. Results indicate that the developed concept may provide a solution to some of the problems currently encountered in video-assisted thoracic surgery. The use of a head-mounted display turned out to be a valuable addition since it ensures the image is always in front of the surgeons eyes.

Pose reconstruction of flexible instruments from endoscopic images using markers

A system is developed that can reconstruct the pose of flexible endoscopic instruments that are used in advanced flexible endoscopes using solely the endoscopic images. Four markers are placed on the instrument, whose positions are measured in the image. These measurements are compared to a three-dimensional rendered model of the instrument. The pseudo-inverse of the interaction matrix between the state of the model and the marker positions in the image is used to update the state such that the model will track the real instrument. An experiment was performed in which the instrument was moved inside a colon model, while the tip position was simultaneously measured with an electromagnetic tracking system. The root mean square errors of the position estimation were 2.3 mm, 2.2 mm and 1.7 mm in the horizontal (x), vertical (y) and away-from-camera (z) directions, respectively.

Three-dimensional pose reconstruction of flexible instruments from endoscopic images

A position and orientation sensing system is developed for the feedback control of endoscopic instruments in advanced flexible endoscopes. The images that are taken by the endoscopic camera are used to match a kinematic model to the observed instrument. Using the pseudo-inverse of the Jacobian of the forward kinematics, the estimated state of the model is continuously updated so as to match feature points from the images to the model. An experiment was performed inside a colon model, in which reference markers with known locations were touched with the instrument. The root mean square position estimation errors were 1.7 mm, 1.2 mm and 3.6 mm in the horizontal (x), vertical (y), and away-from-camera (z) directions, respectively.

Evaluation of robotically controlled advanced endoscopic instruments

Advanced flexible endoscopes and instruments with multiple degrees of freedom enable physicians to perform challenging procedures such as the removal of large sections of mucosal tissue. However, these advanced endoscopes are difficult to control and require several physicians to cooperate.

The Twente humanoid head

This video shows the results of the project on the mechatronic development of the Twente humanoid head. The mechanical structure consists of a neck with four degrees of freedom (DOFs) and two eyes (a stereo pair system) which tilt on a common axis and rotate sideways freely providing a three more DOFs. The motion control algorithm is designed to receive, as an input, the output of a biological-inspired vision processing algorithm and to exploit the redundancy of the joints for the realization of the movements. The expressions of the humanoid head are implemented by projecting light from the internal part of the translucent plastic cover.