Jiaole Wang

Saliency Based Ulcer Detection for Wireless Capsule Endoscopy Diagnosis

Ulcer is one of the most common symptoms of many serious diseases in the human digestive tract. Especially for the ulcers in the small bowel where other procedures cannot adequately visualize, wireless capsule endoscopy (WCE) is increasingly being used in the diagnosis and clinical management. Because WCE generates large amount of images from the whole process of inspection, computer-aided detection of ulcer is considered an indispensable relief to clinicians. In this paper, a two-staged fully automated computer-aided detection system is proposed to detect ulcer from WCE images. In the first stage, we propose an effective saliency detection method based on multi-level superpixel representation to outline the ulcer candidates. To find the perceptually and semantically meaningful salient regions, we first segment the image into multi-level superpixel segmentations. Each level corresponds to different initial region sizes of the superpixels. Then we evaluate the corresponding saliency according to the color and texture features in superpixel region of each level. In the end, we fuse the saliency maps from all levels together to obtain the final saliency map. In the second stage, we apply the obtained saliency map to better encode the image features for the ulcer image recognition tasks. Because the ulcer mainly corresponds to the saliency region, we propose a saliency max-pooling method integrated with the Locality-constrained Linear Coding (LLC) method to characterize the images. Experiment results achieve promising 92.65% accuracy and 94.12% sensitivity, validating the effectiveness of the proposed method. Moreover, the comparison results show that our detection system outperforms the state-of-the-art methods on the ulcer classification task.

Simultaneous Hand–Eye, Tool–Flange, and Robot–Robot Calibration for Comanipulation by Solving the

Multirobot comanipulation shows great potential in surpassing the limitations of single-robot manipulation in complicated tasks such as robotic surgeries. However, a dynamic multirobot setup in unstructured environments poses great uncertainties in robot configurations. Therefore, the coordination relationships between the end-effectors and other devices, such as cameras (hand-eye calibration) and tools (tool-flange calibration), as well as the relationships among the base frames (robot-robot calibration) have to be determined timely to enable accurate robotic cooperation for the constantly changing configuration of the systems. We formulated the problem of hand-eye, tool-flange, and robot-robot calibration to a matrix equation AXB=YCZ. A series of generic geometric properties and lemmas were presented, leading to the derivation of the final simultaneous algorithm. In addition to the accurate iterative solution, a closed-form solution was also introduced based on quaternions to give an initial value. To show the feasibility and superiority of the simultaneous method, two nonsimultaneous methods were compared through thorough simulations under various robot movements and noise levels. Comprehensive experiments on real robots were also performed to further validate the proposed methods. The comparison results from both simulations and experiments demonstrated the superior accuracy and efficiency of the proposed simultaneous calibration method.

\mathbf{AXB=YCZ}

Tasks that are too hard for single robot can be easily carried out by multiple robots in a cooperative manner. If some/all robots have mobile bases, the cooperation is subjected to great uncertainties in both the robotic system and environment. Therefore, the relationships among all the base frames (robot-robot calibration) and the relationships between the end-effectors and the other devices such as cameras and tools (hand-eye and tool-flange calibrations) have to be calculated to enable the robots to cooperate. To address these challenges, in this paper, we propose a simultaneous hand-eye, tool-flange and robot-robot calibration method. Thorough simulations are conducted to show the superiority of the proposed simultaneous method under different noise levels and various numbers of robot movements. Furthermore, the comparison to two non-simultaneous calibration methods has also been carried out to show the efficiency and robustness of the proposed simultaneous method.

Problem

Optical means of instrument tracking technology have been widely used in image-guided surgery and regarded as the de facto standard for tracking rigid bodies under the constraint of direct line-of-sight. It remains unresolved for optical tracking to have the innate drawbacks, such as the bulky volume, the line-of-sight requirement and the occlusion constraint, etc., although they have been successful in clinical scenarios. To address these challenges in this article, we propose a surgical instrument tracking system based on dynamic configuration of multiple monocular pose estimation modules. The main approach is to enable the system to dynamically reconfigure the multiple vision sensors when occlusion occurs partially within the workspace. The corresponding multi-camera calibration algorithm and multi-camera based instrument tracking method are proposed and the evaluation experiments are carried out.

Towards simultaneous coordinate calibrations for cooperative multiple robots.

Optical means of instrument tracking has been widely used in image-guided interventions and considered the de facto standard for tracking rigid bodies with a direct line-of-sight. However, the occlusion problem which remains unresolved in current systems frustrates surgeons during the operation. To address this challenge, we propose a surgical instrument tracking system based on multiple reconfigurable monocular modules. The main approach is to enable the system to dynamically reconfigure the multiple monocular modules when occlusion occurs partially within the workspace. In this paper, we focus on the system architecture and an agile multicamera calibration method which only uses the customized tool for the surgical instrument tracking scenario. Additionally, two fast non-iterative algorithms are proposed and studied. In order to show the feasibility and superiority of the corresponding multicamera calibration algorithm, comparison experiments have carried out. The intensive investigation results give a practical instruction to the real implementation of the proposed system in image-guided interventions.

A preliminary study on surgical instrument tracking based on multiple modules of monocular pose estimation

Magnetic tracking technology is emerging to provide an occlusion-free tracking scheme for the estimation of full pose of various instruments. This brings substantial benefits for intra-corporeal applications, such as tracking of flexible robots or wireless endoscopic devices, and thus is helpful for further computer-assisted diagnosis, interventions, and surgeries. Towards efficient magnetic tracking, we propose a magnetic localization and orientation system in this paper. By modeling the cylindrical magnet, the magnetic models have been compared and analyzed. Moreover, we present a sensor layout strategy based on grid method and its optimization method for better performance. Based on the magnetic model and the layout optimization results, we have built a sensor array for magnetic positioning. Extensive simulations and experiments have shown the feasibility of the proposed system with the average positional and orientational errors of

Towards Occlusion-Free Surgical Instrument Tracking: A Modular Monocular Approach and an Agile Calibration Method

1.4 mm

Design and Optimization Strategy of Sensor Array Layout for Magnetic Localization System

and 3.4°, respectively.

Real-Time Tracking and Navigation for Magnetically Manipulated Untethered Robot

Magnetically manipulated untethered robot, such as an active wireless capsule endoscope, has shown great potential for controlled inspection inside the gastrointestinal tract. To enable the effective manipulation of the robot, real-time pose (position and orientation) information of the robot must be obtained as an important sensory feedback. Usually, a magnetic tracking method is used to provide the pose information. Due to the magnetic disturbance, a traditional magnetic tracking method cannot work simultaneously with the magnetic manipulation system. In this paper, a simultaneously tracking and navigation method is proposed to realize a closed-loop control of the magnetically manipulated untethered robot. The main approach is to conduct a multi-object tracking of the involved magnetic objects. With the proposed method, real-time pose information can be estimated during the manipulation, and both the tracking and manipulation are carried out by a magnetic manner. Therefore, the mobile navigation of the magnetically manipulated untethered robot can be achieved by using a pre-defined map. Experimental results verified the proposed method, and a mean position error of 1.2 mm for the passive magnet has been obtained.

Evolving hidden Markov model based human intention learning and inference

To effectively facilitate human robot cooperation, human intention should be recognized by robot accurately and effectively. Teaching the robot human intentions in advance could be well suitable for a static environment with limited tasks. Nevertheless, in an dynamic environment that requires task update, the pre-teaching approach cannot satisfy the evolving knowledge of human intention. The unknown human intentions which have not been taught in advance, will not be understood by robot. This problem limits the human robot cooperation in a real dynamic environment. In this paper, we proposed a human intention learning and inference method to improve the intuitive cooperative capability of the robot. An evolving hidden Markov model (EHMM) approach has been developed to learn and infer human intentions according to the observation. Assembly tasks with ten different configurations have been designed and simulation experiments were carried out. Four assembly configurations have been used for known human intention recognition experiment and six configurations have been used for unknown human intention learning and inference experiment. The accurate and robust results obtained from the experiments have shown the feasibility of the proposed EHMM for human intention learning and inference.