Ying Mao

An integrated vision-based robotic manipulation system for sorting surgical tools

In this paper, we introduced a robotic system using a humanoid robot, Baxter research robot, to pick-up surgical tools from a tray and place the tools into different trays according to the types of the surgical tools. The pick-n-place manipulation is integrated with a vision component and a special magnet gripper and governed by a finite state machine. This vision-based manipulation system allows the robot to check which tool is on top of the tools in a tray, to find the grasping points on the tools, to grab the tools using a magnet gripper, and to place them into different trays. Major technologies used in this system include: vision, magnet force control, force feedback, motion trajectory planning, and decision-making. We tested our system in a lab-based environment and the system performance satisfies the requirements of the project.

Robotic Handling of Surgical Instruments in a Cluttered Tray

We developed a unique robotic manipulation system that accurately singulates surgical instruments in a cluttered environment. A novel single-view computer vision algorithm identifies the next instrument to grip from a cluttered pile and a compliant electromagnetic gripper picks up the identified instrument. System is validated through extensive experiments. This research was motivated by the challenges of perioperative process in hospitals today. Current process of instrument counting, sorting, and sterilization is highly labor intensive. Improperly sterilized instruments have resulted in many cases of infections. To address these challenges, an integrated robotic system for sorting instruments in a cluttered tray is designed and implemented. A digital camera is used to capture an image of a cluttered tray. A novel single-view vision algorithm is used to detect the instruments and determine the top instrument. Position and orientation of the top instrument is transferred to a robot. A compliant electromagnetic gripper is developed to complete the gripping. Experiments have demonstrated high success rate of both instrument recognition and manipulation. In the future, error handling needs to be further reinforced under various exceptions for better robustness.

A vision-guided robot manipulator for surgical instrument singulation in a cluttered environment

The logistics of counting, sorting, sterilizing, and transporting surgical instruments is labor and capital intensive. Furthermore, infection due to improper sterilization is a critical safety hazard. To address these problems, we have developed a unique robotic manipulation system that is capable of accurately singulating surgical instruments in a cluttered environment. Our solution is comprised of two parts. First, we use a single-view vision algorithm for identifying surgical instruments from a pile and estimating their poses. Occlusion reasoning is performed to determine the next instrument to grip using a contrast invariant feature descriptor. Second, we design a compliant electromagnetic gripper that is capable of picking up the identified surgical instrument based on its estimated pose. We validate our solution through instrument singulation experiments demonstrating identification, localization accuracy, and robustness of occlusion reasoning as well as the flexibility of the electromagnetic gripper.

An integrated robotic system for transporting surgical tools in hospitals

The performance of a hospitals sterile processing center (SPC) significantly impacts patient safety and overall productivity. Key to automating this process is to reliably transport instruments throughout the process. In this paper, we detail a robust integrated system for enabling mobile robots to autonomously perform manipulation of assets; specifically, transporting reusable surgical instrument trays in the SPC of a hospital. Our method is based on a cognitive decision making mechanism that plans and coordinates the motions of the robot base and the robot manipulator at specific processing locations. A vision-based manipulator control algorithm was developed for the robot to reliably locate and subsequently pick up surgical tool trays. Further, to compensate for perception and navigation errors, we developed a robust self-aligning end-effector that allows for improved error-tolerance in larger workspaces. We evaluated the developed integrated system using an Adept PowerBot mobile robot equipped with a 6-DOF Schunk PowerCube arm and our customized end-effector in an SPC-like environment. The experiment results validate the effectiveness and robustness of our system for handling surgical instrument trays in tight and constrained environments.

Human-Supervisory Distributed Robotic System Architecture for Healthcare Operation Automation

This paper proposes a human-supervisory distributed robotic software architecture, which has been applied in a multi-agent robotic system to automate the daily and repeated sterilization process at hospitals of US Department of Veteran Affairs. Each robot is considered as an independent agent to perform assigned tasks with its own capability and coordinate their operations with other robots to ensure that the main process of the work flow to satisfy the overall operation requirements. This layered architecture highlights human factors in the automation work flow to provide a flexible and robust human-knowledge-based supervision and control for safe, reliable, and automated process for healthcare industry. The proposed architecture and the implemented system were tested in a practical project to validate its effectiveness and robustness.