Lynn Ann DeRose

Towards safe robot-human collaboration systems using human pose detection

This paper proposes a human detection-based cognitive system for robots to work in human-existing environment and keep the safety of humans. An integrated system is implemented with perception, recognition, reasoning, decision-making, and action. Without using any traditional safety cages, a vision-based detection system is implemented for robots to monitor the environment and to detect humans. Subsequently, reasoning and decision making enables robots to evaluate the current safety-related situation for humans and provide corresponding safety signals. The decision making is based on maximizing the productivity of the robot in the manipulation process and keep the safety of humans in the environment. The system is implemented with a Baxter humanoid robot and a PowerBot mobile robot. Practical experiments and simulation experiments are carried out to validate our design.

An integrated vision-based robotic manipulation system for sorting surgical tools

In this paper, we introduced a robotic system using a humanoid robot, Baxter research robot, to pick-up surgical tools from a tray and place the tools into different trays according to the types of the surgical tools. The pick-n-place manipulation is integrated with a vision component and a special magnet gripper and governed by a finite state machine. This vision-based manipulation system allows the robot to check which tool is on top of the tools in a tray, to find the grasping points on the tools, to grab the tools using a magnet gripper, and to place them into different trays. Major technologies used in this system include: vision, magnet force control, force feedback, motion trajectory planning, and decision-making. We tested our system in a lab-based environment and the system performance satisfies the requirements of the project.

Robotic Handling of Surgical Instruments in a Cluttered Tray

We developed a unique robotic manipulation system that accurately singulates surgical instruments in a cluttered environment. A novel single-view computer vision algorithm identifies the next instrument to grip from a cluttered pile and a compliant electromagnetic gripper picks up the identified instrument. System is validated through extensive experiments. This research was motivated by the challenges of perioperative process in hospitals today. Current process of instrument counting, sorting, and sterilization is highly labor intensive. Improperly sterilized instruments have resulted in many cases of infections. To address these challenges, an integrated robotic system for sorting instruments in a cluttered tray is designed and implemented. A digital camera is used to capture an image of a cluttered tray. A novel single-view vision algorithm is used to detect the instruments and determine the top instrument. Position and orientation of the top instrument is transferred to a robot. A compliant electromagnetic gripper is developed to complete the gripping. Experiments have demonstrated high success rate of both instrument recognition and manipulation. In the future, error handling needs to be further reinforced under various exceptions for better robustness.

Integration of evolutionary computing and reinforcement learning for robotic imitation learning.

This paper proposes an evolutionary reinforcement learning method by combining Estimation of Distribution Algorithm and Reinforcement Learning. The Reinforcement Learning method in our method is based on Policy Improvement with Path Integrals (PI2). Estimation of Distribution Algorithm is incorporated into this reinforcement learning method to improve the generation of roll outs with certain noises. This method can accelerate the converging of the learning results and improve the overall system performance. Additionally, this method provides a potential solution to integrate the exploratory evolutionary algorithms and the greedy policy learning method. The proposed method is applied in a robotic imitation learning experiment in this paper and the experimental results demonstrate the effectiveness and robustness of our proposed algorithm.

A vision-guided robot manipulator for surgical instrument singulation in a cluttered environment

The logistics of counting, sorting, sterilizing, and transporting surgical instruments is labor and capital intensive. Furthermore, infection due to improper sterilization is a critical safety hazard. To address these problems, we have developed a unique robotic manipulation system that is capable of accurately singulating surgical instruments in a cluttered environment. Our solution is comprised of two parts. First, we use a single-view vision algorithm for identifying surgical instruments from a pile and estimating their poses. Occlusion reasoning is performed to determine the next instrument to grip using a contrast invariant feature descriptor. Second, we design a compliant electromagnetic gripper that is capable of picking up the identified surgical instrument based on its estimated pose. We validate our solution through instrument singulation experiments demonstrating identification, localization accuracy, and robustness of occlusion reasoning as well as the flexibility of the electromagnetic gripper.

An integrated robotic system for transporting surgical tools in hospitals

The performance of a hospitals sterile processing center (SPC) significantly impacts patient safety and overall productivity. Key to automating this process is to reliably transport instruments throughout the process. In this paper, we detail a robust integrated system for enabling mobile robots to autonomously perform manipulation of assets; specifically, transporting reusable surgical instrument trays in the SPC of a hospital. Our method is based on a cognitive decision making mechanism that plans and coordinates the motions of the robot base and the robot manipulator at specific processing locations. A vision-based manipulator control algorithm was developed for the robot to reliably locate and subsequently pick up surgical tool trays. Further, to compensate for perception and navigation errors, we developed a robust self-aligning end-effector that allows for improved error-tolerance in larger workspaces. We evaluated the developed integrated system using an Adept PowerBot mobile robot equipped with a 6-DOF Schunk PowerCube arm and our customized end-effector in an SPC-like environment. The experiment results validate the effectiveness and robustness of our system for handling surgical instrument trays in tight and constrained environments.

Cloud Computing-Based Marketplace for Collaborative Design and Manufacturing

This paper introduces an open-source, interoperable platform for real-time collaboration in complex product lifecycle development across multiple companies. Each segment of the lifecycle, including product conception, design, analysis, prototyping, component sourcing, manufacturing and assembly, logistics and delivery, and services from installation to maintenance, repair and overhaul, can benefit from this collaboration through easy access, development, deployment, and integration of heterogeneous models and data. The platform is built on an elastic cloud-computing environment, which provides efficient scaling of computational performance needs to support the collaboration platform. We believe that this platform will enable organizations of all sizes to enter a new digital age of integrated product design, manufacturing and service systems.

Human-Supervisory Distributed Robotic System Architecture for Healthcare Operation Automation

This paper proposes a human-supervisory distributed robotic software architecture, which has been applied in a multi-agent robotic system to automate the daily and repeated sterilization process at hospitals of US Department of Veteran Affairs. Each robot is considered as an independent agent to perform assigned tasks with its own capability and coordinate their operations with other robots to ensure that the main process of the work flow to satisfy the overall operation requirements. This layered architecture highlights human factors in the automation work flow to provide a flexible and robust human-knowledge-based supervision and control for safe, reliable, and automated process for healthcare industry. The proposed architecture and the implemented system were tested in a practical project to validate its effectiveness and robustness.

RFID for air cargo operations: return on investment analysis through process modeling and simulation

In the airline industry, radio frequency identification (RFID) can enhance profitability of cargo operations by improving asset utilization and reducing system inefficiencies. However, the adoption of such technology is hindered by costs of infrastructure, hardware and software development, and integration. In order to analyze the return on investment (ROI) from an RFID solution, this paper describes an approach to predict process improvement from the RFID implementation using discrete event simulation. The simulation measures improvements in decreasing processing times and required resources through a comparison analysis of the “as-is” system and the “to-be” RFID enabled system. The metrics are then used as inputs to the ROI calculation. A complete ROI analysis for RFID feasibility in the studied air cargo terminal is also presented.