Balajee Kannan

Predictive indoor navigation using commercial smart-phones

Low-cost navigation solutions for indoor environments have a variety of real-world applications ranging from emergency evacuation to mobility aids for people with disabilities. Challenges for commercial indoor navigation solutions include robust localization, intuitive recognition of user navigation goals, and efficient route-planning and re-planning techniques for resource-constrained platforms like smart-phones and mobile phones. In this paper, we present an architecture for indoor navigation using an Android smartphone that integrates observed behavior for recognizing user navigation goals and estimating future paths without direct input from the user. Our architecture contains three core components: plan recognition, map representation and route planning, and effective localization. To evaluate the feasibility of our solution, we develop a prototype application on a commercial smart-phone and tested it in multiple indoor environments.

Localization, Route Planning, and Smartphone Interface for Indoor Navigation

Low-cost navigation solutions for indoor environments have a variety of real-world applications ranging from emergency evacuation to mobility aids for people with disabilities. Primary challenges for commercial indoor navigation solutions include robust localization in the absence of GPS, efficient route-planning and re-planning techniques, and effective user interfaces for resource-constrained platforms like smartphones and mobile phones. In this chapter, we present an architecture for indoor navigation using an Android smartphone that integrates three core components of localization, map-representation, and user interface towards a robust and effective solution for guiding a variety of users, from sighted to the visually impaired to their intended destination. Specifically, we developed a navigation solution that combines complementary localization algorithms [10] of dead reckoning (DR) and WiFi signal strength fingerprinting (SSI) with enhanced route-planning algorithms to account for the sensory and mobility constraints of the user to efficiently plan safe routes and communicate the route information with sufficient resolution to address the needs of the users. To evaluate the feasibility of our solution, we develop a prototype application on a commercial smartphone and tested it in multiple indoor environments. The results show that the system was able to accurately estimate user location to within 5 m and subsequently provide effective navigation guidance to the user.

Towards safe robot-human collaboration systems using human pose detection

This paper proposes a human detection-based cognitive system for robots to work in human-existing environment and keep the safety of humans. An integrated system is implemented with perception, recognition, reasoning, decision-making, and action. Without using any traditional safety cages, a vision-based detection system is implemented for robots to monitor the environment and to detect humans. Subsequently, reasoning and decision making enables robots to evaluate the current safety-related situation for humans and provide corresponding safety signals. The decision making is based on maximizing the productivity of the robot in the manipulation process and keep the safety of humans in the environment. The system is implemented with a Baxter humanoid robot and a PowerBot mobile robot. Practical experiments and simulation experiments are carried out to validate our design.

An integrated vision-based robotic manipulation system for sorting surgical tools

In this paper, we introduced a robotic system using a humanoid robot, Baxter research robot, to pick-up surgical tools from a tray and place the tools into different trays according to the types of the surgical tools. The pick-n-place manipulation is integrated with a vision component and a special magnet gripper and governed by a finite state machine. This vision-based manipulation system allows the robot to check which tool is on top of the tools in a tray, to find the grasping points on the tools, to grab the tools using a magnet gripper, and to place them into different trays. Major technologies used in this system include: vision, magnet force control, force feedback, motion trajectory planning, and decision-making. We tested our system in a lab-based environment and the system performance satisfies the requirements of the project.

Robotic Handling of Surgical Instruments in a Cluttered Tray

We developed a unique robotic manipulation system that accurately singulates surgical instruments in a cluttered environment. A novel single-view computer vision algorithm identifies the next instrument to grip from a cluttered pile and a compliant electromagnetic gripper picks up the identified instrument. System is validated through extensive experiments. This research was motivated by the challenges of perioperative process in hospitals today. Current process of instrument counting, sorting, and sterilization is highly labor intensive. Improperly sterilized instruments have resulted in many cases of infections. To address these challenges, an integrated robotic system for sorting instruments in a cluttered tray is designed and implemented. A digital camera is used to capture an image of a cluttered tray. A novel single-view vision algorithm is used to detect the instruments and determine the top instrument. Position and orientation of the top instrument is transferred to a robot. A compliant electromagnetic gripper is developed to complete the gripping. Experiments have demonstrated high success rate of both instrument recognition and manipulation. In the future, error handling needs to be further reinforced under various exceptions for better robustness.

Integration of evolutionary computing and reinforcement learning for robotic imitation learning.

This paper proposes an evolutionary reinforcement learning method by combining Estimation of Distribution Algorithm and Reinforcement Learning. The Reinforcement Learning method in our method is based on Policy Improvement with Path Integrals (PI2). Estimation of Distribution Algorithm is incorporated into this reinforcement learning method to improve the generation of roll outs with certain noises. This method can accelerate the converging of the learning results and improve the overall system performance. Additionally, this method provides a potential solution to integrate the exploratory evolutionary algorithms and the greedy policy learning method. The proposed method is applied in a robotic imitation learning experiment in this paper and the experimental results demonstrate the effectiveness and robustness of our proposed algorithm.

A vision-guided robot manipulator for surgical instrument singulation in a cluttered environment

The logistics of counting, sorting, sterilizing, and transporting surgical instruments is labor and capital intensive. Furthermore, infection due to improper sterilization is a critical safety hazard. To address these problems, we have developed a unique robotic manipulation system that is capable of accurately singulating surgical instruments in a cluttered environment. Our solution is comprised of two parts. First, we use a single-view vision algorithm for identifying surgical instruments from a pile and estimating their poses. Occlusion reasoning is performed to determine the next instrument to grip using a contrast invariant feature descriptor. Second, we design a compliant electromagnetic gripper that is capable of picking up the identified surgical instrument based on its estimated pose. We validate our solution through instrument singulation experiments demonstrating identification, localization accuracy, and robustness of occlusion reasoning as well as the flexibility of the electromagnetic gripper.

Mapping planetary caves with an autonomous, heterogeneous robot team

Caves on other planetary bodies offer sheltered habitat for future human explorers and numerous clues to a planets past for scientists. While recent orbital imagery provides exciting new details about cave entrances on the Moon and Mars, the interiors of these caves are still unknown and not observable from orbit. Multi-robot teams offer unique solutions for exploration and modeling subsurface voids during precursor missions. Robot teams that are diverse in terms of size, mobility, sensing, and capability can provide great advantages, but this diversity, coupled with inherently distinct low-level behavior architectures, makes coordination a challenge. This paper presents a framework that consists of an autonomous frontier and capability-based task generator, a distributed market-based strategy for coordinating and allocating tasks to the different team members, and a communication paradigm for seamless interaction between the different robots in the system. Robots have different sensors, (in the representative robot team used for testing: 2D mapping sensors, 3D modeling sensors, or no exteroceptive sensors), and varying levels of mobility. Tasks are generated to explore, model, and take science samples. Based on an individual robots capability and associated cost for executing a generated task, a robot is autonomously selected for task execution. The robots create coarse online maps and store collected data for high resolution offline modeling. The coordination approach has been field tested at a mock cave site with highly-unstructured natural terrain, as well as an outdoor patio area. Initial results are promising for applicability of the proposed multi-robot framework to exploration and modeling of planetary caves.

An integrated robotic system for transporting surgical tools in hospitals

The performance of a hospitals sterile processing center (SPC) significantly impacts patient safety and overall productivity. Key to automating this process is to reliably transport instruments throughout the process. In this paper, we detail a robust integrated system for enabling mobile robots to autonomously perform manipulation of assets; specifically, transporting reusable surgical instrument trays in the SPC of a hospital. Our method is based on a cognitive decision making mechanism that plans and coordinates the motions of the robot base and the robot manipulator at specific processing locations. A vision-based manipulator control algorithm was developed for the robot to reliably locate and subsequently pick up surgical tool trays. Further, to compensate for perception and navigation errors, we developed a robust self-aligning end-effector that allows for improved error-tolerance in larger workspaces. We evaluated the developed integrated system using an Adept PowerBot mobile robot equipped with a 6-DOF Schunk PowerCube arm and our customized end-effector in an SPC-like environment. The experiment results validate the effectiveness and robustness of our system for handling surgical instrument trays in tight and constrained environments.

Human-Supervisory Distributed Robotic System Architecture for Healthcare Operation Automation

This paper proposes a human-supervisory distributed robotic software architecture, which has been applied in a multi-agent robotic system to automate the daily and repeated sterilization process at hospitals of US Department of Veteran Affairs. Each robot is considered as an independent agent to perform assigned tasks with its own capability and coordinate their operations with other robots to ensure that the main process of the work flow to satisfy the overall operation requirements. This layered architecture highlights human factors in the automation work flow to provide a flexible and robust human-knowledge-based supervision and control for safe, reliable, and automated process for healthcare industry. The proposed architecture and the implemented system were tested in a practical project to validate its effectiveness and robustness.