Nima Najmaei

Visual servoing in medical robotics: a survey. Part I: endoscopic and direct vision imaging – techniques and applications

Intra‐operative imaging is widely used to provide visual feedback to a clinician when he/she performs a procedure. In visual servoing, surgical instruments and parts of tissue/body are tracked by processing the acquired images. This information is then used within a control loop to manoeuvre a robotic manipulator during a procedure.

Image-guided techniques in renal and hepatic interventions

Development of new imaging technologies and advances in computing power have enabled the physicians to perform medical interventions on the basis of high‐quality 3D and/or 4D visualization of the patients organs. Preoperative imaging has been used for planning the surgery, whereas intraoperative imaging has been widely employed to provide visual feedback to a clinician when he or she is performing the procedure. In the past decade, such systems demonstrated great potential in image‐guided minimally invasive procedures on different organs, such as brain, heart, liver and kidneys. This article focuses on image‐guided interventions and surgery in renal and hepatic surgeries.

Suitability of Small-Scale Magnetorheological Fluid-Based Clutches in Haptic Interfaces for Improved Performance

Haptic information provides important cues in teleoperated systems. It enables the user to feel the interaction with a remote or virtual environment during teleoperation. The two main objectives in designing a haptic interface are stability and transparency. An actuator with poor dynamics, high inertia, large size, and heavy weight can significantly undermine these goals. In this article, the potential benefits of magnetorheological fluid (MRF)-based actuators to the field of haptics are discussed. Devices developed with such fluids are known to possess superior mechanical characteristics over conventional servo systems. These characteristics significantly contribute to improved stability and transparency of haptic devices. In this study, this idea is evaluated from both theoretical and experimental points of view. First, the properties of such actuators which motivated this research are discussed. Next, two single degree-of-freedom (DOF) haptic interfaces, one with an MRF-based clutch and the other with a brushless DC motor, are compared in a virtual wall experiment in order to show the superiority of the MRF-based clutch. In addition, the design of a small-scale MRF-based clutch, suitable for a multi-DOF haptic interface, is discussed and its torque capacity, inertia, and mass are compared with conventional servo systems. Conclusions drawn from this investigation indicate that MRF-based actuation approaches can indeed be developed to design haptic interfaces with improved stability and transparency.

Applications of Artificial Intelligence in Safe Human–Robot Interactions

The integration of industrial robots into the human workspace presents a set of unique challenges. This paper introduces a new sensory system for modeling, tracking, and predicting human motions within a robot workspace. A reactive control scheme to modify a robots operations for accommodating the presence of the human within the robot workspace is also presented. To this end, a special class of artificial neural networks, namely, self-organizing maps (SOMs), is employed for obtaining a superquadric-based model of the human. The SOM network receives information of the humans footprints from the sensory system and infers necessary data for rendering the human model. The model is then used in order to assess the danger of the robot operations based on the measured as well as predicted human motions. This is followed by the introduction of a new reactive control scheme that results in the least interferences between the human and robot operations. The approach enables the robot to foresee an upcoming danger and take preventive actions before the danger becomes imminent. Simulation and experimental results are presented in order to validate the effectiveness of the proposed method.

Prediction-based reactive control strategy for human-robot interactions

In this paper a reactive control strategy intended for human-robot interactions (HRI) is presented. A conventional reactive control scheme is reviewed first. This is followed by the introduction of a new prediction-based reactive control strategy. The new control strategy considers foreseeable dangerous events by predicting human motion using artificial neural networks, based on the previous pattern of the motion. This approach enables a robot to foresee an upcoming danger in order to take preventive actions before the danger is immanent. Experimental results for a CRS-F3 robot manipulator are presented in order to demonstrate and validate the effectiveness of this method.

Design and Performance Evaluation of a Prototype MRF-Based Haptic Interface for Medical Applications

This paper describes the construction and stability and transparency evaluation of a prototype two degrees-of-freedom (DoF) haptic interface, which takes advantage of magneto-rheological fluid (MRF)-based clutches for actuation. These small-scale clutches were designed in our lab, and their evaluation were reported previously [1], [2]. MRF-based actuators exhibit superior characteristics, which can significantly contribute to transparency and stability of haptic devices. Based on these actuators, a distributed antagonistic configuration is used to develop the 2-DoF haptic interface. This device is incorporated in a master-slave teleoperation setup intended for medical percutaneous interventions and soft-tissue palpation. Preliminary studies on the stability and transparency of the haptic interface in this setup using phantom and ex vivo samples show the great potential of MRF-based actuators for integration in haptic devices that require reliable, safe, accurate, highly transparent, and stable force reflection.

Human factors for robot safety assessment

This article presents the results of a study on including human factors in assessment of the level of danger involved in a robot operation. A dynamic risk assessment method, which considers all effective factors in a collision, is essential for planning and control of the human-safe robots. In this regard, the visibility of the robot in the human eyes can significantly effect the probability of collision. To this end, we propose a risk index which not only considers the physical factors effective in a collision, but also includes the direction of eye gaze and human body orientation. Also two sensory systems for measuring these values are introduced. To this effect, a non-invasive brain-machine-interface prototype system which allows the simple control of a switch is presented. This system creates a feedback signal that is a function of the position of the human eyes for the modulation of the risk assessment. Initial results suggest possible benefits of using this approach in human-safe systems.

Application of Magneto-Rheological Fluid based clutches for improved performance in haptic interfaces

The two main objectives in designing a haptic interface are stability and transparency. The dynamics of the actuators employed in a haptic interface have a significant effect on these goals. In this article, the potential benefits of Magneto-Rheological Fluid (MRF) based actuators to the field of haptics are discussed. Devices developed with such fluids are known to possess superior mechanical characteristics over conventional servo systems. This contributes significantly to improved stability and transparency of haptic devices. In this study, this idea is evaluated from both theoretical and experimental points of view. First, the properties of such actuators which motivated this research are discussed. Next, two single degrees-of-freedom (DOF) haptic interfaces are used in a virtual wall experiment. These devices take advantage of an MRF-based clutch and a brushless DC motor at their core, respectively. The results of both devices are compared and show the superiority of the MRF-based clutch. In addition, design and analysis of a small-scale MRF-based clutch, suitable for a multi-DOF haptic interface, is given and its torque capacity, inertia, and mass are compared with those of conventional servo systems. Conclusions drawn from this investigation indicate that MRF clutch actuation approaches can indeed be developed to design haptic interfaces with improved stability and transparency.

A New Sensory System for Modeling and Tracking Humans Within Industrial Work Cells

This paper introduces a new sensory system for the 2-D detection, tracking, and prediction of human motions. Using the data acquired through the sensory system, a model of a human based on a superquadric modeling technique is obtained. The sensory system requires a minimum computational burden for its measurements and almost no reengineering efforts for the integration with existing systems. The sensory system comprises a floor mat for the repeated measurement of the footprints of a human either standing or walking. An intelligent firmware for inferring the required information and identification of the human orientation and for rendering a real-time human model based on primitive data received from the mat is discussed. Furthermore, the integration of the floor mat in a multisensory system for obtaining a 3-D human model is discussed. Several results are presented to validate the effectiveness of the sensory system.

Visual servoing in medical robotics: a survey. Part II: tomographic imaging modalities--techniques and applications.

Intraoperative application of tomographic imaging techniques provides a means of visual servoing for objects beneath the surface of organs.