XiaoQi Chen

Human-Robot Collaboration: A Literature Review and Augmented Reality Approach in Design

NASAs vision for space exploration stresses the cultivation of human-robotic systems. Similar systems are also envisaged for a variety of hazardous earthbound applications such as urban search and rescue. Recent research has pointed out that to reduce human workload, costs, fatigue driven error and risk, intelligent robotic systems will need to be a significant part of mission design. However, little attention has been paid to joint human-robot teams. Making human-robot collaboration natural and efficient is crucial. In particular, grounding, situational awareness, a common frame of reference and spatial referencing are vital in effective communication and collaboration. Augmented Reality (AR), the overlaying of computer graphics onto the real worldview, can provide the necessary means for a human-robotic system to fulfill these requirements for effective collaboration. This article reviews the field of human-robot interaction and augmented reality, investigates the potential avenues for creating natural human-robot collaboration through spatial dialogue utilizing AR and proposes a holistic architectural design for human-robot collaboration.

An experimentally validated double-mass piezoelectric cantilever model for broadband vibration–based energy harvesting

Narrow bandwidth is the major challenge to today’s vibration-based energy harvesters. Compared with other broadband approaches that involve moving parts and control electronics, a double-mass piezoelectric cantilever beam provides a simple and reliable solution to widen the effective bandwidth as a vibration energy harvester. In this article, a continuum model of a double-mass lead zirconate titanate cantilever subject to sinusoidal base excitation is presented. First, the undamped equation of motion along with boundary and transition conditions is derived from Hamilton’s principle, followed by modal analysis that determines the eigenfunctions and natural frequencies. Next, the coupled electromechanical equations for sinusoidal base excitation are obtained. The output voltage and relative displacement are solved analytically. The frequency response function and mode shapes predicted by the model are validated against experiments.

Evaluating the Augmented Reality Human-Robot Collaboration System

This paper discusses an experimental comparison of three user interface techniques for interaction with a mobile robot located remotely from the user. A typical means of operating a robot in such a situation is to teleoperate the robot using visual cues from a camera that displays the robots view of its work environment. However, the operator often has a difficult time maintaining awareness of the robot in its surroundings due to this single ego-centric view. Hence, a multi-modal system has been developed that allows the remote human operator to view the robot in its work environment through an Augmented Reality (AR) interface. The operator is able to use spoken dialog, reach into the 3D graphic representation of the work environment and discuss the intended actions of the robot to create a true collaboration. This study compares the typical ego-centric driven view to two versions of an AR interaction system for an experiment remotely operating a simulated mobile robot. One interface provides an immediate response from the remotely located robot. In contrast, the Augmented Reality Human-Robot Collaboration (AR-HRC) system interface enables the user to discuss and review a plan with the robot prior to execution. The AR-HRC interface was most effective, increasing accuracy by 30% with tighter variation, while reducing the number of close calls in operating the robot by factors of ~3x. It thus provides the means to maintain spatial awareness and give the users the feeling they were working in a true collaborative environment.

Cell image recognition and visual servo control for automated cell injection

This paper presents a micro-robotic bio-manipulation system for automated cell injection, addressing two key challenges. First, this paper reports a human interference-free determination of material deposition destinations based on image processing of the cell structures. Depending on the applications, foreign materials could be deposited inside the nucleus of the cell or outside but still inside the cytoplasm of the cell. Recognition of the nucleoli boundary brings us basic cell structure information, which consequently provides high flexibility to determine the deposition destinations, with no need of the destinations to be selected by an operator. Second, the paper presents a visual servo control design which moves the micropipette tip to the targeted position and deposits foreign materials to that point autonomously. The visual servo control allows the motion settle time within 0.5 sec., with an accuracy of one pixel, which are highly desirable in automated cell injection applications.

Haptic Microrobotic Cell Injection System

Microrobotic cell injection is the subject of increasing research interest. At present, an operator relies completely on visual information and can be subject to low success rates, poor repeatability, and extended training times. This paper focuses on increasing operator performance during cell injection in two ways. First, our completed haptic cell injection system aims to increase the operators performance during real-time cell injection. Haptic bilateralism is investigated and a mapping framework provides an intuitive method for manoeuvring the micropipette in a manner similar to handheld needle insertion. Volumetric virtual fixtures are then introduced to haptically assist the operator to penetrate the cell at the desired location. The performance of the volumetric virtual fixtures is also discussed. Second, the haptically enabled cell injection system is replicated as a virtual environment facilitating virtual offline operator training. Virtual operator training utilizes the same mapping framework and haptic virtual fixtures as the physical system allowing the operator to train offline and then directly transfer their skills to real-time cell injection.

Artificial Neural Network–Based System Identification for a Single-Shaft Gas Turbine

—During recent decades, artificial intelligence has been employed as a powerful tool for identification of complex industrial systems with nonlinear dynamics, such as gas turbines (GT). In this study, a methodology based on artificial neural network (ANN) techniques was developed for offline system identification of a low-power gas turbine. The processed data was obtained from a SIMULINK model of a gas turbine in MATLAB environment. A comprehensive computer program code was generated and run in MATLAB environment for creating and training different ANN models with two-layer feed-forward multi-layer perceptron (MLP) structure. The code consisted of various training functions, different number of neurons as well as a variety of transfer (activation) functions for hidden and output layers of the network. It was shown that the optimal model for a two-layer network with MLP structure, consisted of 20 neurons in its hidden layer and used trainlm as its training function, as well as tansig and logsid as its transfer functions for the hidden and output layers. It was also observed that trainlm has a superior performance in terms of minimum MSE, compared with each of the other training functions. The resulting model could predict performance of the system with high accuracy. The methodology provides a comprehensive view of the performance of over 18720 ANN models for system identification of single shaft GT. One can use the optimal ANN model from this study when training from real data obtained from this type of GT. This is particularly useful when real data is only available over a limited operational range.

A two-mass cantilever beam model for vibration energy harvesting applications

While vibration energy harvesting has become a viable means to power wireless sensors, narrow bandwidth is still a hurdle to the practical use of the technology. For conventional piezoelectric or electromagnetic harvesters, having multiple proof masses mounted on a beam is one way to widen the effective bandwidth. This is because the addition of proof masses increases the number of resonant modes within the same frequency range. Based on the assumptions of the Euler-Bernoulli beam theory, this paper presents a continuum-based model for a two-mass cantilever beam. First, the equation of motion is derived from Hamiltons principle. Next, the modal analysis is presented and a steady state solution for harmonic base excitation is derived. The two-mass beam is considered as two serially connected beam segments. In the derivation, emphasis is given to the transition conditions, which would otherwise not appear in the traditional single mass beam model. Experimental validation on a stainless steel beam confirms that the model can accurately predict both natural frequencies and the frequency response of an arbitrary point along the beam. The derivation procedure presented in this paper is applicable to a beam with any number of proof masses. Lastly, it is demonstrated how the model can be applied to a piezoelectric energy harvester

Haptic guidance for microrobotic intracellular injection

The ability for a bio-operator to utilise a haptic device to manipulate a microrobot for intracellular injection offers immense benefits. One significant benefit is for the bio-operator to receive haptic guidance while performing the injection process. In order to address this, this paper investigates the use of haptic virtual fixtures for cell injection and proposes a novel force field virtual fixture. The guidance force felt by the bio-operator is determined by force field analysis within the virtual fixture. The proposed force field virtual fixture assists the bio-operator when performing intracellular injection by limiting the micropipette tips motion to a conical volume as well as recommending the desired path for optimal injection. A virtual fixture plane is also introduced to prevent the bio-operator from moving the micropipette tip beyond the deposition target inside the cell. Simulation results demonstrate the operation of the guidance system.

LMS-based approach to structural health monitoring of nonlinear hysteretic structures

Structural health monitoring (SHM) algorithms based on adaptive least mean squares (LMS) filtering theory can directly identify time-varying changes in structural stiffness in real-time in a computationally efficient fashion. However, better metrics of seismic structural damage and future utility after an event are related to permanent and total plastic deformations. This study presents a modified LMS-based SHM method and a novel two-step structural identification technique using a baseline nonlinear Bouc—Wen structural model to directly identify changes in stiffness due to damage as well as plastic or permanent deflections. The algorithm is designed to be computationally efficient; therefore it can work in real-time. An in silico single-degree-of-freedom (SDOF) nonlinear shear-type structure is used to prove the concept. The efficiency of the proposed SHM algorithm in identifying stiffness changes and plastic/permanent deflections is assessed under different ground motions using a suite of 20 different ground acceleration records. The results show that in a realistic scenario with fixed filter tuning parameters, the proposed LMS-based SHM algorithm identifies stiffness changes to within 10% of true values within 2s. Permanent deflection is identified to within 14% of the actual as-modeled value using noise-free simulation-derived structural responses. This latter value provides important post-event information on the future serviceability, safety, and repair cost.

A Low-Cost Unmanned Underwater Vehicle Prototype for Shallow Water Tasks

Unmanned underwater vehicles (UUVs) have received worldwide attention and been widely used in various applications. In this paper, a developed low cost UUV prototype at the University of Canterbury is introduced, which is designed specifically for shallow water tasks, especially for inspecting and cleaning sea chests of ships for biosecurity purpose. The main hull of the UUV is made of PVC, with a 400 mm diameter and 800 mm length. External frames mount two horizontal propellers, four vertical thrusters, and power is derived from onboard batteries. The maximum thrust force of up to 10 kg that is provided by the propellers can generate a forward/backward speed of up to 1.4 m/s for the 112 kg UUV. The vertical thrusters provide depth control with a max thrust force of 20 kg. The UUV is equipped with a range of sensors capable of sensing its instantaneous temperature, depth, attitude and surrounding environment. Costing less than US