Trygve Thomessen

Cognitive Human-Machine Interface Applied in Remote Support for Industrial Robot Systems

An attempt is currently being made to widely introduce industrial robots to Small-Medium Enterprises (SMEs). Since the enterprises usually employ too small number of robot units to afford specialized departments for robot maintenance, they must be provided with inexpensive and immediate support remotely. This paper evaluates whether the support can be provided by means of Cognitive Info-communication – communication in which human cognitive capabilities are extended irrespectively of geographical distances. The evaluations are given with an aid of experimental system that consists of local and remote rooms, which are physically separated – a six-degree-of-freedom NACHI SH133-03 industrial robot is situated in the local room, while the operator, who supervises the robot by means of audio-visual Cognitive Human-Machine Interface, is situated in the remote room. The results of simple experiments show that Cognitive Info-communication is not only efficient mean to provide the support remotely, but is probably also a powerful tool to enhance interaction with any data-rich environment that require good conceptual understanding of systems state and careful attention management. Furthermore, the paper discusses data presentation and reduction methods for data-rich environments, as well as introduces the concepts of Naturally Acquired Data and Cognitive Human-Machine Interfaces.

An approach to path planning and real-time redundancy control for human-robot collaboration

This paper presents a control system for redundant robot manipulators that handles the path planning and redundancy control in two separate loops. The goal is to simplify industrial programming of redundant robots, by decomosing it to a familiar path planning loop, and an autonomous redundant control loop. The familiar path planning is done using an industrial robots native control system, which is already known to the operator. An external feedback loop acts as a secondary task control loop so that the redundancy in the robot is used automatically. The secondary task control is expanded from the null space projection method to a method where a parallel space to task space is modeled in joint space. The system was implemented and experimentally verified on a NACHI MR20 7-axes industrial robot with AX20 Controller. The secondary task in the experiments was an obstacle avoidance task that was tested in unknown dynamic and static environments.

New approach for industrial robot controller user interface

In this paper the main focus is on the user interfaces available for robot controllers and how they comply with the requirements of flexibility. An overview is given of the most important robot manufacturers teach pendant interfaces. The conclusion is that the process of creating customized user interfaces is still complicated and the how-to is unclear. We introduce the concept of flexible user interface based on service oriented robot cell operation. In order to offer more user friendly solutions industrial robot systems should use human-centred approach. Service oriented robot cell operation creates an abstract level for human robot interaction which is adapted to human cognition. A flexible graphical user interface is the final link between the robot cell and the operator, which offers a solution for issues faced in creation of customized user interfaces by its flexibility. One implementation called FlexGui is described in details and the most important features are demonstrated in a case study.

Vibrotactile information for supporting pick and place task using industrial robot remote operation

This paper evaluates a wristband-type tactile interface as a user interface for controlling an industrial robot remotely. When operating a remote monitoring system, information delivery from a local site to an operator at a remote site is important. Visual and auditory feedback is often used. However, perception of the information might be limited if different kinds of information in particular sense modality is delivered at the same time. To solve the problem, we proposed a tactile interface which uses tactile information and we evaluate it. First, we explain the remote operation system for industrial robot and interface, then after that, we present an experiment of vibrotactile information of industrial robot.

Towards virtual presence applying multi-modal man-machine communication; soon full remote operation of industrial robot systems

Development of the international industry is towards smaller and smaller batch sizes in the production. This has riced a need for cost efficient support of robot users. In this paper, new methodology for remote operation of industrial robot systems is introduced. The method is based upon multi-modal man-machine communication and sensor bridging to transfer the huge amount of information from the industrial robot cell to the brain of the remote operator.

Multi-Robot Assembly of High-Performance Aerospace Components

Abstract This paper presents an automated assembly and tack welding operation of high performance aerospace components using two real-time controlled industrial robots and a high-precision non-contact measurement system. The system was able to align two components with an offset distance less than 0.1 mm, well within manufacturing tolerances.

Multimodal human-machine interface for remote operation of robot systems

Abstract The trend in robotization is to use industrial robot more and more frequently in small and medium sized enterprises (SMEs). Wide spread of the enterprises over considerable area and small number of applied robot units per enterprise, emerges need to provide an efficient, inexpensive and immediate support remotely. Since there are a lot of important information to be communicated to efficiently support an industrial robot system, traditional data acquisition and human-machine-interfaces (HMI) cannot provide complete and effectively comprehended overview of the state of the robot cell. The goal of the paper is to demonstrate new methodology and technology for remote HMI based upon Cognitive Informatics Communication (CogInfoCom 2011) that augments efficiency of communication between an industrial robot system and remote operator exercising control over the system. For the purposes of evaluation a remote operating laboratory is developed. The laboratory consists of multimodal HMI interfacing an operator with physically separated seven axes NACHI MR20 robot. By means of multi-camera vision, multi-speaker sound system, and advanced 3D simulation, the robot is controlled remotely. The results show that HMI based upon CogInfoCom can be beneficially applied to communicate complex states of an industrial robot system to a remote operator.

Joint Level Collision Avoidance for Industrial Robots

Abstract A system concept for industrial collision avoidance is presented. The paper searches solutions for small and medium enterprises which have strictly limited resources for robot installations. Key factor is to use commercially available components and user-friendly algorithms. The experimental setup consists of a 7-axis industrial robot and an industrial robot controller. The obstacle detecting sensors are applied on the robotic arm. Designing guidelines are formed according to industrial considerations and preliminary experiments are presented.

Towards multimodal interactions: robot jogging in mixed reality

The recent progress made in the field of Augmented Reality/Mixed Reality (AR/MR) has opened new possibilities and approaches to research areas that can benefit from 3D visualization of digital content in the real world. In fact, human-robot interaction design and the design of user interfaces have very much to gain from MR technologies. Nonetheless, designing the user-robot interaction and processing multimodal feedbacks are very challenging tasks. In this paper we focus in particular on interactions in mixed reality. The main contribution of this paper is the implementation of a control system for an industrial manipulator through the users interactions with MR content displayed with the Microsoft HoloLens. The system is based on the communication between Unity3D (used to design the user experience) and ROS, therefore extendible to any ROS-compatible robotic hardware.

A vibrotactile navigation aid for remote operation of an industrial robot

Recently industrial robot systems are introduced widely even in the small and medium size enterprises. However, it is difficult for them to employ professional engineers permanently. Then, they need to depend on outside to maintain industrial robots. Therefore a remote operation support system for industrial robots is proposed. Visual and auditory feedback is often used. However, a perception of the information might be limited if different kinds of information in particular sense modality is delivered at the same time. To solve the problem, we proposed a tactile interface which uses tactile information. This paper evaluates the usefulness of the vibrotactile information using the glove-type interface. The experimental results showed that vibrotactile information could not improve work efficiency. However it enable an operator to concentrate other information without increasing a workload. Therefore vibrotactile information is useful.