Jeffrey Too Chuan Tan

Human-robot collaboration in cellular manufacturing: Design and development

The challenge of this work is to study the design and development of human-robot collaboration (HRC) in cellular manufacturing. Based on the concept of human collaborative design, four main design factors are being identified and developed in an active HRC prototype production cell for cable harness assembly. Human collaborative design aims to optimize the system design for the advantage of collaboration between human and robots based on human considerations. Task modeling approach is developed to study and analyze the task in order to identify the collaboration tasks to develop the collaboration planning. In the collaboration safety development, five safety designs, cover both hardware and control design, are proposed and developed in the prototype system. Risk assessment is conducted to verify the safety design. Two main experiments were conducted as preliminary study to investigate mental workload in HRC. A multimodal information support system is developed in the study of man-machine interface in this work to provide a comprehensive human-robot interface to facilitate human operator. The system performance evaluation had proven the improvement of prototype production cell with HRC design for cellular manufacturing.

Triple stereo vision system for safety monitoring of human-robot collaboration in cellular manufacturing

In a close proximity of a human-robot collaboration production system, safety monitoring has a paramount importance to ensure the human operator is being well protected throughout the collaborative operation with the robot manipulator. Due to the requirement to allow overlapping of working envelopes between these two parties, physical separation or two-dimensional sensory system is not effective as the safety measure for the production system. In the early development, safety monitoring by stereo vision system with two cameras was introduced to track the human operators motion throughout the operation. Camera is used to capture images for tracking of color areas on the human operator. The image coordinates by particle filter and human body model are combined to estimate the 3D positions for the human motion monitoring. However, several weaknesses were observed in this development. For instance, due to the fixed camera viewing direction, occlusion of the detecting areas can severely affect the effectiveness of the safety monitoring. Therefore, one additional camera is added into the system to produce three pairs of stereo vision to improve the robustness towards lost tracking and occlusion tolerance. Hand position tracking experiment is conducted to evaluate the performance of the 3D position estimation.

SIGVerse - A cloud computing architecture simulation platform for social human-robot interaction

The aim of this work is to propose a cloud computing architecture simulation platform for social human-robot interaction. This paper explains the design and development of this system named SIGVerse in four main components, namely (1) SIGServer as the central server, (2) Agent Controller for user applications, (3) Service Provider, and (4) SIGViewer as the client terminal, and web based development interface, in addressing the two main “human” issues in human-robot interaction with cloud computing architecture, (a) Distributed development platform, and (b) Large scale of human-robot simulation. Three current applications are discussed for the validation of the cloud computing architecture in social human-robot interaction simulations.

Safety design and development of human-robot collaboration in cellular manufacturing

The key challenge of this work is to research the safety design and development of a close range active human-robot collaboration system in which no governing safety standards and regulations are currently present. Active robot assistance in a human-centered cellular manufacturing system is very promising proposal, however safety is the uppermost consideration before this idea can be materialized. In this development, five main safety designs in both hardware and control levels have been proposed and discussed in this work. In the prototype system development, human area and robot area are setup as safe working areas but border crossing is allowed and monitored for safety. At the close region of collaboration, three safety robot working zones are defined by light curtains to control the behavior of the robot based on the collaboration requirements. The assisting mobile robot manipulators system is designed with additional safety control strategies in term of robot gripper speed and travel area to minimize the collaboration risks. Vision system by IP cameras is developed to monitor the human operator safety conditions by measuring the body posture and position. The overall control system is developed to coordinate the collaboration flow between the human operator and mobile robot manipulators system to ensure the safety of the assembly operation. Risk assessments are conducted to make evaluations before and after implementation of safety design into the prototype system. Various safety improvements are observed and discussed. It has proven that the safety designs can reduce the potential collaboration risks in the system.

Assembly Information System for Operational Support in Cell Production

In cell production, the output depends heavily on the performance of the worker. Assembly information support is useful to support the worker. The objective of this study is to develop a framework to organize assembly information to support production operation. It is illustrated by an application of a production operation in laboratory simulation. Assembly motion of skilled worker is extracted and the assembly process is decomposed into operational units, which linked with support information to set up the information database. New assembly process with information support can then be generated and implemented with multimodal system to assist production operation.

Task modeling approach to enhance man-machine collaboration in cell production

The objective of this work is to enhance man-machine collaboration in cell production by task modeling approach. Task modeling provides a method to define and study the collaboration between man and machine. Six requirements have been derived as basis of development. In this work, the task modeling has extended classical HTA method to represent collaborative operation in a hierarchical structure with the classification of operation task levels and extended operation details as task properties. A modeling tool is developed to provide a development and data management platform. The man-machine collaboration is well defined in a structural format by the ability of task analysis method to address human task in a more flexible way. The collaboration modeling and operation resources information have significantly facilitated operation planning in both assembly and control levels. From the case study, the improvement in safety aspect has proven the ability of this approach in assisting safety design and development of production system. The integration with operation control system has shown the execution potential in real-time operation. The unique development of operation data management has expanded this work into operation information support. Through the multimodal information support system, the human operator is well guided by the operation information corresponded to the task components. The operation information also can work as evaluation functions to measure the human working performance and system performance as a whole.

Task decomposition of cell production assembly operation for man-machine collaboration by HTA

The main aim of this work is to address task decomposition of man-machine collaboration by hierarchical task analysis (HTA) approach. The success of HTA in ergonomics study has inspired this work to implement this method in practical cable assembly operation in cell production system. In the development, both the cable assembly tasks of human worker and robot manipulator are being modeled in HTA format. By combining both of them, the collaborative working flow is constructed and well presented in the hierarchical diagram. The modeling approach is useful for task sequence design while preserving the collaboration. The cable assembly simulation has proven the practicability of the approach in man-machine collaborative work.

Extending task analysis in HTA to model man-machine collaboration in cell production

The application of robot system to collaborate with human worker in production operation can greatly enhance manufacturing systems. In order to ensure successful implementation of such system, the effectiveness of the man-machine collaboration is the decisive factor. The main aim of this work is to model the collaboration between human and robot in cell production system by task analysis approach. In this work, the production operation is modeled into hierarchical task analysis (HTA) structure with the extension of collaboration modeling. The model is being validated by an actual assembly operation simulation. This work has enabled the study of collaboration modeling in order to design effective man-machine system in cell production.

A study of the human-robot synchronous control system based on skeletal tracking technology

The purpose of this study is to develop a new system to synchronously control the simulated robot and humanoid robot based on the skeletal tracking technology. This system is a human-robot interaction system. Human can control the robot through the skeletal tracking technology without placing any marker on the subjects body. We use the Microsoft Kinect to get the skeleton framework information of the subject and input the data into the program. Therefore, the experiment has no demand on color to distinguish the subjects motions, and the subject can just perform in the front of the Kinect without wearing any control device. After processing the data, the computer controls the simulated robot to regenerate the subjects motions synchronously. In this study, we can control the simulated robot to do three different exercises. To verify the validity of the control system, we did a series of experiments to control a humanoid robot NAO. The experimental results show that the synchronous control system can distinguish the movements of human upper limb efficiently, and the cost is lower than that of the previous studies. This system controls both robot model and humanoid robot NAO based on human motion to achieve the human-machine interaction efficiently and the success rate is more than 90%.

Mobile robot action based on QR code identification

Nowadays, with the developing of the computer science, computer vision is widely used in many areas. It is still a major problem about feature recognition in complex background and uneven intensity of light environment. Some researchers have already achieved in particular conditions. With the robot stepping into family, it is necessary to conduct in-depth study about feature recognition and access to information in complex indoor environmental conditions. In this paper, we present a fast identification approach for QR code in the indoor environment with normal light intensity. In the process of image feature recognition, it is very difficult for the edge while recognizing the slope image. Therefore, we use vector method to express the color characteristics in the image, and use the square minimum method to determine the edge lines of the pattern. In addition, linear scale transformation makes the image pattern more suitable for standard grids, which can correct the tiny sampling error. Besides, in the experiment of emulation process, we add a colored periphery features to positioning the image. It can be captured clearly due to the special colored background and behave well in the distance of 0.5m to 2m. Using an 860-pixel distance camera, the QR code pattern can be captured easily and identified effectively. A feedback control planar mobile robot is used as a platform to verify the identification method. After the robot heading to the front of the QR code pattern about 0.5m, the data of the sample image can be scanned and read accurately. The experiment results show that the approach is feasible and effective.