Ismael Lopez-Juarez

Machine vision approach for robotic assembly

Purpose – Outcome with a novel methodology for online recognition and classification of pieces in robotic assembly tasks and its application into an intelligent manufacturing cell.Design/methodology/approach – The performance of industrial robots working in unstructured environments can be improved using visual perception and learning techniques. The object recognition is accomplished using an artificial neural network (ANN) architecture which receives a descriptive vector called CFD&POSE as the input. Experimental results were done within a manufacturing cell and assembly parts.Findings – Find this vector represents an innovative methodology for classification and identification of pieces in robotic tasks, obtaining fast recognition and pose estimation information in real time. The vector compresses 3D object data from assembly parts and it is invariant to scale, rotation and orientation, and it also supports a wide range of illumination levels.Research limitations/implications – Provides vision guidance...

On the design of intelligent robotic agents for assembly

Robotic agents can greatly be benefited from the integration of perceptual learning in order to monitor and adapt to changing environments. To be effective in complex unstructured environments, robots have to perceive the environment and adapt accordingly. In this paper it is discussed a biology inspired approach based on the adaptive resonance theory (ART) and implemented on an KUKA KR15 industrial robot during real-world operations (e.g. assembly operations). The approach intends to embed naturally the skill learning capability during manufacturing operations (i.e., within a flexible manufacturing system). The integration of machine vision and force sensing has been useful to demonstrate the usefulness of the cognitive architecture to acquire knowledge and to effectively use it to improve its behaviour. Practical results are presented, showing that the robot is able to recognise a given component and to carry out the assembly. Adaptability is validated by using different component geometry during assemblies and also through skill learning which is shown by the robots dexterity.

Knowledge acquisition and learning in unstructured robotic assembly environments

Mechanical assembly by robots has traditionally depended on simple sensing systems and the robot manufacturers programming language. However, this restricts the use of robots in complex manufacturing operations. An alternative to robot programming is the creation of self-adaptive robots based on the adaptive resonance theory (ART) artificial neural network (ANN). The research presented in this paper shows how robots can operate autonomously in unstructured environments. This is achieved by providing the robot with a primitive knowledge base (PKB) of the environment. This knowledge is gradually enhanced online based on the contact force information acquired during operations. The robot resembles a blindfold person performing the same task since no information is provided about the localisation of the fixed assembly component. The design of a novel neural network controller (NNC) based on the Fuzzy ARTMAP network and its implementation results on an industrial robot are presented, which validate the approach.

Learning manipulative skills with ART

The research reported in this paper is related to the creation of self-adapting robots that are capable of learning manipulative skills online. The investigation includes the design of a novel neural network controller (NNC), which is based on the adaptive resonance theory (ART) and a dynamic knowledge base, whose knowledge is regulated by specific assembly operations. A force/torque (F/T) sensor was attached to the robots wrist and this was the only information available to the NNC during the assembly operations, since the precise location of the components was unknown. The knowledge is enhanced online, based on the success in predicting the motion that reduces the constraint forces. The results demonstrate the generalisation capability of the NNC by learning the assembly of different part geometries using the same initial knowledge base. The learning time for a complete new operation was achieved in approximately 1 minute.

A hybrid learning method composed by the orthogonal least-squares and the back-propagation learning algorithms for interval A2-C1 type-1 non-singleton type-2 TSK fuzzy logic systems

The purpose of this paper is to present a hybrid learning method for interval A2-C1 type-1 non-singleton type-2 TSK fuzzy logic system that uses the recursive orthogonal least-squares algorithm to tune the type-1 consequent parameters, and the back-propagation algorithm to tune the interval type-2 antecedent parameters. Based on the combination of these two training algorithms the new hybrid learning method changes the interval type-2 fuzzy model parameters adaptively and minimizes the proposed error function as the new type-1 non-singleton input–output data pairs are processed. Its antecedent sets are interval type-2 fuzzy sets, its consequent sets are type-1 fuzzy sets, and its inputs are type-1 non-singleton fuzzy numbers with uncertain standard deviations. Comparison with the non-hybrid interval A2-C1 type-1 non-singleton type-2 Takagi–Sugeno–Kang fuzzy logic system that only uses the back-propagation algorithm for both antecedent and consequent parameter’s adaptation demonstrates that the proposed hybrid algorithm is a well-performing nonlinear adaptation that enables the interval type-2 fuzzy model to optimally match the nonlinear behavior of the process. The application of the interval type-2 fuzzy logic as adaptable predictor using the proposed hybrid learning method was constructed for the modeling and prediction of the transfer bar surface temperature in an industrial hot strip mill facility. Experimental results demonstrated that this method improves the temperature prediction performance of the interval A2-C1 type-1 non-singleton type-2 Takagi–Sugeno–Kang fuzzy logic system.

Using Object’s Contour, Form and Depth to Embed Recognition Capability into Industrial Robots

Robot vision systems can differentiate parts by pattern matching irrespective of part orientation and location. Somemanufacturers offer 3D guidance systems using robust vision and laser systems so that a 3D programmed point canbe repeated even if the part is moved varying its location, rotation and orientation within the working space. Despitethese developments, current industrial robots are still unable to recognize objects in a robust manner; that is, todistinguish an object among equally shaped objects taking into account not only the object’s contour but also its formand depth information, which is precisely the major contribution of this research. Our hypothesis establishes that it ispossible to integrate a robust invariant object recognition capability into industrial robots by using image features fromthe object’s contour (boundary object information), its form (i.e., type of curvature or topographical surfaceinformation) and depth information (from stereo disparity maps). These features can be concatenated in order to forman invariant vector descriptor which is the input to an artificial neural network (ANN) for learning and recognitionpurposes. In this paper we present the recognition results under different working conditions using a KUKA KR16industrial robot, which validated our approach.

Acquisition of welding skills in industrial robots

Purpose – The purpose of this paper was to propose a method based on an Artificial Neural Network and a real-time vision algorithm, to learn welding skills in industrial robotics. Design/methodology/approach – By using an optic camera to measure the bead geometry (width and height), the authors propose a real-time computer vision algorithm to extract training patterns and to enable an industrial robot to acquire and learn autonomously the welding skill. To test the approach, an industrial KUKA robot and a welding gas metal arc welding machine were used in a manufacturing cell. Findings – Several data analyses are described, showing empirically that industrial robots can acquire the skill even if the specific welding parameters are unknown. Research limitations/implications – The approach considers only stringer beads. Weave bead and bead penetration are not considered. Practical implications – With the proposed approach, it is possible to learn specific welding parameters despite of the material, type of ...

On-line incremental learning for unknown conditions during assembly operations with industrial robots

The assembly operation using industrial robots can be accomplished successfully in well-structured environments where the mating pair location is known in advance. However, in real-world scenarios there are uncertainties associated to sensing, control and modelling errors that make the assembly task very complex. In addition, there are also unmodeled uncertainties that have to be taken into account for an effective control algorithm to succeed. Among these uncertainties, it can be mentioned disturbances, backlash and aging of mechanisms. In this paper, a method to overcome the effect of those uncertainties based on the Fuzzy ARTMAP artificial neural network (ANN) to successfully accomplish the assembly task is proposed. Experimental work is reported using an industrial 6 DOF robot arm in conjunction with a vision system for part location and wrist force/torque sensing data for assembly. Force data is fed into an ANN evolving controller during a typical peg in hole (PIH) assembly operation. The controller uses an incremental learning mechanism that is solely guided by the sensed forces. In this article, two approaches are presented in order to compare the incremental learning capability of the manipulator. The first approach uses a primitive knowledge base (PKB) containing 16 primitive movements to learn online the first insertion. During assembly, the manipulator learns new patterns according to the learning criteria which turn the PKB into an enhanced knowledge base (EKB). During a second insertion the controller uses effectively the EKB and operation improves. The second approach employs minimum information (it contains only the assembly direction) and the process starts from scratch. After several operations, that knowledge base increases by including only the needed patterns to perform the insertion. Experimental results showed that the evolving controller is able to assemble the matting pairs enhancing its knowledge whenever it is needed depending on the part geometry and level of expertise. Our approach is demonstrated through several PIH operations with different tolerances and part geometry. As the robot’s expertise evolves, the PIH operation is carried out faster with shorter assembly trajectories.

On the Implementation of a Robotic Welding Process Using 3D Simulation Environment

This paper presents the simulation of an automated welding process based on an industrial robot KUKA KR16. The main objective is to integrate all devices needed to perform the simulation using Delmia V5 robotics and to avoid any possible mistake when implementing of the real workcell; for example, with physical dimensions, possible collisions, path planning, etc. The feasibility in the implementation can be determined trough 3D simulation of a manufacturing workcell to perform tasks type MIG welding.

Intelligent Task Level Planning for Robotic Assembly: Issues and Experiments

Today’s industrial robots use programming languages that do not allow learning and task knowledge acquisition and probably this is one of the reasons of its restricted used for complex task in unstructured environments. In this paper, results on the implementation of a novel task planner using a 6 DOF industrial robot as an alternative to overcome this limitation are presented. Different Artificial Neural Networks (ANN) models were assessed first to evaluate their learning capabilities, stability and feasibility of implementation in the planner. Simulations showed that the Adaptive Resonance Theory (ART) outperformed other connectionist models during tests and therefore this model was chosen. This work describes initial results on the implementation of the planner showing that the manipulator can acquire manipulative skills to assemble mechanical components using only few clues.