Stefano Tonello

Flexible 3D localization of planar objects for industrial bin-picking with monocamera vision system

In this paper, we present a robust and flexible vision system for 3D localization of planar parts for industrial robots. Our system is able to work with nearly any object with planar shape, randomly placed inside a standard industrial bin or on a conveyor belt. Differently from most systems based on 2D image analysis, which usually can manage parts disposed in single layers, our approach can estimate the 6 degrees of freedom (DoF) pose of planar objects from a single 2D image. The choice of a single camera solution makes our system cheaper and faster with respect to systems using expensive industrial 3D cameras, or laser triangulation systems, or laser range finders. Our system can work virtually with any planar piece, without changing the software parameters, because the input for the recognition and localization algorithm is the CAD data of the planar part. The localization software is based on a two step strategy: i) a candidates selection step based on a well-engineered voting scheme ii) a refinement and best match selection step based on a robust iterative optimize-and-score procedure. During this second step, we employ a novel strategy we called search-in-the-stack that avoids the optimization from being stuck on local minima (representing false positives) created when objects are almost regularly stacked. Our system is currently installed in seven real world industrial plants, with different setups, working with hundreds of different models and successfully guiding the manipulators to pick several hundreds of thousands of pieces per year. In the experiment section, we report statistics about our system at work in real production plants on more than 60000 cycles.

Robust Color-Based Skin Detection for an Interactive Robot

Detection of human skin in an arbitrary image is generally hard. Most color-based skin detection algorithms are based on a static color model of the skin. However, a static model cannot cope with the huge variability of scenes, illuminants and skin types. This is not suitable for an interacting robot that has to find people in different rooms with its camera and without any a priori knowledge about the environment nor of the lighting. In this paper we present a new color-based algorithm called VR filter. The core of the algorithm is based on a statistical model of the colors of the pixels that generates a dynamic boundary for the skin pixels in the color space. The motivation beyond the development of the algorithm was to be able to correctly classify skin pixels in low definition images with moving objects, as the images grabbed by the omnidirectional camera mounted on the robot. However, our algorithm was designed to correctly recognizes skin pixels with any type of camera and without exploiting any information on the camera. In the paper we present the advantages and the limitations of our algorithm and we compare its performances with the principal existing skin detection algorithms on standard perspective images.

WorkCellSimulator: a 3d simulator for intelligent manufacturing

This paper presents WorkCellSimulator, a software platform that allows to manage an environment for the simulation of robot tasks. It uses the most advanced artificial intelligence algorithms in order to define the production process, by controlling one or more robot manipulators and machineries present in the work cell. The main goal of this software is to assist the user in defining customized production processes which involve specific automated cells. It has been developed by IT+Robotics, a spin-off company of the University of Padua, founded in 2005 from the collaboration between young researchers in the field of Robotics and a group of professors from the Department of Information Engineering, University of Padua.

3DComplete: Efficient completeness inspection using a 2.5D color scanner

In this paper, we present a low-cost and highly configurable quality inspection system capable of capturing 2.5D color data, created using off-the-shelf machine vision components, open-source software libraries, and a combination of standard and novel algorithms for 2.5D data processing. The system uses laser triangulation to capture 3D depth, in parallel with a color camera and a line light projector to capture color texture, which are then combined into a color 2.5D model in real-time. Using many examples of completeness inspection tasks that are extremely difficult to solve with current 2D-based methods, we demonstrate how the 2.5D images and point clouds generated by our system can be used to solve these complex tasks effectively and efficiently. Our system is currently being integrated into a real production environment, showing that completeness inspection incorporating 3D technology can be readily achieved in a short time at low costs.

Efficient Completeness Inspection Using Real-Time 3D Color Reconstruction with a Dual-Laser Triangulation System

In this chapter, we present the final system resulting from the European Project “3DComplete” aimed at creating a low-cost and flexible quality inspection system capable of capturing 2.5D color data for completeness inspection. The system uses a single color camera to capture at the same time 3D data with laser triangulation and color texture with a special projector of a narrow line of white light, which are then combined into a color 2.5D model in real time. Many examples of completeness inspection tasks are reported, which are extremely difficult to analyze with the state-of-the-art 2D-based methods. Our system has been integrated into a real production environment, showing that completeness inspection incorporating 3D technology can be readily achieved in a short time at low costs.

Smart Check 3D: An Industrial Inspection System Combining 3D Vision with Automatic Planning of Inspection Viewpoints

In this chapter, we describe an industrial inspection system composed by a 3D vision system, mounted on a manipulator robot arm , able to perform quality and completeness inspection on a complex solid part. The novelty of the system is in the deep integration among three software modules: the visual inspection system , the 3D simulation software, and the motion planning engine of the manipulator robot. This enables an automatic off-line programming of the robot path by specifying in the system the desired inspection tasks. The system automatically generates the needed points of view in order to perform 3D reconstruction and automatic visual inspection. Moreover, the motion planning system can reorder the inspection points in order to optimize the inspection cycle time. The core of this system was developed in the European Project “Thermobot,” and currently, it is been engineered to be deployed in an industrial production plant.

A Constraint Based Motion Optimization System for Quality Inspection Process Improvement

This paper presents a motion optimization system for an industrial quality inspection process where a vision device coupled with a manipulator robot arm is able to perform quality and completeness inspection on a complex solid part. In order to be deployed in an industrial production plant, the proposed system has been engineered and integrated as a module of an offline simulator, called WorkCellSimulator, conceived to simulate robot tasks in industrial environments. The novelty of the paper concerns the introduction of time constraints into the motion planning algorithms. Then, these algorithms have been deeply integrated with artificial intelligence techniques in order to optimize the inspection cycle time. This integration makes the application suitable for time-constrained processes like, e.g., autonomous industrial painting or autonomous thermo-graphic detection of cracks in metallic and composite materials.

A planning system to manipulate metal sheets within a workcell simulator

Our planner allows the extraction of the metal sheets from a press-brake. Since the solution must respect safety standard, this operation can be very complex. For this reason it is useful to usean automatic system to plan these operations. Thus, we have developed a specific planning algorithm, called EFT (Exit from Tools), that utilizes a hybrid approach, combining a probabilistic roadmap with standard operations on the Cartesian configuration space. In addition, our algorithm has the important feature to be automatically reconfigurable. We performed several tests solving a set of critical real cases by WorkCellSimulator, an industrial software system for simulating manufacturing processing.

A robotic sculpture speaking to people

This video shows the interactive robotic sculpture conceived and realized by the artist Albano Guatti. The robotic part was totally developed by people at the IAS-lab and at IT+Robotics according to Guattis concept. This work is the result of the meeting of robotics and art. The video shows that the robot is able to locate people in the environment, to navigate toward them avoiding the obstacles and to approach them as a polite waiter will do. In fact, the sculpture represents a waiter and a waitress (actually their suits, the statue do not have a body, they represent empty suits). The waiters chat among them (by play different pre-recorded voice files) while wandering in the environment. Once the robot locates a person and get close to her, it asks the customer one out of different pre-recorded questions. The more likely is: Would you like a drink?, that is the title of the sculpture.