Matthias Breier

Heightmap generation for printed circuit boards (PCB) using laser triangulation for pre-processing optimization in industrial recycling applications

Electronic devices are nowadays an integral part of our everyday lives. The number of discarded electronical items has grown significantly over the last years. As the amount of precious materials used in the manufacturing of these devices has increased over the last years recycling of these devices is becoming more and more important. Currently the processes to regain some of these precious materials like gold, copper, scarce elements etc. do not differentiate much the input material composition. To enhance these processes as much information about the input material as possible is needed. Especially information used for the classification of the processed printed circuit boards (PCBs) is important as PCBs have been used extensively in electronic devices. One key aspect of this classification process is the acquisition of geometrical properties of the processed PCBs. In this paper employing laser triangulation to gain the height profile of PCBs is discussed. The basic principles of laser triangulation are introduced as well as several laser line detection algorithms. The variability of shapes of the components mounted on PCBs is limited. Due to this limitation the correction of geometrical distortions (called rise extension slope contraction (RESC)) resulting in a systematic error is feasible and discussed in this paper as well. Finally all algorithms presented are evaluated in a comprehensive testing environment and the results are shown in the end.

Active contours for localizing polyps in colonoscopic NBI image data

Colon cancer is the third most common type of cancer in the United States of America. Every year about 140,000 people are newly diagnosed with colon cancer. Early detection is crucial for a successful therapy. The standard screening procedure is called colonoscopy. Using this endoscopic examination physicians can find colon polyps and remove them if necessary. Adenomatous colon polyps are deemed a preliminary stage of colon cancer. The removal of a polyp, though, can lead to complications like severe bleedings or colon perforation. Thus, only polyps diagnosed as adenomatous should be removed. To decide whether a polyp is adenomatous the polyps surface structure including vascular patterns has to be inspected. Narrow-Band imaging (NBI) is a new tool to improve visibility of vascular patterns of the polyps. The first step for an automatic polyp classification system is the localization of the polyp. We investigate active contours for the localization of colon polyps in NBI image data. The shape of polyps, though roughly approximated by an elliptic form, is highly variable. Active contours offer the flexibility to adapt to polyp variation well. To avoid clustering of contour polygon points we propose the application of active rays. The quality of the results was evaluated based on manually segmented polyps as ground truth data. The results were compared to a template matching approach and to the Generalized Hough Transform. Active contours are superior to the Hough transform and perform equally well as the template matching approach.

Text recognition for information retrieval in images of printed circuit boards

In order to achieve an efficient and environment-friendly recycling of printed circuit boards (PCBs), a comprehensive analysis of their material composition is essential. Besides sophisticated chemical and physical methods for a direct material analysis, an indirect method based on information retrieval provides a less costly and more efficient alternative. During the process of information retrieval, PCBs and their components need to be recognized based on their appearance and the corresponding text information. Their material composition is then available through a pre-established database. Therefore, a practical text recognition is necessary for a successful data analysis prior to PCB recycling. Our paper is focusing on two key aspects of text recognition: binarization and final recognition of text objects using optical character recognition (OCR) engines. For binarization of text contents, a novel local thresholding method using an adaptive window size along with background estimation is presented. Several state-of-the-art algorithms and the proposed method were evaluated for comparing their binarization performance on text objects in PCB images. With respect to a data set containing manually created references, our novel method provides superior results. Furthermore, in contrast to previous work on text recognition, an additional evaluation of available open source OCR engines was conducted to asses technical limitations of OCR applications. We show that the quality of text recognition can be significantly improved if the binarization approach accounts for these technical limitations of OCR software. The presented method and results are expected to provide improved OCR performance also in other applications.

Segmentation and classification of THCs on PCBAs

The dramatic increase of electronic waste requires automatic recycling, including technologies from machine vision. A framework for segmentation and classification of THC (through-hole components) mounted on PCBAs is presented, using both RGB and depth frames from the Kinect sensor by Microsoft. A segmentation approach, combining local and global features in a flexible manner, is shown to optimize a freely definable cost function globally. We interleave segmentation and classification as we form the final components using a simple, yet robust shape model.

SMD segmentation for automated PCB recycling

Due to the complex structure and the great variability of printed circuit boards (PCBs), an accurate analysis of their material composition and consequently an efficient recycling is currently hardly possible. Facing these problems, we proposed an automated PCB recycling system with a core component for the information retrieval. As a first step towards automated recycling based on a comprehensive analysis of PCBs, the segmentation of surface-mounted devices (SMDs) was presented in this paper. Inspired by the technical background of SMDs, they were divided into two main groups: 1. small devices, such as resistors, capacitors, and 2. integrated circuits (ICs). Then assembly print-based segmentation and color distribution-based segmentation were carried out for the two groups respectively. In comparison to device template-based and solder joint-based approaches, our approach is more flexible and more reliable with respect to the variability of devices and the erosion or soiling of solder joints. As confirmed by the evaluation, satisfying results were obtained. The approach was also evaluated for different imaging conditions so that improved performance can be achieved using appropriate imaging settings.

Accurate laser triangulation using a perpendicular camera setup to assess the height profile of PCBs

Recycling of printed circuit boards (PCBs) is still subject to current research. For identifying valuable materials in electronical components on the PCBs the height profile of PCBs is a important feature. Laser triangulation with a perpendicular laser is a classical approach to generate height profiles. However, in such a setup it is not possible to acquire different modalities such as panorama images and the height profile simultaneously. The solution is to use a setup in which the laser is slanted and the camera is perpendicular to the PCBs. A new triangulation approach employing raycasting and intersecting a laser plane and a ray of sight is needed to retain accuracy of the height profile generation. Evaluation results show that the accuracy of height measurement with an average error of 0.077 mm is still sufficient for detecting small surface mounted devices on PCBs with a minimal height of about 0.2 mm.

Localizing components on printed circuit boards using 2D information

Information retrieval based re-configurable recycling is crucial for solving the environmental and social problems raised by the dramatically increasing amount of waste printed circuit boards. However, the indispensable localization of mounted components for the desired information retrieval is an unsolved problem for state-of-the-art techniques so far. In this paper, for the first time, a feasible solution addressing this problem by using general 2D information is proposed. In the form of a sophisticated modular analysis pipeline, complementary information sources are combined appropriately for capturing the sought objects. With a novel background removal algorithm and an additional candidate validation step, the proposed approach significantly outperforms state-of-the-art localization approaches in case of PCB images. Moreover, with respect to the intended trade-off between complexity and performance, it is quite straightforward to re-construct the deployed workflow accordingly by considering the knowledge obtained in a comprehensive evaluation of diverse combination possibilities.

Color filter arrays revisited — Evaluation of Bayer pattern interpolation for industrial applications

Modern industrial cameras mainly use the Bayer pattern as color filter array (CFA). However, this filtering limits the resolution of the color space. As interpolation methods cannot reconstruct the original image perfectly, they have to be optimized for a specific application. Therefore, the interpolation should match the purpose of the image processing system. Many standard algorithms are optimized for the subjective impression of a human observer which is not necessarily ideal for industrial applications. This paper introduces a new methodology for evaluating interpolation algorithms in industrial applications. For this purpose, a new dataset based on printed circuit boards as representatives for objects in industrial application is introduced. It shows many distinct features which are typical for industrial scenarios such as high contrast object edges, highly reflective materials and low variety of surface colors. Furthermore, two new error measures based on edge accuracy are presented which are tailored to many measurement tasks which employ edges. It can be shown in an evaluation of common CFA interpolation algorithms that these new error measures are better suited to identify the best interpolation algorithm for retaining edge accuracy than conventional error measures.

Rotation estimation for printed circuit board recycling

Electronic devices are nowadays an integral part of everyday life. The number of discarded electronic items has grown significantly over the last years. Due to the amount of precious materials used in the manufacturing of these devices recycling of electronic devices is becoming more and more important. Currently, the processes to regain some of these precious materials such as gold, copper, scarce elements etc. are not able to differentiate electronic waste according to its material composition. To enhance these processes, as much information as possible needs to be retrieved per electronic waste item. In particular, information used for the classification of the processed printed circuit boards (PCBs) is important as PCBs are extensively used in electronic devices. One key aspect of this classification process is the estimation of the orientation of the PCBs in this process (e.g. on a conveyor belt). In this paper typical properties of PCBs with respect to orientation estimation are introduced and three different orientation estimation algorithms are evaluated and discussed. These approaches comprise the Hough transform, structure tensors and orientation estimation via Fourier transform. Finally, all presented algorithms are evaluated based on images of PCBs with known orientation. The results indicate, that the Hough transform yields the best results for completely visible PCBs while structure tensors performed best on partially visible PCBs.

Vision-based Auto-Teaching for automated PCB depaneling

Machines for automated PCB depaneling have greatly improved the industrial production efficiency of electronic products. But the preparation for automated depaneling could be very complex and time-consuming. In this paper, we propose a novel systematic solution for this problem. Using a vision-based assistant system, called ”Auto-Teaching”, all connection tabs that should be milled off panels are detected automatically. Highly accurate milling curves are generated with respect to the geometry of the corresponding tabs. Then they could be easily converted into CNC code which drive milling cutters later. Moreover, placement suggestions of supporting pins used to fix panels are obtained. All image analysis functions are implemented in C++ and optimized to meet the minimal hardware and time requirements. Thus the whole process of the preparation is simplified for users and the PCB manufactures can benefit from reduced idle running of machines and labor costs.