Jörg Brodersen

High-performance camera module for fast quality inspection in industrial printing applications

Today, printing products which must meet highest quality standards, e.g., banknotes, stamps, or vouchers, are automatically checked by optical inspection systems. Typically, the examination of fine details of the print or security features demands images taken from various perspectives, with different spectral sensitivity (visible, infrared, ultraviolet), and with high resolution. Consequently, the inspection system is equipped with several cameras and has to cope with an enormous data rate to be processed in real-time. Hence, it is desirable to move image processing tasks into the camera to reduce the amount of data which has to be transferred to the (central) image processing system. The idea is to transfer relevant information only, i.e., features of the image instead of the raw image data from the sensor. These features are then further processed. In this paper a color line-scan camera for line rates up to 100 kHz is presented. The camera is based on a commercial CMOS (complementary metal oxide semiconductor) area image sensor and a field programmable gate array (FPGA). It implements extraction of image features which are well suited to detect print flaws like blotches of ink, color smears, splashes, spots and scratches. The camera design and several image processing methods implemented on the FPGA are described, including flat field correction, compensation of geometric distortions, color transformation, as well as decimation and neighborhood operations.

High-Performance Smart Cameras

This chapter describes a high performance smart linescan camera developed to be used in quality inspection systems for high grade printed matter. Such an inspection system has to meet many demanding requirements as very high inspection resolution (better than 100 m) at high production speeds (up to 20 m/s). A total data rate of several Gigabytes per second has to be processed continuously. Under consideration of reasonable system costs, the term high performance is related to the most critical design factors: resolution, speed, throughput, and inspection quality. The smart camera approach overcomes the bottleneck between high speed imager and remote image processing system. Powerful processing units like high end field programmable gate arrays and digital signal processors are integrated into the camera housing. Thereby, it enables outstanding quality inspection in terms of accuracy and economical feasibility. Architectural elements covered in this chapter include the multiple exposure method, which allows the design of high speed linescan cameras based on area scan images, high throughput image processing, high level image processing, as well as a fiberoptic-based 10 Gbit Ethernet used as camera interface. The chapter concludes with an outlook to future developments in the field of high performance smart cameras.

Architecture for hardware driven image inspection based on FPGAs

Requirements for contemporary print inspection systems for industrial applications include, among others, high throughput, examination of fine details of the print, and inspection from various perspectives and different spectral sensitivity. Therefore, an optical inspection system for such tasks has to be equipped with several high-speed/high-resolution cameras, each acquiring hundreds of megabytes of data per second. This paper presents an inspection system which meets the given requirements by exploiting data parallelism and algorithmic parallelism. This is achieved by using complex field-programmable gate arrays (FPGA) for image processing. The scalable system consists of several processing modules, each representing a pair of a FPGA and a digital signal processor. The main chapters of the paper focus on the functionality implemented in the FPGA. The image processing algorithms include flat-field correction, lens distortion correction, image pyramid generation, neighborhood operations, a programmable arithmetic unit, and a geometry unit. Due to shortage of on-chip memory, a multi-port memory concept for buffering streams of data between off-chip and on-chip memories is used. Furthermore, performance measurements of the processing module are presented.

High-speed line-scan camera with multi-line CMOS color sensor

A new camera technology for high-speed color scanning is presented that is based on a novel multi-line CMOS color image sensor. A pixel matrix consisting of multiple red-, green-, and blue-sensitive lines allows acquisition of dense RGB color information at every pixel location including the possibility of improving the signal to noise ratio by summing up multiple exposures in the analog domain of the chip. Therefore, compared to common cameras with Bayer filters, the acquired image data are more robust against color artifacts like Moiré effects. Consequently, images with good quality are possible even at high object speeds, where lighting power may become limiting otherwise. This work describes an algorithm for improving low-light image quality by accumulating multiple exposures acquired by a multi-line color pixel matrix. Furthermore it is shown how this algorithms implementation is effectively split up into an FPGA part and an image sensor part. Finally, the increase of responsivity due to this algorithm is shown, which is regarded as a key requisite for implementing high-speed cameras.

Advanced real-time bus system for concurrent data paths used in high-performance image processing

In this paper we present a new bus protocol satisfying extreme real time demands. It has been applied to a high performance quality inspection system which can involve up to eight sensors of various types. Thanks to the modular configuration this multi-sensor inspection system acts on the outside as a single sensor image processing system. In general, image processing systems comprise three basic functions (i) image acquisition, (ii) image processing and (iii) output of processed data. The data transfers for these three fundamental functions can be accomplished either by individual bus systems or by a single bus. In case of using a single bus the system complexity of the implementation, i.e. Development of protocols, hardware employment and EMC technical considerations, is far smaller. An important goal of the new protocol design is to support extremely fast communication between individual processing modules. For example, the input data (image acquisition) is transferred in real time to individual processing modules. Concurrent to this communication the processed data are being transferred to the output module. Therefore, the key function of this protocol is to realize concurrent data paths (data rates over 1.2 Gbit/s) by using principles of pipeline architectures and methods of time division multiplex. Moreover, the new bus protocol enables concurrent data transfers via a single bus system. In this paper the function of the new bus protocol including hardware layout and innovative bus arbiter are described in details.

FPGA based image processing for optical surface inspection with real time constraints

Today, high-quality printing products like banknotes, stamps, or vouchers, are automatically checked by optical surface inspection systems. In a typical optical surface inspection system, several digital cameras acquire the printing products with fine resolution from different viewing angles and at multiple wavelengths of the visible and also near infrared spectrum of light. The cameras deliver data streams with a huge amount of image data that have to be processed by an image processing system in real time. Due to the printing industry’s demand for higher throughput together with the necessity to check finer details of the print and its security features, the data rates to be processed tend to explode. In this contribution, a solution is proposed, where the image processing load is distributed between FPGAs and digital signal processors (DSPs) in such a way that the strengths of both technologies can be exploited. The focus lies upon the implementation of image processing algorithms in an FPGA and its advantages. In the presented application, FPGAbased image-preprocessing enables real-time implementation of an optical color surface inspection system with a spatial resolution of 100 μm and for object speeds over 10 m/s. For the implementation of image processing algorithms in the FPGA, pipeline parallelism with clock frequencies up to 150 MHz together with spatial parallelism based on multiple instantiations of modules for parallel processing of multiple data streams are exploited for the processing of image data of two cameras and three color channels. Due to their flexibility and their fast response times, it is shown that FPGAs are ideally suited for realizing a configurable all-digital PLL for the processing of camera line-trigger signals with frequencies about 100 kHz, using pure synchronous digital circuit design.

Ultrafast 2K line-scan sensor for industrial inspection applications

Optical inspection systems require fast image acquisition at significantly enhanced resolution when utilized for advanced machine vision tasks. Examples are quality assurance in print inspection, printed circuit board inspection, wafer inspection, real-time surveillance of railroad tracks, and in-line monitoring in flat panel fabrication lines. Ultra-highspeed is an often demanded feature in modern industrial production facilities, especially, where it comes to high volume production. A novel technology in this context is the new high-speed sensor for line-scan camera applications with unmatched line rates up to 200 kHz (tri-linear RGB) and 600 kHz (b/w), presented in this paper. At this speed, the multiline- scan sensor provides full color images with, e.g., a spatial resolution of 50 μm at a transport speed of 10 m/s. In contrast to conventional Bayer pattern or three-chip approaches, the sensor presented here utilizes the tri-linear principle, where the color filters are organized line-wise on the chip. With almost 100% fill-factor, the tri-linear technology assures high image quality because of its robustness against aliasing and Moiré effects leading to improved inspection quality, less false positives and thus less waste in the production lines.