Avinash Burla

Genetic programming applied to automatic algorithm design in multi-scale inspection systems

In recent years image-processing has become a central part of optical inspection and measurement systems. Typically, after measuring the given specimen by utilizing a suitable sensor, image-processing algorithms are used to detect dedicated features such as surface defects. These algorithms are usually designed, optimized, and tested by an image-processing expert according to the task specifications. A methodology (based on genetic programming) is presented to automatically generate, optimize, and test such algorithms without the necessity of an image-processing expert. We also present several examples of inspection tasks to support the concept. For efficiency, an automated multi-scale multi-sensor inspection strategy is employed.

Fourier descriptors for defect indication in a multiscale and multisensor measurement system

Efficient inspection of an object for deformations and defects requires comparison with an existing real or simulated reference model. Fourier descriptor (FDs) based shape analysis is an effective method for describing a shape using the Fourier transform. This shape representation can be easily modified to achieve shift, rotation, and scale invariance. We propose two new methods, namely the ring sampling and the spiral sampling methods, which enable the usage of FDs in order to detect defects on micro-optical elements like microlens arrays. As an example the measurement data obtained from a confocal microscope has been used to show the effectiveness of the two approaches for both indicating and detecting surface defects. Microlens arrays with different types of defects including global (deformed lenses causing aberrations) and local defects (scratches) were simulated using a confocal microscopy simulation tool to test the reliability of the methods. A classifier differentiates between global and local defective lenses. In order to represent other kinds of objects using FDs, the methods can be easily modified or extended. The whole process has been implemented into an automated multiscale multisensor measurement system, which focuses on fast detection of defects on micro-optical and microelectromechanical systems.

Sensor and actuator conditioning for multiscale measurement systems on example of confocal microscopy

Multi-scale measurement systems utilise multiple sensors which differ in resolution and measurement field to pursue an active exploration strategy. The different sensor scales are linked by indicator algorithms for further measurement initiation. A major advantage of this strategy is a reduction of the conflict between resolution, time and field. This reduction is achieved by task specific conditioning of sensors, indicator algorithms and actuators using suitable uncertainty models. This contribution is focused on uncertainty models of sensors and actuators using the example of a prototype multi-scale measurement system. The influence of the sensor parameters, object characteristics and measurement conditions on the measurement reliability is investigated exemplary for the middle-scale sensor, a confocal microscope.

Fast hologram computation and aberration control for holographic tweezers

Holographic tweezers offer a very versatile tool in many trapping applications. Compared to tweezers working with acousto optical modulators or using the generalized phase contrast, holographic tweezers so far were relatively slow. The computation time for a hologram was much longer than the modulation frequency of the modulator. To overcome this drawback we present a method using modified algorithms which run on state of the art graphics boards and not on the CPU. This gives the potential for a fast manipulation of many traps, for cell sorting for example, as well as for a real-time aberration control. The control of aberrations which can vary spatially or temporally is relevant to many real world applications. This can be accomplished by applying an iterative approach based on image processing.

Assistant systems for efficient multiscale measurement and inspection

Optical inspection systems constitute hardware components (e.g. measurement sensors, lighting systems, positioning systems etc.) and software components (system calibration techniques, image processing algorithms for defect detection and classification, data fusion, etc.). Given an inspection task choosing the most suitable components is not a trivial process and requires expert knowledge. For multiscale measurement systems, the optimization of the measurement system is an unsolved problem even for human experts. In this contribution we propose two assistant systems (hardware assistant and software assistant), which help in choosing the most suitable components depending on the task considering the properties of the object (e.g. material, surface roughness, etc.) and the defects (e.g. defect types, dimensions, etc.). The hardware assistant system uses general rules of thumb, sensor models/simulations and stored expert knowledge to specify the sensors along with their parameters and the hierarchy (if necessary) in a multiscale measurement system. The software assistant system then simulates many measurements with all possible defect types for the chosen sensors. Artificial neural networks (ANN) are used for pre-selection and genetic algorithms are used for finer selection of the defect detection algorithms along with their optimized parameters. In this contribution we will show the general architecture of the assistant system and results obtained for the detection of typical defects on technical surfaces in the micro-scale using a multiscale measurement system.

Verlässlichkeitsanalyse von Indikator-Funktionen in einem Automatisierten Multiskalen-Messsystem

Zusammenfassung Innerhalb automatisierter multiskaliger Messsysteme werden im Allgemeinen verschiedene Sensoren in verschiedenen Skalen zur Vermessung/Inspektion verwendet. Ausgehend von der niedrigstauflösenden Skale werden Indikationen für Defekte mittels Indikatoralgorithmen gefunden. Basierend auf diesen Defektindikationen werden Unterregionen für eine Vermessung mit höherer Auflösung spezifiziert. Dieser Vorgang wird iterativ bis zur klaren Defekterkennung durchgeführt. Grundlage einer entsprechenden Vorgehensweise ist die Verwendung zuverlässiger Indikatoralgorithmen. Entsprechende Algorithmen für die Vermessung von Mikrooptiken werden vorgestellt und hinsichtlich ihrer Zuverlässigkeit analysiert. Abstract In an Automated Multiscale Measurement System (AMMS) different sensors at different scales are used for measurement and inspection of a given specimen. Starting from a coarse scale the indicators are detected by the indicator detection algorithms. Based on these indicators the regions of interest are specified for finer measurements in the next scale. This process continues until all indicators are classified. Hence reliable indicator detection algorithms are important. We present suitable indicator detection algorithms for microlens arrays and analyse their reliability.

Automated multiscale measurement system for MEMS characterisation

In former publications we presented an automated multiscale measurement system (AMMS) based on an adaptable active exploration strategy. The system is armed with several sensors linked by indicator algorithms to identify unresolved defects and to trigger finer resolved measurements. The advantage of this strategy in comparison to single sensor approaches is its high flexibility which is used to balance the conflict between measurement range, resolution and duration. For an initial proof of principle we used the system for inspection of microlens arrays. An even higher challenge for inspection systems are modern micro electro-mechanical systems (MEMS). MEMS consist of critical functional components which range from several millimeters down to micrometers and typically have tolerances in sub-micron scale. This contribution is focused on the inspection of MEMS using the example of micro calibration devices. This new class of objects has completely different surface characteristics and features hence it is necessary to adapted the components of the AMMS. Typical defects found on calibration devices are for example broken actuator combs and springs, surface cracks or missing features. These defects have less influence on the optical properties of the surface and the MEMS surface generates more complex intensity distributions in comparison microlense arrays. At the same time, the surface features of the MEMS have a higher variety and less periodicity which reduce the performance of currently used algorithms. To meet these requirements, we present new indicator algorithms for the automated analysis of confocal as well as conventional imaging data and show initial multiscale inspection results.

Extrinsic calibration of a fringe projection sensor based on a zoom stereo microscope in an automatic multiscale measurement system

Multi scale systems offer the opportunity to balance the conflict between execution time, measurement volume and resolution for the inspection of highly complex surface profiles. An example of such a task is the inspection of gears. At first, the coarse position and form of the specimen is registered by a sensor measuring with comparatively low resolution but a large field of view. Possible defects near to the resolution limit are indicated and new regions of interest for higher resolved measurements are identified. As prerequisite for a successful multi-scale inspection, every sampled data set, acquired in different scales and at varying positions, must be registered in one global data model. This is only possible if the extrinsic coordinate transform from the sensors internal coordinate system to the common, global coordinate system of the inspected object and its uncertainties are known. In this paper, we present an approach for the extrinsic calibration using the example of a multi-zoom fringe projection sensor mounted on a multi-axes measurement system. Finally we show the measurement result of a gear, where several sampled patches are merged together into one point cloud with the aid of the presented calibration.

Rapid ideal template creation for the inspection of MEMS based on self-similarity characteristics

Surface metrology of MEMS requires high resolution sensors due to their fine structures. An automated multiscale measurement system with multiple sensors at multiple scales enables fast acquisition of the surface data by utilizing high resolution sensors only at locations required. We propose a technique that depends on the fact that often MEMS have features (e.g. combs) repeating across the surface. These features can be segmented and fused to generate an ideal template. We present an automated similarity search approach based on feature detection, rotation invariant matching, and sum of absolute differences to find similar structures on the specimen. Then, similar segments are fused and replaced in the original image to generate an ideal template.

Fast hologram computation for holographic tweezers

We have shown that it is possible to accelerate considerably the computation of phase-only Fourier holograms by using a consumer graphics board (MSI 6800GT) instead of the ordinary CPU. Our fastest CPU solution (Pentium 4 @ 3.0 GHz) — using handcoded assembly code together with the multimedia extensions SSE — was outperformed by a factor of more than thirty resulting in an average performance of 14.1 GFlops for 100 doughnuts. The presented algorithm can be used for the computation of holograms for an arbitrary (up to 250) number of traps located at different positions in three dimensions and having independent trapping potentials.