Svorad Štolc

Analysis of optically variable devices using a photometric light-field approach

Diffractive Optically Variable Image Devices (DOVIDs), sometimes loosely referred to as holograms, are popular security features for protecting banknotes, ID cards, or other security documents. Inspection, authentication, as well as forensic analysis of these security features are still demanding tasks requiring special hardware tools and expert knowledge. Existing equipment for such analyses is based either on a microscopic analysis of the grating structure or a point-wise projection and recording of the diffraction patterns. We investigated approaches for an examination of DOVID security features based on sampling the Bidirectional Reflectance Distribution Function (BRDF) of DOVIDs using photometric stereo- and light-field-based methods. Our approach is demonstrated on the practical task of automated discrimination between genuine and counterfeited DOVIDs on banknotes. For this purpose, we propose a tailored feature descriptor which is robust against several expected sources of inaccuracy but still specific enough for the given task. The suggested approach is analyzed from both theoretical as well as practical viewpoints and w.r.t. analysis based on photometric stereo and light fields. We show that especially the photometric method provides a reliable and robust tool for revealing DOVID behavior and authenticity.

Depth and all-in-focus images obtained by multi-line-scan light-field approach

We present a light-field multi-line-scan image acquisition and processing system intended for the 2.5/3-D inspection of fine surface structures, such as small parts, security print, etc. in an industrial environment. The system consists of an area-scan camera, that allows for a small number of sensor lines to be extracted at high frame rates, and a mechanism for transporting the inspected object at a constant speed. During the acquisition, the object is moved orthogonally to the camera’s optical axis as well as the orientation of the sensor lines. In each time step, a predefined subset of lines is read out from the sensor and stored. Afterward, by collecting all corresponding lines acquired over time, a 3-D light field is generated, which consists of multiple views of the object observed from different viewing angles while transported w.r.t. the acquisition device. This structure allows for the construction of so-called epipolar plane images (EPIs) and subsequent EPI-based analysis in order to achieve two main goals: (i) the reliable estimation of a dense depth model and (ii) the construction of an all-in-focus intensity image. Beside specifics of our hardware setup, we also provide a detailed description of algorithmic solutions for the mentioned tasks. Two alternative methods for EPI-based analysis are compared based on artificial and real-world data.

Depth and all-in-focus imaging by a multi-line-scan light-field camera

Abstract. We present a multi-line-scan light-field image acquisition and processing system designed for 2.5/3-D inspection of fine surface structures in industrial environments. The acquired three-dimensional light field is composed of multiple observations of an object viewed from different angles. The acquisition system consists of an area-scan camera that allows for a small number of sensor lines to be extracted at high frame rates, and a mechanism for transporting an inspected object at a constant speed and direction. During acquisition, an object is moved orthogonally to the camera’s optical axis as well as the orientation of the sensor lines and a predefined subset of lines is read out from the sensor at each time step. This allows for the construction of so-called epipolar plane images (EPIs) and subsequent EPI-based depth estimation. We compare several approaches based on testing a set of slope hypotheses in the EPI domain. Hypotheses are derived from block matching, namely the sum of absolute differences, modified sum of absolute differences, normalized cross correlation, census transform, and modified census transform. Results for depth estimation and all-in-focus image generation are presented for synthetic and real data.

On Optimal Illumination for DOVID Description Using Photometric Stereo

Diffractive optically variable image devices DOVIDs are popular security features used to protect security documents such as banknotes, ID cards, passports, etc. Nevertheless, checking authenticity of these security features on both user as well as forensic level still remains a challenging task, requiring sophisticated hardware tools and expert knowledge. Based on a photometric acquisition setup comprised of 32 illumination sources from different directions and a recently proposed descriptor capturing the illumination dependent behavior, we investigate the information content, illumination pattern shape and clustering properties of the descriptor. We studied shape and discriminative power of reduced illumination configurations for the task of discrimination applied to DOVIDs using a sample of Euro banknotes.

Multi-view line-scan inspection system using planar mirrors

We demonstrate the design, setup, and results for a line-scan stereo image acquisition system using a single area- scan sensor, single lens and two planar mirrors attached to the acquisition device. The acquired object is moving relatively to the acquisition device and is observed under three different angles at the same time. Depending on the specific configuration it is possible to observe the object under a straight view (i.e., looking along the optical axis) and two skewed views. The relative motion between an object and the acquisition device automatically fulfills the epipolar constraint in stereo vision. The choice of lines to be extracted from the CMOS sensor depends on various factors such as the number, position and size of the mirrors, the optical and sensor configuration, or other application-specific parameters like desired depth resolution. The acquisition setup presented in this paper is suitable for the inspection of a printed matter, small parts or security features such as optical variable devices and holograms. The image processing pipeline applied to the extracted sensor lines is explained in detail. The effective depth resolution achieved by the presented system, assembled from only off-the-shelf components, is approximately equal to the spatial resolution and can be smoothly controlled by changing positions and angles of the mirrors. Actual performance of the device is demonstrated on a 3D-printed ground-truth object as well as two real-world examples: (i) the EUR-100 banknote - a high-quality printed matter and (ii) the hologram at the EUR-50 banknote { an optical variable device.

Binary descriptor-based dense line-scan stereo matching

We present a line-scan stereo system and descriptor-based dense stereo matching for high-performance vision applications. The stochastic binary local descriptor (STABLE) descriptor is a local binary descriptor that builds upon the principles of compressed sensing theory. The most important properties of STABLE are the independence of the descriptor length from the matching window size and the possibility that more than one pair of pixels contributes to a single-descriptor bit. Individual descriptor bits are computed by comparing image intensities over pairs of balanced random subsets of pixels chosen from the whole described area. On a synthetic as well as real-world examples, we demonstrate that STABLE provides competitive or superior performance than other state-of-the-art local binary descriptors in the task of dense stereo matching. The real-world example is derived from line-scan binocular stereo imaging, i.e., two line-scan cameras are observing the same object line and 2-D images are generated due to relative motion. We show that STABLE performs significantly better than the census transform and local binary patterns (LBP) in all considered geometric and radiometric distortion categories to be expected in practical applications of stereo vision. Moreover, we show as well that STABLE provides comparable or better matching quality than the binary robust-independent elementary features descriptor. The low computational complexity and flexible memory footprint make STABLE well suited for most hardware architectures. We present quantitative results based on the Middlebury stereo dataset as well as illustrative results for road surface reconstruction.

Depth Estimation with Light Field and Photometric Stereo Data Using Energy Minimization

Through the fusion of light fields and photometric stereo, two state-of-the-art computational imaging approaches, we improve the 3D reconstruction of objects. Light field imaging techniques observe a scene from different angles, which results in a strong absolute depth estimation of the scene. Photometric stereo uses multiple illuminations to reconstruct the surface of objects, which allows estimating fine local details of objects, even when no surface structure is present. We combine both approaches within a minimization algorithm, which exhibits an accurate absolute depth with a high sensitivity to fine surface details.

Document aging effects and automated security document authentication

Image quality of scanned security documents is crucial to automated authentication systems such as an automated border control (ABC) gate. Due to natural aging effects as well as unavoidable processes such as wear&tear, abrasion, graffiti, or mechanical crumpling, security documents change their appearance within an inspection system. We study the influence of those effects w.r.t. the document age on the image quality delivered by a document reader incorporated in an ABC gate. Results are presented independent from a specific authentication device making use of a number of established image quality indicators.

Depth Estimation within a Multi-Line-Scan Light-Field Framework

We present algorithms for depth estimation from light-field data acquired by a multi-line-scan image acquisition system. During image acquisition a 3-D light field is generated over time, which consists of multiple views of the object observed from different viewing angles. This allows for the construction of so-called epipolar plane images (EPIs) and subsequent EPI-based depth estimation. We compare several approaches based on testing various slope hypotheses in the EPI domain, which can directly be related to depth. The considered methods used in hypothesis assessment, which belong to a broader class of block-matching algorithms, are modified sum of absolute differences (MSAD), normalized cross correlation (NCC), census transform (CT) and modified census transform (MCT). The methods are compared w.r.t. their qualitative results for depth estimation and are presented for artificial and real-world data.

Full BRDF Reconstruction Using CNNs from Partial Photometric Stereo-Light Field Data

The acquisition of partial BRDF measurements using light field cameras and several illumination directions raises critical questions regarding the accuracy of inferences based on that data. Therefore, we attempt to verify the quality of the reconstruction of a full BRDF using partial input data. A dataset that provides a densely sampled BRDF was used, both in viewing and illumination directions. We show the reconstruction of dense BRDFs when the viewing angles are limited to top central regions, while the illumination angles are not reduced and are positioned in the shape of a half sphere around the material object, these properties are characteristic of data provided by plenoptic cameras paired with a photometric light dome. The partial reconstruction of the dense BRDF out of data is achieved by utilizing convolutional neural networks. We obtain a competitive full reconstruction when up to 2/3 of the BRDF is unknown.