Daniel Soukup

Robust Object Recognition under Partial Occlusions Using NMF

In recent years, nonnegative matrix factorization (NMF) methods of a reduced image data representation attracted the attention of computer vision community. These methods are considered as a convenient part-based representation of image data for recognition tasks with occluded objects. A novel modification in NMF recognition tasks is proposed which utilizes the matrix sparseness control introduced by Hoyer. We have analyzed the influence of sparseness on recognition rates (RRs) for various dimensions of subspaces generated for two image databases, ORL face database, and USPS handwritten digit database. We have studied the behavior of four types of distances between a projected unknown image object and feature vectors in NMF subspaces generated for training data. One of these metrics also is a novelty we proposed. In the recognition phase, partial occlusions in the test images have been modeled by putting two randomly large, randomly positioned black rectangles into each test image.

Convolutional Neural Networks for Steel Surface Defect Detection from Photometric Stereo Images

Convolutional neural networks (CNNs) achieved impressive recognition rates in image classification tasks recently. In order to exploit those capabilities, we trained CNNs on a database of photometric stereo images of metal surface defects, i.e. rail defects. Those defects are cavities in the rail surface and are indication for further surface degradation right up to rail break. Due to security issues, defects have to be recognized early in order to take countermeasures in time. By means of differently colored light-sources illuminating the rail surfaces from different and constant directions, those cavities are made visible in a photometric dark-field setup. So far, a model-based approach has been used for image classification, which expressed the expected reflection properties of surface defects in contrast to non-defects. In this work, we experimented with classical CNNs trained in pure supervised manner and also explored the impact of regularization methods such as unsupervised layer-wise pre-training and training data-set augmentation. The classical CNN already distinctly outperforms the model-based approach. Moreover, regularization methods yet yield further improvements.

Demosaicing algorithms for area- and line-scan cameras in print inspection

Most color image sensors use color filter arrays (CFA). With this sensor design the captured information at each sensor pixel position is restricted to a specific spectral portion (typically red, green and blue bands). To obtain the missing color responses at each pixel position, so-called CFA demosaicing algorithms are commonly used. We propose two new CFA demosaicing algorithms, which are well suited for industrial print inspection with respect to the requirements in accuracy and speed. As a main contribution, we introduce novel demosaicing algorithms for specific high-speed color digital time delay and integration (DTDI) CFA line-scan cameras. We compare the suggested CFA demosaicing algorithms to state-of-the art algorithms for area and line-scan camera operation modes. We show that the two new algorithms perform superior to conventional algorithms as indicated by reconstruction error.

Analysis of optically variable devices using a photometric light-field approach

Diffractive Optically Variable Image Devices (DOVIDs), sometimes loosely referred to as holograms, are popular security features for protecting banknotes, ID cards, or other security documents. Inspection, authentication, as well as forensic analysis of these security features are still demanding tasks requiring special hardware tools and expert knowledge. Existing equipment for such analyses is based either on a microscopic analysis of the grating structure or a point-wise projection and recording of the diffraction patterns. We investigated approaches for an examination of DOVID security features based on sampling the Bidirectional Reflectance Distribution Function (BRDF) of DOVIDs using photometric stereo- and light-field-based methods. Our approach is demonstrated on the practical task of automated discrimination between genuine and counterfeited DOVIDs on banknotes. For this purpose, we propose a tailored feature descriptor which is robust against several expected sources of inaccuracy but still specific enough for the given task. The suggested approach is analyzed from both theoretical as well as practical viewpoints and w.r.t. analysis based on photometric stereo and light fields. We show that especially the photometric method provides a reliable and robust tool for revealing DOVID behavior and authenticity.

Invariant characterization of DOVID security features using a photometric descriptor

Diffractive Optically Variable Image Devices (DOVIDs) are popular security features used to protect security documents such as banknotes, ID cards, passports, etc. Checking authenticity of these security features on both user as well as forensic level remains a challenging task, requiring sophisticated hardware tools and expert knowledge. Recently, we proposed a technique exploiting a large-scale photometric behavior of DOVIDs in order to discriminate denominations and detect counterfeits. Here we investigate invariance properties of the proposed method and demonstrate its robustness against various common perturbations, which may have negative impact on the acquisition quality in practice. Presented results show a great potential of this approach primarily for security and forensic purposes, but also for other applications, where automated inspection of DOVIDs is of interest.

Depth and all-in-focus images obtained by multi-line-scan light-field approach

We present a light-field multi-line-scan image acquisition and processing system intended for the 2.5/3-D inspection of fine surface structures, such as small parts, security print, etc. in an industrial environment. The system consists of an area-scan camera, that allows for a small number of sensor lines to be extracted at high frame rates, and a mechanism for transporting the inspected object at a constant speed. During the acquisition, the object is moved orthogonally to the camera’s optical axis as well as the orientation of the sensor lines. In each time step, a predefined subset of lines is read out from the sensor and stored. Afterward, by collecting all corresponding lines acquired over time, a 3-D light field is generated, which consists of multiple views of the object observed from different viewing angles while transported w.r.t. the acquisition device. This structure allows for the construction of so-called epipolar plane images (EPIs) and subsequent EPI-based analysis in order to achieve two main goals: (i) the reliable estimation of a dense depth model and (ii) the construction of an all-in-focus intensity image. Beside specifics of our hardware setup, we also provide a detailed description of algorithmic solutions for the mentioned tasks. Two alternative methods for EPI-based analysis are compared based on artificial and real-world data.

Depth and all-in-focus imaging by a multi-line-scan light-field camera

Abstract. We present a multi-line-scan light-field image acquisition and processing system designed for 2.5/3-D inspection of fine surface structures in industrial environments. The acquired three-dimensional light field is composed of multiple observations of an object viewed from different angles. The acquisition system consists of an area-scan camera that allows for a small number of sensor lines to be extracted at high frame rates, and a mechanism for transporting an inspected object at a constant speed and direction. During acquisition, an object is moved orthogonally to the camera’s optical axis as well as the orientation of the sensor lines and a predefined subset of lines is read out from the sensor at each time step. This allows for the construction of so-called epipolar plane images (EPIs) and subsequent EPI-based depth estimation. We compare several approaches based on testing a set of slope hypotheses in the EPI domain. Hypotheses are derived from block matching, namely the sum of absolute differences, modified sum of absolute differences, normalized cross correlation, census transform, and modified census transform. Results for depth estimation and all-in-focus image generation are presented for synthetic and real data.

Non-negative matrix factorization: a study on influence of matrix sparseness and subspace distance metrics on image object recognition

In recent years non-negative factorization (NMF) methods of a reduced image data representation attracted the attention of computer vision community. These methods are considered as a convenient part-based representation of image data for recognition tasks with occluded objects. In the paper two novel modifications of the NMF are proposed which utilize the matrix sparseness control used by Hoyer. We have analyzed the influence of sparseness on recognition rates (RR) for various dimensions of subspaces generated for two image databases. We have studied the behaviour of four types of distances between a projected unknown image object and feature vectors in NMF-subspaces generated for training data. For occluded ORL face data, Euclidean and diffusion distances perform better than Riemannian ones, not following the overall expactation that Euclidean metric is suitable only for orthogonal basis vectors. In the case of occluded USPS digit data, the RR obtained for the modified NMF algorithm show in comparison to the conventional NMF algorithms very close values for all four distances over all dimensions and sparseness constraints. In this case Riemannian distances provide higher RR than Euclidean and diffusion ones. The proposed modified NMF method has a relevant computational benefit, since it does not require calculation of feature vectors which are explicitly generated in the NMF optimization process.

On Optimal Illumination for DOVID Description Using Photometric Stereo

Diffractive optically variable image devices DOVIDs are popular security features used to protect security documents such as banknotes, ID cards, passports, etc. Nevertheless, checking authenticity of these security features on both user as well as forensic level still remains a challenging task, requiring sophisticated hardware tools and expert knowledge. Based on a photometric acquisition setup comprised of 32 illumination sources from different directions and a recently proposed descriptor capturing the illumination dependent behavior, we investigate the information content, illumination pattern shape and clustering properties of the descriptor. We studied shape and discriminative power of reduced illumination configurations for the task of discrimination applied to DOVIDs using a sample of Euro banknotes.

Cross-channel co-occurrence matrices for robust characterization of surface disruptions in 2 1/2D rail image analysis

We present a new robust approach to the detection of rail surface disruptions in high-resolution images by means of 21/2D image analysis. The detection results are used to determine the condition of rails as a precaution to avoid breaks and further damage. Images of rails are taken with color line scan cameras at high resolution of about 0.2 millimeters under specific illumination to enable 21/2D image analysis. Pixel locations fulfilling the anti-correlation property between two color channels are detected and integrated over regions of general background deviations using so called cross-channel co-occurrence matrices, a novel variant of co-occurrence matrices introduced as part of this work. Consequently, the detection of rail surface disruptions is achieved with high precision, whereas the unintentional elimination of valid detections in the course of false and irrelevant detection removal is reduced. In this regard, the new approach is more robust than previous methods.