Rik Fransens

Combined Depth and Outlier Estimation in Multi-View Stereo

In this paper, we present a generative model based approach to solve the multi-view stereo problem. The input images are considered to be generated by either one of two processes: (i) an inlier process, which generates the pixels which are visible from the reference camera and which obey the constant brightness assumption, and (ii) an outlier process which generates all other pixels. Depth and visibility are jointly modelled as a hiddenMarkov Random Field, and the spatial correlations of both are explicitly accounted for. Inference is made tractable by an EM-algorithm, which alternates between estimation of visibility and depth, and optimisation of model parameters. We describe and compare two implementations of the E-step of the algorithm, which correspond to the Mean Field and Bethe approximations of the free energy. The approach is validated by experiments on challenging real-world scenes, of which two are contaminated by independently moving objects.

A Probabilistic Approach to Large Displacement Optical Flow and Occlusion Detection

This paper deals with the computation of optical flow and occlusion detection in the case of large displacements. We propose a Bayesian approach to the optical flow problem and solve it by means of differential techniques. The images are regarded as noisy measurements of an underlying ’true’ image-function. Additionally, the image data is considered incomplete, in the sense that we do not know which pixels from a particular image are occluded in the other images. We describe an EM-algorithm, which iterates between estimating values for all hidden quantities, and optimizing the current optical flow estimates by differential techniques. The Bayesian way of describing the problem leads to more insight in existing differential approaches, and offers some natural extensions to them. The resulting system involves less parameters and gives an interpretation to the remaining ones. An important new feature is the photometric detection of occluded pixels. We compare the algorithm with existing optical flow methods on ground truth data. The comparison shows that our algorithm generates the most accurate optical flow estimates. We further illustrate the approach with some challenging real-world examples.

A Generalized EM Approach for 3D Model Based Face Recognition under Occlusions

This paper describes an algorithm for pose and illumination invariant face recognition from a single image under occlusions. The method iteratively estimates the parameters of a 3D morphable face model to approximate the appearance of a face in an image. Simultaneously, a visibility map is computed which segments the image into visible and occluded regions. The visibility map is incorporated into a probabilistic image formation model as a set of spatially correlated random variables. This leads to a Generalized Expectation-Maximization algorithm in which the estimation of the morphable model related parameters is interleaved with visibility computations. The validity of the algorithm is verified by a face recognition experiment using images from the publicly available AR Face Database.

Parametric stereo for multi-pose face recognition and 3d-face modeling

This paper presents a new method for face modeling and face recognition from a pair of calibrated stereo cameras. In a first step, the algorithm builds a stereo reconstruction of the face by adjusting the global transformation parameters and the shape parameters of a 3D morphable face model. The adjustment of the parameters is such that stereo correspondence between both images is established, i.e. such that the 3D-vertices of the model project on similarly colored pixels in both images. In a second step, the texture information is extracted from the image pair and represented in the texture space of the morphable face model. The resulting shape and texture coefficients form a person specific feature vector and face recognition is performed by comparing query vectors with stored vectors. To validate our algorithm, an extensive image database was built. It consists of stereo-pairs of 70 subjects. For recognition testing, the subjects were recorded under 6 different head directions, ranging from a frontal to a profile view. The face recognition results are very good, with 100% recognition on frontal views and 97% recognition on half-profile views.

A Mean Field EM-algorithm for Coherent Occlusion Handling in MAP-Estimation Prob

This paper presents a generative model based approach to deal with occlusions in vision problems which can be formulated as MAP-estimation problems. The approach is generic and targets applications in diverse domains like model-based object recognition, depth-from-stereo and image registration. It relies on a probabilistic imaging model, in which visible regions and occlusions are generated by two separate processes. The partitioning into visible and occluded regions is made explicit by the introduction of an hidden binary visibility map, which, to account for the coherent nature of occlusions, is modelled as a Markov Random Field. Inference is made tractable by a mean field EMalgorithm, which alternates between estimation of visibility and optimisation of model parameters. We demonstrate the effectiveness of the approach with two examples. First, in a N-view stereo experiment, we compute a dense depth map of a scene which is contaminated by multiple occluding objects. Finally, in a 2D-face recognition experiment, we try to identify people from partially occluded facial images.

A Probabilistic Approach to Optical Flow based Super-Resolution

This paper deals with the computation of a single super-resolution image from a set of low-resolution images, where the motion fields are not constrained to be parametric. In our approach, the inversion process, in which the super-resolved image is inferred from the input data, is interleaved with the computation of a set of dense optical flow fields. The case of arbitrary motion presents several significant challenges. First of all, the super-resolution setting dictates that the optic flow computations must be very precise. Furthermore, we have to consider the possibility that certain parts of the scene, which are visible in the super-resolved image, are occluded in some of the input images. Such occlusions must be identified and dealt with in the restoration process. We propose a Bayesian approach to tackle these problems. In this framework, the input images are regarded as sub-sampled and noisy versions of the unknown high-quality image. Also, the input data is considered incomplete, in the sense that we do not know which pixels from the evolving super-resolution image are occluded in particular images from the input set. This will be modeled by introducing so-called visibility maps, which are treated as hidden variables. We describe an EM-algorithm, which iterates between estimating values for the hidden quantities, and optimizing the flow-fields and the super-resolution image. The approach is illustrated with a synthetic and a challenging real-world example.

Robust Estimation in the Presence of Spatially Coherent Outliers

We present a generative model based approach to deal with spatially coherent outliers. The model assumes that image pixels are generated by either one of two distinct processes: an inlier process which is responsible for the generation of the majority of the data, and an outlier process which generates pixels not adhering to the inlier model. The partitioning into inlier and outlier regions is made explicit by the introduction of a hidden binary map. To account for the coherent nature of outliers this map is modelled as a Markov Random Field, and inference is made tractable by a mean field EM-algorithm. We make a connection with classical robust estimation theory, and derive the analytic expressions of the equivalent M-estimator for two limiting cases of our model. The effectiveness of the proposed method is demonstrated with two examples. First, in a synthetic linear regression problem, we compare our approach with different M-estimators. Next, in a 2D-face recognition experiment, we try to identify people from partially occluded facial images.

Analysis of Human Locomotion based on Partial Measurements

A lot of computer vision applications have to deal with occlusions. In such settings only a subset of the features of interest can be observed, i.e. only incomplete or partial measurements are available. In this article we show how a learned statistical model can be used to make a prediction of the unknown (occluded) features. The probabilistic nature of the framework also allows to compute the remaining uncertainty given an incomplete observation. The resulting posterior probability distribution can then be used for inference. Additional unknowns such as alignment or scale are easily incorporated into the framework. Instead of computing the alignment in a preprocessing step, it is left as an additional uncertainty, similar to the uncertainty introduced by the missing values of the measurement. It is shown how the technique can be applied to the analysis of human loco-motion, when body parts are occluded. Experiments show how the unobserved body locations are predicted and how it can be inferred whether the measurements come from a running or walking sequence.

Using shape to correct for observed nonuniform color in automated egg grading

We report on algorithmic aspects for the automated visual quality control for grading of brown eggs. Using RGB color images of four different views of every egg enabled to analyze the entire eggshell. The scene was illuminated using a set of white fluorescent tubes placed in a rectangular grid. After detection and approximation of the egg contour (ellipse fitted), the color was corrected to compensate for the elliptical shape of the eggs. A second order polynomial was fitted through points taken from subsequent horizontal lines inside the egg. Iteration was used to reject outliers (most likely points with visual defects). The shape- corrected intensity was calculated as the signed difference between polynomial and measured value, increased with the average egg intensity. Based on the corrected color, dirt regions like yolk, manure, blood, and red mite spots were segmented from the egg-background. Features based on color and shapes were calculated for every segmented region as the combined space and color moments of zeroth, first and second order. A classifier identified most of the defective eggs. Elimination of false rejects due to mirror reflection of the light tubes on some eggs (segmented because of the different color) is currently under investigation.