Zsolt Jankó

Using a genetic algorithm to register an uncalibrated image pair to a 3D surface model

In this paper we present a successful application of genetic algorithms to the registration of uncalibrated optical images to a 3D surface model. The problem is to find the projection matrices corresponding to the images in order to project the texture on the surface as precisely as possible. Recently, we have proposed a novel method that generalises the photo-consistency approach by Clarkson et al. to the case of uncalibrated cameras by using a genetic algorithm. In previous studies we focus on the computer vision aspects of the method, while here we analyse the genetic part. In particular, we use semi-synthetic data to study the performance of different GAs and various types of selector, mutation and crossover. New experimental results on real data are also presented to demonstrate the efficiency of the method.

Photo-consistency based registration of an uncalibrated image pair to a 3D surface model using genetic algorithm

We consider the following data fusion problem. A 3D object with textured Lambertian surface is measured and independently photographed. A triangulated model of the object and two uncalibrated images are obtained. The goal is to precisely register the images to the model. Solving this problem is necessary for building a geometrically accurate, photorealistic model from laser-scanned 3D data and high quality images. Recently, we have proposed a novel method that generalises the photo-consistency approach by Clarkson et al. [2001] to the case of uncalibrated cameras, when both intrinsic and extrinsic parameters are unknown. This gives a user the freedom of taking the pictures by a conventional digital camera, from arbitrary positions and with varying zoom. The method is based on manual pre-registration followed by a genetic optimisation algorithm. A brief description of the pilot version of the method [Z. Janko et al. (2004)] has been given together with the results of a few initial tests. In this paper, we report on some new significant developments in this project. The critical issue of robustness against illumination changes is addressed and various colour representations and cost functions are tested and compared. Natural constraints are introduced and experimentally validated to simplify the camera model and accelerate the algorithm. Finally, we present synthetic and real data with ground truth, apply the improved method to the data and measure the quality of the results.

Registration of an uncalibrated image pair to a 3D surface model

The following data fusion problem is considered: Given a 3D geometric model of an object and two uncalibrated images of the same object, and assuming that the object surface is textured and Lambertian, precisely register the images to the model. Solving this problem is necessary for building a geometrically accurate, photorealistic model from laser-scanned 3D data and high quality images. We generalise the photo-consistency approach by Clarkson et al. to the case of uncalibrated cameras, when both intrinsic and extrinsic parameters are unknown. This gives a user the freedom of taking the pictures by a conventional digital camera, from arbitrary positions and with varying zoom. We discuss a number of possible approaches to the problem and propose a method based on manual preregistration followed by a genetic optimisation algorithm. The issues of speed and robustness are addressed. Results for real data are shown.

Finding region correspondences for wide baseline stereo

This study addresses the problem of finding correspondences for wide baseline stereo. Texture has traditionally been utilised as a single-image cue for 3D shape reconstruction (shape-front-texture); at the same time, its role in multiview scene reconstruction has been very limited. In stereo image matching, repetitive patterns are usually considered as disturbing factor since they tend to produce multiple peaks of correlation, which results in matching ambiguity. We argue that presence and proper analysis of distinct, compact periodic texture areas can facilitate wide baseline matching by providing periodic distinguished regions (PDRs) that efficiently constrain the search for correspondences. We demonstrate how PDRs can be used to find a few initial correspondences in a wide baseline stereo pair and to establish precise correspondences for building the epipolar geometry. Experimental results for various wide baseline stereo pairs are shown.

Creating photorealistic models by data fusion with genetic algorithms

Building photorealistic 3D models of real-world objects is a fundamental problem in computer vision and computer graphics. Such models require precise geometry as well as detailed texture on the surface. Textures allow one to obtain visual effects that are essential for high-quality rendering. Photorealism is further enhanced by adding surface roughness in form of the so-called 3D texture represented by a bump map. Typical applications of precise photorealistic 3D models are:

Data fusion for photorealistic 3d models

This study aims at building photorealistic 3D models of real-world objects. We discuss the problem of combining a 3D textureless model obtained by 3D scanner, with optical images that provide textural information of the object. Recently, we have proposed a novel method to register an uncalibrated image pair to a 3D surface model. After registration, the images are mapped to the surface. However, as the images show different parts of the objects, partial overlapping textures can only be extracted from them. Combining the images into a complete texture map that covers the entire object is not trivial. We present a method to build photorealistic 3D models that includes algorithms for data registration and for merging multiple texture maps using surface flattening. Experimental results on real and synthetic data are shown.