Jean-Denis Durou

Numerical methods for shape-from-shading: A new survey with benchmarks

Many algorithms have been suggested for the shape-from-shading problem, and some years have passed since the publication of the survey paper by Zhang et al. [R. Zhang, P.-S. Tsai, J.E. Cryer, M. Shah, Shape from shading: a survey, IEEE Transactions on Pattern Analysis and Machine Intelligence 21 (8) (1999) 690-706]. In this new survey paper, we try to update their presentation including some recent methods which seem to be particularly representative of three classes of methods: methods based on partial differential equations, methods using optimization and methods approximating the image irradiance equation. One of the goals of this paper is to set the comparison of these methods on a firm basis. To this end, we provide a brief description of each method, highlighting its basic assumptions and mathematical properties. Moreover, we propose some numerical benchmarks in order to compare the methods in terms of their efficiency and accuracy in the reconstruction of surfaces corresponding to synthetic, as well as to real images.

Integrating the Normal Field of a Surface in the Presence of Discontinuities

We show how to integrate the normal field of a surface in the presence of discontinuities by three different ways. We obtain very satisfactory 3D-reconstructions, from the point of view of the accuracy of the reconstructions. As an important consequence, no prior segmentation of the scene into parts without discontinuity is required anymore. Finally, we test the three proposed methods of integration in the framework of photometric stereo, a technique which aims at computing the normal field of a scene surface from several images of this scene lighted under different directions.

Shape from shading for the digitization of curved documents

Document digitization is faster and more affordable using digital cameras than scanners. On the other hand, if we aim at extending the basic digital camera functionalities for such a purpose, post-processings will be of first importance, at least to improve the text legibility. In this paper, we address the specific problem of the virtual flattening of curved documents, as for example the pages of an opened book lying on its spine. In order to compute the document shape, we use the shape from shading technique and discuss why, in some cases, it is more suitable than other 3D single-view reconstruction techniques. We extend the seminal work by Wada etxa0al. (Proceedings of the IAPR Workshop on machine vision and applications, Tokyo, Japan, pp. 591–594, 1992) and consecutive papers, reformulating the problem in terms of perspective shape from shading. Finally, we design a complete post-processing algorithm and test it on real images. Even if the documents are much curved, it is shown that the restored images are almost identical to scanned images of the flattened documents.

Ambiguous Shape from Shading with Critical Points

The eikonal equation can have an infinite number of solutions when the image has critical points. We exhibit a family, indexed by a continuous parameter, of non isomorphic surfaces with one critical point, which give the same simple image. Hence, shape from shading can be an ill-posed problem when no additional condition on the shape is imposed, even when the image has critical points. Also, deformations without distortion are possible, i.e., there can exist a continuous deformation of the surface which does not modify its image.

3D-spline reconstruction using shape from shading: Spline from shading

In this work, we describe an original method of solving the shape from shading (SFS) problem, which relies on the marriage of two simple ideas. On the one hand, we propose to modelize the scene by a parametric surface, namely a 3D-spline. The key advantage is that boundary conditions are no longer required to render the problem well-posed. On the other hand, we introduce the concept of useful domain: this is a sufficient set of pixels, whose greylevels are in accordance with the SFS hypotheses, which allows to solve the SFS problem on the whole reconstruction domain. The proposed method is described for both formulations of SFS, corresponding to orthographic and perspective projections. We can thus validate our method on synthetic as well as on real images.

Non-visible Deformations

The number of non isomorphic solutions of the eikonal equation can be infinite. We show that there can exist a whole family of non isomorphic solutions, indexed by a continuous parameter. This implies, first, that the general problem of shape from shading can be ill-posed when no additional condition on the shape is imposed. This is in contradiction with what is sometimes stated in the literature of shape from shading. Furthermore, this implies that there can exist non visible deformations of a given surface, i. e., continuous deformations of the surface which do not modify the image.

Creation of real images which are valid for the assumptions made in shape from shading

Iterative methods have shown up to now encouraging results for resolving shape from shading. These kinds of methods generally work on synthetic images, and occasionally on real images, even if such images do not agree in general with the hypotheses of shape from shading. In this article, we describe the assumptions mentioned above, and propose an original methodology, which enables the production of real images and the process allowing to correct most of their defects in order to make them correspond to most of these assumptions. Moreover we propose a descent gradient iterative scheme, thanks to which we prove that shape from shading can work as well on real images as on synthetic images.