Fabio Remondino

Turning images into 3-D models

In this article developments and performance analysis of image matching for detailed surface reconstruction of heritage objects is discussed. Three dimensional image-based modeling of heritages is a very interesting topic with many possible applications. In this article we propose a multistage image-based modeling approach that requires only a limited amount of human interactivity and is capable of capturing the fine geometric details with similar accuracy as close-range active range sensors. It can also cope with wide baselines using several advancements over standard stereo matching techniques. Our approach is sequential, starting from a sparse basic segmented model created with a small number of interactively measured points. This model, specifically the equation of each surface, is then used as a guide to automatically add the fine details. The following three techniques are used, each where best suited, to retrieve the details: 1) for regularly shaped patches such as planes, cylinders, or quadrics, we apply a fast relative stereo matching technique. 2) For more complex or irregular segments with unknown shape, we use a global multi-image geometrically constrained technique. 3) For segments unsuited for stereo matching, we employ depth from shading (DFS). The goal is not the development of a fully automated procedure for 3D object reconstruction from image data or a sparse stereo approach, but we aim at the digital reconstruction of detailed and accurate surfaces from calibrated and oriented images for practical daily documentation and digital conservation of wide variety of heritage objects.

3D modeling of close-range objects: photogrammetry or laser scanning?

Photogrammetry has dealt since many years with the 3D reconstruction of objects from images. It provides for accurate sensor calibration and object modeling using analog or digital imageries, it is very portable and many commercial software is available for image processing and 3D modeling. On the other hand, laser scanning technology and all the related reverse engineering software are becoming a very promising alternative for many kind of surveying and modeling applications. Laser scanners allow to acquire very quickly a huge amount of 3D data which can be often combined with color high-resolution digital images. Among the plenty of works so far presented, in particular on the use of laser scanning for cultural heritage survey, some modeling and accuracy related issues have been not yet solved and discussed in details. In this contribution we report about two case studies realized with photogrammetry and laser scanner and we provide some advices and suggestions about the more suitable 3D modeling method for a given object, taking into account its size and shape complexity, the required accuracy and the target application.

Range imaging technology: new developments and applications for people identification and tracking

Range Imaging (RIM) is a new suitable choice for measurement and modeling in many different applications. RIM is a fusion of two different technologies. According to the terminology, it integrates distance measurement as well as imaging aspects. The distance measurement principle is dominated by the time-of-flight principle while the imaging array (e.g. CMOS sensor) enables each pixel to store also the distance towards the corresponding object point. Due to the technologys relatively new appearance on the market, with a few different realizations, the knowledge of its capabilities is very low. In this paper we present our investigations on the range imaging camera SwissRangerTM (realized by the Swiss Center for Electronics and Microtechnology, CSEM). Different calibration procedures are performed, including a photogrammetric camera calibration and a distance system calibration with respect to the reflectivity and the distance itself. Furthermore we report about measurement applications in the field of surveillance and biometrics. In particular, range imaging data of moving people are analyzed, to identify humans, detect their movements and recover 3D trajectories.

Human figure reconstruction and modeling from single image or monocular video sequence

We firstly review some approaches to recover the 3D shape of human figures and the related movements. Then we present an improved and reliable version of an easy approach to recover 3D model of humans using just one frame or a monocular video sequence. A simplification of the camera model based on the collinearity condition is required, due to the absence of stereo view. This leads to a perspective or orthographic projection and requires additional information or constraints to solve the problem. The human figure is reconstructed in a skeleton form and visualized with the portable VRML language. Moreover a surface fitting is performed using a predefined 3D model obtained from laser scanner measurements.

3D Reconstruction of Human Skeleton from Single Images or Monocular Video Sequences

In this paper, we first review the approaches to recover 3D shape and related movements of a human and then we present an easy and reliable approach to recover a 3D model using just one image or monocular video sequence. A simplification of the perspective camera model is required, due to the absence of stereo view. The human figure is reconstructed in a skeleton form and to improve the visual quality, a pre-defined human model is also fitted to the recovered 3D data.

Photogrammetry for geological applications: automatic retrieval of discontinuity orientation in rock slopes

The stability of a rock slope depends on the rock mass geo-structure and its discontinuities. Discontinuities show up at the rock surface as smooth and often plane surfaces. From their location and orientation the main families of fractures can be inferred and a stability analysis performed. To gather information on their distribution, surveys are typically carried out with geological compass and tape along scan lines, with obvious limitations and drawbacks. Here an highly automated image-based approach is presented to compute the required rock parameters: an accurate high resolution Digital Surface Model of the area of interest is generated from an image sequence and segmented in plane surfaces within a multi resolution RANSAC search, which returns location and orientation of each plane. To avoid measuring ground control points, the camera may be interfaced to a GPS receiver. Multiple overlapping and convergent images are captured to achieve good accuracy over the whole network, minimize occlusions and avoid poor object-camera relative geometry. The method is applied to the rock face of Corma di Machaby (Italy): the results are compared to those of a traditional survey with compass and to those of a laser scanner survey.

Performance evaluation of a coded structured light system for cultural heritage applications

D documentation and visualization of Cultural Heritage objects is an expanding application area. The selection of the right technology for these kinds of applications is very important and strictly related to the project requirements, budget and users experience. Active sensors, i.e. triangulation based laser scanners and structured light systems are used for many kinds of 3D object reconstruction tasks and in particular for 3D documentation of cultural heritage objects. This study presents some experiences in the results of two case studies in which a close-range structured light system is used for the 3D digitization. The paper includes all necessary steps of the 3D object modeling pipeline from data acquisition to 3D visualization.

Photogrammetric bridging of GPS outages in mobile mapping

A photogrammetric strategy for the orientation of image sequences acquired by Mobile Mapping Vehicles (MMV) is presented. The motivations for this are twofold: to allow image georeferencing in short GPS outages for a MMV under development at the University of Parma, currently without an IMU; to improve the consistency of image georeferencing between asynchronous frames. The method may also contribute to limit the drift errors of low-cost integrated IMU/GPS systems in GPS outages. Drawing on techniques developed for Structure and Motion (S&M) reconstruction from image sequences and accounting for the specific conditions of the MMV imaging geometry, highly reliable multi-image matches are found, refining image orientation with a final bundle adjustment. Dealing with scenes containing poor image texture, the automation of the convergence of the bundle to the solution is still a problem. After a successful orientation of an image sequences of ca 70 m, the accuracy of the orientation and reconstruction process was checked in a test field by means of GPS observations and check points. Although not all constraints between synchronous image pairs are yet enforced, the accuracy degradation along the sequence was found to be still well within the specifications for the MMV.

The potential of 3D techniques for Cultural Heritage object documentation

The generation of 3D models of objects has become an important research point in many fields of application like industrial inspection, robotics, navigation and body scanning. Recently the techniques for generating photo-textured 3D digital models have interested also the field of Cultural Heritage, due to their capability to combine high precision metrical information with a qualitative and photographic description of the objects. In fact this kind of product is a fundamental support for documentation, studying and restoration of works of art, until a production of replicas by fast prototyping techniques. Close-range photogrammetric techniques are nowadays more and more frequently used for the generation of precise 3D models. With the advent of automated procedures and fully digital products in the 1990s, it has become easier to use and cheaper, and nowadays a wide range of commercial software is available to calibrate, orient and reconstruct objects from images. This paper presents the complete process for the derivation of a photorealistic 3D model of an important basalt stela (about 70 x 60 x 25 cm) discovered in the archaeological site of Tilmen Höyük, in Turkey, dating back to 2nd mill. BC. We will report the modeling performed using passive and active sensors and the comparison of the achieved results.

3D reconstruction of articulated objects from uncalibrated images

The goal of computing realistic 3-D models from image sequences is still a challenging problem. In recent years the demand for realistic reconstruction and modeling of objects and human bodies is increasing both for animation and medical applications. In this paper a system for the reconstruction of 3-D models of articulated objects, like human bodies, from uncalibrated images is presented. The scene is seen from different viewpoints and no pre-set knowledge is considered. To extract precise 3-D information from imagery, a calibration procedure must be performed. Therefore, first a camera calibration with Direct Linear Transformation (DLT) is done assuming few control points on the subject. The initial approximations of the interior and exterior orientation computed with DLT are then improved in a general photogrammetric bundle adjustment with self-calibration. Additionally a stereo matching process based on least squares matching extracts a dense set of image correspondences from the sequence. Finally a 3-D point cloud is computed by forward intersection using the calibration data and the matched points. The resulting 3-D model of human body is presented.