Vítor Sequeira

Automated reconstruction of 3D models from real environments

Abstract This paper describes an integrated approach to the construction of textured 3D scene models of building interiors from laser range data and visual images. This approach has been implemented in a collection of algorithms and sensors within a prototype device for 3D reconstruction, known as the EST (Environmental Sensor for Telepresence). The EST can take the form of a push trolley or of an autonomous mobile platform. The Autonomous EST (AEST) has been designed to provide an integrated solution for automating the creation of complete models. Embedded software performs several functions, including triangulation of the range data, registration of video texture, registration and integration of data acquired from different capture points. Potential applications include facilities management for the construction industry and creating reality models to be used in general areas of virtual reality, for example, virtual studios, virtualised reality for content-related applications (e.g., CD-ROMs), social telepresence, architecture and others. The paper presents the main components of the EST/AEST, and presents some example results obtained from the prototypes. The reconstructed model is encoded in VRML format so that it is possible to access and view the model via the World Wide Web.

Registration and fusion of intensity and range data for 3D modelling of real world scenes

A novel technique combining intensity and range data is presented. Passive (intensity based) and active (range based) techniques used for 3D reconstruction have their limitations and separately, none of these techniques can solve all the problems inherent to the modelling of real environments. Our technique aims to demonstrate how both intensity and range data can be registered and combined into a long-range 3D system. The procedure needs an initial estimation for internal and external camera parameters for two or more intensity images. The technique uses passive triangulation of the intensity data to refine the initial camera calibrations and ensure a good registration of range and video data sets. Once a reliable calibration is achieved, corresponding points from the intensity images are triangulated and introduced in the original range data. With our technique, it is possible to complete the models in areas where data is missing or to increase the resolution in areas of high interest and 3D contents.

3D reality modelling: photo-realistic 3D models of real world scenes

This paper presents a 3D reality modelling system, applying lasers and digital photography to create super-accurate photo-realistic three-dimensional worlds. A key feature of the 3D reality system is its flexibility. It can be used with all common long-range scanners and digital or video cameras, without the need of any pre-calibration or static arrangement between the different sensors. The system can be used for a wide range of applications in industry and science, as well as for environmental modelling and virtual reality presentations. The 3D models are sufficiently lifelike to be used in sectors such as real estate, cultural heritage and construction. Results can be exported to all major 3D software and CAD packages or viewed via a Web browser.

3D environment modelling using laser range sensing

This paper describes a technique for constructing a geometric model of an unknown environment based on data acquired by a Laser Range Finder on board of a mobile robot. The geometric model would be most useful both for navigation and verification purposes. The paper presents all the steps needed for the description of the environment, including the range image acquisition and processing, 3D surface reconstruction and the problem of merging multiple images in order to obtain a complete model.

Active view selection for efficient 3D scene reconstruction

This paper presents an active sensing process for the efficient reconstruction of a real world environment using multiple views of a scene captured by a laser range finder onboard a mobile robot. To resolve the ambiguities caused by occlusions there is the need to acquire multiple range views at different viewpoints. A strategy for optimising the selection of the next sensing position and orientation based on previously detected occlusions is proposed. No a priori knowledge about the scene is used for planning the next views. The system evaluates the set of capture points from which all the necessary data can be acquired aiming at minimising its number. Among these points it then selects the one(s) maximising the occluded areas to be resolved.

Automatic registration of laser reflectance and colour intensity images for 3D reconstruction

Abstract The objective of the work presented in this paper is to generate complete, high-resolution models of real world scenes from passive intensity images and active range sensors. In previous work, an automatic method has been developed in order to compute 3D models of real world scenes from laser range data. The aim of this project is to improve these existing models by fusing range and intensity data. The paper presents different techniques in order to find correspondences between the different sets of data. Based on these control points, a robust camera calibration is computed with a minimal user intervention in order to avoid the fastidious point and click phase that is still necessary in many systems. The intensity images are then re-projected into the laser coordinate frame to produce an image that combines the laser reflectance and the available video intensity images into a colour texture map.

Multisensor fusion for volumetric reconstruction of large outdoor areas

This paper presents techniques for the merging of 3D data coming from different sensors, such as ground and aerial laser range scans. The 3D models created are reconstructed to give a photo-realistic scene enabling interactive virtual walkthroughs, measurements and scene change analysis. The reconstructed model is based on a weighted integration of all available data based on sensor-specific parameters such as noise level, accuracy, inclination and reflectivity of the target, spatial distribution of points. The geometry is robustly reconstructed with a volumetric approach. Once registered and weighed, all data is re-sampled in a multi-resolution distance field using out-of-core techniques. The final mesh is extracted by contouring the iso-surface with a feature preserving dual contouring algorithm. The paper shows results of the above technique applied to Verona (Italy) city centre.

Acquisition, modelling and rendering of very large urban environments

We describe a vehicle borne data acquisition system for urban environments and associated 3D data management and interactive rendering software. The data acquisition system is capable of acquiring 3D data from urban areas with centimetre resolution including automatic capturing of colour. The system includes a management and interactive rendering software which is designed to cope with the huge quantities of data generated by the acquisition system. It uses out-of-core preprocessing to transform data into octrees. Real-time interactive rendering is achieved by using novel techniques such as front-to-back octree traversal, occlusion query and speculative prefetching. The paper presents the results of the described techniques applied to large public areas including the City Centre of Verona, Italy.

Combining intensity and range images for 3D modelling

This paper presents a process combining range and intensity based techniques, in order to get better 3D models than those obtained using these techniques separately. The procedure needs an initial estimation for internal and external camera parameters for two or more intensity images. The technique uses passive triangulation to refine initial camera calibrations and ensure a good registration of range and video data sets. Afterwards, corresponding points from the intensity images are triangulated and introduced in the original range cloud of points. The objective is to complete the models in areas where data is missing or to increase the resolution in areas of high interest and 3D contents.

View planning for the 3D modelling of real world scenes

In recent years, several methods of reconstructing 3D scenes from range images have been proposed. Currently, the factor limiting a fully automated reconstruction process is the lack of a sufficiently scalable planning algorithm for the acquisition step. We present a technique of iteratively planning the next view for a range scanner in an initially unknown, large indoor environment. When designing an algorithm suitable for architectural environments, two main problems have to be addressed. First, an objective function has to be defined to optimise the parameters of the next view. We propose an objective function based on the analysis of occlusions which takes into account both a quality criterion and the cost of the next acquisition. Second, the parameters of the next view have to be computed efficiently for a large search space with eight degrees of freedom. Our technique exploits hardware-accelerated rendering (OpenGL) in order to perform the expensive visibility computation, which reduces the computation time of one planning step to a few minutes. Results are shown for two large indoor scenes-an artificial scene and a real world room-with numerous self occlusions.