João G. M. Gonçalves

Automated reconstruction of 3D models from real environments

Abstract This paper describes an integrated approach to the construction of textured 3D scene models of building interiors from laser range data and visual images. This approach has been implemented in a collection of algorithms and sensors within a prototype device for 3D reconstruction, known as the EST (Environmental Sensor for Telepresence). The EST can take the form of a push trolley or of an autonomous mobile platform. The Autonomous EST (AEST) has been designed to provide an integrated solution for automating the creation of complete models. Embedded software performs several functions, including triangulation of the range data, registration of video texture, registration and integration of data acquired from different capture points. Potential applications include facilities management for the construction industry and creating reality models to be used in general areas of virtual reality, for example, virtual studios, virtualised reality for content-related applications (e.g., CD-ROMs), social telepresence, architecture and others. The paper presents the main components of the EST/AEST, and presents some example results obtained from the prototypes. The reconstructed model is encoded in VRML format so that it is possible to access and view the model via the World Wide Web.

Registration and fusion of intensity and range data for 3D modelling of real world scenes

A novel technique combining intensity and range data is presented. Passive (intensity based) and active (range based) techniques used for 3D reconstruction have their limitations and separately, none of these techniques can solve all the problems inherent to the modelling of real environments. Our technique aims to demonstrate how both intensity and range data can be registered and combined into a long-range 3D system. The procedure needs an initial estimation for internal and external camera parameters for two or more intensity images. The technique uses passive triangulation of the intensity data to refine the initial camera calibrations and ensure a good registration of range and video data sets. Once a reliable calibration is achieved, corresponding points from the intensity images are triangulated and introduced in the original range data. With our technique, it is possible to complete the models in areas where data is missing or to increase the resolution in areas of high interest and 3D contents.

3D reality modelling: photo-realistic 3D models of real world scenes

This paper presents a 3D reality modelling system, applying lasers and digital photography to create super-accurate photo-realistic three-dimensional worlds. A key feature of the 3D reality system is its flexibility. It can be used with all common long-range scanners and digital or video cameras, without the need of any pre-calibration or static arrangement between the different sensors. The system can be used for a wide range of applications in industry and science, as well as for environmental modelling and virtual reality presentations. The 3D models are sufficiently lifelike to be used in sectors such as real estate, cultural heritage and construction. Results can be exported to all major 3D software and CAD packages or viewed via a Web browser.

3D environment modelling using laser range sensing

This paper describes a technique for constructing a geometric model of an unknown environment based on data acquired by a Laser Range Finder on board of a mobile robot. The geometric model would be most useful both for navigation and verification purposes. The paper presents all the steps needed for the description of the environment, including the range image acquisition and processing, 3D surface reconstruction and the problem of merging multiple images in order to obtain a complete model.

Perception maps for the local navigation of a mobile robot: a neural network approach

Sensorial perception is a key issue for the problem of robot local navigation, that is, the immediate or short-range motion planning, reacting only to the free space around the robot, without requiring a pre-defined trajectory plan. Therefore, local navigation requires no environment model and relies entirely on sensorial data. Commonly used sensors such as ultrasonic ranging devices, are known for their associated problems: specular reflections and crosstalk, essentially. However if sensors are used in an appropriate number and geometric lay-outs, the resulting spatial redundancy offers the possibility of overcoming some of those problems. This paper deals with these problems by means of special perception maps using ultrasound data. A generalised grid serves as the base of maps, and its cells have simply binary values: free or occupied. The relation between the topology of the perception map and the environment is a determinant factor for accurate reasoning. A 3-layer feed-forward neural network is used to perform the mapping between sensorial scans and grid occupancy. It was verified that the neural network handles most of the situations of specular reflections, and gives good perception maps for mid-range distances. Changes in environment, such as obstacles in vehicles trajectory, have also been detected, which stresses the networks ability to generalise.<<ETX>>

Automatic registration of laser reflectance and colour intensity images for 3D reconstruction

Abstract The objective of the work presented in this paper is to generate complete, high-resolution models of real world scenes from passive intensity images and active range sensors. In previous work, an automatic method has been developed in order to compute 3D models of real world scenes from laser range data. The aim of this project is to improve these existing models by fusing range and intensity data. The paper presents different techniques in order to find correspondences between the different sets of data. Based on these control points, a robust camera calibration is computed with a minimal user intervention in order to avoid the fastidious point and click phase that is still necessary in many systems. The intensity images are then re-projected into the laser coordinate frame to produce an image that combines the laser reflectance and the available video intensity images into a colour texture map.

Multisensor fusion for volumetric reconstruction of large outdoor areas

This paper presents techniques for the merging of 3D data coming from different sensors, such as ground and aerial laser range scans. The 3D models created are reconstructed to give a photo-realistic scene enabling interactive virtual walkthroughs, measurements and scene change analysis. The reconstructed model is based on a weighted integration of all available data based on sensor-specific parameters such as noise level, accuracy, inclination and reflectivity of the target, spatial distribution of points. The geometry is robustly reconstructed with a volumetric approach. Once registered and weighed, all data is re-sampled in a multi-resolution distance field using out-of-core techniques. The final mesh is extracted by contouring the iso-surface with a feature preserving dual contouring algorithm. The paper shows results of the above technique applied to Verona (Italy) city centre.

XMPP based Context Management Architecture

In recent years several authors and projects have addressed the need for an architecture for collecting and managing context information. Most of the approaches have either been purely based in middlewares unable to address the heterogeneity of components and services platforms, or have been built around Web based technologies that are unable to cope with the realtime and reactiveness requirements that many context-aware use cases have. In this paper the authors propose a new Context Management Architecture able to solve these problems. Furthermore, this architecture, exploring the XMPP protocol, can be used as the basis for new developments in this area.

Acquisition, modelling and rendering of very large urban environments

We describe a vehicle borne data acquisition system for urban environments and associated 3D data management and interactive rendering software. The data acquisition system is capable of acquiring 3D data from urban areas with centimetre resolution including automatic capturing of colour. The system includes a management and interactive rendering software which is designed to cope with the huge quantities of data generated by the acquisition system. It uses out-of-core preprocessing to transform data into octrees. Real-time interactive rendering is achieved by using novel techniques such as front-to-back octree traversal, occlusion query and speculative prefetching. The paper presents the results of the described techniques applied to large public areas including the City Centre of Verona, Italy.

Combining intensity and range images for 3D modelling

This paper presents a process combining range and intensity based techniques, in order to get better 3D models than those obtained using these techniques separately. The procedure needs an initial estimation for internal and external camera parameters for two or more intensity images. The technique uses passive triangulation to refine initial camera calibrations and ensure a good registration of range and video data sets. Afterwards, corresponding points from the intensity images are triangulated and introduced in the original range cloud of points. The objective is to complete the models in areas where data is missing or to increase the resolution in areas of high interest and 3D contents.