Pilar Merchán

3D scene retrieval and recognition with Depth Gradient Images

The intention of the strategy proposed in this paper is to solve the object retrieval problem in highly complex scenes using 3D information. In the worst case scenario the complexity of the scene includes several objects with irregular or free-form shapes, viewed from any direction, which are self-occluded or partially occluded by other objects with which they are in contact and whose appearance is uniform in intensity/color. This paper introduces and analyzes a new 3D recognition/pose strategy based on DGI (Depth Gradient Images) models. After comparing it with current representative techniques, we can affirm that DGI has very interesting prospects.The DGI representation synthesizes both surface and contour information, thus avoiding restrictions concerning the layout and visibility of the objects in the scene. This paper first explains the key concepts of the DGI representation and shows the main properties of this method in comparison to a set of known techniques. The performance of this strategy in real scenes is then reported. Details are also presented of a wide set of experimental tests, including results under occlusion, performance with injected noise and experiments with cluttered scenes of a high level of complexity.

Cultural Heritage: A hybrid human-computer approach for recovering incomplete cultural heritage pieces

The automatic reconstruction of archeological pieces through the integration of a set of unknown segments is a highly complex problem which is still being researched. When only a few segments of the original piece are available, solutions exclusively based on computational algorithms are inefficient when attempting to create a credible whole restoration. Incomplete 3D puzzles must consequently be tackled by considering hybrid human/computer strategies. This paper presents a reconstruction approach in which the knowledge of human experts and computational solutions coexist together. Hypotheses, models and integration solutions originating from both humans and computers are thus continuously updated until an agreement is reached. This semi-automatic restoration approach has been tested on a set of ancient fractured pieces belonging to the remains of Roman sculptures at the well known Merida site (Spain), and promising results have been obtained. The successful results and applicability of this method have led us to believe that computational solutions should evolve towards hybrid human-computer strategies.

3D scene analysis from a single range image through occlusion graphs

This paper presents a new strategy to extract knowledge about the objects and their relative location in a complex scene when a single range image is taken. The analysis process is based on a range data distributed segmentation technique, which separates the components of the scene, and on a silhouette segmentation method, which classified the silhouette in real (non occluded) and false (occluded) parts. Finally, an occlusion graph provides a compact representation about the layout and relationship of the objects in the scene. This information is essential before higher level tasks in complex scenes - like recognition, understanding and robot interaction - are carried out. An extensive experimentation has been accomplished under real conditions in scenes of up to 12 objects yielding a very good performance. The experiments and results carried out validate the goodness of this approach in 3D environments.

Exploration trees on highly complex scenes: A new approach for 3D segmentation

A new strategy for automatic object extraction in highly complex scenes is presented in this paper. The method proposed gives a solution for 3D segmentation avoiding most restrictions imposed in other techniques. Thus, our technique is applicable on unstructured 3D information (i.e. cloud of points), with a single view of the scene, scenes consisting of several objects where contact, occlusion and shadows are allowed, objects with uniform intensity/texture and without restrictions of shape, pose or location. In order to have a fast segmentation stopping criteria, the number of objects in the scene is taken as input. The method is based on a new distributed segmentation technique that explores the 3D data by establishing a set of suitable observation directions. For each exploration viewpoint, a strategy [3D data]-[2D projected data]-[2D segmentation]-[3D segmented data] is accomplished. It can be said that this strategy is different from current 3D segmentation strategies. This method has been successfully tested in our lab on a set of real complex scenes. The results of these experiments, conclusions and future improvements are also shown in the paper.

3D Complex Scenes Segmentation from a Single Range Image Using Virtual Exploration

In this paper we present a method for automatic segmentation of 3D complex scenes from a single range image. A complex scene includes several objects with: irregular shapes, occlusion, the same colour or intensity level and placed in any pose. Unlike most existing methods which proceed with a set of images obtained from different viewpoints, in this work a single view is used and a 3D segmentation process is developed to separate the constituent parts of a complex scene. The method is based on establishing suitable virtualviewpoints in order to carry out a new range data segmentation technique. For a virtual-viewpoint a strategy [3D range data] – [2D projected range data] – [2D segmentation] – [3D segmented range data], is accomplished. The proposed method has been applied to a set of complex scenes and it can be said that the results guarantee the benefits of the method.

Restitution of Sculptural Groups Using 3D Scanners

Imagine for a moment that you have to solve a 3D jigsaw of which you have lost several pieces. You have also lost the original box-top showing the final picture, and as if that were not enough, some of the pieces you do have may belong to some other jigsaw. This is in essence the sort of challenge that we faced in the novel project that we shall be describing in this paper. The final aim of the project was, with the help of 3D scanners, to digitalize and reconstruct multi-piece classical sculptures. Particularly, we tackle the restitution of the so-called “Aeneas Group”, a famous iconographic reference during the Roman Empire. We have undertaken this ambitious project in collaboration with the research department of the Spanish National Museum of Roman Art (MNAR). This paper summarizes the real problems that arose and had to be solved, the innovations, and the main results of the work that we have carried out over these recent years.

Geometric and Colour Data Fusion for Outdoor 3D Models

This paper deals with the generation of accurate, dense and coloured 3D models of outdoor scenarios from scanners. This is a challenging research field in which several problems still remain unsolved. In particular, the process of 3D model creation in outdoor scenes may be inefficient if the scene is digitalized under unsuitable technical (specific scanner on-board camera) and environmental (rain, dampness, changing illumination) conditions. We address our research towards the integration of images and range data to produce photorealistic models. Our proposal is based on decoupling the colour integration and geometry reconstruction stages, making them independent and controlled processes. This issue is approached from two different viewpoints. On the one hand, given a complete model (geometry plus texture), we propose a method to modify the original texture provided by the scanner on-board camera with the colour information extracted from external images taken at given moments and under specific environmental conditions. On the other hand, we propose an algorithm to directly assign external images onto the complete geometric model, thus avoiding tedious on-line calibration processes. We present the work conducted on two large Roman archaeological sites dating from the first century A.D., namely, the Theatre of Segobriga and the Fori Porticus of Emerita Augusta, both in Spain. The results obtained demonstrate that our approach could be useful in the digitalization and 3D modelling fields.

A Robust Method for Filling Holes in 3D Meshes Based on Image Restoration

In this work a method for filling holes in 3D meshes based on a 2D image restoration algorithm is expounded. Since 3D data must be converted to a suitable input format, a 3D to 2D transformation is executed by projecting the 3D surface onto a grid. The storage of the depth information in every grid provides the 2D image which the restoration algorithms is applied in. Finally, an inverse transformation 2D to 3D is performed and the new produced data added to the damaged mesh. To test the method, artificial holes have been generated on a set of 3D surfaces. The distances between 3D original surfaces (before damaging it) and 3D repaired ones have been measured and a comparison with a commercial software has been established. Furthermore, the relation between holes areas and success rates has been also studied. This method has been applied to the sculptures of the collection from the National Museum of Roman Art in Spain with good results.

Reconocimiento de Objetos de Forma Libre a Partir de los Datos de Rango de una Vista Parcial Usando Cono Curvaturas Ponderadas

Este trabajo se ha desarrollado gracias a la financiacion del Ministerio de Educacion y Ciencia, a traves de los proyectos de la CICYT DPI2005-03769 y DPI2006-14794-C02.

Using Non Local Features for 3D Shape Grouping

This work faces the problem of 3D shape clustering when the whole surface information is available. The key of our method is to use a flexible feature, called Cone-Curvature, which provides local and extended information around every node of the mesh that represents the object. Thus as we increase the region around a node a new order of CC can be calculated. This feature, which was originally defined on spherical representation, has been adapted to work with standard triangular meshes and it is used for defining a similarity measure between shapes. Through a PCA technique, the dimensionality of the shape representation is drastically reduced and the hierarchical grouping can be efficiently carried out. This method has been tested under real conditions for a wide set of free shapes yielding promising results. We present a discussion of the clustering comparing human and computer results.