Santiago Salamanca

3D scene retrieval and recognition with Depth Gradient Images

The intention of the strategy proposed in this paper is to solve the object retrieval problem in highly complex scenes using 3D information. In the worst case scenario the complexity of the scene includes several objects with irregular or free-form shapes, viewed from any direction, which are self-occluded or partially occluded by other objects with which they are in contact and whose appearance is uniform in intensity/color. This paper introduces and analyzes a new 3D recognition/pose strategy based on DGI (Depth Gradient Images) models. After comparing it with current representative techniques, we can affirm that DGI has very interesting prospects.The DGI representation synthesizes both surface and contour information, thus avoiding restrictions concerning the layout and visibility of the objects in the scene. This paper first explains the key concepts of the DGI representation and shows the main properties of this method in comparison to a set of known techniques. The performance of this strategy in real scenes is then reported. Details are also presented of a wide set of experimental tests, including results under occlusion, performance with injected noise and experiments with cluttered scenes of a high level of complexity.

Improved Method for Object Recognition in Complex Scenes by Fusioning 3-D Information and RFID Technology

This work analyzes a new method for object recognition in complex scenes combining vision-based techniques applied to the 3-D data obtained using range sensors and object identification coming from radio frequency tags (radio frequency identification (RFID) technology). Three-dimensional vision-based algorithms for object recognition have many restrictions in practical applications, i.e., uncertainty, incapability for real-time tasks, etc., but they work well for pose determination once the object is recognized. On the other hand, RFID technology allows us to detect the presence of specific objects in a scene, but it cannot provide their localization, at least not with the accuracy required in applications such as ours. In this paper, we present a new and powerful recognition method obtained by fusing both techniques. The phases of the method are described, and abundant experimentation results are included. An in-depth performance analysis has been carried out to demonstrate the recognition improvements achieved by the algorithm when RFID assistance is considered. It helps to confirm the robustness of this fusion approach and prove its effectiveness. A final discussion is included, concerning what should be the most adequate size of the object database for optimal algorithm exploitation.

Cultural Heritage: A hybrid human-computer approach for recovering incomplete cultural heritage pieces

The automatic reconstruction of archeological pieces through the integration of a set of unknown segments is a highly complex problem which is still being researched. When only a few segments of the original piece are available, solutions exclusively based on computational algorithms are inefficient when attempting to create a credible whole restoration. Incomplete 3D puzzles must consequently be tackled by considering hybrid human/computer strategies. This paper presents a reconstruction approach in which the knowledge of human experts and computational solutions coexist together. Hypotheses, models and integration solutions originating from both humans and computers are thus continuously updated until an agreement is reached. This semi-automatic restoration approach has been tested on a set of ancient fractured pieces belonging to the remains of Roman sculptures at the well known Merida site (Spain), and promising results have been obtained. The successful results and applicability of this method have led us to believe that computational solutions should evolve towards hybrid human-computer strategies.

3D scene analysis from a single range image through occlusion graphs

This paper presents a new strategy to extract knowledge about the objects and their relative location in a complex scene when a single range image is taken. The analysis process is based on a range data distributed segmentation technique, which separates the components of the scene, and on a silhouette segmentation method, which classified the silhouette in real (non occluded) and false (occluded) parts. Finally, an occlusion graph provides a compact representation about the layout and relationship of the objects in the scene. This information is essential before higher level tasks in complex scenes - like recognition, understanding and robot interaction - are carried out. An extensive experimentation has been accomplished under real conditions in scenes of up to 12 objects yielding a very good performance. The experiments and results carried out validate the goodness of this approach in 3D environments.

3D Complex Scenes Segmentation from a Single Range Image Using Virtual Exploration

In this paper we present a method for automatic segmentation of 3D complex scenes from a single range image. A complex scene includes several objects with: irregular shapes, occlusion, the same colour or intensity level and placed in any pose. Unlike most existing methods which proceed with a set of images obtained from different viewpoints, in this work a single view is used and a 3D segmentation process is developed to separate the constituent parts of a complex scene. The method is based on establishing suitable virtualviewpoints in order to carry out a new range data segmentation technique. For a virtual-viewpoint a strategy [3D range data] – [2D projected range data] – [2D segmentation] – [3D segmented range data], is accomplished. The proposed method has been applied to a set of complex scenes and it can be said that the results guarantee the benefits of the method.

Restitution of Sculptural Groups Using 3D Scanners

Imagine for a moment that you have to solve a 3D jigsaw of which you have lost several pieces. You have also lost the original box-top showing the final picture, and as if that were not enough, some of the pieces you do have may belong to some other jigsaw. This is in essence the sort of challenge that we faced in the novel project that we shall be describing in this paper. The final aim of the project was, with the help of 3D scanners, to digitalize and reconstruct multi-piece classical sculptures. Particularly, we tackle the restitution of the so-called “Aeneas Group”, a famous iconographic reference during the Roman Empire. We have undertaken this ambitious project in collaboration with the research department of the Spanish National Museum of Roman Art (MNAR). This paper summarizes the real problems that arose and had to be solved, the innovations, and the main results of the work that we have carried out over these recent years.

Moving surface extraction based on unordered hexagonal perfect submaps projection: Applications to 3D feature tracking

In this paper, we present a structured light technique based on the projection of a color coded hexagonal array that is able to obtain range images from moving scenes. Repetition and disorder is allowed in the codeword which implies several advantages: the mean Hamming distance between contiguous codewords of the pattern increases, the code loss due to occlusions and discontinuities can be efficiently handled and the computational cost in the pattern-image correspondence phase is highly reduced. The structured light projection system has been tested under real moving scenes on medium resolution range images and for slow controlled movements. In order to validate the performance of our range vision system we have used it to identify and track several 3D feature points as the scene moves. To measure the accuracy of the tracking a 6 DOF manipulator robot has been included in the experimental setup. All this experimental work, the results and the main contributions of our method compared to other perfect map and submap based techniques are detailed in the paper.

Geometric and Colour Data Fusion for Outdoor 3D Models

This paper deals with the generation of accurate, dense and coloured 3D models of outdoor scenarios from scanners. This is a challenging research field in which several problems still remain unsolved. In particular, the process of 3D model creation in outdoor scenes may be inefficient if the scene is digitalized under unsuitable technical (specific scanner on-board camera) and environmental (rain, dampness, changing illumination) conditions. We address our research towards the integration of images and range data to produce photorealistic models. Our proposal is based on decoupling the colour integration and geometry reconstruction stages, making them independent and controlled processes. This issue is approached from two different viewpoints. On the one hand, given a complete model (geometry plus texture), we propose a method to modify the original texture provided by the scanner on-board camera with the colour information extracted from external images taken at given moments and under specific environmental conditions. On the other hand, we propose an algorithm to directly assign external images onto the complete geometric model, thus avoiding tedious on-line calibration processes. We present the work conducted on two large Roman archaeological sites dating from the first century A.D., namely, the Theatre of Segobriga and the Fori Porticus of Emerita Augusta, both in Spain. The results obtained demonstrate that our approach could be useful in the digitalization and 3D modelling fields.

HWM: a new spherical representation structure for modeling partial views of an object

Presents a modeling strategy (half modelling wave, HMW) for the situation when only partial information of the 3D object surface is available. The method is applicable to unstructured range data. It means that no previous knowledge of the sensor used or the rough position of the object with respect to the sensor is required to compute the model. The developed partial modeling method has been conceived as a tool to be applied in new solutions for conventional 3D computer vision problems. Main contributions concerning partial viewing and 3D occlusion are also presented.

A Robust Method for Filling Holes in 3D Meshes Based on Image Restoration

In this work a method for filling holes in 3D meshes based on a 2D image restoration algorithm is expounded. Since 3D data must be converted to a suitable input format, a 3D to 2D transformation is executed by projecting the 3D surface onto a grid. The storage of the depth information in every grid provides the 2D image which the restoration algorithms is applied in. Finally, an inverse transformation 2D to 3D is performed and the new produced data added to the damaged mesh. To test the method, artificial holes have been generated on a set of 3D surfaces. The distances between 3D original surfaces (before damaging it) and 3D repaired ones have been measured and a comparison with a commercial software has been established. Furthermore, the relation between holes areas and success rates has been also studied. This method has been applied to the sculptures of the collection from the National Museum of Roman Art in Spain with good results.