Remco C. Veltkamp

A survey of content based 3D shape retrieval methods

Recent developments in techniques for modeling, digitizing and visualizing 3D shapes has led to an explosion in the number of available 3D models on the Internet and in domain-specific databases. This has led to the development of 3D shape retrieval systems that, given a query object, retrieve similar 3D objects. For visualization, 3D shapes are often represented as a surface, in particular polygonal meshes, for example in VRML format. Often these models contain holes, intersecting polygons, are not manifold, and do not enclose a volume unambiguously. On the contrary, 3D volume models, such as solid models produced by CAD systems, or voxels models, enclose a volume properly. This paper surveys the literature on methods for content based 3D retrieval, taking into account the applicability to surface models as well as to volume models. The methods are evaluated with respect to several requirements of content based 3D shape retrieval, such as: (1) shape representation requirements, (2) properties of dissimilarity measures, (3) efficiency, (4) discrimination abilities, (5) ability to perform partial matching, (6) robustness, and (7) necessity of pose normalization. Finally, the advantages and limitations of the several approaches in content based 3D shape retrieval are discussed.

State of the art in shape matching

Large image databases are used in an extraordinary number of multimedia applications in fields such as entertainment, business, art, engineering, and science. Retrieving images by their content, as opposed to external features, has become an important operation. A fundamental ingredient for content-based image retrieval is the technique used for comparing images. There are two general methods for image comparison: intensity based (color and texture) and geometry based (shape). A recent user survey about cognition aspects of image retrieval shows that users are more interested in retrieval by shape than by color and texture [62]. However, retrieval by shape is still considered one of the most difficult aspects of content-based search. Indeed, systems such as IBM’s Query By Image Content, QBIC [57], perhaps one of the most advanced image retrieval systems to date, is relatively successful in retrieving by color and texture, but performs poorly when searching on shape. A similar behavior is exhibited in the new Alta Vista photo finder [10].

Content-Based Music Information Retrieval: Current Directions and Future Challenges

The steep rise in music downloading over CD sales has created a major shift in the music industry away from physical media formats and towards online products and services. Music is one of the most popular types of online information and there are now hundreds of music streaming and download services operating on the World-Wide Web. Some of the music collections available are approaching the scale of ten million tracks and this has posed a major challenge for searching, retrieving, and organizing music content. Research efforts in music information retrieval have involved experts from music perception, cognition, musicology, engineering, and computer science engaged in truly interdisciplinary activity that has resulted in many proposed algorithmic and methodological solutions to music search using content-based methods. This paper outlines the problems of content-based music information retrieval and explores the state-of-the-art methods using audio cues (e.g., query by humming, audio fingerprinting, content-based music retrieval) and other cues (e.g., music notation and symbolic representation), and identifies some of the major challenges for the coming years.

Shape matching: similarity measures and algorithms

Shape matching is an important ingredient in shape retrieval, recognition and classification, alignment and registration, and approximation and simplification. This paper treats various aspects that are needed to solve shape matching problems: choosing the precise problem, selecting the properties of the similarity measure that are needed for the problem, choosing the specific similarity measure, and constructing the algorithm to compute the similarity. The focus is on methods that lie close to the field of computational geometry.

State-of-the-art in content-based image and video retrieval

Preface. 1. Image Content Analysis and Description X. Zabulis, S.C. Orphanoudakis. 2. Local Features for Image Retrieval L. Van Gool, et al. 3. Fast Invariant Feature Extraction for Image Retrieval S. Siggelkow, H. Burkhardt. 4. Shape Description and Search for Similar Objects in Image Databases L.J. Latecki, R. Lakaemper. 5. Features in Content-based Image Retrieval Systems: a Survey R.C. Veltkamp, et al. 6. Probablistic Image Models for Object Recognition and Pose Estimation J. Hornegger, H. Niemann. 7. Distribution-based Image Similarity J. Puzicha. 8. Distribution Free Statistics for Segmentation G. Frederix, E.J. Pauwels. 9. Information Retrieval Methods for Multimedia Objects N. Fuhr. 10. New descriptors for image and video indexing P. Gros, et al. 11. Facial and Motion Analysis for Image and Video Retrieval M. Tistarelli, E. Grosso. 12. Asymmetric Similarity Measures for Video Summarisation S.M. Iacob, et al. 13. Video Retrieval using Semantic Data A. Del Bimbo. 14. Adaptable Similarity Search in Large Image Databases T. Seidl, H.-P. Kriegel. 15. Parallel NN-search for large multimedia repositories R. Weber, et al.

Efficient image retrieval through vantage objects

We describe a new indexing structure for general image retrieval that relies solely on a distance function giving the similarity between two images. For each image object in the database, its distance to a set of m predetermined vantage objects is calculated; the m-vector of these distances specifies a point in the m-dimensional vantage space. The database objects that are similar (in terms of the distance function) to a given query object can be determined by means of an efficient nearest-neighbor search on these points. We demonstrate the viability of our approach through experimental results obtained with two image databases, one consisting of about 5200 raster images of stamps, the other containing about 72,000 hieroglyphic polylines.

A Survey of Content-Based Image Retrieval Systems

In this chapter we survey some technical aspects of current content-based image retrieval systems.

POLYHEDRAL MODEL RETRIEVAL USING WEIGHTED POINT SETS

Due to the recent improvements in laser scanning technology, 3D visualization and modeling, there is an increasing need for tools supporting the automatic search for 3D objects in archives. In this paper we describe a new geometric approach to 3D shape comparison and retrieval for arbitrary objects described by 3D polyhedral models that may contain gaps. In contrast with the existing approaches, our approach takes the overall relative spatial location into account by representing the 3D shape as a weighted point set. To compare two objects geometrically, we enclose each object by a 3D grid and generate a weighted point set, which represents a salient point for each non-empty grid cell. We compare three methods to obtain a salient point and a weight in each grid cell: (1) choosing the vertex in the cell with the highest Gaussian curvature, and choosing a measure as weight for that curvature, (2) choosing the area-weighted mean of the vertices in the cell, and choosing a measure as weight denoting the normal variation of the facets in the cell and (3) choosing the center of mass of all vertices in the cell, and choosing one as weight. Finally, we compute the similarity between two shapes by comparing their weighted point sets using a new shape similarity measure based on weight transportation that is a variation of the Earth Movers Distance. Unlike the Earth Movers Distance, the new shape similarity measure satisfies the triangle inequality. This property makes it suitable for use in indexing schemes, that depend on the triangle inequality, such as the one we introduce, based on the so-called vantage objects. The strength of our approach is proven through experimental results using a database consisting of 133 models such as mugs, cars and boats, and a database consisting of 512 models, mostly air planes, classified into conventional air planes, delta-jets, multi-fuselages, biplanes, helicopters and other models. The results show that the retrieval performance is better than related shape matching methods.

Properties and Performance of Shape Similarity Measures

This paper gives an overview of shape dissimilarity measure properties, such as metric and robustness properties, and of retrieval performance measures. Fifteen shape similarity measures are shortly described and compared. Their retrieval results on the MPEG-7 Core Experiment CE-Shape-1 test set as reported in the literature and obtained by a reimplementation are compared and discussed.

Reliable and Efficient Pattern Matching Using an Affine Invariant Metric

We present a new pattern similarity measure that behaves well under affine transformations. Our similarity measure is useful for pattern matching since it is defined on patterns with multiple components, satisfies the metric properties, is invariant under affine transformations, and is robust with respect to perturbation and occlusion. We give an algorithm, based on hierarchical subdivision of transformation space, which minimises our measure under the group of affine transformations, given two patterns. In addition, we present results obtained using an implementation of this algorithm.