Ron Barber

The QBIC Project : Querying Images by Content Using Color, Texture, and Shape

In the query by image content (QBIC) project we are studying methods to query large on-line image databases using the images content as the basis of the queries. Examples of the content we use include color, texture, and shape of image objects and regions. Potential applications include medical (`Give me other images that contain a tumor with a texture like this one), photo-journalism (`Give me images that have blue at the top and red at the bottom), and many others in art, fashion, cataloging, retailing, and industry. Key issues include derivation and computation of attributes of images and objects that provide useful query functionality, retrieval methods based on similarity as opposed to exact match, query by image example or user drawn image, the user interfaces, query refinement and navigation, high dimensional database indexing, and automatic and semi-automatic database population. We currently have a prototype system written in X/Motif and C running on an RS/6000 that allows a variety of queries, and a test database of over 1000 images and 1000 objects populated from commercially available photo clip art images. In this paper we present the main algorithms for color texture, shape and sketch query that we use, show example query results, and discuss future directions.

Efficient and effective querying by image content

In the QBIC (Query By Image Content) project we are studying methods to query large on-line image databases using the images content as the basis of the queries. Examples of the content we use include color, texture, shape, position, and dominant edges of image objects and regions. Potential applications include medical (“Give me other images that contain a tumor with a texture like this one”), photo-journalism (“Give me images that have blue at the top and red at the bottom”), and many others in art, fashion, cataloging, retailing, and industry. We describe a set of novel features and similarity measures allowing query by image content, together with the QBIC system we implemented. We demonstrate the effectiveness of our system with normalized precision and recall experiments on test databases containing over 1000 images and 1000 objects populated from commercially available photo clip art images, and of images of airplane silhouettes. We also present new methods for efficient processing of QBIC queries that consist of filtering and indexing steps. We specifically address two problems: (a) non Euclidean distance measures; and (b) the high dimensionality of feature vectors. For the first problem, we introduce a new theorem that makes efficient filtering possible by bounding the non-Euclidean, full cross-term quadratic distance expression with a simple Euclidean distance. For the second, we illustrate how orthogonal transforms, such as Karhunen Loeve, can help reduce the dimensionality of the search space. Our methods are general and allow some “false hits” but no false dismissals. The resulting QBIC system offers effective retrieval using image content, and for large image databases significant speedup over straightforward indexing alternatives. The system is implemented in X/Motif and C running on an RS/6000.

Automatic and semiautomatic methods for image annotation and retrieval in query by image content (QBIC)

Advances in technologies for scanning, networking, and CD-ROM, lower prices for large disk storage, and acceptance of common image compression and file formats have contributed to an increase in the number, size, and uses of on-line image collections. New tools are needed to help users create, manage, and retrieve images from these collections. We are developing QBIC (query by image content), a prototype system that allows a user to create and query image databases in which the image content -- the colors, textures, shapes, and layout of images and the objects they contain -- is used as the basis of queries. This paper describes two sets of algorithms in QBIC. The first are methods that allow `query by color drawing, a form of query in which a user draws an approximate color version of an image, and similar images are retrieved. These are automatic algorithms in the sense that no user action is necessary during database population. Secondly, we describe algorithms for semi-automatic identification of image objects during database population, improving the speed and usability of this manually-intensive step. Once outlined, detailed queries on the content-properties of these individual objects can be made at query time.

Interactive outlining : an improved approach using active contours

The purpose of our work is to outline objects on images in an interactive environment. We use an improved method based on energy minimizing active contours or `snakes. Kass et al., proposed a variational technique; Amini used dynamic programming; and Williams and Shah introduced a fast, greedy algorithm. We combine the advantages of the latter two methods in a two-stage algorithm. The first stage is a greedy procedure that provides fast initial convergence. It is enhanced with a cost term that extends over a large number of points to avoid oscillations. The second stage, when accuracy becomes important, uses dynamic programming. This step is accelerated by the use of alternating search neighborhoods and by dropping stable points from the iterations. We have also added several features for user interaction. First, the user can define points of high confidence. Mathematically, this results in an extra cost term and, in that way, the robustness in difficult areas (e.g., noisy edges, sharp corners) is improved. We also give the user the possibility of incremental contour tracking, thus providing feedback on the refinement process. The algorithm has been tested on numerous photographic clip art images and extensive tests on medical images are in progress.

Query by image content using multiple objects and multiple features: user interface issues

On-line collections of images are growing larger and more common, and tools are needed to efficiently manage, organize, and navigate through them. The authors have developed a prototype system called QBIC which allows complex multi-object and multi-feature queries of large image databases. The queries are based on image content-the colors, textures, shapes, and positions of images and the objects/regions they contain. The system computes numeric features to represent the image properties and uses similarity measures based on these features for image retrieval. The focus of the paper is its user interface which allows a user to graphically pose and refine queries based on multiple visual properties of images and their objects.<<ETX>>

Indexing for complex queries on a query-by-content image database

We describe how the QBIC (Query By Image Content) system handles multi-* queries-queries on large image collections involving multifeatures of each image as a whole and of multiple objects within each image. The queries are based on properties of image content-such as colors, textures, shapes, and edges. The system computes a set of features to describe the above properties, uses distance-like measures on the features to provide similarity based retrieval, and has a graphical interface that enable users pose queries visually. In this paper, we present QBIC indexing algorithms that allow these multi-* queries to run efficiently.

Ultimedia Manager: Query By Image Content and its applications

IBM Almaden Research Centers project on Query By Image Content (QBIC) is studying means to retrieve images from large image databases using image contents such as color, texture, shape and layout. In this paper, we describe the beta version of the PC-based Ultimedia Manager product, which is based on QBIC technology. We outline the product philosophy and give a demonstration of the current version. The product is expected to be announced soon, together with an OEM offering of the QBIC search and query engine.<<ETX>>