Gianluigi Ciocca

Image annotation using SVM

The paper describes an innovative image annotation tool for classifying image regions in one of seven classes - sky, skin, vegetation, snow, water, ground, and buildings - or as unknown. This tool could be productively applied in the management of large image and video databases where a considerable volume of images/frames there must be automatically indexed. The annotation is performed by a classification system based on a multi-class Support Vector Machine. Experimental results on a test set of 200 images are reported and discussed.

An innovative algorithm for key frame extraction in video summarization

Video summarization, aimed at reducing the amount of data that must be examined in order to retrieve the information desired from information in a video, is an essential task in video analysis and indexing applications. We propose an innovative approach for the selection of representative (key) frames of a video sequence for video summarization. By analyzing the differences between two consecutive frames of a video sequence, the algorithm determines the complexity of the sequence in terms of changes in the visual content expressed by different frame descriptors. The algorithm, which escapes the complexity of existing methods based, for example, on clustering or optimization strategies, dynamically and rapidly selects a variable number of key frames within each sequence. The key frames are extracted by detecting curvature points within the curve of the cumulative frame differences. Another advantage is that it can extract the key frames on the fly: curvature points can be determined while computing the frame differences and the key frames can be extracted as soon as a second high curvature point has been detected. We compare the performance of this algorithm with that of other key frame extraction algorithms based on different approaches. The summaries obtained have been objectively evaluated by three quality measures: the Fidelity measure, the Shot Reconstruction Degree measure and the Compression Ratio measure.

Color-based image retrieval using spatial-chromatic histograms

Abstract The paper describes a new indexing methodology for image databases integrating color and spatial information for content-based image retrieval. This methodology, called Spatial-Chromatic Histogram (SCH), synthesizing in few values information about the location of pixels having the same color and their arrangement within the image, can be more satisfactory than standard techniques when the user would like to retrieve from the database the images that actually resemble the query image selected in their color distribution characteristics. Experimental trials on a database of about 3000 images are reported and compared with more standard techniques, like Color Coherence Vectors, on the basis of human perceptual judgments.

A relevance feedback mechanism for content-based image retrieval

Abstract Content-based image retrieval systems require the development of relevance feedback mechanisms that allow the user to progressively refine the systems response to a query. In this paper a new relevance feedback mechanism is described which evaluates the feature distributions of the images judged relevant, or not relevant, by the user and dynamically updates both the similarity measure and the query in order to accurately represent the users particular information needs. Experimental results demonstrate the effectiveness of this mechanism.

Self-Adaptive Image Cropping for Small Displays

We propose a new self-adaptive image cropping algorithm where the processing steps are driven by the classification of the images into semantic classes. The algorithm exploits both visual and semantic information. Visual information is obtained from a visual attention model, while semantic information relates to the automatically assigned image genre and to the detection of face and skin regions.

Quicklook2: An Integrated Multimedia System

tion retrieval engine of Quicklook 2 . Quicklook 2 allows the user to query image and multimedia databases with the aid of sample images, or an impromptu sketch and/or textual descriptions, and progressively refine the system’s response by indicating the relevance, or non-relevance of the retrieved items. The major innovation of the system is its relevance feedback mechanism that performs a statistical analysis of both the image and textual feature distributions of the retrieved items the user has judged relevant, or not relevant to identify what features the user has taken into account (and to what extent) in formulating this judgement, and then weigh their influence in the overall evaluation of similarity, as well as in the formulation of a new, single query that better expresses the user’s multimedia information needs. Another important contribution is the design and integration with the relevance feedback mechanism of an indexing scheme based on triangle inequality to improve retrieval efficiency. The performance of the system is illustrated with examples from various application domains and for different types of queries (target search as well as similarity search). ( 2001 Academic Press

Content-based similarity retrieval of trademarks using relevance feedback

This paper addresses the problemof how to e

AUTOMATIC CLASSIFICATION OF DIGITAL PHOTOGRAPHS BASED ON DECISION FORESTS

ciently and e!ectively retrieve images similar to a query from a trademark database purely on the basis of low-level feature analysis. It investigates the hypothesis that the low-level image features used to index the trademark images can be correlated with image contents by applying a relevance feedback mechanism that evaluates the feature distributions of the images the user has judged relevant, or not relevant and dynamically updates both the similarity measure and query in order to better represent the users particular information needs. Experimental results on a database of 1100 trademarks are reported and commented. 2001

Content-based color image retrieval with relevance feedback

Annotating photographs with broad semantic labels can be useful in both image processing and content-based image retrieval. We show here how low-level features can be related to semantic photo categories, such as indoor, outdoor and close-up, using decision forests consisting of trees constructed according to CART methodology. We also show how the results can be improved by introducing a rejection option in the classification process. Experimental results on a test set of 4,500 photographs are reported and discussed.

How Far Can You Get by Combining Change Detection Algorithms

We describe the main features of Quicklook, a prototype system that allows the user to query an image database and progressively refine the systems response by indicating the relevance, or nonrelevance of the retrieved items. Our experience with Quicklook demonstrates that user interaction with the system greatly improves retrieval results, making it possible, with no significant effort by the user, to tune the similarity measure used by the system to the users notion of image similarity.