Stéphane Bres

Wavelet-based salient points for image retrieval

The use of interest points in content-based image retrieval allows the image index to represent local properties of the image. Classic corner detectors can be used for this purpose. However, they have drawbacks when applied to various natural images for image retrieval, because visual features need not be corners and corners may gather in small regions. We present a salient point detector that extract points where variations occur in the image, whether they are corner-like or not. The detector is based on the wavelet transform to detect global variations as well as local ones. The wavelet-based salient points are evaluated for image retrieval with a retrieval system using texture features. In this experiment our method provides better retrieval performance compared with other point detectors.

Detection of Interest Points for Image Indexation

This paper addresses the problem of detection and delineation of interest points in images as part of an automatic image and video indexing for search by content purposes project. We propose a novel key point detector based on multiresolution contrast information. We compare this detector to the Plessey feature point detector as well as the detector introduced in the SUSAN project. As we are interested in common database applications, we focus this comparison on robustness versus coding noise like Jpeg noise.

Document page similarity based on layout visual saliency: application to query by example and document classification

In this paper we propose to define a measure of visualsimilarity to compare different pages in a corpus. Thismeasure is based on the analysis of the visual layoutsaliency of the page composition. This similarity iscomputed using both the document layout andcharacteristics of the text itself. The text characterizationuses statistical features derived from textural primitives.Our purpose is to establish perceptive links betweendocuments in order to facilitate their storage and theirretrieval. In this paper we present two possibleapplications of this measure of similarity: the query ofthe corpus by example and the documents classification.In the first application, we extract documents that are themost visually similar to a document, given as query. Inthe second application, the similarity measure is used toclassify the document under investigation using its visualsimilarity to a reference set of documents. Our test corpusis extracted from the Finland MTDB Oulu multi-genredatabase that provides a great diversity of page layoutsand contents.

Hermite and Gabor transforms for noise reduction and handwriting classification in ancient manuscripts

In this paper, we propose a biologically inspired, global and segmentation free methodology for manuscript noise reduction and classification. Our method consists of developing well-adapted tools for writing enhancement, background noise, text and drawing separation and handwritten patterns characterization with orientation features. We have used here analysis of handwritten images in the spectral domain by frequency decompositions (Hermite transforms) and Gabor filtering for selective text information extraction. We have tested our approach of writing classification on ancient manuscripts corpus, mainly composed of 18th century authors’ documents. The current results are very promising: they show that our biologically inspired methodology can be efficiently used for handwriting analysis without any a priori grapheme segmentation.

Conditions of Similarity between Hermite and Gabor Filters as Models of the Human Visual System

Among the suggested mathematical models for receptive field profiles, the Gabor model is well known and widely used. Another less used model that agrees with the Gaussian derivative model for human vision is the Hermite model which is based on analysis filters of the Hermite transform. It has the advantage of an orthogonal basis and a better fit to cortical data. In this paper we present an analytical comparison based on minimization of the energy error between the two models, and so the optimal parameters letting the two models be close to each other are found. The results show that both models are equivalent and extract about the same frequency information. Actually, we can implement a Hermite filter with an equivalent Gabor filter and vice versa, provided that conditions leading to error minimization are held.

Multimodal recognition of visual concepts using histograms of textual concepts and selective weighted late fusion scheme

The text associated with images provides valuable semantic meanings about image content that can hardly be described by low-level visual features. In this paper, we propose a novel multimodal approach to automatically predict the visual concepts of images through an effective fusion of textual features along with visual ones. In contrast to the classical Bag-of-Words approach which simply relies on term frequencies, we propose a novel textual descriptor, namely the Histogram of Textual Concepts (HTC), which accounts for the relatedness of semantic concepts in accumulating the contributions of words from the image caption toward a dictionary. In addition to the popular SIFT-like features, we also evaluate a set of mid-level visual features, aiming at characterizing the harmony, dynamism and aesthetic quality of visual content, in relationship with affective concepts. Finally, a novel selective weighted late fusion (SWLF) scheme is proposed to automatically select and weight the scores from the best features according to the concept to be classified. This scheme proves particularly useful for the image annotation task with a multi-label scenario. Extensive experiments were carried out on the MIR FLICKR image collection within the ImageCLEF 2011 photo annotation challenge. Our best model, which is a late fusion of textual and visual features, achieved a MiAP (Mean interpolated Average Precision) of 43.69% and ranked 2nd out of 79 runs. We also provide comprehensive analysis of the experimental results and give some insights for future improvements.

Writer Identification Using Steered Hermite Features and SVM

Writer recognition is considered as a difficult problem to solve due to variations found in the writing, even from the same writer. In this paper, steered Hermite features are used to identify writer from a written document. We will show that steered Hermite features are highly useful for text images because they extract lot of information, notably for data characterized by oriented features, curves and segments. The algorithm we propose here, first calculates the steered Hermite features of the images which are then passed on to support vector machine for training and testing. The base of tests consists of sample of some lines of writings (five at most) of primarily diversified writings of authors from IAM database. With the proposed algorithm based on steered Hermite features, we were able to achieve an accuracy of around 83% percent for a set of 30 authors with non overlapping images of written text.

Curvelets Based Queries for CBIR Application in Handwriting Collections

This paper presents a new use of the curvelet transform as a multiscale method for indexing linear singularities and curved handwritten shapes in documents images. As it belongs to the wavelet family, this representation can be useful at several scales of details. The proposed scheme for handwritten shape characterization targets to detect oriented and curved fragments at different scales so as to compose an unique signature for each handwritten analyzed samples. In this way, curvelets coefficients are used as a representation tool for handwriting when searching in large manuscripts databases by finding similar handwritten samples. Current results of ancient manuscripts retrieval are very promising with very satisfying precisions and recalls.

Analysis and interpretation of visual saliency for document functional labeling

Abstract.In this paper we propose a complete methodology of printed text characterization for document labeling using texture features that have been inspired by a psychovisual approach. This approach considers visual human-based predicates to describe and identify text units according to their visual saliency and their perceptual attraction power on the reader’s eye. It supports a quick and robust process of functional labeling used to characterize text regions of document pages. The test databases are the Finland MTDB Oulu base J. Sauvola and H. Kauniskangas (1999) MediaTeam Document Database II, a CD-ROM document image collection, Oulu University, Finland that provides a great panel of document layouts and contents and our laboratory corpus that contains a large variety of composite documents (about 200 pages). The performance of the method gives very promising results.

Spatio-temporal Primitive Extraction Using Hermite and Laguerre Filters for Early Vision Video Indexing

In this paper we integrate spatial and temporal information, which are extracted separately from a video sequence, for indexing and retrieval purposes. We focus on two filter families that are suitable models of the human visual system for spatial and temporal information encoding. They are special cases of polynomial transforms that perform local decompositions of a signal. Spatial primitives are extracted using Hermite filters, which agree with the Gaussian derivative model of receptive field profiles. Temporal events are characterized by Laguerre filters, which preserve the causality constraint in the temporal domain. Integration of both models gives a spatio-temporal feature extractor based on early vision. They are efficiently implemented as two independent sets of discrete channels, Krawtchouk and Meixner, whose outputs are combined for indexing a video sequence. Results encourage our model for video indexing and retrieval.