Sezer Karaoglu

Object reading: text recognition for object recognition

We propose to use text recognition to aid in visual object class recognition. To this end we first propose a new algorithm for text detection in natural images. The proposed text detection is based on saliency cues and a context fusion step. The algorithm does not need any parameter tuning and can deal with varying imaging conditions. We evaluate three different tasks: 1. Scene text recognition, where we increase the state-of-the-art by 0.17 on the ICDAR 2003 dataset. 2. Saliency based object recognition, where we outperform other state-of-the-art saliency methods for object recognition on the PASCAL VOC 2011 dataset. 3. Object recognition with the aid of recognized text, where we are the first to report multi-modal results on the IMET set. Results show that text helps for object class recognition if the text is not uniquely coupled to individual object instances.

A Novel Algorithm for Text Detection and Localization in Natural Scene Images

Text data in an image present useful information for annotation, indexing and structuring of images. The gathered information from images can be applied for devices for impaired people, navigation, tourist assistance or georeferencing business. In this paper we propose a novel algorithm for text detection and localization from outdoor/indoor images which is robust against different font size, style, uneven illumination, shadows, highlights, over exposed regions, low contrasted images, specular reflections and many distortions which makes text localization task harder. A binarization algorithm based on difference of gamma correction and morphological reconstruction is realized to extract the connected components of an image. These connected components are classified as text and non test using a Random Forest classifier. After that text regions are localized by a novel merging algorithm for further processing.

Color correction: a novel weighted Von Kries model based on memory colors

In this paper we present an automatic color correction framework based on memory colors. Memory colors for 3 different objects: grass, snow and sky are obtained using psychophysical experiments under different illumination levels and later modeled statistically. While supervised image segmentation method detects memory color objects, a luminance level predictor classifies images as dark, dim or bright. This information along with the best memory color model that fits to the data is used to do the color correction using a novel weighted Von Kries formula. Finally, a visual experiment is conducted to evaluate color corrected images. Experimental results suggest that the proposed weighted von Kries model is an appropriate color correction model for natural images.

Con-text: text detection using background connectivity for fine-grained object classification

This paper focuses on fine-grained classification by detecting photographed text in images. We introduce a text detection method that does not try to detect all possible foreground text regions but instead aims to reconstruct the scene background to eliminate non-text regions. Object cues such as color, contrast, and objectiveness are used in corporation with a random forest classifier to detect background pixels in the scene. Results on two publicly available datasets ICDAR03 and a fine-grained Building subcategories of ImageNet shows the effectiveness of the proposed method.

Detect2Rank: Combining Object Detectors Using Learning to Rank

Object detection is an important research area in the field of computer vision. Many detection algorithms have been proposed. However, each object detector relies on specific assumptions of the object appearance and imaging conditions. As a consequence, no algorithm can be considered universal. With the large variety of object detectors, the subsequent question is how to select and combine them. In this paper, we propose a framework to learn how to combine object detectors. The proposed method uses (single) detectors like Deformable Part Models, Color Names and Ensemble of Exemplar-SVMs, and exploits their correlation by high-level contextual features to yield a combined detection list. Experiments on the PASCAL VOC07 and VOC10 data sets show that the proposed method significantly outperforms single object detectors, DPM (8.4%), CN (6.8%) and EES (17.0%) on VOC07 and DPM (6.5%), CN (5.5%) and EES (16.2%) on VOC10. We show with an experiment that there are no constraints on the type of the detector. The proposed method outperforms (2.4%) the state-of-the-art object detector (RCNN) on VOC07 when Regions with Convolutional Neural Network is combined with other detectors used in this paper.

Detecting text in natural scenes based on a reduction of photometric effects: problem of color invariance

In this paper, we propose a novel method for detecting and segmenting text layers in complex images. This method is robust against degradations such as shadows, non-uniform illumination, low-contrast, large signaldependent noise, smear and strain. The proposed method first uses a geodesic transform based on a morphological reconstruction technique to remove dark/light structures connected to the borders of the image and to emphasize on objects in center of the image. Next uses a method based on difference of gamma functions approximated by the Generalized Extreme Value Distribution (GEVD) to find a correct threshold for binarization. The main function of this GEVD is to find the optimum threshold value for image binarization relatively to a significance level. The significance levels are defined in function of the background complexity. In this paper, we show that this method is much simpler than other methods for text binarization and produces better text extraction results on degraded documents and natural scene images.

Detecting text in natural scenes based on a reduction of photometric effects: problem of text detection

In this paper, we propose a novel method for detecting and segmenting text layers in complex images. This method is robust against degradations such as shadows, non-uniform illumination, low-contrast, large signaldependent noise, smear and strain. The proposed method first uses a geodesic transform based on a morphological reconstruction technique to remove dark/light structures connected to the borders of the image and to emphasize on objects in center of the image. Next uses a method based on difference of gamma functions approximated by the Generalized Extreme Value Distribution (GEVD) to find a correct threshold for binarization. The main function of this GEVD is to find the optimum threshold value for image binarization relatively to a significance level. The significance levels are defined in function of the background complexity. In this paper, we show that this method is much simpler than other methods for text binarization and produces better text extraction results on degraded documents and natural scene images.

Con-Text: Text Detection for Fine-Grained Object Classification

This paper focuses on fine-grained object classification using recognized scene text in natural images. While the state-of-the-art relies on visual cues only, this paper is the first work which proposes to combine textual and visual cues. Another novelty is the textual cue extraction. Unlike the state-of-the-art text detection methods, we focus more on the background instead of text regions. Once text regions are detected, they are further processed by two methods to perform text recognition, i.e., ABBYY commercial OCR engine and a state-of-the-art character recognition algorithm. Then, to perform textual cue encoding, bi- and trigrams are formed between the recognized characters by considering the proposed spatial pairwise constraints. Finally, extracted visual and textual cues are combined for fine-grained classification. The proposed method is validated on four publicly available data sets: ICDAR03, ICDAR13, Con-Text, and Flickr-logo. We improve the state-of-the-art end-to-end character recognition by a large margin of 15% on ICDAR03. We show that textual cues are useful in addition to visual cues for fine-grained classification. We show that textual cues are also useful for logo retrieval. Adding textual cues outperforms visual- and textual-only in fine-grained classification (70.7% to 60.3%) and logo retrieval (57.4% to 54.8%).

Per-patch metric learning for robust image matching

We propose a patch-specific metric learning method to improve matching performance of local descriptors. Existing methodologies typically focus on invariance, by completely considering, or completely disregarding all variations. We propose a metric learning method that is robust to only a range of variations. The ability to choose the level of robustness allows us to fine-tune the trade-off between invariance and discriminative power. We learn a distance metric for each patch independently by sampling from a set of relevant image transformations. These transformations give a-priori knowledge about the behavior of the query patch under the applied transformation in feature space. We learn the robust metric by either fully generating only the relevant range of transformations, or by a novel direct metric. The matching between query patch and data is performed with this new metric. Results on the ALOI dataset show that the proposed method improves performance of SIFT by 6.22% for geometric and 4.43% for photometric transformations.

Age estimation under changes in image quality: An experimental study

In this paper, we investigate the influence of image quality on the performance of aging features. Age estimation systems used or designed a number of aging features to capture the aging cues from the face such as skin texture and wrinkles. These aging cues are sensitive to small changes in the imaging conditions which suggests considering the imaging quality when extracting such information. Although interesting performances are reported on various datasets, the effect of image quality has not been addressed. We introduce a scheme to explore the influence of image quality on the performance of appearance aging features. A number of datasets are experimented on where artifacts resulted from different types of noise are considered. Finally, we propose a method to automatically apply the most suitable features based on the quality of the image. The results show that better or comparable performance is obtained when automatically applying different features, based on image quality, in comparison to a single (best) feature type.