Sebastian Sudholt

PHOCNet: A Deep Convolutional Neural Network for Word Spotting in Handwritten Documents

In recent years, deep convolutional neural networks have achieved state of the art performance in various computer vision tasks such as classification, detection or segmentation. Due to their outstanding performance, CNNs are more and more used in the field of document image analysis as well. In this work, we present a CNN architecture that is trained with the recently proposed PHOC representation. We show empirically that our CNN architecture is able to outperform state-of-the-art results for various word spotting benchmarks while exhibiting short training and test times.

A Modified Isomap Approach to Manifold Learning in Word Spotting

Word spotting is an effective paradigm for indexing document images with minimal human effort. Here, the use of the Bag-of-Features principle has been shown to achieve competitive results on different benchmarks. Recently, a spatial pyramid approach was used as a word image representation to improve the retrieval results even further. The high dimensionality of the spatial pyramids was attempted to be countered by applying Latent Semantic Analysis. However, this leads to increasingly worse results when reducing to lower dimensions. In this paper, we propose a new approach to reducing the dimensionality of word image descriptors which is based on a modified version of the Isomap Manifold Learning algorithm. This approach is able to not only outperform Latent Semantic Analysis but also to reduce a word image descriptor to up to \(0.12\,\%\) of its original size without losing retrieval precision. We evaluate our approach on two different datasets.

Handwritten Word Image Categorization with Convolutional Neural Networks and Spatial Pyramid Pooling

The extraction of relevant information from historical document collections is one of the key steps in order to make these documents available for access and searches. The usual approach combines transcription and grammars in order to extract semantically meaningful entities. In this paper, we describe a new method to obtain word categories directly from non-preprocessed handwritten word images. The method can be used to directly extract information, being an alternative to the transcription. Thus it can be used as a first step in any kind of syntactical analysis. The approach is based on Convolutional Neural Networks with a Spatial Pyramid Pooling layer to deal with the different shapes of the input images. We performed the experiments on a historical marriage record dataset, obtaining promising results.

Learning local image descriptors for word spotting

The Bag-of-Features paradigm has enjoyed great success in computer vision as well as document image analysis applications. By far the most common approach here is to power the Bag-of-Features pipeline with SIFT descriptors which are then clustered into a visual vocabulary using Lloyds algorithm. In contrast to using handcrafted descriptors, many researches have started to use descriptors that have been learned from data. While descriptor learning is common in other computer vision tasks, there has been little work on learning descriptors for document analysis purposes. In this work we propose a descriptor learning pipeline designed for word spotting. Evaluation results on the well known George Washington database demonstrate that word-spotting results can effectively be improved by learning specialized local image descriptors.

Attribute CNNs for word spotting in handwritten documents

Word spotting has become a field of strong research interest in document image analysis over the last years. Recently, AttributeSVMs were proposed which predict a binary attribute representation (Almazán et al. in IEEE Trans Pattern Anal Mach Intell 36(12):2552–2566, 2014). At their time, this influential method defined the state of the art in segmentation-based word spotting. In this work, we present an approach for learning attribute representations with convolutional neural networks(CNNs). By taking a probabilistic perspective on training CNNs, we derive two different loss functions for binary and real-valued word string embeddings. In addition, we propose two different CNN architectures, specifically designed for word spotting. These architectures are able to be trained in an end-to-end fashion. In a number of experiments, we investigate the influence of different word string embeddings and optimization strategies. We show our attribute CNNs to achieve state-of-the-art results for segmentation-based word spotting on a large variety of data sets.