Leonard Rothacker

Bag-of-Features HMMs for Segmentation-Free Word Spotting in Handwritten Documents

Recent HMM-based approaches to handwritten word spotting require large amounts of learning samples and mostly rely on a prior segmentation of the document. We propose to use Bag-of-Features HMMs in a patch-based segmentation-free framework that are estimated by a single sample. Bag-of-Features HMMs use statistics of local image feature representatives. Therefore they can be considered as a variant of discrete HMMs allowing to model the observation of a number of features at a point in time. The discrete nature enables us to estimate a query model with only a single example of the query provided by the user. This makes our method very flexible with respect to the availability of training data. Furthermore, we are able to outperform state-of-the-art results on the George Washington dataset.

Segmentation-free query-by-string word spotting with Bag-of-Features HMMs

Word spotting allows to explore document images without requiring a full transcription. In the query-by-string scenario considered in this paper, it is possible to search arbitrary keywords while only limited prior information about the documents is required. We learn context-dependent character models from a training set that is small with respect to the number of models. This is possible due to the use of Bag-of-Features HMMs that are especially suited for estimating robust models from limited training material. In contrast to most query-by-string methods we consider a fully segmentation-free decoding framework that does not require any pre-segmentation on word or line level. Experiments on the well-known George Washington benchmark demonstrate the high accuracy of our method.

Bag-of-Features Representations for Offline Handwriting Recognition Applied to Arabic Script

Due to the great variabilities in human writing, unconstrained handwriting recognition is still considered an open research topic. Recent trends in computer vision, however, suggest that there is still potential for better recognition by improving feature representations. In this paper we focus on feature learning by estimating and applying a statistical bag-of-features model. These models are successfully used in image categorization and retrieval. The novelty here is the integration with a Hidden Markov Model (HMM) that we use for recognition. Our method is evaluated on the IFN/ENIT database consisting of images of handwritten Arabic town and village names.

Bag-of-features representations using spatial visual vocabularies for object classification

This paper presents a novel method for combining local image features and spatial information for object classification tasks using the Bag-of-Features principle. The feature descriptor is extended by additional spatial information. Hence, similar feature descriptors do not only describe similar image patches, but similar patches in roughly the same region. Different spatial measures are evaluated on the Caltech 101 dataset showing the improvement by incorporating spatial information into the feature descriptor. Furthermore, the method achieves better classification rates than the comparable Spatial Pyramids with lower a dimensional representation.

Novel Sub-character HMM Models for Arabic Text Recognition

Hidden Markov Model (HMM) is one of the most widely used classifier for text recognition. In this paper we are presenting novel sub-character HMM models for Arabic text recognition. Modeling at sub-character level allows sharing of common patterns between different contextual forms of Arabic characters as well as between different characters. The number of HMMs gets reduced considerably while still capturing the variations in shape patterns. This results in a compact and efficient recognizer with reduced model set and is expected to be more robust to the imbalance in data distribution. Experimental results using the sub-character model based recognition of handwritten Arabic text as well printed Arabic text are reported.

A method for camera-based interactive whiteboard reading

Recognizing mind maps written on a whiteboard is a challenging task due to the unconstrained handwritten text and the different graphical elements -- i.e. lines, circles and arrows -- available in a mind map. In this paper we propose a prototype system to recognize and visualize such mind maps written on whiteboards. After the image acquisition by a camera, a binarization process is performed, and the different connected components are extracted. Without presuming any prior knowledge about the document, its style, layout, etc., the analysis starts with connected components, labeling them as text, lines, circles or arrows based on a neural network classifier trained on some statistical features extracted from the components. Once the text patches are identified, word detection is performed, modeling the text patches by their gravity centers and grouping them into possible words by density based clustering. Finally, the grouped connected components are recognized by a Hidden Markov Model based recognizer. The paper also presents a software tool integrating all these processing stages, allowing a digital transcription of the mind map and the interaction between the user, the mind map, and the whiteboard.

Retrieving Cuneiform Structures in a Segmentation-free Word Spotting Framework

Cuneiform tablets are an invaluable documentation of early human history. Efforts are being made in digitizing large tablet collections for preserving their content and making them available to a global research community. However, there are hardly any automated computer aided methods for supporting philologists in their analysis. In this paper we present an approach for automatically retrieving cuneiform wedge constellations from digitized cuneiform tablet collections. Compelling results could be achieved in our qualitative and quantitative evaluation on a challenging benchmark consisting of 3D-scanned cuneiform tablets.

Bag-of-features HMMs for segmentation-free Bangla word spotting

In this paper we present how Bag-of-Features Hidden Markov Models can be applied to printed Bangla word spotting. These statistical models allow for an easy adaption to different problem domains. This is possible due to the integration of automatically estimated visual appearance features and Hidden Markov Models for spatial sequential modeling. In our evaluation we are able to report high retrieval scores on a new printed Bangla dataset. Furthermore, we outperform state-of-the-art results on the well-known George Washington word spotting benchmark. Both results have been achieved using an almost identical parametric method configuration.

Robust Output Modeling in Bag-of-Features HMMs for Handwriting Recognition

Bag-of-Features HMMs have been successfully applied to handwriting recognition and word spotting. In this paper we extend our previous work and present methods for modeling sequences of Bag-of-Features representations with Hidden Markov Models. We will discuss our previous approach that uses a pseudo-discrete model. Afterwards, we present a novel semi-continuous integration. The method is effective for probabilistic text clustering and is suitable for statistically modeling the characteristics of Bag-of-Features representations extracted from document images. Furthermore, its statistical expectation-maximization estimation can directly be integrated in Baum-Welch HMM training. In our experiments we present competitive results on the IfN/ENIT word recognition benchmark and state-of-the-art results for word spotting on the George Washington benchmark. Our evaluation gives insights into the properties of the models from the perspectives of modern as well as historic document analysis.

Learning local image descriptors for word spotting

The Bag-of-Features paradigm has enjoyed great success in computer vision as well as document image analysis applications. By far the most common approach here is to power the Bag-of-Features pipeline with SIFT descriptors which are then clustered into a visual vocabulary using Lloyds algorithm. In contrast to using handcrafted descriptors, many researches have started to use descriptors that have been learned from data. While descriptor learning is common in other computer vision tasks, there has been little work on learning descriptors for document analysis purposes. In this work we propose a descriptor learning pipeline designed for word spotting. Evaluation results on the well known George Washington database demonstrate that word-spotting results can effectively be improved by learning specialized local image descriptors.