Douglas J. Kennard

Separating lines of text in free-form handwritten historical documents

We present an approach to finding (and separating) lines of text in free-form handwritten historical document images. After preprocessing, our method uses the count of foreground/background transitions in a binarized image to determine areas of the document that are likely to be text lines. Alternatively, an adaptive local connectivity map (ALCM) found in the literature can be used for this step of the process. We then use a min-cut/max-flow graph cut algorithm to split up text areas that appear to encompass more than one line of text. After removing text lines containing relatively little text information (or merging them with nearby text lines), we create output images for each line. A grayscale output image is created, as well as a special mask image containing both the foreground and information flagging ambiguous pixels. Foreground pixels that belong to other text lines are removed from the output images to provide cleaner line images useful for further processing. While some refinement is still necessary, the result of early experimentation with our method is encouraging

Combining multiple thresholding binarization values to improve OCR output

For noisy, historical documents, a high optical character recognition (OCR) word error rate (WER) can render the OCR text unusable. Since image binarization is often the method used to identify foreground pixels, a body of research seeks to improve image-wide binarization directly. Instead of relying on any one imperfect binarization technique, our method incorporates information from multiple simple thresholding binarizations of the same image to improve text output. Using a new corpus of 19th century newspaper grayscale images for which the text transcription is known, we observe WERs of 13.8% and higher using current binarization techniques and a state-of-the-art OCR engine. Our novel approach combines the OCR outputs from multiple thresholded images by aligning the text output and producing a lattice of word alternatives from which a lattice word error rate (LWER) is calculated. Our results show a LWER of 7.6% when aligning two threshold images and a LWER of 6.8% when aligning five. From the word lattice we commit to one hypothesis by applying the methods of Lund et al. (2011) achieving an improvement over the original OCR output and a 8.41% WER result on this data set.

Linking the past: discovering historical social networks from documents and linking to a genealogical database

Historical social networks (HSNs) can be used to inform historical research, including family history and genealogy. In some cases, clues about the structure of an HSN can be found in artifacts of family history such as personal diaries or autobiographical sketches. However, manual inference of such networks can require significant time and effort, including pooling and cross-referencing many different data sources. We present our current research into facilitating that process by automatically finding names in document transcriptions, relating those names to the names found on a roster/list of people who may be talked about in the documents, and automatically generating a social network graph from the result. We link individuals in the social network to a global genealogical database so that people researching their own family histories can easily find their ancestors within the HSNs created in this manner. We also provide examples of how the linked HSNs may be used to inform research about people and situations even when direct information is scarce.

Word Warping for Offline Handwriting Recognition

We present a novel method of offline whole-word handwriting recognition. We use automatic image morphing to compute 2-D geometric warps that align the strokes of each word image with the strokes of word images of training examples. Once the strokes of a given word are aligned to a training example, we use distance maps to compare how similar the two words are. Like 1-D Dynamic Programming (DP) methods, our warp-based method is robust to limited variation in word length and letter spacing. However, due to its 2-D nature, our method is also more robust than 1-D DP methods in handling variations caused by additional inconsistencies in character shape and stroke placement. Although we use DP for coarse alignment, the novel contribution of this paper is not 2-D DP, but morphing to automatically discover an actual 2-D mesh-based warp, followed by the use of distance maps to compute similarity between words. Early results are encouraging. On two datasets (1,000 training and 1,000 test words each), we get 88.77% and 89.33% recognition accuracy for in-vocabulary words. These are increases of 7.89% and 17.16% above the results of a 1-D DP approach.

Automated recognition and extraction of tabular fields for the indexing of census records

We describe a system for indexing of census records in tabular documents with the goal of recognizing the content of each cell, including both headers and handwritten entries. Each document is automatically rectified, registered and scaled to a known template following which lines and fields are detected and delimited as cells in a tabular form. Whole-word or whole-phrase recognition of noisy machine-printed text is performed using a glyph library, providing greatly increased efficiency and accuracy (approaching 100%), while avoiding the problems inherent with traditional OCR approaches. Constrained handwriting recognition results for a single author reach as high as 98% and 94.5% for the Gender field and Birthplace respectively. Multi-author accuracy (currently 82%) can be improved through an increased training set. Active integration of user feedback in the system will accelerate the indexing of records while providing a tightly coupled learning mechanism for system improvement.

Why multiple document image binarizations improve OCR

Our previous work has shown that the error correction of optical character recognition (OCR) on degraded historical machine-printed documents is improved with the use of multiple information sources and multiple OCR hypotheses including from multiple document image binarizations. The contributions of this paper are in demonstrating how diversity among multiple binarizations makes those improvements to OCR accuracy possible. We demonstrate the degree and breadth to which the information required for correction is distributed across multiple binarizations of a given document image. Our analysis reveals that the sources of these corrections are not limited to any single binarization and that the full range of binarizations holds information needed to achieve the best result as measured by the word error rate (WER) of the final OCR decision. Even binarizations with high WERs contribute to improving the final OCR. For the corpus used in this research, fully 2.68% of all tokens are corrected using hypotheses not found in the OCR of the binarized image with the lowest WER. Further, we show that the higher the WER of the OCR overall, the more the corrections are distributed among all binarizations of the document image.

Improving historical research by linking digital library information to a global genealogical database

Journals, letters, and other writings are of great value to historians and those who research their own family history; however, it can be difficult to find writings by specific people, and even harder to find what others wrote about them. We present a prototype web-based system that enables users to discover information about historical people (including their own ancestors) by linking digital library content to unique PersonIDs from a genealogical database. Users can contribute content such as scanned journals or information about where items can be found. They can also transcribe content and tag it with PersonIDs to identify who it is about. Additional features provide tools for users to explore historical contexts and relationships. These include the ability to tag places and to create a historical social network by specifying non-family relationships or by using a mechanism we call rosters to imply participation in some group or event.

Interactive training for handwriting recognition in historical document collections

We present a method of interactive training for handwriting recognition in collections of documents. As the user transcribes (labels) the words in the training set, words are automatically skipped if they appear to match words that are already transcribed. By reducing the amount of redundant training, better coverage of the data is achieved, resulting in more accurate recognition. Using word-level features for training and recognition in a collection of George Washingtons manuscripts, the recognition ratio is approximately 2%-8% higher after training with our interactive method than after training the same number of words sequentially. Using our approach, less training is required to achieve an equivalent recognition ratio. A slight improvement in recognition ratio is also observed when using our method on a second data set, which consists of several pages from a diary written by Jennie Leavitt Smith.