Ehry MacRostie

Multi-lingual offline handwriting recognition using hidden Markov models: a script-independent approach

This paper introduces a script-independent methodology for multilingual offline handwriting recognition (OHR) based on the use of Hidden Markov Models (HMM). The OHR methodology extends our script-independent approach for OCR of machine-printed text images. The feature extraction, training, and recognition components of the system are all designed to be script independent. The HMM training and recognition components are based on our Byblos continuous speech recognition system. The HMM parameters are estimated automatically from the training data, without the need for laborious hand-written rules. The system does not require pre-segmentation of the data, neither at the word level nor at the character level. Thus, the system can handle languages with cursive handwritten scripts in a straightforward manner. The script independence of the system is demonstrated with experimental results in three scripts that exhibit significant differences in glyph characteristics: English, Chinese, and Arabic. Results from an initial set of experiments are presented to demonstrate the viability of the proposed methodology.

The BBN Byblos Hindi OCR System

The BBN Byblos OCR system implements a script-independent methodology for OCR using hidden Markov models (HMMs). We have successfully ported the system to Arabic, English, Chinese, Pashto, and Japanese. In this chapter, we report on our recent effort in training the system to perform recognition of Hindi (Devanagari) documents. The initial experiments reported in this chapter were performed using a corpus of synthetic (computer-generated) document images along with slightly degraded versions of the same that were generated by scanning printed versions of the document images and by scanning faxes of the printed versions. On a fair test set consisting of synthetic images alone we measured a character error rate of 1.0%. The character error rate on a fair test set consisting of scanned images (scans of printed versions of the synthetic images) was 1.40% while the character error rate on a fair test set of fax images (scans of printed and faxed versions of the synthetic images) was 8.7%.

The BBN Byblos Pashto OCR system

The BBN Byblos OCR system implements a script-independent methodology for OCR using Hidden Markov Models (HMMs). We have successfully tested the system with Arabic, English, and Chinese documents. In this paper, we describe our recent effort in training the system to perform recognition of documents in Pashto, one of the national languages of Afghanistan. We discuss the availability and characteristics of suitable experimental data and the methods we used to assemble Pashto training and test corpora. We modeled each form of each Pashto character with an HMM and tested the models on several varieties of document images. On a fair test set consisting of synthetic images alone we measured a character error rate of 1.6%. The character error rate on a fair test set consisting of scanned pages was 2.1%, and the character error rate on a fair test set of faxed pages was 3.1%. On other types of document images, character error rates increased in rough proportion to the level of degradation of the image.

Performance improvements to the BBN Byblos OCR system

In this paper, we describe four recent enhancements to the BBN Byblos OCR system, a multilingual HMM-based character recognition system which has been demonstrated on a variety of languages, including English, Arabic, Chinese, and Japanese. These enhancements are implemented as optional extensions to the system and provide improved performance for certain scripts or domains. Projection-based re-estimation of line boundaries reduces instability in the presence of some types of noise. An alternate modeling strategy used in the first of two recognition search passes substantially increases speed on languages with a large number of characters. Another speed improvement comes from automatic discovery and modeling of sub-characters. The use of heteroschedastic linear discriminant analysis (HLDA) makes modeling more tractable by reducing feature-space dimensionality.

Large-scale, real-time logo recognition in broadcast videos

Robust, real-time, multi-class logo detection in high resolution broadcast videos presents several difficult challenges. For most logos we only have a few training samples, which makes training robust classifiers hard. Also, logos could potentially occur anywhere in the image, and traditional sliding window approaches for logo/object detection are computationally intensive. We present a system that addresses these issues by first identifying a small set of possible logo locations in a frame, based on temporal continuity and multi-resolution search, and then successively pruning these locations for each logo template, using a cascade of color and edge based features. We present experimental results that demonstrate our system for detecting a total of 270 different logo classes in broadcast video from 5 different languages (English, Indonesian, Malay, Simplified and Traditional Chinese).

Robust named entity detection in videotext using character lattices

Text in video sequences can provide key indexing information. In particular, videotext is rich in named entities (NEs) and detection of such entities is critical for search applications. Traditional approaches for detecting NEs in OCR output look for these NEs in the single-best recognition results. Due to inevitable presence of recognition errors in the single-best output, such approaches usually result in low recall. Given that a lattice is more likely to contain the correct answer, we explore NE detection from character lattices produced by our videotext OCR system. Furthermore, we use an approximate match criterion that allows insertion of punctuations during lookup. Experimental results show a 50% relative improvement in NE recall using lattices over exact lookup in the 1-best hypothesis. Since the improvement in recall is accompanied by a large number of false positives, we present techniques for reducing false alarms. In addition, we describe efficient techniques for reducing the time for detecting NEs.

Multi-frame combination for robust videotext recognition

Optical character recognition (OCR) of overlaid text in video streams is a challenging problem due to various factors including the presence of dynamic backgrounds, color, and low resolution. In video feeds such as Broadcast News, a particular overlaid text region usually persists for multiple frames during which the background may or may not vary. In this paper we explore two innovative techniques that exploit such multi-frame persistence of videotext. The first technique uses multiple instances to generate a single enhanced image for recognition. The second technique uses the NIST ROVER algorithm developed for speech recognition to combine 1-best hypotheses from different frames of a text region. Significant improvement in the word error rate (WER) is obtained by using ROVER when compared to recognizing a single instance. The WER is further reduced by combining hypotheses from frame instances, which were generated using character models trained with different binarization thresholds. A 20% relative reduction in the WER was achieved for multi-frame combination over decoding a single frame instance.

End-to-End Trainable Thai OCR System Using Hidden Markov Models

In this paper we present an end-to-end trainable optical character recognition (OCR) system for recognizing machine-printed text in Thai documents. The end-to-end OCR system is based on a script-independent methodology using hidden Markov models. Our system provides an integrated workflow beginning with annotation and transcription of training images to performing OCR on new images with models trained on transcribed training images. The efficacy of our end-to-end OCR system is demonstrated by rapidly configuring our OCR engine for the Thai script. We present experimental results on Thai documents to highlight the specific challenges posed by the Thai script and analyze the recognition performance as a function of amount of training data.

The BBN Byblos Japanese OCR system

The BBN Byblos OCR system implements a script-independent methodology for OCR using hidden Markov models (HMMs). We have successfully ported the system to Arabic, Pashto, English, and Chinese. We discuss our effort in configuring the system to perform recognition of noisy machine printed Japanese documents. The data for our experimentation was taken from the University of Washington (UW-II) Japanese OCR corpus and the LDC Japanese Business News Supplement corpus. We evaluated the performance of a whole-character configuration in which each character was modeled using a separate HMM. As in the case of our Chinese OCR system [P. Natarajan et al., 2001], we also used a sub-character modeling approach [P. Natarajan et al., 2003] in which each Japanese character was spelled using a shared set of automatically generated sub-characters. We experimentally evaluated the performance of different sub-character clusters as well as different HMM topologies to identify the best overall system configuration. On a fair test using noisy/degraded images from the UW-II corpus, the best sub-character configuration resulted in a character error rate of 20.13%, On relatively cleaner data, consisting of scanned newspaper images, the system delivered an error rate of 7.85%. Using a whole-character configuration the corresponding error rates were 11.94% and 4.55% respectively.

The BBN document analysis service: a platform for multilingual document translation

In this paper, we introduce a new operational platform for end-to-end document image analysis, recognition, and machine translation. The Raytheon BBN Document Analysis Service (BBN DAS) performs the following operations on scanned machine-print document images: (1) image pre-processing and segmentation to identify homogenous zones of text, (2) text recognition to convert the text zones into electronic text, (3) machine translation for converting the text from the native language of the document into English, and (4) document archiving and indexing for effective content-based search. BBN DAS uses a service-oriented architecture (SOA), which offers modularity and scalability for operation on hardware configurations ranging from a laptop to distributed multi-node server environments. This paper describes the platform architecture, the process of configuring it for Arabic newsprint documents and resulting performance results of the Arabic system.