Harish Srinivasan

Document image retrieval using signatures as queries

In searching a repository of business documents, a task of interest is that of using a query signature image to retrieve from a database, other signatures matching the query. The signature retrieval task involves a two-step process of extracting all the signatures from the documents and then performing a match on these signatures. This paper presents a novel signature retrieval strategy, which includes a technique for noise and printed text removal from signature images, previously extracted from business documents. Signature matching is based on a normalized correlation similarity measure using global shape-based binary feature vectors. In a retrieval task involving a database of 447 signatures, on an average 4.43 out of the top 5 choices were signatures belonging to the writer of the queried signature. On considering the Top 10 ranks, a F-measure value of 76.3 was obtained and the precision and recall values at this F-measure were 74.5% and 78.28% respectively

Biometric and Forensic Aspects of Digital Document Processing

Signatures and handwriting have long played a role in dayto-day business transactions and in forensics, e.g., to authenticate documents, as evidence to establish crime or innocence, etc. The individuality of handwriting and signatures is the basis for their relevance to authentication and forensics. This very individuality makes them also potentially useful as a biometric modality. This chapter is concerned with automatic methods for verifying the writership of handwritten documents and signatures. The discussion consists of the individuality of handwriting, image pre-processing and interactive tools for forensic document examination, discriminating characteristics of handwriting, a statistical model of writer verification, and application of the model to signature verification.

Search engine for handwritten documents

Search aspects of a system for analyzing handwritten documents are described. Documents are indexed using global image features, e.g., stroke width, slant as well as local features that describe the shapes of words and characters. Image indexing is done automatically using page analysis, page segmentation, line separation, word segmentation and recognition of words and characters. Two types of search are permitted: search based on global features of entire document and search using features at local level. For the second type of search, i.e., local, all the words in the document are characterized and indexed by various features and it forms the basis of different search techniques. The paper focuses on local search and describes four tasks: word/phrase spotting, text to image, image to text and plain text. Performance in terms of precision/recall and word ranking is reported on a database of handwriting samples from about 1,000 individuals.

Use of ridge points in partial fingerprint matching

Matching of partial fingerprints has important applications in both biometrics and forensics. It is well-known that the accuracy of minutiae-based matching algorithms dramatically decrease as the number of available minutiae decreases. When singular structures such as core and delta are unavailable, general ridges can be utilized. Some existing highly accurate minutiae matchers do use local ridge similarity for fingerprint alignment. However, ridges cover relatively larger regions, and therefore ridge similarity models are sensitive to non-linear deformation. An algorithm is proposed here to utilize ridges more effectively- by utilizing representative ridge points. These points are represented similar to minutiae and used together with minutiae in existing minutiae matchers with simple modification. Algorithm effectiveness is demonstrated using both full and partial fingerprints. The performance is compared against two minutiae-only matchers (Bozorth and k-minutiae). Effectiveness with full fingerprint matching is demonstrated using the four databases of FVC2002- where the error rate decreases by 0.2-0.7% using the best matching algorithm. The effectiveness is more significant in the case of partial fingerprint matching- which is demonstrated with sixty partial fingerprint databases generated from FVC2002 (with five levels of numbers of minutiae available). When only 15 minutiae are available the error rate decreases 5-7.5%. Thus the method, which involves selecting representative ridge points, minutiae matcher modification, and a group of minutiae matchers, demonstrates improved performance on full and especially partial fingerprint matching.

Automatic scoring of short handwritten essays in reading comprehension tests

Reading comprehension is largely tested in schools using handwritten responses. The paper describes computational methods of scoring such responses using handwriting recognition and automatic essay scoring technologies. The goal is to assign to each handwritten response a score which is comparable to that of a human scorer even though machine handwriting recognition methods have high transcription error rates. The approaches are based on coupling methods of document image analysis and recognition together with those of automated essay scoring. Document image-level operations include: removal of pre-printed matter, segmentation of handwritten text lines and extraction of words. Handwriting recognition is based on a fusion of analytic and holistic methods together with contextual processing based on trigrams. The lexicons to recognize handwritten words are derived from the reading passage, the testing prompt, answer rubric and student responses. Recognition methods utilize childrens handwriting styles. Heuristics derived from reading comprehension research are employed to obtain additional scoring features. Results with two methods of essay scoring-both of which are based on learning from a human-scored set-are described. The first is based on latent semantic analysis (LSA), which requires a reasonable level of handwriting recognition performance. The second uses an artificial neural network (ANN) which is based on features extracted from the handwriting image. LSA requires the use of a large lexicon for recognizing the entire response whereas ANN only requires a small lexicon to populate its features thereby making it practical with current word recognition technology. A test-bed of essays written in response to prompts in statewide reading comprehension tests and scored by humans is used to train and evaluate the methods. End-to-end performance results are not far from automatic scoring based on perfect manual transcription, thereby demonstrating that handwritten essay scoring has practical potential.

COMPARISON OF ROC AND LIKELIHOOD DECISION METHODS IN AUTOMATIC FINGERPRINT VERIFICATION

The biometric verification task is to determine whether or not an input and a template belong to the same individual. In the context of automatic fingerprint verification the task consists of three steps: feature extraction, where features (typically minutiae) are extracted from each fingerprint, scoring, where the degree of match between the two sets of features is determined, and decision, where the score is used to accept or reject the hypothesis that the input and template belong to the same individual. The paper focuses on the final decision step, which is a binary classification problem involving a single score variable. The commonly used decision method is to learn a score threshold from a labeled set of inputs and templates, by first determining the receiver operating characteristic (ROC) of the task. The ROC method works well when there is a well-registered fingerprint image. The paper shows that when there is uncertainty due to fingerprint quality, e.g. the input is a latent print or a partial print, the decision method can be improved by using the likelihood ratio of match/non match. The likelihood ratio is obtained by modeling the distributions of same finger and different finger scores using parametric distributions. The parametric forms considered are Gaussian and Gamma distributions whose parameters are learnt from labeled training samples. The performances of the likelihood and ROC methods are compared for varying numbers of minutiae points available for verification. Using either Gaussian or Gamma parametric distributions, the likelihood method has a lower error rate than the ROC method when few minutiae points are available. Likelihood and ROC methods converge to the same accuracy as more minutiae points are available.

Generative Models for Fingerprint Individuality using Ridge Types

Generative models of pattern individuality attempt to represent the distribution of observed quantitative features, e.g., by learning parameters from a database, and then use such distributions to determine the probability of two random patterns being the same. Considering fingerprint patterns, Gaussian distributions have been previously used for minutiae location and von-Mises distributions for minutiae orientation so as to determine the probability of random correspondence (PRC) between two fingerprints. Motivated by the fact that ridges have not been modeled in generative models, and using representative ridge points in fingerprint matching, ridge information is incorporated into the generative model by using a third distribution for ridge types. The joint probability of minutiae location, minutiae orientation and ridge type is modeled as a mixture distribution. The proposed model offers a more accurate fingerprint representation from which more reliable PRCs can be computed. Based on parameters estimated from fingerprint databases, PRCs using ridge types are seen to be much smaller than PRCs computed with only minutiae.

Versatile search of scanned Arabic handwriting

Searching handwritten documents is a relatively unexplored frontier for documents in any language. Traditional approaches use either image-based or text-based techniques. This paper describes a framework for versatile search where the query can be either text or image, and the retrieval method fuses text and image retrieval methods. A UNICODE and an image query are maintained throughout the search, with the results being combined by a neural network. Preliminary results show positive results that can be further improved by refining the component pieces of the framework (text transcription and image search).

Handwritten Word Recognition Using Conditional Random Fields

The paper describes a lexicon driven approach for word recognition on handwritten documents using conditional random fields (CRFs). CRFs are discriminative models and do not make any assumptions about the underlying data and hence are known to be superior to hidden Markov models (HMMs) for sequence labeling problems. For word recognition, the document is first segmented into word images using an existing neural network based algorithm. Each word image is then over segmented into a number of small segments such that the combination of segments forms character images. Segment(s) is/are labeled as characters with probability evaluated from the CRF model. The total probability of a word image representing an entry from the lexicon is computed using a dynamic programming algorithm which evaluates the optimal combination of segments.

A statistical approach to line segmentation in handwritten documents

A new technique to segment a handwritten document into distinct lines of text is presented. Line segmentation is the first and the most critical pre-processing step for a document recognition/analysis task. The proposed algorithm starts, by obtaining an initial set of candidate lines from the piece-wise projection profile of the document. The lines traverse around any obstructing handwritten connected component by associating it to the line above or below. A decision of associating such a component is made by (i) modeling the lines as bivariate Gaussian densities and evaluating the probability of the component under each Gaussian or (ii)the probability obtained from a distance metric. The proposed method is robust to handle skewed documents and those with lines running into each other. Experimental results show that on 720 documents (which includes English, Arabic and childrens handwriting) containing a total of 11, 581 lines, 97.31% of the lines were segmented correctly. On an experiment over 200 handwritten images with 78, 902 connected components, 98.81% of them were associated to the correct lines.