Yousri Kessentini

Off-line handwritten word recognition using multi-stream hidden Markov models

In this paper, we present a multi-stream approach for off-line handwritten word recognition. The proposed approach combines low level feature streams namely, density based features extracted from 2 different sliding windows with different widths, and contour based features extracted from upper and lower contours. The multi-stream paradigm provides an interesting framework for the integration of multiple sources of information and is compared to the standard combination strategies namely fusion of representations and fusion of decisions. We investigate the extension of 2-stream approach to N streams (N=2,...,4) and analyze the improvement in the recognition performance. The computational cost of this extension is discussed. Significant experiments have been carried out on two publicly available word databases: IFN/ENIT benchmark database (Arabic script) and IRONOFF database (Latin script). The multi-stream framework improves the recognition performance in both cases. Using 2-stream approach, the best recognition performance is 79.8%, in the case of the Arabic script, on a 2100-word lexicon consisting of 946 Tunisian town/village names. In the case of the Latin script, the proposed approach achieves a recognition rate of 89.8% using a lexicon of 196 words.

A Dempster-Shafer Theory based combination of handwriting recognition systems with multiple rejection strategies

Dempster-Shafer Theory (DST) is particularly efficient in combining multiple information sources providing incomplete, imprecise, biased, and conflictive knowledge. In this work, we focused on the improvement of the accuracy rate and the reliability of a HMM based handwriting recognition system, by the use of Dempster-Shafer Theory (DST). The system proceeds in two steps: First, an evidential combination method is proposed to finely combine the probabilistic outputs of the HMM classifiers. Second, a global post-processing module is proposed to improve the reliability of the system thanks to a set of acceptance/rejection decision strategies. In the end, an alternative treatment of the rejected samples is proposed using multi-stream HMM to improve the word recognition rate as well as the reliability of the recognition system, while not causing significant delays in the recognition process. Experiments carried out on two publically available word databases (RIMES for Latin script and IFN/ENIT for Arabic script) show the benefit of the proposed strategies. HighlightsA two level handwritten word recognition system is proposed.Combining Hidden Markov models classifier based on Dempster-Shafer Theory.A post-processing module based on different DST based acceptance/rejection strategies.We improve the accuracy rate and the reliability of a Handwriting recognition system.

A deep HMM model for multiple keywords spotting in handwritten documents

In this paper, we propose a query by string word spotting system able to extract arbitrary keywords in handwritten documents, taking both segmentation and recognition decisions at the line level. The system relies on the combination of a HMM line model made of keyword and non-keyword (filler) models, with a deep neural network that estimates the state-dependent observation probabilities. Experiments are carried out on RIMES database, an unconstrained handwritten document database that is used for benchmarking different handwriting recognition tasks. The obtained results show the superiority of the proposed framework over the classical GMM–HMM and standard HMM hybrid architectures.

Word Spotting and Regular Expression Detection in Handwritten Documents

In this paper, we propose a novel system for word spotting and regular expression detection in Handwritten documents. The proposed approach is lexicon-free, i.e., able to spot arbitrary keywords that are not required to be known at the training stage. Furthermore, the proposed system is segmentation-free, i.e., text lines are not required to be segmented into words. The originalities of our approach is twofold. First we propose a new filler model which allows to speed-up the decoding process. Second, we extend the methodology to search for regular expressions. The system has been evaluated on a public handwritten document database used for the 2011 ICDAR handwriting recognition competitions.

Multi-script handwriting recognition with N-streams low level features

The multi-stream paradigm provides an interesting framework for the integration of multiple sources of information. In this paper, we present our multi-script recognition system using the multi-stream formalism to combine low level feature streams. We Analyze how the combination of n streams (n=2,....,4) can improve the recognition performance. Significant experiments have been carried out on two publicly available word databases: IFN/ENIT benchmark database (Arabic script) and IRONOFF database (Latin script). The proposed framework shows interesting results in both cases thanks to the use of low level features in a segmentation free approach which ensure its applicability to various scripts.

A Multi-stream Approach to Off-Line Handwritten Word Recognition

We present in this paper a new approach based on multi-stream hidden Markov models (HMM) for the recognition of off-line handwriting. Every word is presented by two HMM models: the first one is learned with features extracted from upper contour, the second with features extracted from lower contour. The combination of these two sources of information is studied using the multi-stream framework. We present experiment results obtained on a database composed of isolated words extracted from incoming mail documents.

Handwritten document segmentation using hidden Markov random fields

In this paper we present a method based on hidden Markov random fields and 2D dynamic programming image decoding, for segmenting pages of complex handwritten manuscripts such as novelist drafts. After a formal description of the theoretical framework and the principles of the decoding method, we describe the implementation of the model and the decoding method. Then we discuss the results obtained with this approach on the drafts of the French novelist Gustave Flaubert.

Dempster-Shafer Based Rejection Strategy for Handwritten Word Recognition

In this paper, a novel rejection strategy is proposed to optimize the reliability of an handwritten word recognition system. The proposed approach is based on several steps. First, we combine the outputs of several HMM classifiers using the Dempster-Shafer theory (DST). Then, we take advantage of the expressivity of mass functions (the counter part of probability distributions in DST) to characterize the quality/reliability of the classification. Finally, we use this characterization to decide whether a test word is rejected or not. Experiments carried out on RIMES and IFN/ENIT datasets show that the proposed approach outperforms other state-of-the-art rejection methods.

Evidential combination of multiple HMM classifiers for multi-script handwriting recognition

In this work, we focus on an improvement of a multiscript handwritting recognition system using a HMM based classifiers combination. The improvement relies on the use of Dempster-Shafer theory to combine in a finer way the probabilistic outputs of the HMM classifiers. The experiments are conducted on two public databases written on two different scripts : IFN/ENIT (latin script) and RIMES (arabic script). The obtained results are compared with the classical algorithms of the field and the superiority of the proposed approach is shown.

A Multi-Lingual Recognition System for Arabic and Latin Handwriting

Generally, handwritten word recognition systems use script specific methodologies. In this paper, we present a unified approach for multi-lingual recognition of alphabetic scripts. The proposed system operates independently of the nature of the script using the multi-stream paradigm. The experiments have been carried out on a multi-script database composed of Arabic and Latin handwritten words from the IFN/ENIT and the IRONOFF public databases and show interesting recognition performances with only1.5% of script confusion and an overall word recognition rate of 84.5% using a multi-script lexicon of 1142 words.