Handwritten Text Recognition using Fully Convolutional Network

Abstract

Handwritten text recognition from images is challenging because there are many variations in handwriting as each person has a different writing style. This research implements multilevel recognition to solve this problem. In the first level, a Lexicon Convolutional Neural Network (CNN) model is used to recognize words (containing a set of word) that often appear in the text. If a word is not recognized by the Lexicon CNN model, which is designed for a limited number of words, then it goes to the next level. A character sequence recognition consisting of predicting the number of characters using CNN, cropping each character using a sliding window, and character recognition using Fully Convolutional Network (FCN), is applied at the second level. Experiments show that the system performance is promising. The experiment conducted using NIST Special Database 19 as a training dataset and a handwritten text on screen as a testing dataset. The best accuracy of word recognition, character number prediction, and character recognition is 99.98%, 98.56%, and 83.52%, respectively.