Effective offline handwritten text recognition model based on a sequence-to-sequence approach with CNN–RNN networks

Abstract

Automatic text recognition system might serve as an important factor in creating a paperless environment through digitizing and processing the existing paper documents in the upcoming days. Handwritten recognition using deep learning methods has been widely explored by many researchers. The existence of large quantity of data and a variety of algorithmic innovations enable the ease of training deep neural networks. Different techniques have been initiated in the literature for recognizing text from handwritten documents. This paper proposes a hybrid handwritten text recognition (H2TR) model using deep neural networks that use the sequence-to-sequence (Seq2Seq) approach. This hybrid model makes use of the salient features of convolution neural network (CNN) and recurrent neural network (RNN) with long–short-term memory network (LSTM). It uses CNN to extract the features from the handwritten image. The features that are extracted are later modelled with a sequence-to-sequence approach and fed to RNN–LSTM for encoding the visual features and decoding the sequence of letters that are available in the handwritten image. The proposed model is tested with IAM and RIMES handwritten databases, which shows competitive letter accuracy and word accuracy results.