Anh Duc Le

A System for Recognizing Online Handwritten Mathematical Expressions and Improvement of Structure Analysis

This paper presents a system for recognizing online handwritten mathematical expressions (MEs) and improvement of structure analysis. We represent MEs in Context Free Grammars (CFGs) and employ the Cocke-Younger-Kasami (CYK) algorithm to parse 2D structure of on-line handwritten MEs and select the best interpretation in terms of symbol segmentation, recognition and structure analysis. We propose a method to learn structural relations from training patterns without any heuristic decisions by using two SVM models. We employ stroke order to reduce the complexity of the parsing algorithm. Moreover, we revise structure analysis. Even though CFG does not resolve ambiguities in some cases, our method still gives users a list of candidates that contain expecting result. We evaluate our method in the CROHME 2013 database and demonstrate the improvement of our system in recognition rate as well as processing time.

Deep neural networks for recognizing online handwritten mathematical symbols

This paper presents application of deep learning to recognize online handwritten mathematical symbols. Recently various deep learning architectures such as Convolution neural network (CNN), Deep neural network (DNN) and Long short term memory (LSTM) RNN have been applied to fields such as computer vision, speech recognition and natural language processing where they have been shown to produce state-of-the-art results on various tasks. In this paper, we apply max-out-based CNN and BLSTM to image patterns created from online patterns and to the original online patterns, respectively and combine them. We also compare them with traditional recognition methods which are MRF and MQDF by carrying out some experiments on CROHME database.

Comparison of Parsing Algorithms for Recognizing Online Handwritten Mathematical Expressions

Parsing process is the most important process in recognition of online handwritten mathematical expressions. There are two basic approaches: stroke order dependent (SOD) and stroke order free (SOF) approaches. The SOD approach depends on stroke order while the SOF approach is free from stroke order. Although both approaches have shown high recognition rates in recently competitions, there are a few of works that analyze the complexities of parsing algorithms in the same experimental conditions. In this work, we have tested and analyzed recognition rate, recognition speed and memory space required by parsing algorithms on CROHME 2014. SOF is slightly superior to SOD in recognition rate, but SOD is faster in processing time and lower in memory space than SOF.

A system for recognizing online handwritten mathematical expressions by using improved structural analysis

A system for recognizing online handwritten mathematical expressions (MEs), by applying improved structural analysis, is proposed and experimentally evaluated on two databases. With this system, MEs are represented in the form of stochastic context-free grammar (SCFG), and the Cocke–Younger–Kasami (CYK) algorithm is used to parse two-dimensional (2D) structures of online handwritten MEs and select the best interpretation in terms of the results of symbol segmentation and recognition as well as structural analysis. A concept of “body box” is proposed, and two SVM models are applied for learning and analyzing structural relations from training patterns without the need for any heuristic decisions. Stroke order is used to reduce the complexity of the parsing algorithm. Even though SCFG does not resolve ambiguities in some cases, the proposed system still gives users a list of candidates that contains the expected result. The results of experimental evaluations of the proposed system on the CROHME 2013 and CROHME 2014 databases and on an in-house (“Hand-Math”) database show that the recognition rate of the proposed system is improved, while the processing time on a common CPU is kept to a practical level.

An incremental recognition method for online handwritten mathematical expressions

This paper presents an incremental recognition method for online handwritten mathematical expressions (MEs), which is used for busy recognition interface (recognition while writing) without large waiting time. We employ local processing strategy and focus on recent strokes. For the latest stroke, we perform segmentation, recognition and update of Cocke-Younger-Kasami (CYK) table. We also reuse the segmentation and recognition candidates in the previous processes. Moreover, using multi-thread reduces the waiting time. Experiments on our data set show the effectiveness of the incremental method not only in small waiting time but also keeping almost the same recognition rate of the batch recognition method without significant decrease. We also propose the combination of the two methods which succeeds the advantages of the both.

Semi-incremental Recognition of Online Handwritten Mathematical Expressions

This paper presents a semi-incremental recognition method for online handwritten mathematical expressions (MEs). The method reduces the waiting time after an ME is written until the result of recognition is output. Our method has two main processes, one is to process the latest stroke, the other is to find and correct wrong recognitions in the strokes up to the latest stroke. In the first process, the segmentation, recognition and Cocke-Younger-Kasami (CYK) algorithm are only executed for the latest stroke. In the second process, all the previous segmentations are updated if they are significantly changed after the latest stroke is input, and then, all the symbols related to the updated segmentations will be updated with their recognition scores. These changes are reflected into the CYK table. In addition, the waiting time is further reduced by employing multi-thread processes. Experiments on our data set show the effectiveness of this semi-incremental method which not only has higher recognition rate than our previous pure-incremental method but also keeps the waiting time unnoticeable.

Augmented incremental recognition of online handwritten mathematical expressions

This paper presents an augmented incremental recognition method for online handwritten mathematical expressions (MEs). If an ME is recognized after all strokes are written (batch recognition), the waiting time increases significantly when the ME becomes longer. On the other hand, the pure incremental recognition method recognizes an ME whenever a new single stroke is input. It shortens the waiting time but degrades the recognition rate due to the limited context. Thus, we propose an augmented incremental recognition method that not only maintains the advantage of the two methods but also reduces their weaknesses. The proposed method has two main features: one is to process the latest stroke, and the other is to find the erroneous segmentations and recognitions in the recent strokes and correct them. In the first process, the segmentation and the recognition by Cocke–Younger–Kasami (CYK) algorithm are only executed for the latest stroke. In the second process, all the previous segmentations are updated if they are significantly changed after the latest stroke is input, and then, all the symbols related to the updated segmentations are updated with their recognition scores. These changes are reflected in the CYK table. In addition, the waiting time is further reduced by employing multi-thread processes. Experiments on our dataset and the CROHME datasets show the effectiveness of this augmented incremental recognition method, which not only maintains recognition rate even compared with the batch recognition method but also reduces the waiting time to a very small level.

Modified X-Y Cut for Re-Ordering Strokes of Online Handwritten Mathematical Expressions

This paper proposes a modified X-Y cut method for reordering strokes of online handwritten mathematical expression (ME) in order to make stroke order free recognition. To deal with overlapping, which causes problems in the X-Y cut method, we determine vertically ordered strokes by detecting vertical symbols and its upper/lower MEs. An upper ME and a lower ME are treated as MEs which are reordered recursively. Unordered strokes on the left side of a vertical symbol are reordered as horizontally ordered strokes. The remaining strokes are reordered recursively. The horizontally ordered strokes are reordered from left to right and the vertically ordered strokes are reordered from top to bottom. The results of evaluations of the reordering method on the CROHME 2014 database show that our ME recognition system incorporating this method outperforms all other systems that use only CROHME 2014 for training while the processing time is kept to a practical level.