Harold Mouchère

CROHME2011: Competition on Recognition of Online Handwritten Mathematical Expressions

A competition on recognition of online handwritten mathematical expressions is organized. Recognition of mathematical expressions has been an attractive problem for the pattern recognition community because of the presence of enormous uncertainties and ambiguities as encountered during parsing of the two-dimensional structure of expressions. The goal of this competition is to bring out a state of the art for the related research. Three labs come together to organize the event and six other research groups participated the competition. The competition defines a standard format for presenting information, provides a training set of 921 expressions and supplies the underlying grammar for understanding the content of the training data. Participants were invited to submit their recognizers which were tested with a new set of 348 expressions. Systems are evaluated based on four different aspects of the recognition problem. However, the final rating of the systems is done based on their correct expression recognition accuracies. The best expression level recognition accuracy (on the test data) shown by the competing systems is 19.83% whereas a baseline system developed by one of the organizing groups reports an accuracy 22.41% on the same data set.

Towards Handwritten Mathematical Expression Recognition

In this paper, we propose a new framework for online handwritten mathematical expression recognition. The proposed architecture aims at handling mathematical expression recognition as a simultaneous optimization of symbol segmentation, symbol recognition, and 2D structure recognition under the restriction of a mathematical expression grammar. To achieve this goal, we consider a hypothesis generation mechanism supporting a 2D grouping of elementary strokes, a cost function defining the global likelihood of a solution, and a dynamic programming scheme giving at the end the best global solution according to a 2D grammar and a classifier. As a classifier, a neural network architecture is used; it is trained within the overall architecture allowing rejecting incorrect segmented patterns. The proposed system is trained with a set of synthetic online handwritten mathematical expressions. When tested on a set of real complex expressions, the system achieves promising results at both symbol and expression interpretation levels.

A global learning approach for an online handwritten mathematical expression recognition system

Despite the recent advances in handwriting recognition, handwritten two-dimensional (2D) languages are still a challenge. Electrical schemas, chemical equations and mathematical expressions (MEs) are examples of such 2D languages. In this case, the recognition problem is particularly difficult due to the two dimensional layout of the language. This paper presents an online handwritten mathematical expression recognition system that handles mathematical expression recognition as a simultaneous optimization of expression segmentation, symbol recognition, and 2D structure recognition under the restriction of a mathematical expression grammar. The originality of the approach is a global strategy allowing learning mathematical symbols and spatial relations directly from complete expressions. A new contextual modeling is proposed for combining syntactic and structural information. Those models are used to find the most likely combination of segmentation/recognition hypotheses proposed by a 2D segmentation scheme. Thus, models are based on structural information concerning the symbol layout. The system is tested with a new public database of mathematical expressions which was used in the CHROME competition. We have also produced a large base of semi-synthetic expressions which are used to train and test the global learning approach. We obtain very promising results on both synthetic and real expressions databases, as well as in the recent CHROME competition.

ICFHR 2012 Competition on Recognition of On-Line Mathematical Expressions (CROHME 2012)

This paper presents an overview of the second Competition on Recognition of Online Handwritten Mathematical Expressions, CROHME 2012. The objective of the contest is to identify current advances in mathematical expression recognition using common evaluation performance measures and datasets. This paper describes the contest details including the evaluation measures used as well as the performance of the 7 submitted systems along with a short description of each system. Progress as compared to the 1st version of CROHME is also documented.

A Unified Strategy to Deal with Different Natures of Reject

The interest of reject for classifier optimization has been shown many times. The diversity of the applications requiring this concept makes us to distinguish two main natures of reject with distinct goals: the confusion reject and the distance reject. After the description of this two kinds of reject, we present a unified formalism to define them using reliability functions and reject thresholds. Then we present a generic algorithm dedicated to the automatic learning of these thresholds. Finally, we compare various possibilities of reject to achieve application goals

Stroke-Based Performance Metrics for Handwritten Mathematical Expressions

Evaluating mathematical expression recognition involves a complex interaction of input primitives (e.g. pen/finger strokes), recognized symbols, and recognized spatial structure. Existing performance metrics simplify this problem by separating the assessment of spatial structure from the assessment of symbol segmentation and classification. These metrics do not characterize the overall accuracy of a pen-based mathematics recognition, making it difficult to compare math recognition algorithms, and preventing the use of machine learning algorithms requiring a criterion function characterizing overall system performance. To address this problem, we introduce performance metrics that bridge the gap from handwritten strokes to spatial structure. Our metrics are computed using bipartite graphs that represent classification, segmentation and spatial structure at the stroke level. Overall correctness of an expression is measured by counting the number of relabelings of nodes and edges needed to make the bipartite graph for a recognition result match the bipartite graph for ground truth. This metric may also be used with other primitive types (e.g. image pixels).

ICFHR 2014 Competition on Recognition of On-line Handwritten Mathematical Expressions (CROHME 2014)

We present the outcome of the latest edition of the CROHME competition, dedicated to on-line handwritten mathematical expression recognition. In addition to the standard full expression recognition task from previous competitions, CROHME 2014 features two new tasks. The first is dedicated to isolated symbol recognition including a reject option for invalid symbol hypotheses, and the second concerns recognizing expressions that contain matrices. System performance is improving relative to previous competitions. Data and evaluation tools used for the competition are publicly available.

HAMEX - A Handwritten and Audio Dataset of Mathematical Expressions

In this paper, we present HAMEX, a new public dataset that contains mathematical expressions available in their on-line handwritten form and in their audio spoken form. We have designed this dataset so that, given a mathematical expression, its handwritten signal and its audio signal can be used jointly to design multimodal recognition systems. Here, we describe the different steps that allowed us to acquire this dataset, from the creation of the mathematical expression corpora (including expressions from Wikipedia pages) to the segmentation and the transcription of the collected data, via the data collection process itself. Currently, the dataset contains 4 350 on-line handwritten mathematical expressions written by 58 writers, and the corresponding audio expressions (in French) spoken by 58 speakers. The ground truth is also provided both for the handwritten expressions (as INKML files with the digital ink, the symbol segmentation, and the MATHML structure) and for the audio expressions (as XML files with the transcriptions of the spoken expressions).

WRITER STYLE ADAPTATION IN ONLINE HANDWRITING RECOGNIZERS BY A FUZZY MECHANISM APPROACH: THE ADAPT METHOD

This study presents an automatic online adaptation mechanism to the handwriting style of a writer for the recognition of isolated handwritten characters. The classifier we use here is based on a Fuzzy Inference System (FIS) similar to those we have designed for handwriting recognition. In this FIS each premise rule is composed of a fuzzy prototype which represents intrinsic properties of a class. Furthermore, the conclusion part of rules associates a score to the prototype for each class. The adaptation mechanism affects both the conclusions of the rules and the fuzzy prototypes by recentering and reshaping them thanks to a new approach called ADAPT inspired by the Learning Vector Quantization. Thus the FIS is automatically fitted to the handwriting style of the writer that currently uses the system. Our adaptation mechanism is compared with well known adaptation techniques. The tests were based on eight different writers and the results illustrate the benefits of the method in terms of error rate reduction (86% in average). This allows such kind of simple classifiers to achieve up to 98.4% of recognition accuracy on the 26 Latin letters in a writer dependent context.

The Problem of Handwritten Mathematical Expression Recognition Evaluation

We discuss in this paper some issues related to the problem of mathematical expression recognition. The very first important issue is to define how to ground truth a dataset of handwritten mathematical expressions, and next we have to face the problem of benchmarking systems. We propose to define some indicators and the way to compute them so as they reflect the actual performances of a given system.