Daniel Martín-Albo

Gestures à Go Go: Authoring Synthetic Human-Like Stroke Gestures Using the Kinematic Theory of Rapid Movements

Training a high-quality gesture recognizer requires providing a large number of examples to enable good performance on unseen, future data. However, recruiting participants, data collection, and labeling, etc., necessary for achieving this goal are usually time consuming and expensive. Thus, it is important to investigate how to empower developers to quickly collect gesture samples for improving UI usage and user experience. In response to this need, we introduce Gestures à Go Go (g3), a web service plus an accompanying web application for bootstrapping stroke gesture samples based on the kinematic theory of rapid human movements. The user only has to provide a gesture example once, and g3 will create a model of that gesture. Then, by introducing local and global perturbations to the model parameters, g3 generates from tens to thousands of synthetic human-like samples. Through a comprehensive evaluation, we show that synthesized gestures perform equally similar to gestures generated by human users. Ultimately, this work informs our understanding of designing better user interfaces that are driven by gestures.

Training of On-Line Handwriting Text Recognizers with Synthetic Text Generated Using the Kinematic Theory of Rapid Human Movements

A method for automatic generation of synthetic handwritten words is presented which is based in the Kinematic Theory and its Sigma-lognormal model. To generate a new synthetic sample, first a real word is modelled using the Sigma-lognormal model. Then the Sigma-lognormal parameters are randomly perturbed within a range, introducing human-like variations in the sample. Finally, the velocity function is recalculated taking into account the new parameters. The synthetic words are then used as training data for a Hidden Markov Model based on-line handwritten recognizer. The experimental results confirm the great potential of the kinematic theory of rapid human movements applied to writer adaptation.

Synthesizing Stroke Gestures Across User Populations: A Case for Users with Visual Impairments

We introduce a new principled method grounded in the Kinematic Theory of Rapid Human Movements to automatically generate synthetic stroke gestures across user populations in order to support ability-based design of gesture user interfaces. Our method is especially useful when the target user population is difficult to sample adequately and, consequently, when there is not enough data to train gesture recognizers to deliver high levels of accuracy. To showcase the relevance and usefulness of our method, we collected gestures from people without visual impairments and successfully synthesized gestures with the articulation characteristics of people with visual impairments. We also show that gesture recognition accuracy improves significantly when using our synthetic gesture samples for training. Our contributions will benefit researchers and practitioners that wish to design gesture user interfaces for people with various abilities by helping them prototype, evaluate, and predict gesture recognition performance without having to expressly recruit and involve people with disabilities in long, time-consuming gesture collection experiments.

Improving sigma-lognormal parameter extraction

A fully automatic framework based on the kinematic theory of rapid human movements was recently introduced for analyzing and modeling complex human movements patterns such as those involved in handwriting. In this paper, we present a new approach to better extract and estimate the lognormal primitives and parameters. Through a comprehensive evaluation using 32,000 words from a public database, we show that our approach greatly improves the state-of-the-art extractor.

The Kinematic Theory Produces Human-Like Stroke Gestures

We show that the Kinematic Theory produces synthesized stroke gestures that “look and feel” the same and hold the same statistical characteristics as human-generated gestures. Previous research in this vein has conducted such comparison from the classification accuracy performance, which is a legitimate though indirect measure. In this article, we synthesized two well-known public datasets comprising unistroke and multistroke gestures. We then compared geometric, kinematic, and articulation aspects of human and synthetic gestures, and found no practical differences between both populations. We also conducted an online survey involving 236 participants and found that it is very difficult to tell human and synthetic gestures apart. We can finally be confident that synthesized gestures are actually reflective of how users produce stroke gestures. In sum, this work enables a deeper understanding of synthetic gestures’ production, which can inform the design of better gesture sets and development of more accurate recognizers. Author

G3: bootstrapping stroke gestures design with synthetic samples and built-in recognizers

Stroke gestures are becoming increasingly important with the ongoing success of touchscreen-capable devices. However, training a high-quality gesture recognizer requires providing a large number of examples to enable good performance on unseen, future data. Furthermore, recruiting participants, data collection and labeling, etc. necessary for achieving this goal are usually time-consuming and expensive. In response to this need, we introduce G3, a mobile-first web application for bootstrapping unistroke, multistroke, or multitouch gestures. The user only has to provide a gesture example once, and G3 will create a kinematic model of that gesture. Then, by introducing local and global perturbations to the model parameters, G3 will generate any number of synthetic human-like samples. In addition, the user can get a gesture recognizer together with the synthesized data. As such, the outcome of G3 can be directly incorporated into production-ready applications.

Interactive Off-Line Handwritten Text Transcription Using On-Line Handwritten Text as Feedback

Handwritten Text Recognition is a problem that has gained attention in the last years mainly due to the interest in the transcription of historical documents. However, the automatic transcription is ineffectual in unconstrained handwritten documents. Thus, human intervention is typically needed to correct the results. Given that a post-editing approach is inefficient and uncomfortable, multimodal interactive approaches have begun to emerge in the last years. In this scheme, the user interacts with the system by means of an e-pen. This multimodal feedback, on the one hand, allows to improve the accuracy of the system and, on the other hand, increases user acceptability. In this work, we present a new approach on interaction based on character sequences. Here we present developments that allow taking advantage of interaction-derived context to significantly improve feedback decoding accuracy. Empirical tests suggest that, despite the loss of the deterministic accuracy of traditional peripherals, this approach can save significant amounts of user effort with respect to non-interactive post-editing correction.

An Experimental Study of Pruning Techniques in Handwritten Text Recognition Systems

Handwritten Text Recognition is a problem that has gained attention in the last years mainly due to the interest in the transcription of historical documents. However, the automatic transcription of handwritten documents is not error free and human intervention is typically needed to correct the results of such systems. This interactive scenario demands real-time response. In this paper, we present a study comparing how different pruning techniques affect the performance of two freely available decoding systems, HTK and iATROS. These two systems are based on Hidden Markov Models and n-gram language models. However, while HTK only considers 2-gram language models, iATROS works with n-grams of any order. In this paper, we also carried out a study about how the use of n-grams of size greater than two can enhance results over 2-grams. Experiments are reported with the publicly available ESPOSALLES database.

Multimodal Computer-Assisted Transcription of Text Images at Character-Level Interaction

Currently, automatic handwriting recognition systems are ineffectual in unconstrained handwriting documents. Therefore, to obtain perfect transcriptions, heavy human intervention is required to validate and correct the results of such systems. Given that this post-editing process is inefficient and uncomfortable, a multimodal interactive approach has been proposed in previous works, which aims at obtaining correct transcriptions with the minimum human effort. In this approach, the user interacts with the system by means of an e-pen and/or more traditional methods such as keyboard or mouse. This users feedback allows to improve system accuracy and multimodality increases system ergonomics and user acceptability. Until now, multimodal interaction has been considered only at whole-word level. In this work, multimodal interaction at character-level is studied, that may lead to more effective interactivity, since it is faster and easier to write only one character rather than a whole word. Here we study this kind of fine-grained multimodal interaction and present developments that allow taking advantage of interaction-derived context to significantly improve feedback decoding accuracy. Empirical tests on three cursive handwritten tasks suggest that, despite losing the deterministic accuracy of traditional peripherals, this approach can save significant amounts of user effort with respect to fully manual transcription as well as to noninteractive post-editing correction.

On the Design of Personal Digital Bodyguards: Impact of Hardware Resolution on Handwriting Analysis

Handheld touch-capable devices have become one of the most popular and fastest growing consumer products. It seems logical therefore to think of such devices as Personal Digital Bodyguards (PDBs) in charge for example of biometrical, biomedical, and neurocognitive monitoring by just inspecting the users handwriting activity. However, it is unclear whether the hardware of todays devices is capable to handle this task. To this end, we conducted a comparative study regarding the capabilities of past and current tablets to allow for the design of PDBs based on the exploitation of the Kinematic Theory. Our study shows that, while some improvements are still necessary at the sampling frequency level, the conclusions drawn by the Kinematic Theory can be directly transferred to PDBs.