Daniel Martin Keysers

GyroPen: Gyroscopes for Pen-Input With Mobile Phones

We present GyroPen, a method to reconstruct the motion path for pen-like interaction from standard built-in sensors in modern smartphones. The key idea is to reconstruct a representation of the trajectory of the phones corner that is touching a writing or drawing surface from the measurements obtained from the phones gyroscopes and accelerometers. We propose to directly use the angular trajectory for this reconstruction, which removes the necessity for accurate absolute 3-D position estimation, a task that can be difficult using low-cost accelerometers. We connect GyroPen to a handwriting recognition system and perform two proof-of-concept experiments to demonstrate that the reconstruction accuracy of GyroPen is accurate enough to be a promising approach to text entry. In a first experiment, the average novice participant (n=10) was able to write the first word only 37 s after the starting to use GyroPen for the first time. In a second experiment, experienced users (n=2) were able to write at the speed of 3-4 s for one English word and with a character error rate of 18%.

Multi-Language Online Handwriting Recognition

We describe Googles online handwriting recognition system that currently supports 22 scripts and 97 languages. The systems focus is on fast, high-accuracy text entry for mobile, touch-enabled devices. We use a combination of state-of-the-art components and combine them with novel additions in a flexible framework. This architecture allows us to easily transfer improvements between languages and scripts. This made it possible to build recognizers for languages that, to the best of our knowledge, are not handled by any other online handwriting recognition system. The approach also enabled us to use the same architecture both on very powerful machines for recognition in the cloud as well as on mobile devices with more limited computational power by changing some of the settings of the system. In this paper we give a general overview of the system architecture and the novel components, such as unified time- and position-based input interpretation, trainable segmentation, minimum-error rate training for feature combination, and a cascade of pruning strategies. We present experimental results for different setups. The system is currently publicly available in several Google products, for example in Google Translate and as an input method for Android devices.