Per Ola Kristensson

SHARK 2 : a large vocabulary shorthand writing system for pen-based computers

Zhai and Kristensson (2003) presented a method of speed-writing for pen-based computing which utilizes gesturing on a stylus keyboard for familiar words and tapping for others. In SHARK²:, we eliminated the necessity to alternate between the two modes of writing, allowing any word in a large vocabulary (e.g. 10,000-20,000 words) to be entered as a shorthand gesture. This new paradigm supports a gradual and seamless transition from visually guided tracing to recall-based gesturing. Based on the use characteristics and human performance observations, we designed and implemented the architecture, algorithms and interfaces of a high-capacity multi-channel pen-gesture recognition system. The systems key components and performance are also reported.

Shorthand writing on stylus keyboard

We propose a method for computer-based speed writing, SHARK (shorthand aided rapid keyboarding), which augments stylus keyboarding with shorthand gesturing. SHARK defines a shorthand symbol for each word according to its movement pattern on an optimized stylus keyboard. The key principles for the SHARK design include high efficiency stemmed from layout optimization, duality of gesturing and stylus tapping, scale and location independent writing, Zipfs law, and skill transfer from tapping to shorthand writing due to pattern consistency. We developed a SHARK system based on a classic handwriting recognition algorithm. A user study demonstrated the feasibility of the SHARK method.

Memorability of pre-designed and user-defined gesture sets

We studied the memorability of free-form gesture sets for invoking actions. We compared three types of gesture sets: user-defined gesture sets, gesture sets designed by the authors, and random gesture sets in three studies with 33 participants in total. We found that user-defined gestures are easier to remember, both immediately after creation and on the next day (up to a 24% difference in recall rate compared to pre-designed gestures). We also discovered that the differences between gesture sets are mostly due to association errors (rather than gesture form errors), that participants prefer user-defined sets, and that they think user-defined gestures take less time to learn. Finally, we contribute a qualitative analysis of the tradeoffs involved in gesture type selection and share our data and a video corpus of 66 gestures for replicability and further analysis.

Improving two-thumb text entry on touchscreen devices

We study the design of split keyboards for fast text entry with two thumbs on mobile touchscreen devices. The layout of KALQ was determined through first studying how users should grip a device with two hands. We then assigned letters to keys computationally, using a model of two-thumb tapping. KALQ minimizes thumb travel distance and maximizes alternation between thumbs. An error correction algorithm was added to help address linguistic and motor errors. Users reached a rate of 37 words per minute (with a 5% error rate) after a training program.

The word-gesture keyboard: reimagining keyboard interaction

The basic concepts and initial prototype of a word-gesture keyboard are discussed. In early 1980s, Montgomery conceived the idea of using sliding gestures on a touch keyboard to enter characters. He designed a wipe activated keyboard with a flat touch sensitive surface. The positions of the letter keys were carefully arranged to make consecutive letters commonly appear in words connected on the keyboard. Since a gesture keyboard enhances, rather than replaces, a conventional touchscreen keyboard, out of vocabulary (OOV) letter sequences can always be entered by typing the individual letter keys. In using a word-gesture keyboard, the production of movements increasingly changes from focusing on individual letters to connecting multiple letters into a word gesture. Conceptually, gesture recognition is done by identifying the word which has the highest probability given the users gesture. Exact or statistical modeling of gesture keyboards speed-accuracy trade-off incorporating human control behavior is under research.

In search of effective text input interfaces for off the desktop computing

It is generally recognized that todays frontier of HCI research lies beyond the traditional desktop computers whose GUI interfaces were built on the foundation of display—pointing device—full keyboard. Many interface challenges arise without such a physical UI foundation. Text writing—ranging from entering URLs and search queries, filling forms, typing commands, to taking notes and writing emails and chat messages—is one of the hard problems awaiting for solutions in off-desktop computing. This paper summarizes and synthesizes a research program on this topic at the IBM Almaden Research Center. It analyzes various dimensions that constitute a good text input interface; briefly reviews related literature; discusses the evaluation methodology issues of text input; presents the major ideas and results of two systems, ATOMIK and SHARK; and points out current and future directions in the area from our current vantage point.

A versatile dataset for text entry evaluations based on genuine mobile emails

Mobile text entry methods are typically evaluated by having study participants copy phrases. However, currently there is no available phrase set that has been composed by mobile users. Instead researchers have resorted to using invented phrases that probably suffer from low external validity. Further, there is no available phrase set whose phrases have been verified to be memorable. In this paper we present a collection of mobile email sentences written by actual users on actual mobile devices. We obtained our sentences from emails written by Enron employees on their BlackBerry mobile devices. We provide empirical data on how easy the sentences were to remember and how quickly and accurately users could type these sentences on a full-sized keyboard. Using this empirical data, we construct a series of phrase sets we suggest for use in text entry evaluations.

Shapewriter on the iphone: from the laboratory to the real world

We present our experience in bringing ShapeWriter, a novel HCI research product, from the laboratory to real world users through iPhones App Store.

Foundational Issues in Touch-Surface Stroke Gesture Design: An Integrative Review

The potential for using stroke gestures to enter, retrieve and select commands and text has been recently unleashed by the popularity of touchscreen devices. This monograph provides a state-of-the-art integrative review of a body of human–computer interaction research on stroke gestures. It begins with an analysis of the design dimensions of stroke gestures as an interaction medium. The analysis classifies gestures into analogue versus abstract gestures, gestures for commands versus for symbols, gestures with different orders of complexity, visual-spatial dependent and independent gestures, and finger versus stylus drawn gestures. Gesture interfaces such as the iOS interface, the Graffiti text entry method for Palm devices, marking menus, and the SHARK/ShapeWriter word-gesture keyboard, make different choices in this multi-dimensional design space. The main body of this work consists of reviewing and synthesizing some of the foundational studies in the literature on stroke gesture interaction, particularly those done by the authors in the last decade. The human performance factors covered include motor control complexity, visual and auditory feedback, and human memory capabilities in dealing with gestures. Based on these foundational studies this review presents a set of design principles for creating stroke gesture interfaces. These include making gestures analogous to physical effects or cultural conventions, keeping gestures simple and distinct, defining stroke gestures systematically, making them self-revealing, supporting appropriate levels of chunking, and facilitating progress from visually guided performance to recall-driven performance. The overall theme is on making learning gestures easier while designing for long-term efficiency. Important system implementation issues of stroke gesture interfaces such as gesture recognition algorithms and gesture design toolkits are also covered in this review. The monograph ends with a few call-to-action research topics.

Five Challenges for Intelligent Text Entry Methods

For text entry methods to be useful they have to deliver high entry rates and low error rates. At the same time they need to be easy-to-learn and provide effective means of correcting mistakes. Intelligent text entry methods combine AI techniques with HCI theory to enable users to enter text as efficiently and effortlessly as possible. Here I sample a selection of such techniques from the research literature and set them into their historical context. I then highlight five challenges for text entry methods that aspire to make an impact in our society: localization, error correction, editor support, feedback, and context of use.