Shumin Zhai

Characterizing computer input with Fitts' law parameters: the information and non-information aspects of pointing

Throughput (TP), also known as index of performance or bandwidth in Fitts law tasks, has been a fundamental metric in quantifying input system performance. The operational definition of TP is varied in the literature. In part thanks to the common interpretations of International Standard ISO 9241-9, the Ergonomic requirements for office work with visual display terminals-- Part 9: Requirements for non-keyboard input devices, the measurements of throughput have increasingly converged onto the average ratio of index of difficulty (ID) and trial completion time (MT), i.e. TP=ID/MT. In lieu of the complete Fitts law regression results that can only be represented by both slope (b) and intercept (a) (or MT=a+b ID), TP has been used as the sole performance characteristic of input devices, which is problematic. We show that TP defined as ID/ MT is an ill-defined concept that may change its value with the set of ID values used for the same input device and cannot be generalized beyond specific experimental target distances and sizes. The greater the absolute value of a is, the more variable TP (=ID/MT) is. ID/MT only equals a constant 1/b when a = 0. We suggest that future studies should use the complete Fitts law regression characterized by (a, b) parameters to characterize an input system, a reflects the noninformational aspect and b the informational aspect of input performance. For convenience, 1/b can be named as throughput which, unlike ID/MT, is conceptually a true constant.

Speed-accuracy tradeoff in Fitts' law tasks: on the equivalency of actual and nominal pointing precision

Pointing tasks in human-computer interaction obey certain speed-accuracy tradeoff rules. In general, the more accurate the task to be accomplished, the longer it takes and vice versa. Fitts law models the speed-accuracy tradeoff effect in pointing as imposed by the task parameters, through Fitts index of difficulty (Id) based on the ratio of the nominal movement distance and the size of the target. Operating with different speed or accuracy biases, performers may utilize more or less area than the target specifies, introducing another subjective layer of speed-accuracy tradeoff relative to the task specification. A conventional approach to overcome the impact of the subjective layer of speed-accuracy tradeoff is to use the a posteriori effective pointing precision We in lieu of the nominal target width W. Such an approach has lacked a theoretical or empirical foundation. This study investigates the nature and the relationship of the two layers of speed-accuracy tradeoff by systematically controlling both Id and the index of target utilization Iu in a set of four experiments. Their results show that the impacts of the two layers of speed-accuracy tradeoff are not fundamentally equivalent. The use of We could indeed compensate for the difference in target utilization, but not completely. More logical Fitts law parameter estimates can be obtained by the We adjustment, although its use also lowers the correlation between pointing time and the index of difficulty. The study also shows the complex interaction effect between Id and Iu, suggesting that a simple and complete model accommodating both layers of speed-accuracy tradeoff may not exist.

Touch behavior with different postures on soft smartphone keyboards

Text entry on smartphones is far slower and more error-prone than on traditional desktop keyboards, despite sophisticated detection and auto-correct algorithms. To strengthen the empirical and modeling foundation of smartphone text input improvements, we explore touch behavior on soft QWERTY keyboards when used with two thumbs, an index finger, and one thumb. We collected text entry data from 32 participants in a lab study and describe touch accuracy and precision for different keys. We found that distinct patterns exist for input among the three hand postures, suggesting that keyboards should adapt to different postures. We also discovered that participants touch precision was relatively high given typical key dimensions, but there were pronounced and consistent touch offsets that can be leveraged by keyboard algorithms to correct errors. We identify patterns in our empirical findings and discuss implications for design and improvements of soft keyboards.

A comparative evaluation of finger and pen stroke gestures

This paper reports an empirical investigation in which participants produced a set of stroke gestures with varying degrees of complexity and in different target sizes using both the finger and the pen. The recorded gestures were then analyzed according to multiple measures characterizing many aspects of stroke gestures. Our findings were as follows: (1) Finger drawn gestures were quite different to pen drawn gestures in basic measures including size ratio and average speed. Finger drawn gestures tended to be larger and faster than pen drawn gestures. They also differed in shape geometry as measured by, for example, aperture of closed gestures, corner shape distance and intersecting points deviation; (2) Pen drawn gestures and finger drawn gestures were similar in several measures including articulation time, indicative angle difference, axial symmetry and proportional shape distance; (3) There were interaction effects between gesture implement (finger vs. pen) and target gesture size and gesture complexity. Our findings show that half of the features we tested were performed well enough by the finger. This finding suggests that finger friendly systems should exploit these features when designing finger interfaces and avoid using the other features in which the finger does not perform as well as the pen.

Personalized Digital Television

This chapter presents the recommendation techniques applied in Personal Program Guide (PPG). This is a system generating personalized Electronic Program Guides for Digital TV. The PPGmanages a user model that stores the estimates of the individual user’s preferences for TV program categories. This model results from the integration of di¡erent preference acquisitionmodules that handle explicit user preferences, stereotypical information about TV viewers, and information about the user’s viewing behavior. The observation of the individual viewing behavior is particularly easy because the PPG runs on the set-top box and is deeply integrated with the TV playing and the video recording services o¡ered by that type of device.

Making touchscreen keyboards adaptive to keys, hand postures, and individuals: a hierarchical spatial backoff model approach

We propose a new approach for improving text entry accuracy on touchscreen keyboards by adapting the underlying spatial model to factors such as input hand postures, individuals, and target key positions. To combine these factors together, we introduce a hierarchical spatial backoff model (SBM) that consists of submodels with different levels of complexity. The most general model includes no adaptive factors, whereas the most specific model includes all three. Considering that in practice people may switch hand postures (e.g., from two-thumb to one-finger) to better suit a situation, and that the specific submodels may take time to train for each user, a specific submodel should be applied only if its corresponding input posture can be identified with confidence, and if the submodel has enough training data from the user. We introduce the backoff mechanism to fall back to a simpler model if either of these conditions are not met. We implemented a prototype system capable of reducing the language-model-independent error rate by 13.2% using an online posture classifier with 86.4% accuracy. Further improvements in error rate may be possible with even better posture classification.

Bayesian touch: a statistical criterion of target selection with finger touch

To improve the accuracy of target selection for finger touch, we conceptualize finger touch input as an uncertain process, and derive a statistical target selection criterion, Bayesian Touch Criterion, by combining the basic Bayes rule of probability with the generalized dual Gaussian distribution hypothesis of finger touch. The Bayesian Touch Criterion selects the intended target as the candidate with the shortest Bayesian Touch Distance to the touch point, which is computed from the touch point to the target center distance and the target size. We give the derivation of the Bayesian Touch Criterion and its empirical evaluation with two experiments. The results showed that for 2-dimensional circular target selection, the Bayesian Touch Criterion is significantly more accurate than the commonly used Visual Boundary Criterion (i.e., a target is selected if and only if the touch point falls within its boundary) and its two variants.

Bimanual gesture keyboard

Gesture keyboards represent an increasingly popular way to input text on mobile devices today. However, current gesture keyboards are exclusively unimanual. To take advantage of the capability of modern multi-touch screens, we created a novel bimanual gesture text entry system, extending the gesture keyboard paradigm from one finger to multiple fingers. To address the complexity of recognizing bimanual gesture, we designed and implemented two related interaction methods, finger-release and space-required, both based on a new multi-stroke gesture recognition algorithm. A formal experiment showed that bimanual gesture behaviors were easy to learn. They improved comfort and reduced the physical demand relative to unimanual gestures on tablets. The results indicated that these new gesture keyboards were valuable complements to unimanual gesture and regular typing keyboards.

Top-down learning strategies: can they facilitate stylus keyboard learning?

Abstract Learning a new stylus keyboard layout is time-consuming yet potentially rewarding, as optimized virtual keyboards can substantially increase performance for expert users. This paper explores whether the learning curve can be accelerated using top-down learning strategies. In an experiment, one group of participants learned a stylus keyboard layout with top-down methods, such as visuo-spatial grouping of letters and mnemonic techniques, to build familiarity with a stylus keyboard. The other (control) group learned the keyboard by typing sentences. The top-down learning group liked the stylus keyboard better and perceived it to be more effective than the control group. They also had better memory recall performance. Typing performance after the top-down learning process was faster than the initial performance of the control group, but not different from the performance of the control group after they had spent an equivalent amount of time typing. Therefore, top-down learning strategies improved the explicit recall as expected, but the improved memory of the keyboard did not result in quicker typing speeds. These results suggest that quicker acquisition of declarative knowledge does not improve the acquisition speed of procedural knowledge, even during the initial cognitive stage of the virtual keyboard learning. They also suggest that top-down learning strategies can motivate users to learn a new keyboard more than repetitive rehearsal, without any loss in typing performance.

Octopus: evaluating touchscreen keyboard correction and recognition algorithms via

The time and labor demanded by a typical laboratory-based keyboard evaluation are limiting resources for algorithmic adjustment and optimization. We propose Remulation, a complementary method for evaluating touchscreen keyboard correction and recognition algorithms. It replicates prior user study data through real-time, on-device simulation. We have developed Octopus, a Remulation-based evaluation tool that enables keyboard developers to efficiently measure and inspect the impact of algorithmic changes without conducting resource-intensive user studies. It can also be used to evaluate third-party keyboards in a black box fashion, without access to their algorithms or source code. Octopus can evaluate both touch keyboards and word-gesture keyboards. Two empirical examples show that Remulation can efficiently and effectively measure many aspects of touch screen keyboards at both macro and micro levels. Additionally, we contribute two new metrics to measure keyboard accuracy at the word level: the Ratio of Error Reduction (RER) and the Word Score.