Pourang Irani

Detecting and leveraging finger orientation for interaction with direct-touch surfaces

Current interactions on direct-touch interactive surfaces are often modeled based on properties of the input channel that are common in traditional graphical user interfaces (GUI) such as x-y coordinate information. Leveraging additional information available on the surfaces could potentially result in richer and novel interactions. In this paper we specifically explore the role of finger orientation. This property is typically ignored in touch-based interactions partly because of the ambiguity in determining it solely from the contact shape. We present a simple algorithm that unambiguously detects the directed finger orientation vector in real-time from contact information only, by considering the dynamics of the finger landing process. Results of an experimental evaluation show that our algorithm is stable and accurate. We then demonstrate how finger orientation can be leveraged to enable novel interactions and to infer higher-level information such as hand occlusion or user position. We present a set of orientation-aware interaction techniques and widgets for direct-touch surfaces.

Tilt techniques: investigating the dexterity of wrist-based input

Most studies on tilt based interaction can be classified as point-designs that demonstrate the utility of wrist-tilt as an input medium; tilt parameters are tailored to suit the specific interaction at hand. In this paper, we systematically analyze the design space of wrist-based interactions and focus on the level of control possible with the wrist. In a first study, we investigate the various factors that can influence tilt control, separately along the three axes of wrist movement: flexion/extension, pronation/supination, and ulnar/radial deviation. Results show that users can control comfortably at least 16 levels on the pronation/supination axis and that using a quadratic mapping function for discretization of tilt space significantly improves user performance across all tilt axes. We discuss the findings of our results in the context of several interaction techniques and identify several general design recommendations.

Wedge: clutter-free visualization of off-screen locations

To overcome display limitations of small-screen devices, researchers have proposed techniques that point users to objects located off-screen. Arrow-based techniques such as City Lights convey only direction. Halo conveys direction and distance, but is susceptible to clutter resulting from overlapping halos. We present Wedge, a visualization technique that conveys direction and distance, yet avoids overlap and clutter. Wedge represents each off-screen location using an acute isosceles triangle: the tip coincides with the off-screen locations, and the two corners are located on-screen. A wedge conveys location awareness primarily by means of its two legs pointing towards the target. Wedges avoid overlap programmatically by repelling each other, causing them to rotate until overlap is resolved. As a result, wedges can be applied to numbers and configurations of targets that would lead to clutter if visualized using halos. We report on a user study comparing Wedge and Halo for three off-screen tasks. Participants were significantly more accurate when using Wedge than when using Halo.

Consumed endurance: a metric to quantify arm fatigue of mid-air interactions

Mid-air interactions are prone to fatigue and lead to a feeling of heaviness in the upper limbs, a condition casually termed as the gorilla-arm effect. Designers have often associated limitations of their mid-air interactions with arm fatigue, but do not possess a quantitative method to assess and therefore mitigate it. In this paper we propose a novel metric, Consumed Endurance (CE), derived from the biomechanical structure of the upper arm and aimed at characterizing the gorilla-arm effect. We present a method to capture CE in a non-intrusive manner using an off-the-shelf camera-based skeleton tracking system, and demonstrate that CE correlates strongly with the Borg CR10 scale of perceived exertion. We show how designers can use CE as a complementary metric for evaluating existing and designing novel mid-air interactions, including tasks with repetitive input such as mid-air text-entry. Finally, we propose a series of guidelines for the design of fatigue-efficient mid-air interfaces.

Augmenting the mouse with pressure sensitive input

In this paper we investigate the use of a uni-pressure and dual-pressure augmented mouse. With a pressure augmented mouse users can simultaneously control cursor positions as well as multiple levels of discrete selection modes for common desktop application tasks. Two or more independent pressure sensors can be mounted onto several locations on the body of the mouse. To highlight the design potential of a pressure augmented mouse we conducted a multi-part study. In the first part we identified the number of maximum discrete levels controllable with a uni-pressure augmented mouse, the most appropriate locations for installing pressure sensors on the mouse, and the design of new interaction techniques to support selection with pressure-based input. In a follow-up design we introduced an additional sensor and two different types of selection techniques to control a larger number of discrete levels with two pressure sensors. Our results show that users can comfortably control up to 64 modes with a dual-pressure augmented mouse. We discuss the findings of our results in the context of several desktop interaction techniques and identify several design recommendations.

GesText: accelerometer-based gestural text-entry systems

Accelerometers are common on many devices, including those required for text-entry. We investigate how to enter text with devices that are solely enabled with accelerometers. The challenge of text-entry with such devices can be overcome by the careful investigation of the human limitations in gestural movements with accelerometers. Preliminary studies provide insight into two potential text-entry designs that purely use accelerometers for gesture recognition. In two experiments, we evaluate the effectiveness of each of the text-entry designs. The first experiment involves novice users over a 45 minute period while the second investigates the possible performance increases over a four day period. Our results reveal that a matrix-based text-entry system with a small set of simple gestures is the most efficient (5.4wpm) and subjectively preferred by participants.

ARC-Pad: absolute+relative cursor positioning for large displays with a mobile touchscreen

We introduce ARC-Pad (Absolute+Relative Cursor pad), a novel technique for interacting with large displays using a mobile phones touchscreen. In ARC-Pad we combine ab-solute and relative cursor positioning. Tapping with ARC-Pad causes the cursor to jump to the corresponding location on the screen, providing rapid movement across large distances. For fine position control, users can also clutch using relative mode. Unlike prior hybrid cursor positioning techniques, ARC-Pad does not require an explicit switch between relative and absolute modes. We compared ARC-Pad with the relative positioning commonly found on touchpads. Users were given a target acquisition task on a large display, and results showed that they were faster with ARC-Pad, without sacrificing accuracy. Users welcomed the benefits associated with ARC-Pad.

Improving selection of off-screen targets with hopping

Many systems provide the user with a limited viewport of a larger graphical workspace. In these systems, the user often needs to find and select targets that are in the workspace, but not visible in the current view. Standard methods for navigating to the off-screen targets include scrolling, panning, and zooming; however, these are laborious when users cannot see a targets direction or distance. Techniques such as halos can provide awareness of targets, but actually getting to the target is still slow with standard navigation. To improve off-screen target selection, we developed a new technique called hop, which combines halos with a teleportation mechanism that shows proxies of distant objects. Hop provides both awareness of off-screen targets and fast navigation to the target context. A study showed that users are significantly faster at selecting off-screen targets with hopping than with two-level zooming or grab-and-drag panning, and it is clear that hop will be faster than either halos or proxy-based techniques (like drag-and-pop or vacuum filtering) by themselves. Hop both improves on halo-based navigation and extends the value of proxies to small-screen environments.

Putting your best foot forward: investigating real-world mappings for foot-based gestures

Foot-based gestures have recently received attention as an alternative interaction mechanism in situations where the hands are pre-occupied or unavailable. This paper investigates suitable real-world mappings of foot gestures to invoke commands and interact with virtual workspaces. Our first study identified user preferences for mapping common mobile-device commands to gestures. We distinguish these gestures in terms of discrete and continuous command input. While discrete foot-based input has relatively few parameters to control, continuous input requires careful design considerations on how the users input can be mapped to a control parameter (e.g. the volume knob of the media player). We investigate this issue further through three user-studies. Our results show that rate-based techniques are significantly faster, more accurate and result if far fewer target crossings compared to displacement-based interaction. We discuss these findings and identify design recommendations.

Diagramming information structures using 3D perceptual primitives

The class of diagrams known collectively as node-link diagrams are used extensively for many applications, including planning, communications networks, and computer software. The defining features of these diagrams are nodes, represented by a circle or rectangle connected by links usually represented by some form of line or arrow. We investigate the proposition that drawing three-dimensional shaded elements instead of using simple lines and outlines will result in diagrams that are easier to interpret. A set of guidelines for such diagrams is derived from perception theory and these collectively define the concept of the geon diagram. We also introduce a new substructure identification task for evaluating diagrams and use it to test the effectiveness of geon diagrams. The results from five experiments are reported. In the first three experiments geon diagrams are compared to Unified Modeling Language (UML) diagrams. The results show that substructures can be identified in geon diagrams with approximately half the errors and significantly faster. The results also show that geon diagrams can be recalled much more reliably than structurally equivalent UML diagrams. In the final two experiments geon diagrams are compared with diagrams having the same outline but not constructed with shaded solids. This is designed to specifically test the importance of using 3D shaded primitives. The results also show that substructures can be identified much more accurately with shaded components than with 2D outline equivalents and remembered more reliably. Implications for the design of diagrams are discussed.