Johannes Schöning

Falling asleep with Angry Birds, Facebook and Kindle: a large scale study on mobile application usage

While applications for mobile devices have become extremely important in the last few years, little public information exists on mobile application usage behavior. We describe a large-scale deployment-based research study that logged detailed application usage information from over 4,100 users of Android-powered mobile devices. We present two types of results from analyzing this data: basic descriptive statistics and contextual descriptive statistics. In the case of the former, we find that the average session with an application lasts less than a minute, even though users spend almost an hour a day using their phones. Our contextual findings include those related to time of day and location. For instance, we show that news applications are most popular in the morning and games are at night, but communication applications dominate through most of the day. We also find that despite the variety of apps available, communication applications are almost always the first used upon a devices waking from sleep. In addition, we discuss the notion of a virtual application sensor, which we used to collect the data.

Map torchlight: a mobile augmented reality camera projector unit

The advantages of paper-based maps have been utilized in the field of mobile Augmented Reality (AR) in the last few years. Traditional paper-based maps provide high-resolution, large-scale information with zero power consumption. There are numerous implementations of magic lens interfaces that combine high-resolution paper maps with dynamic handheld displays. From an HCI perspective, the main challenge of magic lens interfaces is that users have to switch their attention between the magic lens and the information in the background. In this paper, we attempt to overcome this problem by using a lightweight mobile camera projector unit to augment the paper map directly with additional information. The Map Torchlight is tracked over a paper map and can precisely highlight points of interest, streets, and areas to give directions or other guidance for interacting with the map.

Multi-user interaction on media facades through live video on mobile devices

The increasing number of media facades in urban spaces offers great potential for new forms of interaction especially for collaborative multi-user scenarios. In this paper, we present a way to directly interact with them through live video on mobile devices. We extend the Touch Projector interface to accommodate multiple users by showing individual content on the mobile display that would otherwise clutter the facades canvas or distract other users. To demonstrate our concept, we built two collaborative multi-user applications: (1) painting on the facade and (2) solving a 15-puzzle. We gathered informal feedback during the ARS Electronica Festival in Linz, Austria and found that our interaction technique is (1) considered easy-to-learn, but (2) may leave users unaware of the actions of others.

How to nudge in Situ: designing lambent devices to deliver salient information in supermarkets

There are a number of mobile shopping aids and recommender systems available, but none can be easily used for a weekly shop at a local supermarket. We present a minimal, mobile and fully functional lambent display that clips onto any shopping trolley handle, intended to nudge people when choosing what to buy. It provides salient information about the food miles for various scanned food items represented by varying lengths of lit LEDs on the handle and a changing emoticon comparing the average miles of all the products in the trolley against a social norm. When evaluated in situ, the lambent handle display nudged people to choose products with fewer food miles than the items they selected using their ordinary shopping strategies. People also felt guilty when the average mileage of the contents of their entire shopping trolley was above the social norm. The findings are discussed in terms of how to provide different kinds of product information that people care about, using simple lambent displays.

Using hands and feet to navigate and manipulate spatial data

We demonstrate how multi-touch hand gestures in combination with foot gestures can be used to perform navigation tasks in interactive systems. The geospatial domain is an interesting example to show the advantages of the combination of both modalities because the complex user interfaces of common Geographic Information System (GIS) requires a high degree of expertise from its users. Recent developments in interactive surfaces that enable the construction of low cost multi-touch displays and relatively cheap sensor technology to detect foot gestures allow the deep exploration of these input modalities for GIS users with medium or low expertise. In this paper, we provide a categorization of multitouch hand and foot gestures for the interaction with spatial data on a large-scale interactive wall. In addition we show with an initial evaluation how these gestures can improve the overall interaction with spatial information.

The Geography of Pokémon GO: Beneficial and Problematic Effects on Places and Movement

The widespread popularity of Pokémon GO presents the first opportunity to observe the geographic effects of location-based gaming at scale. This paper reports the results of a mixed methods study of the geography of Pokémon GO that includes a five-country field survey of 375 Pokémon GO players and a large scale geostatistical analysis of game elements. Focusing on the key geographic themes of places and movement, we find that the design of Pokémon GO reinforces existing geographically-linked biases (e.g. the game advantages urban areas and neighborhoods with smaller minority populations), that Pokémon GO may have instigated a relatively rare large-scale shift in global human mobility patterns, and that Pokémon GO has geographically-linked safety risks, but not those typically emphasized by the media. Our results point to geographic design implications for future systems in this space such as a means through which the geographic biases present in Pokémon GO may be counteracted.

Explanatory semantic relatedness and explicit spatialization for exploratory search

Exploratory search, in which a user investigates complex concepts, is cumbersome with todays search engines. We present a new exploratory search approach that generates interactive visualizations of query concepts using thematic cartography (e.g. choropleth maps, heat maps). We show how the approach can be applied broadly across both geographic and non-geographic contexts through explicit spatialization, a novel method that leverages any figure or diagram -- from a periodic table, to a parliamentary seating chart, to a world map -- as a spatial search environment. We enable this capability by introducing explanatory semantic relatedness measures. These measures extend frequently-used semantic relatedness measures to not only estimate the degree of relatedness between two concepts, but also generate human-readable explanations for their estimates by mining Wikipedias text, hyperlinks, and category structure. We implement our approach in a system called Atlasify, evaluate its key components, and present several use cases.

Evaluating automatically generated location-based stories for tourists

Tourism provides over six percent of the worlds gross domestic product. As a result, there have been many efforts to use technology to improve the tourists experience via mobile tour guide systems. One key bottleneck in such location-based systems is content development; existing systems either provide trivial information at a global scale or present quality narratives but at an extremely local scale. The primary reason for this dichotomy is that, although good narrative content is more educationally effective (and more entertaining) than a stream of simple, disconnected facts, it is time-intensive and expensive to develop. However, the WikEar system uses narrative theory-informed data mining methodologies in an effort to produce high-quality narrative content for any location on Earth. It allows tourists to interact with these narratives using their camera-enabled cell phones and an innovative interface designed around a magic lens and paper map metaphor. In this paper, we describe a first evaluation of these narratives and the WikEar interface, which reported promising, but not conclusive, results. We also present ideas for future work that will use this feedback to improve the narratives.

Bimanual Interaction with Interscopic Multi-Touch Surfaces

Multi-touch interaction has received considerable attention in the last few years, in particular for natural two-dimensional (2D) interaction. However, many application areas deal with three-dimensional (3D) data and require intuitive 3D interaction techniques therefore. Indeed, virtual reality (VR) systems provide sophisticated 3D user interface, but then lack efficient 2D interaction, and are therefore rarely adopted by ordinary users or even by experts. Since multi-touch interfaces represent a good trade-off between intuitive, constrained interaction on a touch surface providing tangible feedback, and unrestricted natural interaction without any instrumentation, they have the potential to form the foundation of the next generation user interface for 2D as well as 3D interaction. In particular, stereoscopic display of 3D data provides an additional depth cue, but until now the challenges and limitations for multi-touch interaction in this context have not been considered. In this paper we present new multi-touch paradigms and interactions that combine both traditional 2D interaction and novel 3D interaction on a touch surface to form a new class of multi-touch systems, which we refer to as interscopic multi-touch surfaces (iMUTS). We discuss iMUTS-based user interfaces that support interaction with 2D content displayed in monoscopic mode and 3D content usually displayed stereoscopically. In order to underline the potential of the proposed iMUTS setup, we have developed and evaluated two example interaction metaphors for different domains. First, we present intuitive navigation techniques for virtual 3D city models, and then we describe a natural metaphor for deforming volumetric datasets in a medical context.

Marauders light: replacing the wand with a mobile camera projector unit

Classic paper-based maps provide high-resolution, large-scale information and are ubiquitous in larger cities and even outdoors. In this paper we present a combination of their advantages with a mobile camera projector unit to create a new mobile and intuitive buddy finder interface. Current friend or buddy finder systems on mobile phones such as Google Latitude suffer from the small screen estate and display size of mobile devices. To overcome this problem we use a lightweight mobile camera projector unit to augment the paper map with a projected overlay of the buddies positions. By using digital and geo-referenced representations of public maps no extra preperation for the tracking is needed. The idea is presented by enhancing Google Latitude, which allows users to browse the positions of their friends, to project the positions directly on the paper map without the cumbersome panning and zooming of a digital map on a small display.