Konrad Tollmar

Searching the Web with mobile images for location recognition

We describe an approach to recognizing location from mobile devices using image-based Web search. We demonstrate the usefulness of common image search metrics applied on images captured with a camera-equipped mobile device to find matching images on the World Wide Web or other general-purpose databases. Searching the entire Web can be computationally overwhelming, so we devise a hybrid image-and-keyword searching technique. First, image-search is performed over images and links to their source Web pages in a database that indexes only a small fraction of the Web. Then, relevant keywords on these Web pages are automatically identified and submitted to an existing text-based search engine (e.g. Google) that indexes a much larger portion of the Web. Finally, the resulting image set is filtered to retain images close to the original query. It is thus possible to efficiently search hundreds of millions of images that are not only textually related but also visually relevant. We demonstrate our approach on an application allowing users to browse Web pages matching the image of a nearby location.

Supporting social awareness @ work design and experience

During the last year we have been designing and studying a computer based tool intended to strengthen social group awareness within a research laboratory. While awareness has been a subject of previous research it is still unclear how it should be conceptualized and how it can be provided for a CSCW system. In order to investigate this, and hence to attempt to create a system that would gain acceptance in the user community, we have been using a mixture of user-centered and participatory design methods. This paper presents the design process, the resulting system as well as users’ comments on it. Based on all this, issues related to awareness are discussed and ideas for further studies are suggested.

Health Mashups: Presenting Statistical Patterns between Wellbeing Data and Context in Natural Language to Promote Behavior Change

People now have access to many sources of data about their health and wellbeing. Yet, most people cannot wade through all of this data to answer basic questions about their long-term wellbeing: Do I gain weight when I have busy days? Do I walk more when I work in the city? Do I sleep better on nights after I work out? We built the Health Mashups system to identify connections that are significant over time between weight, sleep, step count, calendar data, location, weather, pain, food intake, and mood. These significant observations are displayed in a mobile application using natural language, for example, “You are happier on days when you sleep more.” We performed a pilot study, made improvements to the system, and then conducted a 90-day trial with 60 diverse participants, learning that interactions between wellbeing and context are highly individual and that our system supported an increased self-understanding that lead to focused behavior changes.

Activity Zones for Context-Aware Computing

Location is a primary cue in many context-aware computing systems, and is often represented as a global coordinate, room number, or a set of Euclidean distances to various landmarks. A user’s concept of location, however, is often defined in terms of regions in which similar activities occur. We discuss the concept of such regions, which we call activity zones, and suggest that such zones can be used to trigger application actions, retrieve information based on previous context, and present information to users. We show how to semi- automatically partition a space into activity zones based on patterns of observed user location and motion. We describe our system and two implemented example applications whose behavior is controlled by users’ entry, exit, and presence in the zones.

Virtually living together

In this paper, we discuss the possibility of a holistic approach in the design of new media for interpersonal communication. The key argument is that if we base our design on daily practice, this may inhibit truly innovative ideas from taking form, and, on the contrary, if we design using pure intuition and visions, the design is likely to fail due to a lack of connection to daily practice. Scenario-based design was hence used to makes us envision new media while field observations such as ethnographic studies, become a tool to retain the ties with everyday life. This duality can also be implemented, as we will describe, in a design that makes a bi-language/bi-levelled understanding of a product possible.

A picture is worth a thousand keywords: image-based object search on a mobile platform

Finding information based on an objects visual appearance is useful when specific keywords for the object are not known. We have developed a mobile image-based search system that takes images of objects as queries and finds relevant web pages by matching them to similar images on the web. Image-based search works well when matching full scenes, such as images of buildings or landmarks, and for matching objects when the boundary of the object in the image is available. We demonstrate the effectiveness of a simple interactive paradigm for obtaining a segmented object boundary, and show how a shape-based image matching algorithm can use the object outline to find similar images on the web.

The power of mobile notifications to increase wellbeing logging behavior

Self-logging is a critical component to many wellbeing systems. However, self-logging often is difficult to sustain at regular intervals over many weeks. We demonstrate the power of passive mobile notifications to increase logging of wellbeing data, particularly food intake, in a mobile health service. Adding notifications increased the frequency of logging from 12% in a one-month, ten-user pilot study without reminders to 63% in the full 60-user study with reminders included. We will discuss the benefits of passive notifications over existing interruptive methods.

Communiplay: a field study of a public display mediaspace

We present Communiplay, a public display media space. People passing by see their own contour mirrored on a public display and can start to play with virtual objects. At the same time, they see others playing at remote displays within the same virtual space. We are interested whether people would use such a public display media space, and if so, how and why. We evaluate Communiplay in a field study in six connected locations and find a remote honey-pot effect, i.e. people interacting at one location attract people at other locations. The conversion rate (percentage of passers-by starting to interact) rose by +136% when people saw others playing at remote locations. We also provide the first quantification of the (local) honey-pot effect (in our case it raised the conversion rate by +604% when people saw others playing at the same location). We conclude that the integration of multiple public displays into a media space is a promising direction for public displays and can make them more attractive and valuable.

Private and public digital domestic spaces

With the introduction of information and communication technologies into our homes and the different physical and communicative expressions this implies for our living spaces the concepts of being private and of beingpublic become crucial. In this paper, we introduce A Pattern Language, developed by Christopher Alexander in the 1970s, in order to handle these problems systematically. The presentation formally follows Alexanders structure in five cases all related to practical experiments on being private and public at home. We start with a number of concrete user situations related to human?computer interaction. Social and communicative phenomena or possibilities end up in novel design patterns at the interface between an architectural and a technological perspective. The novel patterns presented are primarily based on experiences from practical work on the development of a conceptual dwelling of the future, comHOME, designed and constructed as a full-scale model of a flat. By creating different zones for video-mediated communication, comZONES, the user can control the private and public digital areas varying in time and space. The novel patterns refer to two separate levels. On the first level a specific pattern, called “PRIVATE AND PUBLIC DIGITAL SPACES”, is designed as a conceptual floor plan layout. This plan distributes private and public digital spaces for video-mediated communication over the flat. At a second level, four patterns show the integration of the specific comZONES aiming at solving four specific problems with video-mediated communication at home. Our intention is to describe the application of design patterns as a method for analysing and solving novel problems encountered with the introduction of information and communication technologies in our homes. The video-mediated set-ups are not dealt with in depth. They serve mainly as designs that make it possible to apply the design patterns.

IDeixis: image-based Deixis for finding location-based information

We demonstrate an image-based approach to specifying location and finding location-based information from camera-equipped mobile devices. We introduce a point-by-photograph paradigm, where users can specify a location simply by taking pictures. Our technique uses content-based image retrieval methods to search the web or other databases for matching images and their source pages to find relevant location-based information. In contrast to conventional approaches to location detection, our method can refer to distant locations and does not require any physical infrastructure beyond mobile internet service. We have developed a prototype on a camera phone and conducted user studies to demonstrate the efficacy of our approach compared to other alternatives.