Matthias Kranz

Embedded Interaction: Interacting with the Internet of Things

The Internet of Things assumes that objects have digital functionality and can be identified and tracked automatically. The main goal of embedded interaction is to look at new opportunities that arise for interactive systems and the immediate value users gain. The authors developed various prototypes to explore novel ways for human-computer interaction (HCI), enabled by the Internet of Things and related technologies. Based on these experiences, they derive a set of guidelines for embedding interfaces into peoples daily lives.

The mobile fitness coach: Towards individualized skill assessment using personalized mobile devices

We report on our extended research on GymSkill, a smartphone system for comprehensive physical exercising support, from sensor data logging, activity recognition to on-top skill assessment, using the phones built-in sensors. In two iterations, we used principal component breakdown analysis (PCBA) and criteria-based scores for individualized and personalized automated feedback on the phone, with the goal to track training quality and success and give feedback to the user, as well as to engage and motivate regular exercising. Qualitative feedback on the system was collected in a user study, and the system showed good evaluation results in an evaluation against manual expert assessments of video-recorded trainings.

A cube to learn: a tangible user interface for the design of a learning appliance

In this paper we introduce the design and development of the Learning Cube as a novel tangible learning appliance. Using the common shape of a cube we implemented a general learning platform that supports test based quizzes where questions and answers can be text or image based. Exploiting the physical affordances of the cube and augmenting it with embedded sensors and LCD displays placed on each face, we present different learning appliances as playful learning interfaces for children. Based on the initial observations of the experience with children, we argue that breaking conventions about how a computer has to look like, and providing children with a playful interface is a promising approach to embed and integrate technology into children’s everyday context and activities.

A middleware for intelligent environments and the internet of things

Interdisciplinary research from the domains of pervasive computing or ubiquitous computing, computer-human-interaction and computer science has led to the development of many intelligent environments, either on lab scale or as live in laboratories. While several middleware have been developed in this field, no standard middleware for intelligent environments or ubiquitous computing has evolved yet. We consider the lack of a de-facto standard middleware for distributed sensor-actuator environments as one of the key issues limiting research on intelligent environment and the proliferation of intelligent environments from research environments to their deployment in our everyday lives. In addition, we expect the advent of personal robotics for health care and ambient assisted living scenarios in the context of ubiquitous computing in the close future. In this paper, we report on the successful application of a robotic middleware as glue between sensors, actuators and services and its application in a deployed example scenario. Thereby, we verify by examples the applicability of robotic middleware for complex ubiquitous computing environments. To foster re-use and potential community-adoption, we share our source code, documentation and data sets (in the future) via https://vmi.lmt.ei.tum.de/ros/.

Interacting with the ubiquitous computer: towards embedding interaction

Computing and communication technology is widely used and integrated in devices, environments, and everyday objects. Even with major advances in technology the vision of ubiquitous computing - from a user perspective - is not yet achieved. In this paper we look at new forms of interaction that will help to interact with the ubiquitous computer. In particular we introduce the concept of embedded interaction, and implicit use. The focus of the research is on embedding information into peoples environments. Currently massive amounts of information are available. However, delivering it to the user in a way that is pleasant and not annoying is still a challenge. Observing mobile phone information push services, it appears that endless information is available; however, much of the information is interesting only in a very specific context of use. We investigate how information can be provided to users - exactly when and where it is needed. Our approach is based on a variety of information displays unobtrusively embedded into the users everyday environment. We place the information displays in context. In contrast to the traditional approach on context-awareness where a context is recognized and then the appropriate information is delivered, we look at providing information already in context. It is up to the user to make use of the provided information or not.

A Capacitive Sensing Toolkit for Pervasive Activity Detection and Recognition

In this paper we present a toolkit for realizing capacitive sensing applications for human-computer interaction in pervasive computing systems. We argue that capacitive sensors - due to their unique properties - are well suited for many pervasive and ubiquitous computing applications and scenarios. We describe the CapToolKit designed to rapidly realize prototypes and systems that are able to detect the presence of humans and objects. Our toolkit also allows the integration of 3D interaction with everyday objects as well as instrumented environments. We illustrate its capabilities by presenting several applications implemented using CapToolKit. The entire system will be open-sourced to allow utilization of our technology within other research projects. By building on the existing toolkit researchers are provided a foundation for developing their own sensor systems, algorithms, and applications

Sensing Technologies and the Player-Middleware for Context-Awareness in Kitchen Environments

In this paper, we report on a use case of networked sensing technologies in the context of smart homes and specifically the kitchen as scenario for our research. We adapt, use and extend an existing middleware originating from robotics for pervasive computing. We report on initial results towards context recognition in this sensor enriched environment.

Context-aware kitchen utilities

We report on approaches for context-awareness in a kitchen environment. Two devices, an augmented cutting board and a sensor-enriched knife, enable the environment to determine the type of food handled during the preparation of meals.

A mobile indoor navigation system interface adapted to vision-based localization

Vision-based approaches for mobile indoor localization do not rely on the infrastructure and are therefore scalable and cheap. The particular requirements to a navigation user interface for a vision-based system, however, have not been investigated so far. Such mobile interfaces should adapt to localization accuracy, which strongly relies on distinctive reference images, and other factors, such as the phones pose. If necessary, the system should motivate the user to point at distinctive regions with the smartphone to improve localization quality. We present a combined interface of Virtual Reality (VR) and Augmented Reality (AR) elements with indicators that help to communicate and ensure localization accuracy. In an evaluation with 81 participants, we found that AR was preferred in case of reliable localization, but with VR, navigation instructions were perceived more accurate in case of localization and orientation errors. The additional indicators showed a potential for making users choose distinctive reference images for reliable localization.

Thracker - Using Capacitive Sensing for Gesture Recognition

In this paper we present Thracker - a low-cost and robust hardware to track hand gestures in front of a screen or small-scale active spaces like public displays or posters. Thracker uses capacitive sensing for tracking user input. Thracker allows for entire new interaction modes like picking and dropping an object on the screen with the hand. We present a fully working prototype and a short user study to confirm our findings.