Johannes Luderschmidt

Detecting interaction above digital tabletops using a single depth camera

Digital tabletop environments offer a huge potential to realize application scenarios where multiple users interact simultaneously or aim to solve collaborative tasks. So far, research in this field focuses on touch and tangible interaction, which only takes place on the tabletop’s surface. First approaches aim at involving the space above the surface, e.g., by employing freehand gestures. However, these are either limited to specific scenarios or employ obtrusive tracking solutions. In this paper, we propose an approach to unobtrusively segment and detect interaction above a digital surface using a depth sensing camera. To achieve this, we adapt a previously presented approach that segments arms in depth data from a front-view to a top-view setup facilitating the detection of hand positions. Moreover, we propose a novel algorithm to merge segments and give a comparison to the original segmentation algorithm. Since the algorithm involves a large number of parameters, estimating the optimal configuration is necessary. To accomplish this, we describe a low effort approach to estimate the parameter configuration based on simulated annealing. An evaluation of our system to detect hands shows that a repositioning precision of approximately 1 cm is achieved. This accuracy is sufficient to reliably realize interaction metaphors above a surface.

Authoring Support for Post-WIMP Applications

Employing post-WIMP interfaces, i.e. user interfaces going beyond the traditional WIMP (Windows, Icons, Menu, Pointer) paradigm, often implies a more complex authoring process for applications. We present a novel authoring method and a corresponding tool that aims to enable developers to cope with the added level of complexity. Regarding the development as a process conducted on different layers, we introduce a specific layer for post-WIMP in addition to layers addressing implementation or traditional GUI elements. We discuss the concept of cross layer authoring that supports different author groups in the collaborative creation of post-WIMP applications permitting them working independently on their respective layer and contributing their specific skills. The concept comprises interactive visualization techniques that highlight connections between code, GUI and post-WIMP functionality. It allows for graphical inspection while transitioning smoothly between layers. A cross layer authoring tool has been implemented and was well received by UI developers during evaluation.

EMIL: a rapid prototyping authoring environment for the design of interactive surface applications

Interactive surfaces (IS) like digital tabletop systems offer a cornucopia of input possibilities like touch gestures or interaction with physical objects. Additionally, multiple users can interact simultaneously allowing for a collaborative setting. These aspects have increased the complexity of designing such interfaces as compared to WIMP interfaces. However, existing UI design approaches fall short of taking these aspects into account and existing design approaches for IS focus on software development. We introduce the EMIL environment that allows authors of design teams to create multi-touch and tangible user interfaces. In its core, EMIL consists of a software framework that provides interaction components (for instance, widgets like images or maps as well as interaction concepts like gestures) that are especially suited for IS. Authors like UI designers collaboratively create software prototypes directly at the IS without the need to write code. For this purpose, they use and adapt the components of the software framework in an authoring application. Authors collect and retrieve information about the interaction components in a knowledge database employing a tablet computer app. In a qualitative evaluation interview, EMIL has been well received by a design team of an advertising agency.