Daniel Kohlsdorf

MAGIC 2.0: A web tool for false positive prediction and prevention for gesture recognition systems

False positives are a common problem for interfaces that rely on gesture recognition. Often a gesture can seem fine in development but is found to trigger accidentally during an initial deployment of the interface, restarting development and increasing expense. In this work we introduce MAGIC 2.0, a technique for false positive prediction and prevention that can be used interactively during the interface design process. To ground our research, we implement MAGIC 2.0 as a web service and develop gesture interfaces using sensors on common Android mobile phone platforms. We use iSAX (indexable Symbolic Aggregate approXimation) to enable interactive searching (<2 sec/example) of a large database (>1,500,000 sec) of everyday user movements on a standard workstation to determine if a candidate gesture will trigger accidentally during use of an interface. We perform a user-independent study that suggests that the number of matches to this Everyday Gesture Library (EGL) database is indeed predictive of a candidate gestures suitability. We compare iSAX to hidden Markov models (HMMs) and nearest neighbor with respect to accuracy and speed for the EGL search. Using iSAX on the EGL, we also develop a “garbage” class and show that including this class in recognition reduces errors.

Mobile Music Touch: The effect of primary tasks on passively learning piano sequences

The Mobile Music Touch (MMT) system allows users to learn to reproduce piano note sequences while performing other tasks. The system consists of a mobile Bluetooth-enabled computing device and a fingerless glove with embedded vibrators corresponding to each finger and thumb. Melodies to be learned are played over the users headphones repeatedly. As each note is played, the finger corresponding to the appropriate piano key is stimulated. Past experiments have shown that users could learn simple note sequences even though they were performing a reading comprehension test. Here, we investigate different primary tasks to determine which, if any, interfere with the Passive Haptic Learning (PHL) effect. In a 12 participant within-subject user study, no overall difference was observed in the number of passive sessions required to learn a random note sequence when users viewed a film, played a memory game, or followed a walking path as their primary task. However, individual differences in scores suggest that the type of primary task may have a greater or lesser effect for a given user.

Learning action failure models from interactive physics-based simulations

Predicting the outcome of an action can help a robot detect failures in advance, and schedule action replanning before an error occurs. We propose using an interactive physics based simulator with the aim of collecting realistic data to be used for learning. We then show how we save and query for specific information from the data more effectively. The data from the simulation is used to learn a failure detection model which is utilized by a real robot performing the same actions. We show that learning from simulation data is realistic enough to be applied on a real robot. The learning algorithm is more simple in design and outperforms the more complex one from our previous work.

Learning Task Outcome Prediction for Robot Control from Interactive Environments

In order to manage complex tasks such as cooking, future robots need to be action-aware and posses common sense knowledge. For example flipping a pancake requires a robot to know that a spatula has to be under a pancake in order to succeed. We present a novel approach for the extraction and learning of action and common sense knowledge, and developed a game using a robot-simulator with realistic physics for data acquisition. The game environment is a virtual kitchen, in which a user has to create a pancake by pouring pancake-mix on an oven and flipping it using a spatula. The interaction is done by controlling a virtual robot hand with a 3D input sensor. We incorporate a realistic fluid simulation in order to gather appropriate data of the pouring action. Furthermore, we present a task outcome prediction algorithm for this specific system and show how to learn a failure model for the pouring and flipping action.

Probabilistic extraction and discovery of fundamental units in dolphin whistles

The study of dolphin cognition involves intensive research of animal vocalizations. Marine mammalogists commonly study a specific sound type known as the whistle found in dolphin communication. However, one of the main problems arises from noisy underwater environments. Often waves and splash noises will partially distort the whistle making analysis or extraction difficult. Another problem is discovering fundamental units that allow research of the composition of whistles. We propose a method for whistle extraction from noisy underwater recordings using a probabilistic approach. Furthermore, we investigate discovery algorithms for fundamental units using a mixture of hidden Markov models. We evaluate our findings with a marine mammalogist on data collected in the field. Furthermore, we have evidence that our algorithms enable researchers to form hypotheses about the composition of whistles.

IT-ASSIST: Towards Usable Applications for Elderly People

IT-ASSIST is a twenty months research project which has the goal to give elderly people the opportunity to profit from digital media. Suffering from age related impairments concerning vision, hearing, or dexterity and bad hand-eye coordination are challenges when designing user interfaces for elderly people. Common approaches are trying to model systems for specific impairments. In this project, the authors follow the approach to set up interfaces and systems that can be used independent from personal impairments. Customization has adapted these systems to be in accordance with personnel impairments. Common applications like photo editing, digital mailing or internet browsing in a redesigned form provide social communication accordingly. In this article, a prototype of a customized user interface, its implementation, and results of user studies are presented and discussed.

Feature Learning and Automatic Segmentation for Dolphin Communication Analysis.

The study of dolphin cognition involves intensive research of animal vocalizations recorded in the field. We address the automated analysis of audible dolphin communication and propose a system that automatically discovers patterns in dolphin signals. These patterns are invariant to frequency shifts and time warping transformations. The discovery algorithm is based on feature learning and unsupervised time series segmentation using hidden Markov models. Researchers can inspect the patterns visually and interactively run comparative statistics between the distribution of dolphin signals in different behavioral contexts. Our results indicate that our system provides meaningful patterns to the marine biologist and that the comparative statistics are aligned with the biologists domain knowledge.

A Mobile Application for Survey Reports: An Evaluation

In manufacturing processes damages occur caused by humans or machines. These damages have to be reported and documented, e.g. to enable a manufacturer to react in quality circles. The first part of this paper describes the process of creating survey reports. Furthermore a customized solution designed for mobile survey reports is introduced. In the second part this paper describes and discusses the advantages and disadvantages of this mobile solution in an automotive industry setting.