Eugene M. Taranta

Jackknife: A Reliable Recognizer with Few Samples and Many Modalities

Despite decades of research, there is yet no general rapid prototyping recognizer for dynamic gestures that can be trained with few samples, work with continuous data, and achieve high accuracy that is also modality-agnostic. To begin to solve this problem, we describe a small suite of accessible techniques that we collectively refer to as the Jackknife gesture recognizer. Our dynamic time warping based approach for both segmented and continuous data is designed to be a robust, go-to method for gesture recognition across a variety of modalities using only limited training samples. We evaluate pen and touch, Wii Remote, Kinect, Leap Motion, and sound-sensed gesture datasets as well as conduct tests with continuous data. Across all scenarios we show that our approach is able to achieve high accuracy, suggesting that Jackknife is a capable recognizer and good first choice for many endeavors.

Exploring the Benefits of Context in 3D Gesture Recognition for Game-Based Virtual Environments

We present a systematic exploration of how to utilize video game context (e.g., player and environmental state) to modify and augment existing 3D gesture recognizers to improve accuracy for large gesture sets. Specifically, our work develops and evaluates three strategies for incorporating context into 3D gesture recognizers. These strategies include modifying the well-known Rubine linear classifier to handle unsegmented input streams and per-frame retraining using contextual information (CA-Linear); a GPU implementation of dynamic time warping (DTW) that reduces the overhead of traditional DTW by utilizing context to evaluate only relevant time sequences inside of a multithreaded kernel (CA-DTW); and a multiclass SVM with per-class probability estimation that is combined with a contextually based prior probability distribution (CA-SVM). We evaluate each strategy using a Kinect-based third-person perspective VE game prototype that combines parkour-style navigation with hand-to-hand combat. Using a simple gesture collection application to collect a set of 57 gestures and the game prototype that implements 37 of these gestures, we conduct three experiments. In the first experiment, we evaluate the effectiveness of several established classifiers on our gesture set and demonstrate state-of-the-art results using our proposed method. In our second experiment, we generate 500 random scenarios having between 5 and 19 of the 57 gestures in context. We show that the contextually aware classifiers CA-Linear, CA-DTW, and CA-SVM significantly outperform their non--contextually aware counterparts by 37.74%, 36.04%, and 20.81%, respectively. On the basis of the results of the second experiment, we derive upper-bound expectations for in-game performance for the three CA classifiers: 96.61%, 86.79%, and 96.86%, respectively. Finally, our third experiment is an in-game evaluation of the three CA classifiers with and without context. Our results show that through the use of context, we are able to achieve an average in-game recognition accuracy of 89.67% with CA-Linear compared to 65.10% without context, 79.04% for CA-DTW compared to 58.1% without context, and 90.85% with CA-SVM compared to 75.2% without context.

A Rapid Prototyping Approach to Synthetic Data Generation for Improved 2D Gesture Recognition

Training gesture recognizers with synthetic data generated from real gestures is a well known and powerful technique that can significantly improve recognition accuracy. In this paper we introduce a novel technique called gesture path stochastic resampling (GPSR) that is computationally efficient, has minimal coding overhead, and yet despite its simplicity is able to achieve higher accuracy than competitive, state-of-the-art approaches. GPSR generates synthetic samples by lengthening and shortening gesture subpaths within a given sample to produce realistic variations of the input via a process of nonuniform resampling. As such, GPSR is an appropriate rapid prototyping technique where ease of use, understandability, and efficiency are key. Further, through an extensive evaluation, we show that accuracy significantly improves when gesture recognizers are trained with GPSR synthetic samples. In some cases, mean recognition errors are reduced by more than 70%, and in most cases, GPSR outperforms two other evaluated state-of-the-art methods.

A

We explore the benefits of intelligent prototype selection for

-Family Friendly Approach to Prototype Selection

-family recognizers. Currently, the state of the art is to randomly select a subset of prototypes from a dataset without any processing. This results in reduced computation time for the recognizer, but also increases error rates. We propose applying optimization algorithms, specifically random mutation hill climb and a genetic algorithm, to search for reduced sets of prototypes that minimize recognition error. After an evaluation, we found that error rates could be reduced compared to random selection and rapidly approached the baseline accuracies for a number of different

Streamlined and accurate gesture recognition with Penny Pincher

-family recognizers.

Math Boxes: A Pen-Based User Interface for Writing Difficult Mathematical Expressions

Penny Pincher is a recently introduced template matching

Macro 64-regions for uniform grids on GPU

-family gesture recognizer that exhibits competitive accuracy with even just one template. However, our recognizer is also able to rapidly compare a candidate gesture against numerous templates in a short amount of time, as compared to other recognizers, in order to achieve higher accuracy within a given time budget. Penny Pincher achieves this goal by reducing the template matching process to merely addition and multiplication; by avoiding translation, scaling, and rotation; and by avoiding calls to expensive geometric functions. In an evaluation compared against four other

A comparison of eye-head coordination between virtual and physical realities

-family recognizers, in three of our six datasets, Penny Pincher achieves the highest accuracy of all recognizers reaching 97.5%, 99.8%, and 99.9% user independent recognition accuracy, while remaining competitive with the three remaining datasets. Further, when a time constraint is imposed, our recognizer always exhibits the highest accuracy, realizing a reduction in recognition error of between 83% and 99% in most cases as Penny Pincher is able to process five times as many templates in the same amount of time as its closest competitor. Further, in this extended work, we also evaluate the effectiveness of Penny Pincher in a stressful setting using a video game prototype that makes heavy use of gestures, so that rushed and malformed gesture articulation is more likely. Our evaluation was conducted with a 24 participant between-subject user study of Protractor and Penny Pincher. Training data and in-game data collected during the user study was further used to evaluate several

Sketch Based Interaction Techniques for Chart Creation and Manipulation

-family recognizers. Again we find that our recognizer is on par with or better than the others, reducing the recognition error by as much as 5.8% to 10.4% with just a small number of templates per gesture. Graphical abstractDisplay Omitted HighlightsFast and accurate gesture recognition using only dot products.Compared to other methods, ours yields the highest accuracy in a time constraint.Evaluated with six different well-known datasets.Highest accuracy in prototype game user study.