Christopher J. Tralie

(Quasi)Periodicity Quantification in Video Data, Using Topology

This work introduces a novel framework for quantifying the presence and strength of recurrent dynamics in video data. Specifically, we provide continuous measures of periodicity (perfect repetition) and quasiperiodicity (superposition of periodic modes with non-commensurate periods), in a way which does not require segmentation, training, object tracking or 1-dimensional surrogate signals. Our methodology operates directly on video data. The approach combines ideas from nonlinear time series analysis (delay embeddings) and computational topology (persistent homology), by translating the problem of finding recurrent dynamics in video data, into the problem of determining the circularity or toroidality of an associated geometric space. Through extensive testing, we show the robustness of our scores with respect to several noise models/levels, we show that our periodicity score is superior to other methods when compared to human-generated periodicity rankings, and furthermore, we show that our quasiperiodicity score clearly indicates the presence of biphonation in videos of vibrating vocal folds, which has never before been accomplished end to end quantitatively.

High-Dimensional Geometry of Sliding Window Embeddings of Periodic Videos

We explore the high dimensional geometry of sliding windows of periodic videos. Under a reasonable model for periodic videos, we show that the sliding window is necessary to disambiguate all states within a period, and we show that a video embedding with a sliding window of an appropriate dimension lies on a topological loop along a hypertorus. This hypertorus has an independent ellipse for each harmonic of the motion. Natural motions with sharp transitions from foreground to background have many harmonics and are hence in higher dimensions, so linear subspace projections such as PCA do not accurately summarize the geometry of these videos. Noting this, we invoke tools from topological data analysis and cohomology to parameterize motions in high dimensions with circular coordinates after the embeddings. We show applications to videos in which there is obvious periodic motion and to videos in which the motion is hidden.

Scaffoldings and Spines: Organizing High-Dimensional Data Using Cover Trees, Local Principal Component Analysis, and Persistent Homology

We propose a flexible and multi-scale method for organizing, visualizing, and understanding point cloud datasets sampled from or near stratified spaces. The first part of the algorithm produces a cover tree for a dataset using an adaptive threshold that is based on multi-scale local principal component analysis. The resulting cover tree nodes reflect the local geometry of the space and are organized via a scaffolding graph. In the second part of the algorithm, the goals are to uncover the strata that make up the underlying stratified space using a local dimension estimation procedure and topological data analysis, as well as to ultimately visualize the results in a simplified spine graph. We demonstrate our technique on several synthetic examples and then use it to visualize song structure in musical audio data.

In-hand radio frequency identification (RFID) for robotic manipulation

We present a unique multi-antenna RFID reader (a sensor) embedded in a robots manipulator that is designed to operate with ordinary UHF RFID tags in a short-range, near-field electromagnetic regime. Using specially designed near-field antennas enables our sensor to obtain spatial information from tags at ranges of less than 1 meter. In this work, we characterize the near-field sensors ability to detect tagged objects in the robots manipulator, present robot behaviors to determine the identity of a grasped object, and investigate how additional RF signal properties can be used for “pre-touch” capabilities such as servoing to grasp an object. The future combination of long-range (far-field) and short-range (near-field) UHF RFID sensing has the potential to enable roboticists to jump-start applications by obviating or supplementing false-positive-prone visual object recognition. These techniques may be especially useful in the healthcare and service sectors, where mis-identification of an object (for example, a medication bottle) could have catastrophic consequences.

Geometric Models for Musical Audio Data

We study the geometry of sliding window embeddings of audio features that summarize perceptual information about audio, including its pitch and timbre. These embeddings can be viewed as point clouds in high dimensions, and we add structure to the point clouds using a cover tree with adaptive thresholds based on multi-scale local principal component analysis to automatically assign points to clusters. We connect neighboring clusters in a scaffolding graph, and we use knowledge of stratified space structure to refine our estimates of dimension in each cluster, demonstrating in our music applications that choruses and verses have higher dimensional structure, while transitions between them are lower dimensional. We showcase our technique with an interactive web-based application powered by Javascript and WebGL which plays music synchronized with a principal component analysis embedding of the point cloud down to 3D. We also render the clusters and the scaffolding on top of this projection to visualize the transitions between different sections of the music.