David Gerhard

Creating synergy between usability courses and open source software projects

In this paper, we discuss our experience in offering a usability course with projects taken from an active open source software development project. We describe what was done in the class inside the larger context of the usability of open source software. We conclude with an invitation for others to adopt this model and use it for their own purposes.

Perceptual features for a fuzzy speech-song classification

Human speech and song seem disparate, but a range of utterances between speech and song are evident, such as poetry, chant, and rap, which have features of both singing and speaking. This work seeks to identify and characterize the perceptual features relevant for a fuzzy classification of utterances between speech and singing. The speech-ness or song-ness of an utterance depends on the speech or song features evident in that utterance. This paper presents a brief discussion of the collection and annotation of the corpus of sound clips used in this work, followed by a description of the perceptual features expected to be useful, and presentation of preliminary results for two of these features.

A human vocal utterance corpus for perceptual and acoustic analysis of speech, singing, and intermediate vocalizations

In this paper we present the collection and annotation process of a corpus of human utterance vocalizations used for speech and song research. The corpus was collected to fill a void in current research tools, since no corpus currently exists which is useful for the classification of intermediate utterances between speech and monophonic singing. Much work has been done in the domain of speech versus music discrimination, and several corpora exist which can be used for this research. A specific example is the work done by Eric Scheirer and Malcom Slaney [IEEE ICASSP, 1997, pp. 1331–1334]. The collection of the corpus is described including questionnaire design and intended and actual response characteristics, as well as the collection and annotation of pre‐existing samples. The annotation of the corpus consisted of a survey tool for a subset of the corpus samples, including ratings of the clips based on a speech–song continuum, and questions on the perceptual qualities of speech and song, both generally an...

Collaborative computer-aided parameter exploration for music and animation

Although many artists have worked to create associations between music and animation, this has traditionally be done by developing one to suit the pre-existing other, as in visualization or sonification. The approach we employ in this work is to enable the simultaneous development of both music and sound from a common and rather generic central parameter variation, which may simply indicate a structure for periodic repetitions. This central parameter variation is then simultaneously mapped to appropriate musical and graphical variables by the musician and the animator, thereby contributing their own interpretations. The result of this mapping is then rendered in an intermediate form where music and animation are allowed to iteratively influence each other. The main piece of software in this development is the system which allows exploration of parameter mappings. The software interface allows both musician and animator to meaningfully experiment with the others mappings since the interface permits access in a common form, without requiring additional skills to interpret.

Stride length determination during overground running using a single foot-mounted inertial measurement unit

From a research perspective, detailed knowledge about stride length (SL) is important for coaches, clinicians and researchers because together with stride rate it determines the speed of locomotion. Moreover, individual SL vectors represent the integrated output of different biomechanical determinants and as such provide valuable insight into the control of running gait. In recent years, several studies have tried to estimate SL using body-mounted inertial measurement units (IMUs) and have reported promising results. However, many studies have used systems based on multiple sensors or have only focused on estimating SL for walking. Here we test the concurrent validity of a single foot-mounted, 9-degree of freedom IMU to estimate SL for running. We employed a running-specific, Kalman filter based zero-velocity update (ZUPT) algorithm to calculate individual SL vectors with the IMU and compared the results to SLs that were simultaneously recorded by a 6-camera 3D motion capture system. The results showed that the analytical procedures were able to successfully identify all strides that were recorded by the camera system and that excellent levels of absolute agreement (ICC(3,1) = 0.955) existed between the two methods. The findings demonstrate that individual SL vectors can be accurately estimated with a single foot-mounted IMU when running in a controlled laboratory setting.

Waveform-Aligned Adaptive Windows for Spectral Component Tracking and Noise Rejection

A new Short-Time Fourier Transform (STFT) pipeline is presented which uses a two-pass method to adapt the size of the analysis window to the period of the signal, which is assumed in this case to be at least pseudo-periodic. The pipeline begins with pitch estimation, followed by upsampling, to construct a new analysis window that matches the length of a single period. This reduces or eliminates the spectral leakage problems which are typical of traditional STFT analysis techniques. The result is a discrete and accurate spectral representation that provides highly accurate location of partials from one analysis frame to the next. We have extended this method to allow noise cancellation by selecting an analysis window that contains a small whole number of complete cycles. We also present a new display method based on this pipeline which greatly improves the spectrogram through enhanced distinction among partials. Finally, validation is performed by signal restoration on 40 clips, showing the superiority of the pipeline for true periodic signals and comparability for pseudo-periodic signals.

Mixed Reality MIDI Keyboard Demonstration

The Mixed Reality MIDI Keyboard is a prototype designed to augment virtual reality experiences through the inclusion of a physical interface which aligns the users senses with the virtual environment. It also serves as a platform on which the uses of virtual reality in music interaction and art installations can be experimented with. The main problem is that of synchronizing the real and virtual environments in a convincing way that makes the user feel more connected to the experience. To accomplish this a system of devices including an HTC Vive, Leap Motion hand tracker, and MIDI Keyboard are used together to produce a convincing mixed reality instrument that aligns with the users visual, tactile and proprioceptive senses. The system is being developed as both a mixed reality musical instrument for use with common digital audio workstations, and as an installation piece which allows users to explore the nature of perception which this virtual reality system itself takes advantage of.

Stance Phase Detection for Walking and Running Using an IMU Periodicity-based Approach

This paper presents a novel stance phase detection procedure based on observations from a foot-mounted inertial measurement unit (IMU). A frequency-tracking algorithm from the field of audio analysis was applied to the inertial signal to obtain information about gait cycle duration. Afterwards, this information was used to determine the stance phase point for the next gait cycle. This periodicity-based stance-phase detection procedure was found to be superior to traditional threshold based algorithms, significantly reducing the number of insertion and deletion errors, with less dependence on selected threshold values.

Information Rate for Fast Time-Domain Instrument Classification

In this paper, we propose a novel feature set for instrument classification which is based on the information rate of the signal in the time domain. The feature is extracted by calculating the Shannon entropy over a sliding short-time energy frame and binning statistical features into a unique feature vector. Experimental results are presented, including a comparison to frequency-domain feature sets. The proposed entropy features are shown to be faster than popular frequency-domain methods while maintaining comparable accuracy in an instrument classification task.

Discrete Isomorphic Completeness And A Unified Isomorphic Layout Format

Abstract: An isomorphic layout can be used to position pitches on a grid of hexagons. This has many beneficial musical properties such as consistent fingering and spatial harmonic consistency. A Unified Isomorphic Layout (UIL) format is presented in order to create a common specification for describing hexagonal isomorphic layouts. The UIL format provides an unambiguous description of relative pitch orientations and is easily visualized. The notion of complete and degenerate isomorphic layouts (along with a proof) is introduced to narrow down the number of valid isomorphic layouts used for exhaustive evaluations.