Merlijn Blaauw

Statistical Modeling of Bowing Control Applied to Violin Sound Synthesis

Excitation-continuous music instrument control patterns are often not explicitly represented in current sound synthesis techniques when applied to automatic performance. Both physical model-based and sample-based synthesis paradigms would benefit from a flexible and accurate instrument control model, enabling the improvement of naturalness and realism. We present a framework for modeling bowing control parameters in violin performance. Nearly non-intrusive sensing techniques allow for accurate acquisition of relevant timbre-related bowing control parameter signals. We model the temporal contour of bow velocity, bow pressing force, and bow-bridge distance as sequences of short Be¿zier cubic curve segments. Considering different articulations, dynamics, and performance contexts, a number of note classes are defined. Contours of bowing parameters in a performance database are analyzed at note-level by following a predefined grammar that dictates characteristics of curve segment sequences for each of the classes in consideration. As a result, contour analysis of bowing parameters of each note yields an optimal representation vector that is sufficient for reconstructing original contours with significant fidelity. From the resulting representation vectors, we construct a statistical model based on Gaussian mixtures suitable for both the analysis and synthesis of bowing parameter contours. By using the estimated models, synthetic contours can be generated through a bow planning algorithm able to reproduce possible constraints caused by the finite length of the bow. Rendered contours are successfully used in two preliminary synthesis frameworks: digital waveguide-based bowed string physical modeling and sample-based spectral-domain synthesis.

Modeling and transforming speech using variational autoencoders

Comunicacio presentada al Interspeech 2016, celebrat a San Francisco (California, EUA) els dies 8 a 12 de septembre de 2016, i organitzat per la International Speech Communication Association (ISCA).

Performance Control Driven Violin Timbre Model Based on Neural Networks

The objective of this research is to model the relationship between actions performed by a violinist and the sound which these actions produce. Violinist actions and audio are captured during real performances by means of a newly developed sensing system from which bowing and audio descriptors are computed. A database is built with this data and used to train a generative model based on neural networks. The model is driven by a continuous sequence of bowing and fingering controls and is able to generate their corresponding sequence of spectral envelopes. The model is used for synthesis, either alone as a purely spectral model, by filling the predicted envelopes with harmonic and noisy components, or coupled with a concatenative synthesizer, where the predicted envelopes are used as time-varying filters to transform the concatenated samples. The combination of sample concatenation with the timbre model allows for the preservation of sound quality inherent in samples, while providing a high level of control. Additionally, we perform an analysis of the violinist control space and the influence of the controls on the timbre.

Expressive singing synthesis based on unit selection for the singing synthesis challenge 2016

Comunicacio presentada al Interspeech 2016, celebrat a San Francisco (California, EUA) els dies 8 a 12 de septembre de 2016, i organitzat per la International Speech Communication Association (ISCA).

Generation of growl-type voice qualities by spectral morphing

In this paper we introduce a morph-based approach for generating voice source aperiodicities frequently associated with strong vocal expressions, especially in singing. In our approach the excitation characteristics of one signal are combined the fundamental frequency and spectral envelope characteristics of another signal. An exemplar sustained sample of the target voice quality is looped and resampled in the time domain in order to generate a continuous signal matching the input voices fundamental frequency. This operation preserves most of the samples voice quality characteristics at the cost of scaling time and frequency dimensions. While we found the temporal scaling to be acceptable in many contexts, the frequency scaling has to be inverted in order to generate appropriate spectral content for the source excitations entire bandwidth. This can be accomplished by the phase-locked vocoder method of modulating harmonic bands in frequency. Finally, the input signals harmonic amplitudes and phases are applied to the transformed morph sample, allowing for a simple one-dimensional control of morph amount by linear interpolation with the input signal. The proposed system is evaluated and the results are discussed.