Pietro Polotti

A toolkit for explorations in sonic interaction design

Physics-based sound synthesis represents a promising paradigm for the design of a veridical and effective continuous feedback in augmented everyday contexts. In this paper, we introduce the Sound Design Toolkit (SDT), a software package available as a complete front-end application, providing a palette of virtual lutheries and foley pits, that can be exploited in sonic interaction design research and education. In particular, the package includes polyphonic features and connectivity to multiple external devices and sensors in order to facilitate the embedding of sonic attributes in interactive artifacts. The present release represents an initial version towards an effective and usable tool for sonic interaction designers.

Sound and Music Computing: Research Trends and Some Key Issues

Abstract This contribution attempts to give an overview of current research trends and open research problems in the rich field of Sound and Music Computing (SMC). To that end, the field is roughly divided into three large areas related to Sound, Music, and Interaction, respectively, and within each of these, major research trends are briefly described. In addition, for each sub-field a small number of open research (or research strategy) issues are identified that should be addressed in order to further advance the SMC field.

Gamelunch: forging a dining experience through sound

The Gamelunch is a sonically augmented dining table. By means of the Gamelunch, we aim at investigating the closed loop between interaction, sound and emotion by exploiting the power and flexibility of physically-based sound models for an effective and coherent process of interactive sound design. Continuous interaction gestures are captured by means of various force transducers, providing data that are coherently mapped onto physically-based sound synthesis algorithms. Investigation of the above mentioned loop is carried out by the principle of contradiction: while performing usual dining actions, such as cutting and slicing, dressing the salad, pouring beverages, the user encounters contradicting and unexpected sound feedbacks, thus experiencing -- per absurdum -- the importance of environmental sounds in everyday-life acts.

Analysis and synthesis of pseudo-periodic1/f-like noise by means of wavelets with applications to digital audio

Voiced musical sounds have nonzero energy in sidebands of the frequency partials. Our work is based on the assumption, often experimentally verified, that the energy distribution of the sidebands is shaped as powers of the inverse of the distance from the closest partial. The power spectrum of these pseudo-periodic processes is modeled by means of a superposition of modulated components, that is, by a pseudo-periodic-like process. Due to the fundamental selfsimilar character of the wavelet transform, processes can be fruitfully analyzed and synthesized by means of wavelets. We obtain a set of very loosely correlated coefficients at each scale level that can be well approximated by white noise in the synthesis process.Our computational scheme is based on an orthogonal-band filter bank and a dyadic wavelet transform per channel. The channels are tuned to the left and right sidebands of the harmonics so that sidebands are mutually independent. The structure computes the expansion coefficients of a new orthogonal and complete set of harmonic-band wavelets. The main point of our scheme is that we need only two parameters per harmonic in order to model the stochastic fluctuations of sounds from a pure periodic behavior.

Fractal Additive Synthesis via Harmonic-Band Wavelets

One of the most challenging aspects of sound analysis and representation is the definition of a good model for the noisy part of sounds. We need a good representation of those components of sound whose spectra lie outside the frequencies of the partials. In this article, we subdivide a sound into its deterministic and stochastic components. The deterministic part of sounds provides the pitch and the global timbre of a sound; it is, in a sense, the fundamental structure of the sound. The stochastic part contains the ”life” of a sound—all the ”microfluctuations” with respect to a non-evolving sound plus the noise due to the physical excitation system. The main idea behind our method is that these ”microfluctuations” with respect to a pure harmonic behavior can be reconstructed from the power spectrum. In this article, we describe a model for the particular case of voiced sounds, namely, sounds with a harmonic spectrum. We define a well-suited analysis and resynthesis method for our model based on the Harmonic-Band Wavelet Transform (HBWT). Owing to the mathematical properties of the HBWT, the synthesis of signals with pseudoperiodic 1/f -like power spectra is straightforward. These spectra are very good approximations of those of real-life voiced sounds. In a first approximation, the only thing we need to do is control the energies of white noise coefficients according to very few parameters derived from the analysis of real sounds. As a further insight, the HWBT analysis reveals the existence of a small but non-zero correlation between the coefficients. An autoregressive (AR) analysis and resynthesis model employing white noise as excitation for AR filters, in conjunction with the above-mentioned loose correlation, can substitute for the trivial white noise coefficient model. A refinement of the technique also takes into account scale-dependent time evolution of the resynthesis parameters. The model does not apply to the harmonic components. We preserve the restricted set of analysis wavelet coefficients corresponding to the narrow bands of the harmonics in order to have perfect reconstruction data for the deterministic part, that is, the harmonic amplitudes and their time envelopes. The attack transients are preserved as well. This method can be seen both as a musical tool for sound synthesis able to provide synthetic sounds with a natural timbre dynamic and as a compression technique. In the first part of this article we review ordinary wavelets, Harmonic-Band Wavelets, and the pseudo-periodic 1/f-like model. In the second part we present new developments of the synthesis method from methodological and experimental points of view.

Fractal additive synthesis

In this article, we outlined time-frequency techniques that, when brought together, form the basis for FAS. The method is based on an exact orthogonal transform, the HBWT. The FAS model can be seen as an intelligent spectrogram, i.e., as a spectrogram where the frequency bins (the HBWT subbands) are adapted to the spectrum of the analyzed sound by tuning the number of channels P to the period of the sound. Local cosines replace the classical windowed complex exponentials, whose main property is that the basis elements form sidebands of the harmonics rather than being centered on the harmonics themselves. The wavelet transform nonuniform frequency subdivision characteristic is exploited to synthesize each sideband of the harmonic peaks by means of colored noise, generating an approximate pseudoperiodic 1/f behavior. The pseudoharmonics are modeled by narrow sidebands whose weights can be generated by means of amplitude envelopes and phase functions in a complexified HBWT domain. We also showed how perceptual criteria can be employed to reduce the number of synthesis parameters. The possibility of independently controlling the noisy components and the harmonic terms and the definition of parameters such as volumes and envelopes for all of the components independently provide powerful tools for processing voiced-sound for sound design purposes

Sonic Interaction Design for Paper Wearables

The paper reports a workshop on sonic interaction design conceived and led by the authors in the context of a living lab born from the collaboration between a music conservatory and an IT university department. The main subject was the application of non-verbal sound in the process of product design, focusing on the augmentation of clothes and wearable accessories. The workshop resulted in exercises exploring the interactive role of the sound within three different scenarios: (a) abstract/relational, (b) strictly functional, (c) aesthetic/performative. Each exercise was carried on by a group of four to five participants working as a team. The presentation of the exercises comes along considerations regarding the participatory approach, that matched design techniques and tools with practices and theoretical foundations of electroacoustic music. Results are briefly discussed and improvements and further steps are accounted for.

A Pitch-Synchronous Extension of Fractal Additive Synthesis via Time-Varying Cosine Modulated Filter Banks

Fractal additive synthesis (FAS) is a method for the synthesis-by-analysis of voiced-sounds. FAS is based on a harmonic version of the wavelet transform (WT), and it allows to define two different parametric models for the deterministic and the stochastic components of sounds, respectively. In this letter, we introduce a pitch-synchronous (PS) extension of FAS. By adapting an already existing method for the design of time-varying cosine modulated filter banks (TV-CMFB), we are now able to deal with a much wider class of voiced-sounds, i.e., any voiced-sound showing a variable pitch as in the case, for example, of glissando or vibrato effects.