Bernd Schoner

Cluster-weighted modelling for time-series analysis

The need to characterize and forecast time series recurs throughout the sciences, but the complexity of the real world is poorly described by the traditional techniques of linear time-series analysis. Although newer methods can provide remarkable insights into particular domains, they still make restrictive assumptions about the data, the analyst, or the application. Here we show that signals that are nonlinear, non-stationary, non-gaussian, and discontinuous can be described by expanding the probabilistic dependence of the future on the past around local models of their relationship. The predictors derived from this general framework have the form of the global combinations of local functions that are used in statistics, machine learning and studies of nonlinear dynamics,. Our method offers forecasts of errors in prediction and model estimation, provides a transparent architecture with meaningful parameters, and has straightforward implementations for offline and online applications. We demonstrate our approach by applying it to data obtained from a pseudo-random dynamical system, from a fluctuating laser, and from a bowed violin.

An installation of interactive furniture

We report on a project that explored emerging technologies for intuitive and unobtrusive information interfaces in a compelling setting. An installation at the Museum of Modern Art, New York, was part of a public exhibit and included an interactive table that presented information associated with the exhibit to the gallery visitors without visible conventional computing elements. The enabling devices included noncontact sensing of low-cost tags in physical icons, electrostatic detection of hand location in three dimensions, and sensor fusion through lightweight Internet Protocol access.

An immersive, multi-user, musical stage environment

A multi-user, polyphonic sensor stage environment that maps position and gestures of up to four performers to the pitch and articulation of distinct notes is presented. The design seeks to provide multiple players on a stage with the feeling of a traditional acoustic instrument by giving them complete control over the instruments expressive parameters and a clear causal connection between their actions and the resulting sound. The positions of the performers are determined by a custom ultrasonic tracking system, while hand motions are measured by custom-made gloves containing accelerometer units. Furthermore, juggling clubs are illuminated dynamically to make complex juggling patterns more apparent. The system is currently on tour with the Flying Karamazov Brothers juggling troupe.

Cluster-Weighted Modeling: Probabilistic Time Series Prediction, Characterization, and Synthesis

Cluster-weighted modeling, a mixture density estimator around local models, is presented as a framework for the analysis, prediction and characterization of non-linear time series. First architecture, model estimation and characterization formalisms are introduced. The characterization tools include estimator uncertainty, predictor uncertainty, and the correlation dimension of the data set. in the second part of this chapter the framework is extended to synthesize audio signals and is applied to model a violin in a data-driven input-output approach.

Inference based modeling of nonlinear microwave devices

We present an inference-based algorithm for modeling complex non-linear systems, that integrates current approaches to modeling of microwave devices within a generalized framework. Familiar techniques for characterizing linear and non-linear systems are embedded in an automated non-linear weighting mechanism, so that globally complex behavior is approximated by simple local models.

Determining Feature Relevance in Subject Responses to Musical Stimuli

This paper presents a method that determines the relevance of a set of signals (musical features) given listener judgments of music in an experimental setting. Rather than using linear correlation methods, we allow for nonlinear relationships and multi-dimensional feature vectors. We first provide a methodology based on polynomial functions and the least-mean-square error measure. We then extend the methodology to arbitrary nonlinear function approximation techniques and introduce the Kullback-Leibler Distance as an alternative relevance metric. The method is demonstrated first with simple artificial data and then applied to analyze complex experimental data collected to examine the perception of musical tension.

Phoneme discrimination from MEG data

Abstract We treat magnetoencephalographic (MEG) data in a signal detection framework to discriminate between different phonemes heard by a test subject. Our data set consists of responses evoked by the voiced syllables /bae and /dae/ and the corresponding voiceless syllables /pae/ and /tae/. The data yield well to principal component analysis (PCA), with a reasonable subspace in the order of three components out of 37 channels. To discriminate between responses to the voiced and voiceless versions of a consonant we form a feature vector by either matched filtering or wavelet packet decomposition and use a mixture-of-experts model to classify the stimuli. Both choices of a feature vector lead to a significant detection accuracy. Furthermore, we show how to estimate the onset time of a stimulus from a continuous data stream.

Data‐driven modeling of acoustical instruments

Comprehensive digital analysis and synthesis of musical instruments using direct observations of their physical behavior have been developed and implemented for the violin. In a training session, control input data from unobtrusive bow and finger sensors is recorded simultaneously with the violin’s audio output. These signals are used to train a cluster‐weighted probabilistic prediction model that reproduces the nonlinear relationship between the control inputs and the target audio output data. Cluster‐weighted modeling was developed to apply previous results from linear systems theory and time‐series approximation theory in the broader context of a globally complex and nonlinear model. The presented sound synthesis engine makes use of familiar sound synthesis techniques, but extends them with a complex input/output framework that naturally incorporates dynamic control. The final system predicts audio data based on new control data. While a violinist plays the interface device (a silent violin), the compu...