David H. Jameson

Complementary visualization and sonification of multidimensional data

Interpretation of multi-dimensional complex data usually involves extracting the relationship between several variables. This is typically done with an interactive visual system . Iugh resolution volumetric data imaging, color, animation, and multiple views are effective tools for data interpretation. Sound can provide an additional and complementary perceptual channel. This presentation focuses on the use of sound with a multi-dimensional imaging system to facilitate the interactive interpretation of complex data. Our methods and system are presented with data from a simulation which computes electron density, hole density, and potential throughout the volume of a three-dimensional semiconductor. The spatial changes and relationships of the three scalar fields are the object of study. Normally the field relations would he examined through multiple visualizations. here, sound is used to augment the visualization by permitting a user to visually concentrate on one field, while listening to the other. Two of the three scalar fields from the simulation arc selected for interpretation and visualized. The 3-dimensional vector gradient of one of them is sonified at a selected focal point within the semiconductor solid. As the current focus i interactively moved through the solid, the representative sound is altered accordingly. The sonification is composed such that local minima and maxima of one of the fields can he found without looking at it.

Visual music in a visual programming language

Sonnet was designed as a visual language for implementing real-time processes. Early design and development of behavioral components has largely focused on the domain of music programming. However, Sonnets architecture is well-suited to expressing many kinds of real-time activities. In particular, Sonnet is easily extended with new kinds of data types and behavioral components. We have developed a collection of visual output components for Sonnet, referred to collectively as Sonnet+Imager. Its design embodies aesthetically grounded representations of color, form and rhythm, as well as dynamics for each. Moreover, its value is enhanced by a flexible, modular architecture that treats these graphic entities and operations as first-class objects.

Building real-time music tools visually with Sonnet

We are building a variety of interactive music tools using Sonnet, a visual programming language in use at our center. Originally designed for sonification experiments for monitoring and debugging programs, Sonnet has grown into a more general system with a focus on real-time event-driven applications. In this paper, we describe some of the features of Sonnet followed by some examples of how it is being used.

Events: a structuring mechanism for a real-time runtime system

Recent progress in the implementation of the ORE real-time programming language is reported. A summary of the important features of the language is included. Progress in designing a runtime system for the language is reported. The design issues are discussed. A model of ORE processes is described. The set of operations defined on events, strips, and processes is described. Examples of the implementation of specific ORE features using this mechanism are illustrated.<<ETX>>