Paul Hertz

Synesthetic Art— An Imaginary Number?

Works that fuse the senses are often referred to as synesthetic art. Computers, which offer the possibility of controlling and synchronizing different media and implementing highly abstract compositional structures across media, seem an ideal tool for synesthetic art. This essay argues that a structural approach to such an art form is inadequate, and that it must be grounded in its potential symbolic functions. Starting from a brief examination of synesthesia as a neurological phenomenon and a sketch of the origins and influence of Baudelaires poetics of synesthesia, this essay suggests points of departure for a poetics of multisensory composition.

Dynamic reconstruction of the orophanryngeal swallow using computer based animation

The oropharyngeal swallow was modeled with computer based animation using data from biplane videofluorographic and dynamic CT images of 10 ml liquid swallows of a volunteer subject. Tracings of oropharyngeal structures from synchronized, magnification adjusted, images of the posterior-anterior, lateral, and cross-sectional planes were aligned in three dimensions with graphics animation software. Twenty oropharyngeal configurations were created at 1/15 second intervals to dynamically illustrate the swallow. These three-dimensional reconfigurations could be sequenced into an animation routine. Software analysis of the model permitted quantification of structural movement and intrapharyngeal volume across time. Such analyses can be used to detail both the efficacy of individual functional elements of the swallow as well as global pump function. It is hoped that modeling the oropharyngeal swallow will be useful to analyze mechanisms of dysphagia and the mechanics of compensatory therapeutic strategies.

Orai/Kalos

ARTIST STATEMENT Orai/Kalos (from the Japanese orai – comings and goings, traffic, communication and the Greek kalos – fair, beautiful) presents images and audio in an interactive computer-driven installation that continually varies its content and composition. Orai/Kalos is an intermedia work, where sound and image events are driven by the same underlying structures, and an interactive work, where each visitor generates new configurations.

Digital intermedia composition

Throughout most of its intermittent history, which in the Western tradition we could somewhat arbitrarily position from Pythagoras onward, the imaginative scope of intermedia has far exceeded the available technological support. Much like Charles Babbage’s unfinished Analytical Engine, color organs and other instruments to produce intermedia foreshadowed possibilities which they could at best only approximate. Digital technology has realized and surpassed the visions of Babbage and Ada Lovelace for mathematical computation, and in the process enabled a wide array of intermedia techniques, from the synchronicities of VJ jamming to the intricacies of algorithmic composition. Binary code provides a common representation for media and so enables transcoding of information from one sensory modality to another over multiple scales. High speed computational systems permit exquisitely precise timing of events and so make it possible to perform multimodal events within real-time responsive environments.