Mark Ballora

Heart Rate Sonification: A New Approach to Medical Diagnosis

Ever since 1819, when Theophile Lannec first put a block of wood to a patients chest in order to listen to her heartbeat, physicians have used auscultation to help diagnose cardiopulmonary disorders. Here the authors describe a novel diagnostic method based in music technology. Digital music-synthesis software is used to transform the sequence of time intervals between consecutive heartbeats into an electroacoustic soundtrack. The results show promise as a diagnostic tool and also provide the basis of an interesting musical soundscape.

Songs of cyberspace: an update on sonifications of network traffic to support situational awareness

Building on our previous work, we extend sonification techniques to common network security data. In this current work, we examine packet flow and the creation of socket connections between a requestors IP address and port number with the servers IP address and port number. Our goals for the aural rendering are twofold: to make certain conditions immediately apparent to untrained listeners, and to create a sound model capable of enough nuance that there is the possibility of unexpected patterns becoming apparent to a seasoned listener. This system could be used to potentially provide better cognitive refinement capabilities for data fusion systems, especially when multiple sources of data at various levels of refinement are presented to the human analyst.

Sonification, Science and Popular Music: In search of the ‘wow’

Sonification is described as an under-utilised dimension of the ‘wow!’ factor in science engagement multi-media. It is suggested that sonifications potential value, like much of the scientific visualisation content, probably lies less in hard facts and more in how it may serve as a stimulant for curiosity. Sound is described as a multi-dimensional phenomenon, and a number of approaches to creating sonifications are reviewed. Design strategies are described for five types of phenomena that were sonified for works created by cosmologist George Smoot III and percussionist/ethnomusicologist Mickey Hart, most particularly for their film Rhythms of the Universe (Hart and Smoot 2013).

Beyond visualization of big data: a multi-stage data exploration approach using visualization, sonification, and storification

As the sheer volume of data grows exponentially, it becomes increasingly difficult for existing visualization techniques to keep pace. The sonification field attempts to address this issue by enlisting our auditory senses to detect anomalies or complex events that are difficult to detect via visualization alone. Storification attempts to improve analyst understanding by converting data streams into organized narratives describing the data at a higher level of abstraction than the input stream that they area derived from. While these techniques hold a great deal of promise, they also each have a unique set of challenges that must be overcome. Sonification techniques must represent a broad variety of distributed heterogeneous data and present it to the analyst/listener in a manner that doesn’t require extended listening – as visual “snapshots” are useful but auditory sounds only exist over time. Storification still faces many human-computer interface (HCI) challenges as well as technical hurdles related to automatically generating a logical narrative from lower-level data streams. This paper proposes a novel approach that utilizes a service oriented architecture (SOA)-based hybrid visualization/ sonification / storification framework to enable distributed human-in-the-loop processing of data in a manner that makes optimized usage of both visual and auditory processing pathways while also leveraging the value of narrative explication of data streams. It addresses the benefits and shortcomings of each processing modality and discusses information infrastructure and data representation concerns required with their utilization in a distributed environment. We present a generalizable approach with a broad range of applications including cyber security, medical informatics, facilitation of energy savings in “smart” buildings, and detection of natural and man-made disasters.

Use of sonification in the detection of anomalous events

In this paper, we describe the construction of a soundtrack that fuses stock market data with information taken from tweets. This soundtrack, or auditory display, presents the numerical and text data in such a way that anomalous events may be readily detected, even by untrained listeners. The soundtrack generation is flexible, allowing an individual listener to create a unique audio mix from the available information sources. Properly constructed, the display exploits the auditory systems sensitivities to periodicities, to dynamic changes, and to patterns. This type of display could be valuable in environments that demand high levels of situational awareness based on multiple sources of incoming information.