Lonce Wyse

A neural network architecture for figure-ground separation of connected scenic figures

Abstract A neural network model, called an FBF network, is proposed for automatic parallel separation of multiple image figures from each other and their backgrounds in noisy gray-scale or multicolored images. The figures can then be processed in parallel by an array of self-organizing Adaptive Resonance Theory (ART) neural networks for automatic target recognition. An FBF network can automatically separate the disconnected but interleaved spirals that Minsky and Papert introduced in their book Perceptrons. The networks design also clarifies why humans cannot rapidly separate interleaved spirals, yet can rapidly detect conjunctions of disparity and color, or of disparity and motion, that distinguish target figures from surrounding distractors. Figure-ground separation is accomplished by iterating operations of a Feature Contour System (FCS) and a Boundary Contour System (BCS) in the order FCS-BCS-FCS, hence the term FBF. The FCS operations include the use of nonlinear shunting networks to compensate for variable illumination and nonlinear diffusion networks to control filling-in. A key new feature of an FBF networks is the use of filling-in for figure-ground separation. The BCS operations include oriented filters joined to competitive and cooperative interactions designed to detect, regularize, and complete boundaries in up to 50% noise, while suppressing the noise. A modified CORT-X filter is described. which uses both on-cells and off-cells to generate a boundary segmentation from a noisy image.

Blind separation of fetal ECG from single mixture using SVD and ICA

In this paper, we propose a novel blind-source separation method to extract fetal ECG from a single-channel signal measured on the abdomen of the mother. The signal is a mixture of the fetal ECG, the maternal ECG and noise. The key idea is to project the signal into higher dimensions, and then use an assumption of statistical independence between the components to separate them from the mixtures. This is achieved by applying singular value decomposition (SVD) on the spectrogram, followed by an iterated application of independent component analysis (ICA) on the principle components. The SVD contributes to the separability of each component and the ICA contributes to the independence of the two components. We further refine and adapt the above general idea to ECG by exploiting a-prior knowledge of the maternal ECG frequency distribution and other characteristics of ECG. Experimental studies show that the proposed method is more accurate than using SVD only. Because our method does not exploit extensive domain knowledge of the ECGs, the idea of combining SVD and ICA in this way can be applied to other blind separation problems.

Real-Time Signal Estimation From Modified Short-Time Fourier Transform Magnitude Spectra

An algorithm for estimating signals from short-time magnitude spectra is introduced offering a significant improvement in quality and efficiency over current methods. The key issue is how to invert a sequence of overlapping magnitude spectra (a ldquospectrogramrdquo) containing no phase information to generate a real-valued signal free of audible artifacts. Also important is that the algorithm performs in real-time, both structurally and computationally. In the context of spectrogram inversion, structurally real-time means that the audio signal at any given point in time only depends on transform frames at local or prior points in time. Computationally, real-time means that the algorithm is efficient enough to run in less time than the reconstructed audio takes to play on the available hardware. The spectrogram inversion algorithm is parameterized to allow tradeoffs between computational demands and the quality of the signal reconstruction. The algorithm is applied to audio time-scale and pitch modification and compared to classical algorithms for these tasks on a variety of signal types including both monophonic and polyphonic audio signals such as speech and music.

AudiOdyssey: an accessible video game for both sighted and non-sighted gamers

Despite the growing number and demographics of video game players, most games are still completely inaccessible to disabled populations. To study the issue of gaming accessibility, we created AudiOdyssey, a prototype video game designed to be usable by both sighted and non-sighted audiences. Featuring multiple input control schemes, rhythm based game play, and fully accessible menus and play levels, the prototype allows all individuals to share a common gaming experience, regardless of level of vision.

The viability of the web browser as a computer music platform

The computer music community has historically pushed the boundaries of technologies for music-making, using and developing cutting-edge computing, communication, and interfaces in a wide variety of creative practices to meet exacting standards of quality. Several separate systems and protocols have been developed to serve this community, such as Max/MSP and Pd for synthesis and teaching, JackTrip for networked audio, MIDI/OSC for communication, as well as Max/MSP and TouchOSC for interface design, to name a few. With the still-nascent Web Audio API standard and related technologies, we are now, more than ever, seeing an increase in these capabilities and their integration in a single ubiquitous platform: the Web browser. In this article, we examine the suitability of the Web browser as a computer music platform in critical aspects of audio synthesis, timing, I/O, and communication. We focus on the new Web Audio API and situate it in the context of associated technologies to understand how well they together can be expected to meet the musical, computational, and development needs of the computer music community. We identify timing and extensibility as two key areas that still need work in order to meet those needs.

Generic Audio Classification Using a Hybrid Model Based on GMMs and HMMs

A hybrid model comprised of Gaussian Mixtures Models (GMMs) and Hidden Markov Models (HMMs) is used to model generic sounds with large intra class perceptual variations. Each class has variable number of mixture components in the GMM. The number of mixture components is derived using the Minimum Description Length (MDL) criterion. The overall performance of the hybrid model was compared against models based on HMMs and GMMs with a fixed number of mixture components across all classes. We show that a hybrid model outperforms both class-based GMMs, HMMs, and GMMs based on fixed number of components. Further, our experiments revealed that the contribution of transitions between states in HMMs has no significant effect on the overall classification performance of generic sounds when large intra class perceptual variations are present among sounds in the training and test datasets. Sounds that show multi-event structure with events that tend to be similar (repetitive) indicated improved performance when modeled with HMMs that can be attributed to HMM’s state transition property. Conversely, GMMs indicate better performance when the sound samples show subtle or no repetitive behavior. These results were validated using the MuscleFish sound database.

Enhancing Musical Experience for the Hearing-Impaired Using Visual and Haptic Displays

This article addresses the broad question of understanding whether and how a combination of tactile and visual information could be used to enhance the experience of music by the hearing impaired. Initially, a background survey was conducted with hearing-impaired people to find out the techniques they used to “listen” to music and how their listening experience might be enhanced. Information obtained from this survey and feedback received from two profoundly deaf musicians were used to guide the initial concept of exploring haptic and visual channels to augment a musical experience. The proposed solution consisted of a vibrating “Haptic Chair” and a computer display of informative visual effects. The Haptic Chair provided sensory input of vibrations via touch by amplifying vibrations produced by music. The visual display transcoded sequences of information about a piece of music into various visual sequences in real time. These visual sequences initially consisted of abstract animations corresponding to specific features of music such as beat, note onset, tonal context, and so forth. In addition, because most people with impaired hearing place emphasis on lip reading and body gestures to help understand speech and other social interactions, their experiences were explored when they were exposed to human gestures corresponding to musical input. Rigorous user studies with hearing-impaired participants suggested that musical representation for the hearing impaired should focus on staying as close to the original as possible and is best accompanied by conveying the physics of the representation via an alternate channel of perception. All the hearing-impaired users preferred either the Haptic Chair alone or the Haptic Chair accompanied by a visual display. These results were further strengthened by the fact that user satisfaction was maintained even after continuous use of the system over a period of 3 weeks. One of the comments received from a profoundly deaf user when the Haptic Chair was no longer available (“I am going to be deaf again”), poignantly expressed the level of impact it had made. The system described in this article has the potential to be a valuable aid in speech therapy, and a user study is being carried out to explore the effectiveness of the Haptic Chair for this purpose. It is also expected that the concepts presented in this paper would be useful in converting other types of environmental sounds into a visual display and/or a tactile input device that might, for example, enable a deaf person to hear a doorbell ring, footsteps approaching from behind, or a person calling him or her, or to make understanding conversations or watching television less stressful. Moreover, the prototype system could be used as an aid in learning to play a musical instrument or to sing in tune. This research work has shown considerable potential in using existing technology to significantly change the way the deaf community experiences music. We believe the findings presented here will add to the knowledge base of researchers in the field of human–computer interaction interested in developing systems for the hearing impaired.

Perceptual and conceptual priming of environmental sounds

It is still unknown whether sonic environments influence the processing of individual sounds in a similar way as discourse or sentence context influences the processing of individual words. One obstacle to answering this question has been the failure to dissociate perceptual (i.e., how similar are sonic environment and target sound?) and conceptual (i.e., how related are sonic environment and target?) priming effects. In this study, we dissociate these effects by creating prime–target pairs with a purely perceptual or both a perceptual and conceptual relationship. Perceptual prime–target pairs were derived from perceptual–conceptual pairs (i.e., meaningful environmental sounds) by shuffling the spectral composition of primes and targets so as to preserve their perceptual relationship while making them unrecognizable. Hearing both original and shuffled targets elicited a more positive N1/P2 complex in the ERP when targets were related to a preceding prime as compared with unrelated. Only related original targets reduced the N400 amplitude. Related shuffled targets tended to decrease the amplitude of a late temporo-parietal positivity. Taken together, these effects indicate that sonic environments influence first the perceptual and then the conceptual processing of individual sounds. Moreover, the influence on conceptual processing is comparable to the influence linguistic context has on the processing of individual words.

NMF vs ICA for face recognition

This paper deals with the application of spatially localized, nonoverlapping features for face recognition. The analysis is carried out by using the features generated from two closely related techniques known as independent component analysis (ICA) and nonnegative matrix factorization (NMF). A set of statistically independent basis vectors with sparse features is derived from ICA. Likewise, NMF is used to yield sparse representation of localized features to represent distributed parts over a human face. Similarities between reconstructed faces of test images and a set of synthesised face representations from the basis vectors derived from an image database using the two techniques are measured. The strengths and weaknesses of each method in the context of face recognition are discussed.

Real-Time Iterative Spectrum Inversion with Look-Ahead

In this paper, we present an algorithm for real-time iterative spectrogram inversion (RTISI) with look-ahead (RTISI-LA). RTISI-LA reconstructs a time-domain signal from a given sequence of short-time Fourier transform magnitude (STFTM) spectra without phase information. Whereas RTISI reconstructs the current frame using only magnitude spectra information for previous frames and the current frame, RTISI-LA also uses magnitude spectra for a small number future frames. This allows RTISI-LA to achieve substantially higher signal-to-noise (SNR) performance than either RTISI or the Griffin & Lim method with an equivalent computational load, while retaining the real-time properties of RTISI