Mick Grierson

Better brain interfacing for the masses: progress in event-related potential detection using commercial brain computer interfaces

Event-Related Potential (ERP) techniques are commonly used by researchers from a range of disciplines including psychology and medicine to stimulate meaningful ERP signals from the brain and interpret them through Electroencephalography (EEG). ERP signals are in most cases able to reliably reflect cognitive processes, and are widely used in Brain Computer Interface (BCI) research. We present work in progress towards the application of these techniques to emerging consumer-grade BCI technology. Our approach has an impact on the reliability and usability of consumer Brain Computer Interfaces in commercial contexts, and is already being adopted by our industry partners in the games and entertainment sector. It could also significantly reduce the cost and complexity of certain types of large scale ERP research. This work is being undertaken by the Embodied AudioVisual Interaction (EAVI) group at Goldsmiths, University of London, and is supported by the Arts and Humanities Research Council.

Using Interactive Machine Learning to Support Interface Development Through Workshops with Disabled People

We have applied interactive machine learning (IML) to the creation and customisation of gesturally controlled musical interfaces in six workshops with people with learning and physical disabilities. Our observations and discussions with participants demonstrate the utility of IML as a tool for participatory design of accessible interfaces. This work has also led to a better understanding of challenges in end-user training of learning models, of how people develop personalised interaction strategies with different types of pre-trained interfaces, and of how properties of control spaces and input devices influence peoples customisation strategies and engagement with instruments. This work has also uncovered similarities between the musical goals and practices of disabled people and those of expert musicians.

The effect of exercise-induced arousal on chosen tempi for familiar melodies

Many previous studies have shown that arousal affects time perception, suggesting a direct influence of arousal on the speed of the pacemaker of the internal clock. However, it is unknown whether arousal influences the mental representation of tempo (speed) for highly familiar and complex stimuli, such as well-known melodies, that have long-term representations in memory. Previous research suggests that mental representations of the tempo of familiar melodies are stable over time; the aim of the present study was to investigate whether these representations can be systematically altered via an increase in physiological arousal. Participants adjusted the tempo of 14 familiar melodies in real time until they found a tempo that matched their internal representation of the appropriate tempo for that piece. The task was carried out before and after a physiologically arousing (exercise) or nonarousing (anagrams) manipulation. Participants completed this task both while hearing the melodies aloud and while imagining them. Chosen tempi increased significantly following exercise-induced arousal, regardless of whether a melody was heard aloud or imagined. These findings suggest that a change in internal clock speed affects temporal judgments even for highly familiar and complex stimuli such as music.

NoiseBear: a wireless malleable multiparametric controller for use in assistive technology contexts

NoiseBear is a malleable multiparametric interface, currently being developed in a series of participatory design workshops with disabled children. It follows a soft toy design, using conductive textiles for pressure sensing and circuitry. The system is a highly sensitive deformable controller; it can be used flexibly in a range of scenarios for continuous or discrete control, allowing interaction to be designed at a range of complexity levels. The controller is wireless, and can be used to extend the interactive possibilities of mobile computing devices. Multiple controllers may also be networked together in collaborative scenarios.

Contemporary Approaches to Music BCI Using P300 Event Related Potentials

This chapter is intended as a tutorial for those interested in exploring the use of P300 event related potentials (ERPs) in the creation of brain computer music interfaces (BCMIs). It also includes results of research in refining digital signal processing (DSP) approaches and models of interaction using low-cost, portable BCIs. We will look at a range of designs for BCMIs using ERP techniques. These include the P300 Composer, the P300 Scale Player, the P300 DJ and the P300 Algorithmic Improviser. These designs have all been used in both research and performance, and are described in such a way that they should be reproducible by other researchers given the methods and guidelines indicated. The chapter is not intended to be exhaustive in terms of its neuroscientific detail, although the systems and approaches documented here have been reproduced by many labs, which should be an indication of their quality. Instead, what follows is a basic introduction to what ERPs are, what the P300 is, and how it can be applied in the development of these BCMI designs. This description of ERPs is not intended to be exhaustive, and at best should be thought of as an illustration designed to allow the reader to begin to understand how such approaches can be used for new instrument development. In this way, this chapter is intended to be indicative of what can be achieved, and to encourage others to think of BCMI problems in ways that focus on the measurement and understanding of signals that reveal aspects of human cognition. With this in mind, towards the end of the chapter we look at the results of our most recent research in the area of P300 BCIs that may have an impact on the usability of future BCI systems for music.

STEAM WORKS: Student coders experiment more and experimenters gain higher grades

For the last decade, there has been growing interest in the STEAM approach (essentially combining methods and practices in arts, humanities and social sciences into STEM teaching and research) to develop better research and education, and enable us to produce students who can work most effectively in the current and developing market-place. However, despite this interest, there seems to be little quantitative evidence of the true power of STEAM learning, especially describing how it compares and performs with respect to more established approaches. To address this, we present a comparative, quantitative study of two distinct approaches to teaching programming, one based on STEAM (with an open-ended inquiry-based approach), the other based on a more traditional, non-STEAM approach (where constrained problems are set and solved). Our key results evidence how students exhibit different styles of programming in different types of lessons and, crucially, that students who tend to exhibit more of the style of programming observed in our STEAM lessons also tend to achieve higher grades. We present our claims through a range of visualisations and statistical validations which clearly show the significance of the results, despite the small scale of the study. We believe that this work provides clear evidence for the advantages of STEAM over non-STEAM, and provides a strong theoretical and technological framework for future, larger studies.

Autoencoding Blade Runner : reconstructing films with artificial neural networks

In this paper, the authors explain how they created Blade Runner---Autoencoded, a film made by training an autoencoder---a type of generative neural network---to recreate frames from the film Blade Runner. The autoencoder is made to reinterpret every individual frame, reconstructing it based on its memory of the film. The result is a hazy, dreamlike version of the original film. The authors discuss how the project explores the aesthetic qualities of the disembodied gaze of the neural network and describe how the autoencoder is also capable of reinterpreting films it has not been trained on, transferring the visual style it has learned from watching Blade Runner (1982).

FEATUR.UX.AV: A Live Sound Visualization System Using Multitrack Audio

In this paper, we describe the conceptual design and technical implementation of an audiovisual system whereby multitrack audio is used to generate visualizations in real time. We discuss our motivation within the context of audiovisual practice and present the outcomes of studies conducted to outline design requirements. We then describe the audio and visual components of our multitrack visualization model, and specific parts of the graphical user interface (GUI) which focus on mapping as the primary mechanism to facilitate live multitrack audiovisual performance.

Calligraphic Stylisation Learning with a Physiologically Plausible Model of Movement and Recurrent Neural Networks

We propose a computational framework to learn stylisation patterns from example drawings or writings, and then generate new trajectories that possess similar stylistic qualities. We particularly focus on the generation and stylisation of trajectories that are similar to the ones that can be seen in calligraphy and graffiti art. Our system is able to extract and learn dynamic and visual qualities from a small number of user defined examples which can be recorded with a digitiser device, such as a tablet, mouse or motion capture sensors. Our system is then able to transform new user drawn traces to be kinematically and stylistically similar to the training examples. We implement the system using a Recurrent Mixture Density Network (RMDN) combined with a representation given by the parameters of the Sigma Lognormal model, a physiologically plausible model of movement that has been shown to closely reproduce the velocity and trace of human handwriting gestures.

Light field completion using focal stack propagation

Both light field photography and focal stack photography are rapidly becoming more accessible with Lytros commercial light field cameras and the ever increasing processing power of mobile devices. Light field photography offers the ability of post capturing perspective changes and digital refocusing, but little is available in the way of post-production editing of light field images. We present a first approach for interactive content aware completion of light fields and focal stacks, allowing for the removal of foreground or background elements from a scene.