Kim Vincs

Teaching a Digital Performing Agent: Artificial Neural Network and Hidden Markov Model for recognising and performing dance movement

For a Digital Performing Agent to be able to perform live with a human dancer, it would be useful for the agent to be able to contextualize the movement the dancer is performing and to have a suitable movement vocabulary with which to contribute to the performance. In this paper we will discuss our research into the use of Artificial Neural Networks (ANN) as a means of allowing a software agent to learn a shared vocabulary of movement from a dancer. The agent is able to use the learnt movements to form an internal representation of what the dancer is performing, allowing it to follow the dancer, generate movement sequences based on the dancers current movement and dance independently of the dancer using a shared movement vocabulary. By combining the ANN with a Hidden Markov Model (HMM) the agent is able to recognize short full body movement phrases and respond when the dancer performs these phrases. We consider the relationship between the dancer and agent as a means of supporting the agents learning and performance, rather than developing the agents capability in a self-contained fashion.

Identifying regions of good agreement among responders in engagement with a piece of live dance

This study investigated the issue of subjectivity in dance cognition by examining the amount of agreement in continuous “engagement” responses made by 12 observers to a semi-improvised dance work. Continuous judgments of engagement were collected using the portable Audience Response Facility, which sampled ratings from participants twice a second. Sample-by-sample standard deviation (SD) scores were used as a measure of observer agreement. SD varied from 19% to 34% of the total engagement scale range. Seven regions of good agreement were identified and analyzed. Sections where expectations were not interrupted were more likely to produce good agreement. A further analysis examined the effect of window size (bin width of one sample, up to 11 samples) and alternate measures of observer agreement (50% interpercentile, 90% interpercentile, and median absolute deviation, in addition to SD) on results. Window size made the least difference to the analysis of observer agreement. The results counter the views that dance response is too subjective to be worthy of experimental investigation, or that it lacks subjective, intertextual components.

Snapshots of complexity: using motion capture and principal component analysis to reconceptualise dance

This article brings together the disparate worlds of dance practice, motion capture and statistical analysis. Digital technologies such as motion capture offer dance artists new processes for recording and studying dance movement. Statistical analysis of these data can reveal hidden patterns in movement in ways that are semantically ‘blind’, and are hence able to challenge accepted culturo-physical ‘grammars’ of dance creation. The potential benefit to dance artists is to open up new ways of understanding choreographic movement. However, quantitative analysis does not allow for the uncertainty inherent in emergent, artistic practices such as dance. This article uses motion capture and principal component analysis (PCA), a common statistical technique in human movement recognition studies, to examine contemporary dance movement, and explores how this analysis might be interpreted in an artistic context to generate a new way of looking at the nature and role of movement patterning in dance creation.

Effects of observer experience on continuous measures of engagement with a contemporary dance work

Dance is multifaceted and interculturally complex. Empirical approaches to investigating the ways in which observers respond to dance offer a potentially productive means of understanding the cognitive bases upon which these responses to dance are generated. This study demonstrates one such approach, by measuring the responses of two groups of observers to a single dance performance continuously, as the performance unfolded. Specifically, we asked a group of dance students and a group of dance professionals to record their engagement with the same professional solo contemporary dance performance as a means of investigating whether and how the responses of dance students and professionals differ. Our analysis of these measurements demonstrates both similarities and differences between the two groups. On the basis of this analysis, we speculate that both students and professionals respond to choreographic ‘disjunctures’ in which expectations are overturned, which we term ‘gem moments,’ and that mature artists’ responses to dance differ from those of students through a change in degree – an increased ability to respond to ‘gem moments’ – rather than through a shift in the kinds of choreographic structures that elicit increasing engagement.

Navigating control and illusion: functional interactivity versus ‘faux-interactivity’ in transmedia dance performance

ABSTRACT Interactivity – a networked loop in which a performer’s live data feeds a digital system – can bridge the divide between live performance and digital entities in transmedia dance performances. In the ‘entanglement scene’ of Australian Dance Theatre’s Multiverse (2014), choreographer Garry Stewart and the creative coders and animators at the Deakin Motion.Lab utilise ‘faux-interactivity’, or a perceived relationship between the dancers and digital entities that exists only from the perspective of the audience. The spectre of ‘faux-interactivity’ challenges the spontaneity in live, embodied performance art because it both integrates live performance with prerendered digital content and offers a potential structure for a shared, dispersed creative and choreographic process across numerous and shared artistic and technological platforms. This paper investigates the concept of ‘faux-interactivity’, suggesting that its use can be a catalyst for moving beyond the limitations and values of ‘real’, or functional interactive systems within a theatrical context, and positing that definitions of ‘interactivity’ might be further expanded to accommodate the shifting timelines inherent in the disparate creative processes of human performance and coding.

Haptically-Enabled Dance Visualisation Framework for Deafblind-Folded Audience and Artists

In this paper we propose a framework for communicating performance art to deaf, blind and deaf blind audiences and artists haptically through the sense of touch. This research opens doors for novel artistic trends relying mainly on the sense of touch. The paper investigates the design considerations dictated by solo and group dances as well as stage setup. Implementation scenarios for deaf blind audiences and performers are also discussed.

Recognition: combining human interaction and a digital performing agent

Virtual and augmented environments are often dependent on human intervention for change to occur. However there are times when it would be advantageous for appropriate human-like activity to still occur when there are no humans present. In this paper, we describe the installation art piece Recognition, which uses the movement of human participants to effect change, and the movement of a performing agent when there are no humans present. The agents Artificial Neural Network has learnt appropriate movements from a dancer and is able to generate suitable movement for the main avatar in the absence of human participants.

Feels Like Dancing: Motion capture driven haptic interface as an added sensory experience for dance viewing

This paper describes a system for delivering movement information from a dance performance using a multisensory approach that includes visual, sonic and haptic information. The work builds on previous research into interpreting dance as haptic information for blind, deaf-blind and vision-impaired audience members. This current work is aimed at a general audience, with haptic information being one of a number of sensory experiences of the dance. A prototype haptic device has been developed for use in dance performance research.

The intersection of live and digital: new technical classifications for digital scenography in opera

ABSTRACT Over the past decade, there has been a substantive increase in the number of theatrical productions that utilise digital scenography. Whether in dance, theatre or opera, these works encounter a number of artistic and technical challenges, as well as scrutiny from those who consider digital effects as interfering with live performers in the context of legacy art forms. While the use of digital technology in theatre has been the subject of study by scholars such as Auslander, Baugh and Salter, the use of digital enhancement in opera production has received far less attention, particularly in terms of the scenographic conventions that have emerged as a result of advancing technologies. Because opera is largely defined and constrained by highly traditional production conventions, including a seated orchestra and unamplified singers, the genre provides a unique framework for considering the use of new technologies in the performative space. This article will propose a method for classifying and quantifying digitally enhanced theatrical productions and explore the resulting dramaturgical implications through a close study of three productions of Mozart’s The Magic Flute.