Gilles Comeau

Flexible Multi-Camera Network Calibration for Human Gesture Monitoring

This paper presents the design and implementation of a flexible and easy-to-use multi-camera acquisition setup for markerless human gesture monitoring in unconstrained environments. A robust 2-stage framework is proposed to achieve full calibration of a variable number of synchronized cameras separated by long baselines. In the first stage, the intrinsic parameters are computed for each camera independently. In the second stage, the cameras are registered based on their relative positioning by waving a red light emitting device to produce a set of feature points. Matches are regrouped by camera pair such that pair-wise stereo relations can be found for as many pairs as possible before being scaled to create a consistent weighted camera graph which is used to link all cameras. Experimental results demonstrate the accuracy of the calibration that is achieved and the suitability of the proposed approach for almost any multi-camera configurations. An application is presented for volumetric-reconstruction of human beings to validate the implementation.

Comparative Study of Adaptive Segmentation Techniques for Gesture Analysis in Unconstrained Environments

This paper discusses the importance of providing non- invasive techniques in the analysis of the complex movements performed by musicians and athletes. Current gesture analysis systems are insufficient and do not succeed in providing quality performance measurements without imposing strict environmental and operational constraints on individuals. Computer vision offers the means with which such techniques can be made possible without impeding the performance of musicians or compromising the integrity of the measurements. The following study compares some of the modern image segmentation techniques and discusses their shortcomings with respect to the stated context. A novel statistical method is also introduced in an attempt to improve the resilience of vision-based gesture segmentation. I. INTRODUCTION With the advent of more powerful computing facilities and advances in artificial intelligence, few techniques have been proposed in the evaluation of the complex movements inherent in the performances of musicians and athletes. Such techniques are highly desirable in order to detect problematic situations which often lead to chronic injuries and to provide the capacity with which professors and trainers may measure the evolution and habits of individuals. Gesture analysis provides the means with which quantitative measurements may be obtained with respect to the physical performances of musicians and athletes. Modern gesture analysis methodologies employed by professionals fail to obtain an exhaustive evaluation or a complete comparison between gestures. Current techniques in this type of analysis are not robust enough to operate in the unconstrained environments in which these performances must be evaluated. Many techniques today still rely on the use of encumbering sensor technologies and require the use of cabling or attaching markers on an individual. The techniques also often require a very controlled environment and involve tedious, expensive and complex operating methodologies. The imposed environments are usually foreign to the musicians or athletes resulting in an impeded performance and corruption to the exactitude of the measurements. The undertaken study aims to research and develop new vision-based methodologies suitable for the analysis of musician gestures. Specifically the context being explored consists of analyzing the gestures and postures of pianists with applications to piano pedagogy. Computer vision techniques offer the ideal means for this type of evaluation. Due to their non-invasive nature they do not interfere with the performance of pianists. Computer vision techniques are put to the test in a context where the imposed constraints on the environment and individuals must remain minimal.

Assisted Piano Pedagogy through 3D Visualization of Piano Playing

Having a correct posture at the piano requires a lot of practice to master; however, a visual feedback can help students realize potential problems and adopt a better position when playing the instrument. This paper discusses an innovative application of the techniqes used for the 3D visualization of piano performances in any possible view, with the purpose of comparing them to an ideal piano playing. It includes the capture and reconstruction of the 3D motion and posture of a professional piano player so that it may be compared against the posture and movements of students, by overlaying 2D videos of their recital. Issues with respect to displaying data and adding useful interface features are also discussed. The motion can be adjusted to fit measurements of anthropometrical different students, so that learners can see themselves and better understand what position they should adopt while they are at the piano

Work ethic, motivation, and parental influences in Chinese and North American children learning to play the piano:

This study compared 50 Chinese and 100 North American Caucasian children aged 6 to 17 who were learning piano, in terms of their work ethic, motivation, and parental influences. Compared to North American Caucasians, Chinese children and parents believed more strongly that musical ability requires hard work, and Chinese children were more interested in working hard at piano practice, and practiced nearly twice as much. We also found differences in autonomous motivation, as defined by Self-Determination Theory: compared to Caucasians, Chinese children identified more with playing the piano, found it more intrinsically enjoyable, and pressured themselves less by shame or guilt, though they were more motivated by a desire to please their teachers and parents. Furthermore, Chinese parents more frequently sat in on their child’s piano lessons. These findings suggest several reasons that may contribute to the success of Chinese musicians.

3D visual feedback in learning of piano posture

An overlooked, yet essential problem in piano practice involves body posture. Bending the back, leaning towards the keys, lifting the wrist and other improper positions are usually noticed by piano teachers when instructing their students. If they sit too low, fingers will feel the burden of static arm weight; if they sit too high, the playing might become rigid with tension in wrist and shoulders. In this manner, visual feedback can help in realizing potential problems when playing the instrument. This paper discusses an innovative application of the techniques used for the 3D visualization of piano performances in any possible view, with the purpose of comparing them to a good model of someone playing the piano. It includes the capture and reconstruction of the 3D motion and posture of a professional piano player so that it may be compared against the posture and movements of students, by overlaying 2D videos of their practice session at the piano.

Human gesture quantification: An evaluation tool for somatic training and piano performance

Motion capture technology has been widely adopted by computer gaming and the movie making industries. The technology also found a suitable niche in some areas of medicine. However, while sports and arts also present important opportunities for analysis of the human gesture, whether to improve performance or prevent injuries, the potential of motion capture was not yet fully exploited in these fields. This paper examines the potential of markerless motion capture in the context of piano pedagogy and performance evaluation when somatic training methods are used. An affordable motion capture platform based on Kinect sensor is designed to suit the specific requirements of piano playing, and an early experimental investigation of the suitability of the technology for this application is reported.

Assessment of piano-related injuries using infrared imaging

Playing the piano is a repetitive task that involves the use of the hands and the arms. Pain related to piano-playing can result in extending the tissues and ligaments of the hands and arms beyond their mechanical tolerance. Infrared imaging records the skin temperature and produces a thermal map of the imaged body part; small variations in the skin temperature could be a sign of inflammation or stress of the tissues. In this paper, we used statistical analysis to examine the difference in hand and arm temperatures of pianists with pain and pianists without pain related to piano-playing. We found that there is a statistically significant difference in hand temperatures between the two populations, but not in the lower arm and upper arm temperatures.

Video segmentation for markerless motion capture in unconstrained environments

Segmentation is a first and important step in video-based motion capture applications. A lack of constraints can make this process daunting and difficult to achieve. We propose a technique that makes use of an improved JSEG procedure in the context of markerless motion capture for performance evaluation of human beings in unconstrained environments. In the proposed algorithm a non-parametric clustering of image data is performed in order to produce homogenous colour-texture regions. The clusters are modified using soft - classifications and allow the J-Value segmentation to deal with smooth colour and lighting transitions. The regions are adapted using an original merging and video stack tracking algorithm.

The value of reflective journaling with advanced piano students

Abstract Benefits to using a reflective journal include developing critical awareness and new perspective, problem-solving skills, and independent learning skills. The training of advanced piano students could be enhanced by the addition of journals to assist with their piano practice, specifically when learning new repertoire. Using the model by Plack and colleagues for developing and assessing reflection in reflective journal entries, we examined the journal entries of 18 advanced piano students to explore the development of reflection over a period of four weeks. Results suggest that reflectively trained piano students develop more critical reflection compared to a control group. Reflectively trained students perceive the same benefits to journaling as their counterparts in other disciplines. The most frequently referenced reflective elements include listing practice strategies and expressing feelings about the learning process. The reflective training model used in this study can be implemented by piano teachers and piano students.

Cognitive modelling of early music reading skill acquisition for piano: A comparison of the Middle-C and Intervallic methods

In the classical music tradition, knowing how to read music is an essential skill and is seen as a fundamental component to develop when learning to play the piano. This researchs focus is to study the possible impact of two different teaching approaches to the acquisition of initial reading skills. By using cognitive modeling, we are hoping to observe through computer simulation the problem solving and decision-making tasks involved in decoding a simple musical score. Our model intends to capture a novice initial coordination of music reading and motor operations on a piano keyboard. As such, it does not aim at modelling advanced sight-reading skills. The paper introduces the Middle-C and Intervallic methods followed by a description of an ACT-R cognitive model and simulation results upon learning with each of the reading methods. Inspection of the simulation results reveals differences in terms of declarative memory and cognitive processing demands. In particular, the Intervallic method requires a larger number of declarative knowledge related to notes, and more execution planning than the Middle-C method.