Cecily Morrison

Quantifying Progression of Multiple Sclerosis via Classification of Depth Videos

This paper presents new learning-based techniques for measuring disease progression in Multiple Sclerosis (MS) patients. Our system aims to augment conventional neurological examinations by adding quantitative evidence of disease progression. An off-the-shelf depth camera is used to image the patient at the examination, during which he/she is asked to perform carefully selected movements. Our algorithms then automatically analyze the videos, assessing the quality of each movement and classifying them as healthy or non-healthy. Our contribution is three-fold: We i) introduce ensembles of randomized SVM classifiers and compare them with decision forests on the task of depth video classification; ii) demonstrate automatic selection of discriminative landmarks in the depth videos, showing their clinical relevance; iii) validate our classification algorithms quantitatively on a new dataset of 1041 videos of both MS patients and healthy volunteers. We achieve average Dice scores well in excess of the 80% mark, confirming the validity of our approach in practical applications. Our results suggest that this technique could be fruitful for depth-camera supported clinical assessments for a range of conditions.

Usability and Acceptability of ASSESS MS: Assessment of Motor Dysfunction in Multiple Sclerosis Using Depth-Sensing Computer Vision

Background Sensor-based recordings of human movements are becoming increasingly important for the assessment of motor symptoms in neurological disorders beyond rehabilitative purposes. ASSESS MS is a movement recording and analysis system being developed to automate the classification of motor dysfunction in patients with multiple sclerosis (MS) using depth-sensing computer vision. It aims to provide a more consistent and finer-grained measurement of motor dysfunction than currently possible. Objective To test the usability and acceptability of ASSESS MS with health professionals and patients with MS. Methods A prospective, mixed-methods study was carried out at 3 centers. After a 1-hour training session, a convenience sample of 12 health professionals (6 neurologists and 6 nurses) used ASSESS MS to capture recordings of standardized movements performed by 51 volunteer patients. Metrics for effectiveness, efficiency, and acceptability were defined and used to analyze data captured by ASSESS MS, video recordings of each examination, feedback questionnaires, and follow-up interviews. Results All health professionals were able to complete recordings using ASSESS MS, achieving high levels of standardization on 3 of 4 metrics (movement performance, lateral positioning, and clear camera view but not distance positioning). Results were unaffected by patients’ level of physical or cognitive disability. ASSESS MS was perceived as easy to use by both patients and health professionals with high scores on the Likert-scale questions and positive interview commentary. ASSESS MS was highly acceptable to patients on all dimensions considered, including attitudes to future use, interaction (with health professionals), and overall perceptions of ASSESS MS. Health professionals also accepted ASSESS MS, but with greater ambivalence arising from the need to alter patient interaction styles. There was little variation in results across participating centers, and no differences between neurologists and nurses. Conclusions In typical clinical settings, ASSESS MS is usable and acceptable to both patients and health professionals, generating data of a quality suitable for clinical analysis. An iterative design process appears to have been successful in accounting for factors that permit ASSESS MS to be used by a range of health professionals in new settings with minimal training. The study shows the potential of shifting ubiquitous sensing technologies from research into the clinic through a design approach that gives appropriate attention to the clinic environment.

Vision-based body tracking: turning Kinect into a clinical tool

Abstract Purpose: Vision-based body tracking technologies, originally developed for the consumer gaming market, are being repurposed to form the core of a range of innovative healthcare applications in the clinical assessment and rehabilitation of movement ability. Vision-based body tracking has substantial potential, but there are technical limitations. Method: We use our “stories from the field” to articulate the challenges and offer examples of how these can be overcome. Results: We illustrate that: (i) substantial effort is needed to determine the measures and feedback vision-based body tracking should provide, accounting for the practicalities of the technology (e.g. range) as well as new environments (e.g. home). (ii) Practical considerations are important when planning data capture so that data is analysable, whether finding ways to support a patient or ensuring everyone does the exercise in the same manner. (iii) Home is a place of opportunity for vision-based body tracking, but what we do now in the clinic (e.g. balance tests) or in the home (e.g. play games) will require modifications to achieve capturable, clinically relevant measures. Conclusions: This article articulates how vision-based body tracking works and when it does not to continue to inspire our clinical colleagues to imagine new applications. Implications for Rehabilitation Vision-based body tracking has quickly been repurposed to form the core of innovative healthcare applications in clinical assessment and rehabilitation, but there are clinical as well as practical challenges to make such systems a reality. Substantial effort needs to go into determining what types of measures and feedback vision-based body tracking should provide. This needs to account for the practicalities of the technology (e.g. range) as well as the opportunities of new environments (e.g. the home). Practical considerations need to be accounted for when planning capture in a particular environment so that data is analysable, whether it be finding a chair substitute, ways to support a patient or ensuring everyone does the exercise in the same manner. The home is a place of opportunity with vision-based body tracking, but it would be naïve to think that we can do what we do now in the clinic (e.g. balance tests) or in the home (e.g. play games), without appropriate modifications to what constitutes a practically capturable, clinically relevant measure.

Setwise Comparison: Consistent, Scalable, Continuum Labels for Computer Vision

A growing number of domains, including affect recognition and movement analysis, require a single, real number ground truth label capturing some property of a video clip. We term this the provision of continuum labels. Unfortunately, there is often an uncacceptable trade-off between label consistency and the efficiency of the labelling process with current tools. We present a novel interaction technique, setwise comparison, which leverages the intrinsic human capability for consistent relative judgements and the TrueSkill algorithm to solve this problem. We describe SorTable, a system demonstrating this technique. We conducted a real-world study where clinicians labelled videos of patients with multiple sclerosis for the ASSESS MS computer vision system. In assessing the efficiency-consistency trade-off of setwise versus pairwise comparison, we demonstrated that not only is setwise comparison more efficient, but it also elicits more consistent labels. We further consider how our findings relate to the interactive machine learning literature.

Assessing Multiple Sclerosis With Kinect: Designing Computer Vision Systems for Real-World Use

The use of depth-sensing computer vision to capture bodily movement is increasingly being exploited in healthcare. Yet, there are few descriptions of how real-world practices influence the design of such applications. To this end, we present the development and empirical evaluation of ASSESS MS, a system to support the clinical assessment of Multiple Sclerosis using Kinect. A key issue for developing machine-learning based systems is the need for standardized data on which statistical inferences can be made. We demonstrate that there are many aspects of clinical practice that are at odds with the need to capture standardized data for a computer vision system. We offer three design guidelines so address these: 1) Standardization is a multi-disciplinary issue and needs to be addressed early in the development process; 2) Tools that provide a view into what the camera “sees” can support the achievement of standardized data capture in real environments; 3) Tools to support standardized data capture should maintain the agency of human interaction. More broadly we show that when considering every day contexts, the traditional focus on measurement accuracy is only a small part of the effort needed to make a technology “work” in practice.

Improving Indoor Mobility of the Visually Impaired with Depth-Based Spatial Sound

We present a novel system to help visually impaired people to move efficiently and safely in indoor environments by mapping input from a depth camera to spatially localized auditory cues. We propose a set of context-specific cues which are suitable for use in systems that provide minimal audio feedback and hence reduce masking of natural sounds compared to the audio provided by general-purpose sense substitution devices. Using simple but effective heuristics for detecting the floor and the side walls, we propose auditory cues that encode information about the distances to walls, obstacles, the orientation of the corridor or room, and openings into corridors or rooms. But the key to our system is the use of a spatial sound engine that localizes the generated sounds in 3D. We evaluate our system, comparing with MeloSee. Our preliminary pilot study with ten blindfolded participants suggests that our system was more helpful for spotting smaller obstacles on the floor, though neither system had a significant edge in terms of walking speed or safety.

Body Tracking in Healthcare

Within the context of healthcare, there has been a long-standing interest in understanding the posture and movement of the human body. Gait analysis work over the years has looked to articulate the patterns and parameters of this movement both for a normal healthy body and in a range of movement-based disorders. In recent years, these efforts to understand the moving body have been transformed by significant advances in sensing technologies and computational analysis techniques all offering new ways for the moving body to be tracked, measured, and interpreted. While much of this work has been largely research focused, as the field matures, we are seeing more shifts into clinical practice. As a consequence, there is an increasing need to understand these sensing technologies over and above the specific capabilities to track, measure, and infer patterns of movement in themselves. Rather, there is an imperative to understand how the material form of these technologies enables them also to be situated in everyday healthcare contexts and practices. There are significant mutually interdependent ties between the fundamental characteristics and assumptions of these technologies and the configurations of everyday collaborative practices that are possible them. Our attention then must look to social, clinical, and technical relations pertaining to these various body technologies that may play out in particular ways across a range of different healthcare contexts and stakeholders. Our aim in this book is to explore these issues with key examples illustrating how social contexts of use relate to the properties and assumptions bound up in particular choices of body-tracking technology. We do this through a focus on three core application areas in healthcare-assessment, rehabilitation, and surgical interaction-and recent efforts to apply body-tracking technologies to them.

Reference videos reduce variability of motor dysfunction assessments in multiple sclerosis

Motor dysfunction, particularly ataxia, is one of the predominant clinical manifestations in patients with multiple sclerosis (MS). Assessment of motor dysfunction suffers from a high variability. We investigated whether the clinical rating of ataxia can be improved through the use of reference videos, covering the spectrum of severity degrees as defined in the Neurostatus-Expanded Disability Status Scale. Twenty-five neurologists participated. The variability of their assessments was significantly lower when reference videos were used (SD = 0.12; range = 0.40 vs SD = 0.26; range = 0.88 without reference videos; p = 0.013). Reference videos reduced the variability of clinical assessments and may be useful tools to improve the precision and consistency in the clinical assessment of motor functions in MS.

Visualizing Ubiquitously Sensed Measures of Motor Ability in Multiple Sclerosis: Reflections on Communicating Machine Learning in Practice

Sophisticated ubiquitous sensing systems are being used to measure motor ability in clinical settings. Intended to augment clinical decision-making, the interpretability of the machine-learning measurements underneath becomes critical to their use. We explore how visualization can support the interpretability of machine-learning measures through the case of Assess MS, a system to support the clinical assessment of Multiple Sclerosis. A substantial design challenge is to make visible the algorithms decision-making process in a way that allows clinicians to integrate the algorithms result into their own decision process. To this end, we present a series of design iterations that probe the challenges in supporting interpretability in a real-world system. The key contribution of this article is to illustrate that simply making visible the algorithmic decision-making process is not helpful in supporting clinicians in their own decision-making process. It disregards that people and algorithms make decisions in different ways. Instead, we propose that visualisation can provide context to algorithmic decision-making, rendering observable a range of internal workings of the algorithm from data quality issues to the web of relationships generated in the machine-learning process.

BrightSelf: A Qualitative Design Study of a Mobile Application for the Self-Report of Psychological Wellbeing in Pregnancy (Preprint)

Background Maternal mental health impacts both parental well-being and childhood development. In the United Kingdom, 15% of women are affected by depression during pregnancy or within 1 year of giving birth. Suicide is a leading cause of perinatal maternal mortality, and it is estimated that >50% of perinatal depression cases go undiagnosed. Mobile technologies are potentially valuable tools for the early recognition of depressive symptoms, but complex design challenges must be addressed to enable their use in public health screening. Objective The aim of this study was to explore the issues and challenges surrounding the use of mobile phones for the self-report of psychological well-being during pregnancy. Methods This paper presents design research carried out as part of the development of BrightSelf, a mobile app for the self-report of psychological well-being during pregnancy. Design sessions were carried out with 38 participants, including pregnant women, mothers, midwives, and other health professionals. Overall, 19 hours of audio were fully transcribed and used as the basis of thematic analysis. Results The study highlighted anxieties concerning the pregnancy journey, challenges surrounding current approaches to the appraisal of well-being in perinatal care, and the midwife-patient relationship. Designers should consider the framing of perinatal mental health technologies, the experience of self-report, supporting self-awareness and disclosure, providing value to users through both self-report and supplementary features, and designing for longitudinal engagement. Conclusions This study highlights the needs, motivations, and anxieties of women with respect to technology use in pregnancy and implications for the design of mobile health technologies.