Jessica Burggraaff

Comparing long-term results of PASAT and SDMT scores in relation to neuropsychological testing in multiple sclerosis

Background and objectives: The Symbol Digit Modalities Test (SDMT) shows advantages over the Paced Auditory Serial Addition Test (PASAT) as a cognitive test in patients with multiple sclerosis (MS). To determine which of these tests is most valid and reliable over time as an indicator of the cognitive state of MS patients, long-term test results of both tests were compared in relation to scores of the Brief Repeatable Battery of Neuropsychological tests (BRBN). Methods: For 485 MS patients visiting the VU University Medical Center Amsterdam for different research projects, a total number of 1078 visits with BRBN (including PASAT and SDMT) was planned. Observed and model-based correlations were used to calculate the construct validity of the SDMT and PASAT 3 seconds test (PASAT3) by comparing correlations with the BRBN-sumscore. The test-retest reliability of each test was also computed. Results: For the construct validity, higher correlations were found between SDMT and BRBN compared to PASAT3 and BRBN, especially for the model-based correlations at baseline. The reliability of the measurements was good for all instruments, with the highest coefficients for the SDMT. Conclusion: As a single assessment tool for cognition in MS, the SDMT is more valid and reliable compared to PASAT3.

Quantifying Progression of Multiple Sclerosis via Classification of Depth Videos

This paper presents new learning-based techniques for measuring disease progression in Multiple Sclerosis (MS) patients. Our system aims to augment conventional neurological examinations by adding quantitative evidence of disease progression. An off-the-shelf depth camera is used to image the patient at the examination, during which he/she is asked to perform carefully selected movements. Our algorithms then automatically analyze the videos, assessing the quality of each movement and classifying them as healthy or non-healthy. Our contribution is three-fold: We i) introduce ensembles of randomized SVM classifiers and compare them with decision forests on the task of depth video classification; ii) demonstrate automatic selection of discriminative landmarks in the depth videos, showing their clinical relevance; iii) validate our classification algorithms quantitatively on a new dataset of 1041 videos of both MS patients and healthy volunteers. We achieve average Dice scores well in excess of the 80% mark, confirming the validity of our approach in practical applications. Our results suggest that this technique could be fruitful for depth-camera supported clinical assessments for a range of conditions.

Usability and Acceptability of ASSESS MS: Assessment of Motor Dysfunction in Multiple Sclerosis Using Depth-Sensing Computer Vision

Background Sensor-based recordings of human movements are becoming increasingly important for the assessment of motor symptoms in neurological disorders beyond rehabilitative purposes. ASSESS MS is a movement recording and analysis system being developed to automate the classification of motor dysfunction in patients with multiple sclerosis (MS) using depth-sensing computer vision. It aims to provide a more consistent and finer-grained measurement of motor dysfunction than currently possible. Objective To test the usability and acceptability of ASSESS MS with health professionals and patients with MS. Methods A prospective, mixed-methods study was carried out at 3 centers. After a 1-hour training session, a convenience sample of 12 health professionals (6 neurologists and 6 nurses) used ASSESS MS to capture recordings of standardized movements performed by 51 volunteer patients. Metrics for effectiveness, efficiency, and acceptability were defined and used to analyze data captured by ASSESS MS, video recordings of each examination, feedback questionnaires, and follow-up interviews. Results All health professionals were able to complete recordings using ASSESS MS, achieving high levels of standardization on 3 of 4 metrics (movement performance, lateral positioning, and clear camera view but not distance positioning). Results were unaffected by patients’ level of physical or cognitive disability. ASSESS MS was perceived as easy to use by both patients and health professionals with high scores on the Likert-scale questions and positive interview commentary. ASSESS MS was highly acceptable to patients on all dimensions considered, including attitudes to future use, interaction (with health professionals), and overall perceptions of ASSESS MS. Health professionals also accepted ASSESS MS, but with greater ambivalence arising from the need to alter patient interaction styles. There was little variation in results across participating centers, and no differences between neurologists and nurses. Conclusions In typical clinical settings, ASSESS MS is usable and acceptable to both patients and health professionals, generating data of a quality suitable for clinical analysis. An iterative design process appears to have been successful in accounting for factors that permit ASSESS MS to be used by a range of health professionals in new settings with minimal training. The study shows the potential of shifting ubiquitous sensing technologies from research into the clinic through a design approach that gives appropriate attention to the clinic environment.

Setwise Comparison: Consistent, Scalable, Continuum Labels for Computer Vision

A growing number of domains, including affect recognition and movement analysis, require a single, real number ground truth label capturing some property of a video clip. We term this the provision of continuum labels. Unfortunately, there is often an uncacceptable trade-off between label consistency and the efficiency of the labelling process with current tools. We present a novel interaction technique, setwise comparison, which leverages the intrinsic human capability for consistent relative judgements and the TrueSkill algorithm to solve this problem. We describe SorTable, a system demonstrating this technique. We conducted a real-world study where clinicians labelled videos of patients with multiple sclerosis for the ASSESS MS computer vision system. In assessing the efficiency-consistency trade-off of setwise versus pairwise comparison, we demonstrated that not only is setwise comparison more efficient, but it also elicits more consistent labels. We further consider how our findings relate to the interactive machine learning literature.

Impact of 3 Tesla MRI on interobserver agreement in clinically isolated syndrome: A MAGNIMS multicentre study

Background: Compared to 1.5 T, 3 T magnetic resonance imaging (MRI) increases signal-to-noise ratio leading to improved image quality. However, its clinical relevance in clinically isolated syndrome suggestive of multiple sclerosis remains uncertain. Objectives: The purpose of this study was to investigate how 3 T MRI affects the agreement between raters on lesion detection and diagnosis. Methods: We selected 30 patients and 10 healthy controls from our ongoing prospective multicentre cohort. All subjects received baseline 1.5 and 3 T brain and spinal cord MRI. Patients also received follow-up brain MRI at 3–6 months. Four experienced neuroradiologists and four less-experienced raters scored the number of lesions per anatomical region and determined dissemination in space and time (McDonald 2010). Results: In controls, the mean number of lesions per rater was 0.16 at 1.5 T and 0.38 at 3 T (p = 0.005). For patients, this was 4.18 and 4.40, respectively (p = 0.657). Inter-rater agreement on involvement per anatomical region and dissemination in space and time was moderate to good for both field strengths. 3 T slightly improved agreement between experienced raters, but slightly decreased agreement between less-experienced raters. Conclusion: Overall, the interobserver agreement was moderate to good. 3 T appears to improve the reading for experienced readers, underlining the benefit of additional training.

Three-Tesla MRI does not improve the diagnosis of multiple sclerosis: A multicenter study

Objective In the work-up of patients presenting with a clinically isolated syndrome (CIS), 3T MRI might offer a higher lesion detection than 1.5T, but it remains unclear whether this affects the fulfilment of the diagnostic criteria for multiple sclerosis (MS). Methods We recruited 66 patients with CIS within 6 months from symptom onset and 26 healthy controls in 6 MS centers. All participants underwent 1.5T and 3T brain and spinal cord MRI at baseline according to local optimized protocols and the MAGNIMS guidelines. Patients who had not converted to MS during follow-up received repeat brain MRI at 3–6 months and 12–15 months. The number of lesions per anatomical region was scored by 3 raters in consensus. Criteria for dissemination in space (DIS) and dissemination in time (DIT) were determined according to the 2017 revisions of the McDonald criteria. Results Three-Tesla MRI detected 15% more T2 brain lesions compared to 1.5T (p < 0.001), which was driven by an increase in baseline detection of periventricular (12%, p = 0.015), (juxta)cortical (21%, p = 0.005), and deep white matter lesions (21%, p < 0.001). The detection rate of spinal cord lesions and gadolinium-enhancing lesions did not differ between field strengths. Three-Tesla MRI did not lead to a higher number of patients fulfilling the criteria for DIS or DIT, or subsequent diagnosis of MS, at any of the 3 time points. Conclusion Scanning at 3T does not influence the diagnosis of MS according to McDonald diagnostic criteria.

Reference videos reduce variability of motor dysfunction assessments in multiple sclerosis

Motor dysfunction, particularly ataxia, is one of the predominant clinical manifestations in patients with multiple sclerosis (MS). Assessment of motor dysfunction suffers from a high variability. We investigated whether the clinical rating of ataxia can be improved through the use of reference videos, covering the spectrum of severity degrees as defined in the Neurostatus-Expanded Disability Status Scale. Twenty-five neurologists participated. The variability of their assessments was significantly lower when reference videos were used (SD = 0.12; range = 0.40 vs SD = 0.26; range = 0.88 without reference videos; p = 0.013). Reference videos reduced the variability of clinical assessments and may be useful tools to improve the precision and consistency in the clinical assessment of motor functions in MS.

Tasks of activities of daily living (ADL) are more valuable than the classical neurological examination to assess upper extremity function and mobility in multiple sclerosis

Background: Accurate clinical assessment in multiple sclerosis (MS) is challenging. The Assess MS system is being developed to automatically quantify motor dysfunction in MS, including upper extremity function (UEF) and mobility. Objective: To determine to what extent combinations of standardized movements included in the Assess MS system explain accepted measures of UEF and mobility. Methods: MS patients were recruited at four European MS centres. Eight movements were selected, including tasks of activities of daily living (ADL) and classical neurological tests. Movements were recorded on video and rated by experienced neurologists (n = 5). Subsequently, multivariate linear regression models were performed to explain the variance of the Nine-Hole Peg Test (9HPT), Arm Function in Multiple Sclerosis Questionnaire (AMSQ) and Timed-25 Foot Walk test (T25WT). Results: In total, 257 patients were included. The movements explained 62.9% to 80.1% of the variance of the 9HPT models, 43.3% and 44.3% of the AMSQ models and 70.8% of the T25WT. In all models, tasks of ADL contributed most to the variance. Conclusion: Combinations of movements are valuable to assess UEF and mobility. Incorporating ADL tasks into daily clinical practice and clinical trials may be more valuable than the classical neurological examination of UEF and mobility.