Jeremy Hobart

Measuring the impact of MS on walking ability The 12-Item MS Walking Scale (MSWS-12)

Objective: To develop a patient-based measure of walking ability in MS. Methods: Twelve items describing the impact of MS on walking (12-Item MS Walking Scale [MSWS-12]) were generated from 30 patient interviews, expert opinion, and literature review. Preliminary psychometric evaluation (data quality, scaling assumptions, acceptability, reliability, validity) was undertaken in the data generated by 602 people from the MS Society membership database. Further psychometric evaluation (including comprehensive validity assessment, responsiveness, and relative efficiency) was conducted in two hospital-based samples: people with primary progressive MS (PPMS; n = 78) and people with relapses admitted for IV steroid treatment (n = 54). Results: In all samples, missing data were low (≤3.8%), item test–retest reproducibility was high (≥0.78), scaling assumptions were satisfied, and reliability was high (≥0.94). Correlations between the MSWS-12 and other scales were consistent with a priori hypotheses. The MSWS-12 (relative efficiency = 1.0) was more responsive than the Functional Assessment of Multiple Sclerosis mobility scale (0.72), the 36-Item Short Form Health Survey physical functioning scale (0.33), the Expanded Disability Status Scale (0.03), the 25-ft Timed Walk Test (0.44), and Guy’s Neurologic Disability Scale lower limb disability item (0.10). Conclusions: The MSWS-12 satisfies standard criteria as a reliable and valid patient-based measure of the impact of MS on walking. In these samples, the MSWS-12 was more responsive than other walking-based scales.

Rating scales as outcome measures for clinical trials in neurology: problems, solutions, and recommendations

Have state-of-the-art clinical trials failed to deliver treatments for neurodegenerative diseases because of shortcomings in the rating scales used? This Review assesses two methodological limitations of rating scales that might help to answer this question. First, the numbers generated by most rating scales do not satisfy the criteria for rigorous measurements. Second, we do not really know which variables most rating scales measure. We use clinical examples to highlight concerns about the limitations of rating scales, examine their underlying rationales, clarify their implications, explore potential solutions, and make some recommendations for future research. We show that improvements in the scientific rigour of rating scales can improve the chances of reaching the correct conclusions about the effectiveness of treatments.

Measuring change in disability after inpatient rehabilitation: comparison of the responsiveness of the Barthel Index and the Functional Independence Measure

BACKGROUND The importance of evaluating disability outcome measures is well recognised. The Functional Independence Measure (FIM) was developed to be a more comprehensive and “sensitive” measure of disability than the Barthel Index (BI). Although the FIM is widely used and has been shown to be reliable and valid, there is limited information about its responsiveness, particularly in comparison with the BI. This study compares the appropriateness and responsiveness of these two disability measures in patients with multiple sclerosis and stroke. METHODS Patients with multiple sclerosis (n=201) and poststroke (n=82) patients undergoing inpatient neurorehabilitation were studied. Admission and discharge scores were generated for the BI and the three scales of the FIM (total, motor, and cognitive). Appropriateness of the measures to the study samples was determined by examining score distributions, floor and ceiling effects. Responsiveness was determined using an effect size calculation. RESULTS The BI, FIM total, and FIM motor scales show good variability and have small floor and ceiling effects in the study samples. The FIM cognitive scale showed a notable ceiling effect in patients with multiple sclerosis. Comparable effect sizes were found for the BI, and two FIM scales (total and motor) in both patients with multiple sclerosis and stroke patients. CONCLUSION All measures were appropriate to the study sample. The FIM cognitive scale, however, has limited usefulness as an outcome measure in progressive multiple sclerosis. The BI, FIM total, and FIM motor scales show similar responsiveness, suggesting that both the FIM total and FIM motor scales have no advantage over the BI in evaluating change.

Improving the evaluation of therapeutic interventions in multiple sclerosis: the role of new psychometric methods.

OBJECTIVES In this monograph we examine the added value of new psychometric methods (Rasch measurement and Item Response Theory) over traditional psychometric approaches by comparing and contrasting their psychometric evaluations of existing sets of rating scale data. We have concentrated on Rasch measurement rather than Item Response Theory because we believe that it is the more advantageous method for health measurement from a conceptual, theoretical and practical perspective. Our intention is to provide an authoritative document that describes the principles of Rasch measurement and the practice of Rasch analysis in a clear, detailed, non-technical form that is accurate and accessible to clinicians and researchers in health measurement. REVIEW METHODS A comparison was undertaken of traditional and new psychometric methods in five large sets of rating scale data: (1) evaluation of the Rivermead Mobility Index (RMI) in data from 666 participants in the Cannabis in Multiple Sclerosis (CAMS) study; (2) evaluation of the Multiple Sclerosis Impact Scale (MSIS-29) in data from 1725 people with multiple sclerosis; (3) evaluation of test-retest reliability of MSIS-29 in data from 150 people with multiple sclerosis; (4) examination of the use of Rasch analysis to equate scales purporting to measure the same health construct in 585 people with multiple sclerosis; and (5) comparison of relative responsiveness of the Barthel Index and Functional Independence Measure in data from 1400 people undergoing neurorehabilitation. RESULTS Both Rasch measurement and Item Response Theory are conceptually and theoretically superior to traditional psychometric methods. Findings from each of the five studies show that Rasch analysis is empirically superior to traditional psychometric methods for evaluating rating scales, developing rating scales, analysing rating scale data, understanding and measuring stability and change, and understanding the health constructs we seek to quantify. CONCLUSIONS There is considerable added value in using Rasch analysis rather than traditional psychometric methods in health measurement. Future research directions include the need to reproduce our findings in a range of clinical populations, detailed head-to-head comparisons of Rasch analysis and Item Response Theory, and the application of Rasch analysis to clinical practice.

Quality of Life Measurement After Stroke Uses and Abuses of the SF-36

Background and Purpose— The Medical Outcomes Study 36-item Short-Form Health Survey (SF-36) is widely used to measure health status after stroke. However, a fundamental assumption for its valid use after stroke has not been comprehensively tested: is it legitimate to generate scores for 8 scales and 2 summary measures using the standard algorithms? We tested this assumption. Methods— SF-36 data from 177 people after stroke were examined (71% male; mean age, 62). We tested 6 scaling criteria to determine the legitimacy of generating the 8 SF-36 scale scores using Likert’s method of summed ratings, and we tested 2 scaling criteria to determine the appropriateness of the standard SF-36 algorithms for weighting and combining scale scores to generate 2 summary measures (physical and mental). Results— Scaling assumptions were fully satisfied for 6 of the 8 scales, but 3 of these 6 scales had notable floor and/or ceiling effects. Assumptions for generating 2 SF-36 summary measures were not satisfied. Conclusions— In this sample, 5 of the 8 SF-36 scales had limited validity as outcome measures after stroke, and the reporting of physical and mental summary scores was not supported. Results raise questions about the use of the SF-36 in stroke, and the SF-12 that is developed from it, and highlight the importance of testing scaling assumptions when applying existing scales to new populations.

Evidence-based measurement Which disability scale for neurologic rehabilitation?

Objective: To compare the 10-item Barthel Index (BI), 18-item Functional Independence Measure (FIM), and 30-item Functional Independence Measure + Functional Assessment Measure (FIM+FAM) as measures of disability outcomes for neurologic rehabilitation. Methods: A total of 149 inpatients from two rehabilitation units in South England specializing in neurologic disorders were studied. Traditional psychometric methods were used to evaluate and compare acceptability (score distributions), reliability (internal consistency, intrarater reproducibility), validity (concurrent, convergent and discriminant construct), and responsiveness (standardized response mean). Results: All three rating scales satisfied recommended criteria for reliable and valid measurement of disability, and are acceptable and responsive in this study sample. The FIM and FIM+FAM total scales are psychometrically similar measures of global disability. The BI, FIM, and FIM+FAM motor scales are psychometrically similar measures of physical disability. The FIM and FIM+FAM cognitive scales are psychometrically similar measures of physical disability. Conclusions: In the sample studied, the BI, FIM, FIM+FAM have similar measurement properties, when examined using traditional psychometric analyses. Although instruments with more items and item response categories generate more qualitative information about an outcome, they may not improve its measurement. Results highlight the importance of using recognized techniques of scale construction to develop health outcome measures.

Cannabis for dyskinesia in Parkinson disease A randomized double-blind crossover study

Background: The long-term treatment of Parkinson disease (PD) may be complicated by the development of levodopa-induced dyskinesia. Clinical and animal model data support the view that modulation of cannabinoid function may exert an antidyskinetic effect. The authors conducted a randomized, double-blind, placebo-controlled crossover trial to examine the hypothesis that cannabis may have a beneficial effect on dyskinesia in PD. Methods: A 4-week dose escalation study was performed to assess the safety and tolerability of cannabis in six PD patients with levodopa-induced dyskinesia. Then a randomized placebo-controlled crossover study (RCT) was performed, in which 19 PD patients were randomized to receive oral cannabis extract followed by placebo or vice versa. Each treatment phase lasted for 4 weeks with an intervening 2-week washout phase. The primary outcome measure was a change in Unified Parkinson’s Disease Rating Scale (UPDRS) (items 32 to 34) dyskinesia score. Secondary outcome measures included the Rush scale, Bain scale, tablet arm drawing task, and total UPDRS score following a levodopa challenge, as well as patient-completed measures of a dyskinesia activities of daily living (ADL) scale, the PDQ-39, on-off diaries, and a range of category rating scales. Results: Seventeen patients completed the RCT. Cannabis was well tolerated, and had no pro- or antiparkinsonian action. There was no evidence for a treatment effect on levodopa-induced dyskinesia as assessed by the UPDRS, or any of the secondary outcome measures. Conclusions: Orally administered cannabis extract resulted in no objective or subjective improvement in dyskinesias or parkinsonism.

Inpatient rehabilitation in multiple sclerosis Do the benefits carry over into the community

Objective: To determine the duration and pattern of carry-over of benefits gained after a short period of multidisciplinary inpatient rehabilitation. Background: Few studies have evaluated the outcome of rehabilitation after discharge. Long-term follow-up is required to establish whether gains made during the inpatient stay are sustained over time and in the patient’s own environment. Methods: Prospective single-group longitudinal study. Fifty consecutive patients with progressive MS undergoing inpatient rehabilitation were followed for 12 months after discharge. Assessments were undertaken on admission (A), at discharge, and subsequently at 3-month intervals for 1 year (1Y) with a battery of measures addressing neurologic status, disability, handicap, quality of life, and emotional well-being. The time taken to return to baseline level was calculated using summary measures, and trends in performance levels were plotted. Results: Twelve-month data were collected for 92% of patients. Although neurologic status declined (median Expanded Disability Status Scale scores: A = 6.8, 1Y = 8.0), improvements were maintained in disability and handicap for 6 months, emotional well-being for 7 months, and health-related quality of life (physical component) for 10 months. Conclusions: The benefits gained from rehabilitation were partly maintained after discharge despite worsening neurologic status. Carry-over of benefits, however, declined over time, reinforcing the need for continuity of care between the inpatient setting and the community.

Clinical appropriateness: a key factor in outcome measure selection: the 36 item short form health survey in multiple sclerosis

OBJECTIVES Understanding the properties of an outcome measure is essential in choosing the appropriate instrument and interpreting the information it generates. The MOS 36 item short form health survey questionnaire (SF-36) is widely acknowledged as the gold standard generic measure of health status; few studies however have evaluated its use for clinical trials in multiple sclerosis. Its clinical appropriateness, internal consistency reliability, validity, and responsiveness was investigated across a broad range of patients with multiple sclerosis. METHODS A prospective study in which 150 adults with clinically definite multiple sclerosis completed a battery of questionnaires evaluating generic health status, disability, handicap, and emotional wellbeing. Of these, 44 patients undergoing inpatient rehabilitation completed the questionnaires before and after intervention to evaluate responsiveness. RESULTS Score distributions demonstrated significant floor and ceiling effects in four of the eight dimensions which were particularly marked when patient selection was restricted to a narrow band of disease severity (as is the case in most clinical trials). Internal consistency exceeded the standard for group comparisons for all dimensions. Convergent and discriminant construct validity was supported by the direction, magnitude, and pattern of correlations with other health measures. In comparison with instruments measuring associated constructs, the responsiveness of the SF-36 was poor in evaluating change in moderate to severely disabled patients participating in a programme of inpatient rehabilitation. CONCLUSIONS The SF-36 has some limitations as an outcome measure in multiple sclerosis. The results highlight the need for all instruments to be examined in the specific sample population under question and for the specific research question being investigated. In multiple sclerosis clinical trials, the SF-36 should be supplemented with other relevant measures.

Multiple Sclerosis Impact Scale (MSIS-29): reliability and validity in hospital based samples

Background and aim: The psychometric properties of rating scales are sample dependent and need evaluations in different samples. The Multiple Sclerosis Impact Scale (MSIS-29), a new patient based rating scale for multiple sclerosis (MS) was predominantly developed from a community based sample derived from the MS Society. A number of important patient characteristics of this sample remain unknown. The aim of the study was to evaluate five psychometric properties of the MSIS-29 in three hospital based samples: people admitted for rehabilitation, people admitted for intravenous corticosteroid treatment for MS relapses, and people with primary progressive MS. Methods: People with MS were recruited from the three clinical settings. They completed several health measures. MSIS-29 data were evaluated for data quality, scaling assumptions, acceptability, reliability and validity, and compared with those from a previously reported community based study. Results: A total of 233 people (rehabilitation =53; corticosteroids =76; primary progressive =104) completed questionnaires. In all samples, missing data were low (≤2.2%), scaling assumptions were satisfied, and reliability was high (≥0.91). Correlations between the MSIS-29 and other scales were consistent with a priori hypotheses. Findings were consistent with those from the community samples. Conclusions: The psychometric properties of the MSIS-29 are consistent across three hospital based samples, and similar to those in the community samples. These findings further support its use as an outcome measure in different clinical settings.