Terence J. Quinn

Barthel Index for Stroke Trials Development, Properties, and Application

Background and Purpose— Robust measures of functional outcome are required to determine treatment effects in stroke trials. Of the various measures available, the Barthel index (BI) is one of the more prevalent. We aimed to describe validity, reliability, and responsiveness (clinimetric properties) of the BI in stroke trials. Methods— Narrative review of published articles describing clinimetric properties or use of the BI as a stroke trial end point. Results— Definitive statements on properties of BI are limited by heterogeneity in methodology of assessment and in the content of “BI” scales. Accepting these caveats, evidence suggests that BI is a valid measure of activities of daily living; sensitivity to change is limited at extremes of disability (floor and ceiling effects), and reliability of standard BI assessment is acceptable. However, these data may not be applicable to contemporary multicenter stroke trials. Conclusions— Substantial literature describing BI clinimetrics in stroke is available; however, questions remain regarding certain properties. The “BI” label is used for a number of instruments and we urge greater consistency in methods, content, and scoring. A 10-item scale, scoring 0 to 100 with 5-point increments, has been used in several multicenter stroke trials and it seems reasonable that this should become the uniform stroke trial BI.

Reliability of the Modified Rankin Scale A Systematic Review

Background and Purpose— A perceived weakness of the modified Rankin Scale is potential for interobserver variability. We undertook a systematic review of modified Rankin Scale reliability studies. Methods— Two researchers independently reviewed the literature. Crossdisciplinary electronic databases were interrogated using the following key words: Stroke*; Cerebrovasc*; Modified Rankin*; Rankin Scale*; Oxford Handicap*; Observer variation*. Data were extracted according to prespecified criteria with decisions on inclusion by consensus. Results— From 3461 titles, 10 studies (587 patients) were included. Reliability of modified Rankin Scale varied from weighted &kgr;=0.95 to &kgr;=0.25. Overall reliability of mRS was &kgr;=0.46; weighted &kgr;=0.90 (traditional modified Rankin Scale) and &kgr;=0.62; weighted &kgr;=0.87 (structured interview). Conclusion— There remains uncertainty regarding modified Rankin Scale reliability. Interobserver studies closest in design to large-scale clinical trials demonstrate potentially significant interobserver variability.

Functional outcome measures in contemporary stroke trials

Various instruments are used to describe poststroke functional outcome, with limited consensus as to optimal end-point for clinical trial use. Many of the popular assessment tools are administered with little formal guidance on best practice. Thus there is potential for substantial heterogeneity in functional outcome assessment poststroke, with consequent effects on trial quality. We examined functional assessment methodology in recent stroke trials. We reviewed six journals representing high-impact international publications in the fields of: stroke (Stroke); neurology (Neurology, Lancet Neurology) and internal medicine (Lancet, New England Journal Medicine; Journal of the American Medical Association). Journals were hand searched for all interventional studies in stroke patients between 2001 and 2006 inclusive. Chosen manuscripts were then analyzed for outcome assessment methodology. We identified 126 trials, comprising a mix of early hypothesis generating studies through to multicentre trials (phase I: four trials; phase II: 46 trials; phase III: 20 trials; noninvestigational medicinal product studies: 56 trials). The median number of patients assessed per trial was 100. Across the trials, 47 different outcome measures were used. One hundred trials had functional outcome assessment as the primary study end-point. The median number of outcome measures was two per trial (range 1–9). The modified Rankin scale was the most prevalent outcome assessment (64·3%); followed by Barthel index (40·5%). A minority of trials (33·3%) provided full details on outcome assessment methodology. Among these trials there was substantial heterogeneity in data collection procedures. There is heterogeneity in the use of functional outcome measures in stroke trials. This compromises comparison and meta-analysis. Trialists continue to use poorly validated approaches to outcome assessment. Given the potential effects on data quality, explicit description of methodology should be mandatory for all trials and rigour is desirable.

Reliability (Inter-rater Agreement) of the Barthel Index for Assessment of Stroke Survivors Systematic Review and Meta-analysis

Background and Purpose— The Barthel Index (BI) is a 10-item measure of activities of daily living which is frequently used in clinical practice and as a trial outcome measure in stroke. We sought to describe the reliability (interobserver variability) of standard BI in stroke cohorts using systematic review and meta-analysis of published studies. Methods— Two assessors independently searched various multidisciplinary electronic databases from inception to April 2012 inclusive. Inclusion criteria comprised: original research, human stroke participants, and inter-rater reliability data on equivalent methods of BI administration. Manuscripts were reviewed against prespecified inclusion criteria. Primary outcome for meta-analysis was reliability, measured by weighted &kgr; (&kgr;w). Results— From 20 210 titles, 306 abstracts were reviewed, 12 studies met inclusion criteria, and 10 were included in meta-analysis (n=543 participants; range of participants in studies, 7–21). There was substantial clinical heterogeneity with respect to method of BI application; population studied and assessors. Two papers were graded high quality. Overall interobserver reliability of standard administration of the BI was excellent (&kgr;w, 0.93; 95% confidence interval, 0.90–0.96 random effects modeling). Conclusions— The BI has excellent inter-rater reliability for standard administration after stroke. However, included studies were modest in size, with clinical heterogeneity and variable methodological quality. Despite these limitations, standard BI seems an appropriate outcome measure for stroke trials and practice.

Reporting standards for studies of diagnostic test accuracy in dementia: The STARDdem Initiative

Objective: To provide guidance on standards for reporting studies of diagnostic test accuracy for dementia disorders. Methods: An international consensus process on reporting standards in dementia and cognitive impairment (STARDdem) was established, focusing on studies presenting data from which sensitivity and specificity were reported or could be derived. A working group led the initiative through 4 rounds of consensus work, using a modified Delphi process and culminating in a face-to-face consensus meeting in October 2012. The aim of this process was to agree on how best to supplement the generic standards of the STARD statement to enhance their utility and encourage their use in dementia research. Results: More than 200 comments were received during the wider consultation rounds. The areas at most risk of inadequate reporting were identified and a set of dementia-specific recommendations to supplement the STARD guidance were developed, including better reporting of patient selection, the reference standard used, avoidance of circularity, and reporting of test-retest reliability. Conclusion: STARDdem is an implementation of the STARD statement in which the original checklist is elaborated and supplemented with guidance pertinent to studies of cognitive disorders. Its adoption is expected to increase transparency, enable more effective evaluation of diagnostic tests in Alzheimer disease and dementia, contribute to greater adherence to methodologic standards, and advance the development of Alzheimer biomarkers.

Initial Experience of a Digital Training Resource for Modified Rankin Scale Assessment in Clinical Trials

Background and Purpose— The modified Rankin Scale (mRS) is the preferred measure of disability in cerebrovascular clinical trials, but its value is restricted by interobserver variability. Poor reliability reduces the statistical power of clinical trials and leads to underestimation of effect size. Strategies to improve mRS grading are required. Video training has previously improved application of the National Institutes of Health Stroke Scale in clinical research. We developed an mRS training resource in an attempt to minimize interobserver variability. Methods— We produced a complete training resource comprising an instructional DVD with accompanying written materials and assessment recordings of patient interviews. Formal assessment of training involved grading of real-life cases. Results of initial training and recertification were collected centrally and scored. Results— Data from 1564 assessments are presented. The majority of assessors were participating in 2 large prospective clinical stroke trials. Assessors represented a mixed group of disciplines and nationalities. After training, most trainees (90%) achieved certification in mRS assessment. The majority (85%) of investigators who did not reach an acceptable score on initial testing achieved certification after further exposure to the package. Conclusions— Mass training in mRS assessment for clinical trials is possible. We outline the development of a video-based training package, including technical issues, patient selection procedures, and methods of scoring and assessment. Certification results suggest that use of the resource can improve mRS grading. Acceptability of the training has been demonstrated by its successful use in 2 international acute stroke trials, SAINT 1 and CHANT.

Assessment scales in stroke: clinimetric and clinical considerations

As stroke care has developed, there has been a need to robustly assess the efficacy of interventions both at the level of the individual stroke survivor and in the context of clinical trials. To describe stroke-survivor recovery meaningfully, more sophisticated measures are required than simple dichotomous end points, such as mortality or stroke recurrence. As stroke is an exemplar disabling long-term condition, measures of function are well suited as outcome assessment. In this review, we will describe functional assessment scales in stroke, concentrating on three of the more commonly used tools: the National Institutes of Health Stroke Scale, the modified Rankin Scale, and the Barthel Index. We will discuss the strengths, limitations, and application of these scales and use the scales to highlight important properties that are relevant to all assessment tools. We will frame much of this discussion in the context of “clinimetric” analysis. As they are increasingly used to inform stroke-survivor assessments, we will also discuss some of the commonly used quality-of-life measures. A recurring theme when considering functional assessment is that no tool suits all situations. Clinicians and researchers should chose their assessment tool based on the question of interest and the evidence base around clinimetric properties.

Test accuracy of cognitive screening tests for diagnosis of dementia and multidomain cognitive impairment in stroke.

Background and Purpose— Guidelines recommend screening stroke-survivors for cognitive impairments. We sought to collate published data on test accuracy of cognitive screening tools. Methods— Index test was any direct, cognitive screening assessment compared against reference standard diagnosis of (undifferentiated) multidomain cognitive impairment/dementia. We used a sensitive search statement to search multiple, cross-disciplinary databases from inception to January 2014. Titles, abstracts, and articles were screened by independent researchers. We described risk of bias using Quality Assessment of Diagnostic Accuracy Studies tool and reporting quality using Standards for Reporting of Diagnostic Accuracy guidance. Where data allowed, we pooled test accuracy using bivariate methods. Results— From 19 182 titles, we reviewed 241 articles, 35 suitable for inclusion. There was substantial heterogeneity: 25 differing screening tests; differing stroke settings (acute stroke, n=11 articles), and reference standards used (neuropsychological battery, n=21 articles). One article was graded low risk of bias; common issues were case–control methodology (n=7 articles) and missing data (n=22). We pooled data for 4 tests at various screen positive thresholds: Addenbrooke’s Cognitive Examination-Revised (<88/100): sensitivity 0.96, specificity 0.70 (2 studies); Mini Mental State Examination (<27/30): sensitivity 0.71, specificity 0.85 (12 studies); Montreal Cognitive Assessment (<26/30): sensitivity 0.95, specificity 0.45 (4 studies); MoCA (<22/30): sensitivity 0.84, specificity 0.78 (6 studies); Rotterdam-CAMCOG (<33/49): sensitivity 0.57, specificity 0.92 (2 studies). Conclusions— Commonly used cognitive screening tools have similar accuracy for detection of dementia/multidomain impairment with no clearly superior test and no evidence that screening tools with longer administration times perform better. MoCA at usual threshold offers short assessment time with high sensitivity but at cost of specificity; adapted cutoffs have improved specificity without sacrificing sensitivity. Our results must be interpreted in the context of modest study numbers: heterogeneity and potential bias.

Exploring the Reliability of the Modified Rankin Scale

Background and Purpose— The modified Rankin Scale (mRS) is the most prevalent outcome measure in stroke trials. Use of the mRS may be hampered by variability in grading. Previous estimates of the properties of the mRS have used diverse methodologies and may not apply to contemporary trial populations. We used a mock clinical trial design to explore inter- and intraobserver variability of the mRS. Methods— Consenting patients with stroke attending for outpatient review had the mRS performed by 2 independent assessors with pairs of assessors selected from a team of 3 research nurses and 4 stroke physicians. Before formal assessment, interviewers estimated disability based only on initial patient observation. Each patient was then randomized to undergo the mRS using standard assessment or a prespecified structured interview. The second interviewer in the pair reassessed the patient using the same method blinded to the colleague’s score. For each patient assessed, one rater was randomly assigned to video record their interview. After 3 months, this interviewer reviewed and regraded their original video assessment. Results— Across 100 paired assessments, interobserver agreement was moderate (k=0.57). Intraobserver variability was good (k=0.72) but less than would be expected from previous literature. Forty-nine assessments were performed using the structured interview approach with no significant difference between structured and standard mRS. Researchers were unable to reliably predict mRS from initial limited patient assessment (k=0.16). Conclusions— Despite availability of training and structured interview, there remains substantial interobserver variability in mRS grades awarded even by experienced researchers. Additional methods to improve mRS reliability are required.

Thyroid Hormone Therapy for Older Adults with Subclinical Hypothyroidism

BACKGROUND The use of levothyroxine to treat subclinical hypothyroidism is controversial. We aimed to determine whether levothyroxine provided clinical benefits in older persons with this condition. METHODS We conducted a double‐blind, randomized, placebo‐controlled, parallel‐group trial involving 737 adults who were at least 65 years of age and who had persisting subclinical hypothyroidism (thyrotropin level, 4.60 to 19.99 mIU per liter; free thyroxine level within the reference range). A total of 368 patients were assigned to receive levothyroxine (at a starting dose of 50 μg daily, or 25 μg if the body weight was <50 kg or the patient had coronary heart disease), with dose adjustment according to the thyrotropin level; 369 patients were assigned to receive placebo with mock dose adjustment. The two primary outcomes were the change in the Hypothyroid Symptoms score and Tiredness score on a thyroid‐related quality‐of‐life questionnaire at 1 year (range of each scale is 0 to 100, with higher scores indicating more symptoms or tiredness, respectively; minimum clinically important difference, 9 points). RESULTS The mean age of the patients was 74.4 years, and 396 patients (53.7%) were women. The mean (±SD) thyrotropin level was 6.40±2.01 mIU per liter at baseline; at 1 year, this level had decreased to 5.48 mIU per liter in the placebo group, as compared with 3.63 mIU per liter in the levothyroxine group (P<0.001), at a median dose of 50 μg. We found no differences in the mean change at 1 year in the Hypothyroid Symptoms score (0.2±15.3 in the placebo group and 0.2±14.4 in the levothyroxine group; between‐group difference, 0.0; 95% confidence interval [CI], ‐2.0 to 2.1) or the Tiredness score (3.2±17.7 and 3.8±18.4, respectively; between‐group difference, 0.4; 95% CI, ‐2.1 to 2.9). No beneficial effects of levothyroxine were seen on secondary‐outcome measures. There was no significant excess of serious adverse events prespecified as being of special interest. CONCLUSIONS Levothyroxine provided no apparent benefits in older persons with subclinical hypothyroidism. (Funded by European Union FP7 and others; TRUST ClinicalTrials.gov number, NCT01660126.)