Stig Brorson

Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed.

OBJECTIVE Results of reliability and agreement studies are intended to provide information about the amount of error inherent in any diagnosis, score, or measurement. The level of reliability and agreement among users of scales, instruments, or classifications is widely unknown. Therefore, there is a need for rigorously conducted interrater and intrarater reliability and agreement studies. Information about sample selection, study design, and statistical analysis is often incomplete. Because of inadequate reporting, interpretation and synthesis of study results are often difficult. Widely accepted criteria, standards, or guidelines for reporting reliability and agreement in the health care and medical field are lacking. The objective was to develop guidelines for reporting reliability and agreement studies. STUDY DESIGN AND SETTING Eight experts in reliability and agreement investigation developed guidelines for reporting. RESULTS Fifteen issues that should be addressed when reliability and agreement are reported are proposed. The issues correspond to the headings usually used in publications. CONCLUSION The proposed guidelines intend to improve the quality of reporting.

Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors

Background: Clinical trials are commonly done without blinded outcome assessors despite the risk of bias. We wanted to evaluate the effect of nonblinded outcome assessment on estimated effects in randomized clinical trials with outcomes that involved subjective measurement scales. Methods: We conducted a systematic review of randomized clinical trials with both blinded and nonblinded assessment of the same measurement scale outcome. We searched PubMed, EMBASE, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press and Google Scholar for relevant studies. Two investigators agreed on the inclusion of trials and the outcome scale. For each trial, we calculated the difference in effect size (i.e., standardized mean difference between nonblinded and blinded assessments). A difference in effect size of less than 0 suggested that nonblinded assessors generated more optimistic estimates of effect. We pooled the differences in effect size using inverse variance random-effects meta-analysis and used metaregression to identify potential reasons for variation. Results: We included 24 trials in our review. The main meta-analysis included 16 trials (involving 2854 patients) with subjective outcomes. The estimated treatment effect was more beneficial when based on nonblinded assessors (pooled difference in effect size −0.23 [95% confidence interval (CI) −0.40 to −0.06]). In relative terms, nonblinded assessors exaggerated the pooled effect size by 68% (95% CI 14% to 230%). Heterogeneity was moderate (I2 = 46%, p = 0.02) and unexplained by metaregression. Interpretation: We provide empirical evidence for observer bias in randomized clinical trials with subjective measurement scale outcomes. A failure to blind assessors of outcomes in such trials results in a high risk of substantial bias.

Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors

Objective To evaluate the impact of non-blinded outcome assessment on estimated treatment effects in randomised clinical trials with binary outcomes. Design Systematic review of trials with both blinded and non-blinded assessment of the same binary outcome. For each trial we calculated the ratio of the odds ratios—the odds ratio from non-blinded assessments relative to the corresponding odds ratio from blinded assessments. A ratio of odds ratios <1 indicated that non-blinded assessors generated more optimistic effect estimates than blinded assessors. We pooled the individual ratios of odds ratios with inverse variance random effects meta-analysis and explored reasons for variation in ratios of odds ratios with meta-regression. We also analysed rates of agreement between blinded and non-blinded assessors and calculated the number of patients needed to be reclassified to neutralise any bias. Data Sources PubMed, Embase, PsycINFO, CINAHL, Cochrane Central Register of Controlled Trials, HighWire Press, and Google Scholar. Eligibility criteria for selecting studies Randomised clinical trials with blinded and non-blinded assessment of the same binary outcome. Results We included 21 trials in the main analysis (with 4391 patients); eight trials provided individual patient data. Outcomes in most trials were subjective—for example, qualitative assessment of the patient’s function. The ratio of the odds ratios ranged from 0.02 to 14.4. The pooled ratio of odds ratios was 0.64 (95% confidence interval 0.43 to 0.96), indicating an average exaggeration of the non-blinded odds ratio by 36%. We found no significant association between low ratios of odds ratios and scores for outcome subjectivity (P=0.27); non-blinded assessor’s overall involvement in the trial (P=0.60); or outcome vulnerability to non-blinded patients (P=0.52). Blinded and non-blinded assessors agreed in a median of 78% of assessments (interquartile range 64-90%) in the 12 trials with available data. The exaggeration of treatment effects associated with non-blinded assessors was induced by the misclassification of a median of 3% of the assessed patients per trial (1-7%). Conclusions On average, non-blinded assessors of subjective binary outcomes generated substantially biased effect estimates in randomised clinical trials, exaggerating odds ratios by 36%. This bias was compatible with a high rate of agreement between blinded and non-blinded outcome assessors and driven by the misclassification of few patients.

Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies

BACKGROUND Blinding patients in clinical trials is a key methodological procedure, but the expected degree of bias due to nonblinded patients on estimated treatment effects is unknown. METHODS Systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving nonblinded patients. Within each trial, we compared the difference in effect sizes (i.e. standardized mean differences) between the sub-studies. A difference <0 indicates that nonblinded patients generated a more optimistic effect estimate. We pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity. RESULTS Our main analysis included 12 trials (3869 patients). The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2)=60%, P=0.004), i.e. nonblinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding. There was a larger effect size difference in 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding also increased attrition and use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43). CONCLUSIONS This study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.

Benefits and harms of locking plate osteosynthesis in intraarticular (OTA Type C) fractures of the proximal humerus: A systematic review

INTRODUCTION Locking plate osteosynthesis of proximal humeral fractures are widely recommended and used, even in complex intraarticular fracture patterns such as AO/OTA Type C fractures. We systematically reviewed clinical studies assessing the benefits and harms of osteosynthesis with angle stable plates in AO/OTA Type C fractures of the proximal humerus. METHODS We conducted an iterative search in PubMed, Embase, Cochrane Library, Web of Science, Cinahl, and PEDro in all languages from 1999 to November 2010. Eligible studies should study the outcome for Type C fractures after primary osteosynthesis with locking plate within two weeks of injury, and a follow-up period of six months or more. Patients should be evaluated with the Constant-Murley Score (CS). Two observers extracted data independently. RESULTS Twelve studies and 282 Type C fractures were included. Results were categorised according to study type and synthesised qualitatively. No randomised clinical trials were identified. Two comparative, observational studies reported a mean CS of 71 (relative to contralateral shoulder) and 75 (non-adjusted Constant Score) for Type C fractures. For all studies mean non-adjusted CS ranged from 53 to 75. Mean age- and sex-adjusted CS ranged from 60 to 88. Mean CS relative to the contralateral shoulder ranged from 71 to 85. The most common complications were avascular necrosis (range, 4-33%), screw perforations (range, 5-20%), loss of fixation (range, 3-16%), impingement (range, 7-11%) and infections range 4-19%. Reoperation rate ranged from 6 to 44%. CONCLUSIONS Insufficient study designs and unclear reporting preclude safe treatment recommendations. Complication and reoperation rates were unexpected high. Based on the studies included we cannot routinely recommend the use of locking plates in AO/OTA Type C fractures.

The Danish Shoulder Arthroplasty Registry: clinical outcome and short-term survival of 2,137 primary shoulder replacements.

The Danish Shoulder Arthroplasty Registry (DSR) was established in 2004. Data are reported electronically by the surgeons. Patient-reported outcome is collected 10–14 months postoperatively using the Western Ontario osteoarthritis of the shoulder index (WOOS). 2,137 primary shoulder arthroplasties (70% women) were reported to the registry between January 2006 and December 2008. Mean age at surgery was 69 years (SD 12). The most common indications were a displaced proximal humeral fracture (54%) or osteoarthritis (30%). 61% were stemmed hemiarthroplasties, 28% resurfacing hemiarthroplasties, 8% reverse shoulder arthroplasties, and 3% total arthroplasties. Median WOOS was 59% (IQR: 37–82). 5% had been revised by the end of June 2010. The most frequent indications for revision were dislocation or glenoid attrition.

Improved interobserver variation after training of doctors in the Neer system: A RANDOMISED TRIAL

We investigated whether training doctors to classify proximal fractures of the humerus according to the Neer system could improve interobserver agreement. Fourteen doctors were randomised to two training sessions, or to no training, and asked to categorise 42 unselected pairs of plain radiographs of fractures of the proximal humerus according to the Neer system. The mean kappa difference between the training and control groups was 0.30 (95% CI 0.10 to 0.50, p = 0.006). In the training group the mean kappa value for interobserver variation improved from 0.27 (95% CI 0.24 to 0.31) to 0.62 (95% CI 0.57 to 0.67). The improvement was particularly notable for specialists in whom kappa increased from 0.30 (95% CI 0.23 to 0.37) to 0.79 (95% CI 0.70 to 0.88). These results suggest that formal training in the Neer system is a prerequisite for its use in clinical practice and research.

Effect of osteosynthesis, primary hemiarthroplasty, and non-surgical management for displaced four-part fractures of the proximal humerus in elderly: a multi-centre, randomised clinical trial

BackgroundFractures of the proximal humerus are common injuries and account for 4–5 percent of all fractures, second only to hip and wrist fractures. The incidence is positively correlated with age and osteoporosis, and is likely to increase. Displaced four-part fractures are among the most severe injuries, accounting for 2–10 percent of proximal humeral fractures. The optimal intervention is disputed. Two previous randomised trials were very small and involved a noticeable risk of bias, and systematic reviews consequently conclude that there is inadequate basis for evidence-based treatment decisions. We aim to compare the effect of osteosynthesis with angle-stable plate with non-surgical management, and the effect of primary hemiarthroplasty with both osteosynthesis and non-surgical management.Methods/DesignWe will conduct a randomised, multi-centre, clinical trial including patients from ten national shoulder units within a two-year period. We plan to include 162 patients. A central randomisation unit will allocate patients. All patients will receive a standardised three-month rehabilitation program of supervised physiotherapy regardless of treatment allocation. Patients will be followed at least one year. The primary outcomes will be the overall score on the Constant Disability Scale, and its pain subscale, measured at 12 months. A blinded physiotherapist will carry out the assessments. Other secondary outcomes are Oxford Shoulder Score, and general health status (Short Form-36).

Reverse shoulder arthroplasty in acute fractures of the proximal humerus: A systematic review

The indications for surgical intervention in complex fractures of the proximal humerus are disputed. In elderly patients with poor bone stock it may be impossible to obtain satisfactory fixation of the tuberosities to a hemiarthroplasty (HA). In such cases primary insertion of a reverse shoulder arthroplasty (RSA) has been suggested. We aimed to review clinical studies reporting benefits and harms of RSA in acute fractures. A systematic review. We included 18 studies containing 430 RSA in acute fractures. We found no randomized clinical trials. Four studies compared outcome after RSA with a historical control group of HA. The median constant score was 58 (range 44-68) which is comparable to previous reviews of HA in 4-part fractures. Complications included dislocation, infection, hematoma, instability, neurological injury, reflex sympathetic dystrophy, intraoperative fractures, periprosthetic fractures, and baseplate failure. Scapular notching was reported in 11 studies with a median value of 25% (range 0-94). Heterogeneity of study designs and lack of primary data precluded statistical pooling of data. No high quality evidence was identified. Based on the available evidence the use of RSA in acute fractures is questionable. The complication rate was high and the clinical implications of long term scapular notching are worrying. Randomized studies with long term follow up using the latest techniques of tubercular reinsertion in RSA toward HA should be encouraged.

Reliability of patient-reported functional outcome in a joint replacement registry. A comparison of primary responders and non-responders in the Danish Shoulder Arthroplasty Registry.

Background and purpose Patient-reported outcome measures (PROMs) are used by some arthroplasty registries to evaluate results after surgery, but non-response may bias the results. The aim was to identify a potential bias in the outcome scores of subgroups in a cohort of patients from the Danish Shoulder Arthroplasty Registry (DSR) and to characterize non-responders. Methods Patient-reported outcome of 787 patients operated in 2008 was assessed 12 months postoperatively using the Western Ontario Osteoarthritis of the Shoulder (WOOS) index. In January 2012, non-responders and incomplete responders were sent a postal reminder. Non-responders to the postal reminder were contacted by telephone. Total WOOS score and WOOS subscales were compared for initial responders (n = 509), responders to the postal reminder (n = 156), and responders after telephone contact (n = 27). The predefined variables age, sex, diagnosis, geographical region, and reoperation rate were compared for responding and non-responding cohorts. Results A postal reminder increased the response rate from 65% (6% incomplete) to 80% (3% incomplete) and telephone contact resulted in a further increase to 82% (2% incomplete). We did not find any statistically significant differences in total WOOS score or in any of the WOOS subscales between responders to the original questionnaire, responders to the postal reminder, and responders after telephone contact. However, a trend of worse outcome for non-responders was found. The response rate was lower in younger patients. Interpretation Non-responders did not appear to bias the overall results after shoulder replacement despite a trend of worse outcome for a subgroup of non-responders. As response rates rose markedly by the use of postal reminders, we recommend the use of reminders in arthroplasty registries using PROMs.