Identifying the ‘incredible’! Part 1: assessing the risk of bias in outcomes included in systematic reviews

Abstract

Systematic reviews fulfil a vital role in modern medicine.1 However, the results of systematic reviews are only as valid as the studies they include.2 Pooling flawed, or biased, results from different studies can compromise the credibility of systematic review findings. Bias is a systematic deviation from the truth in the results of a research study that can manifest due to limitations in study design, conduct, or analysis.3\n\nThe results of sport and exercise medicine research, like results in other fields, are vulnerable to bias.4 It is important that systematic review authors assess for bias in a way that enables a judgement about whether a review outcome is at risk of bias due to methodological limitations in included studies. This two-part education primer focuses on how systematic review authors can perform and interpret risk of bias assessments to avoid misleading systematic review conclusions. In this editorial, we introduce the concept of risk of bias, and the principles of assessing risk of bias.\n\nDifferent biases have effects that vary in direction and magnitude.3 5 It is challenging to precisely determine how bias may overestimate or underestimate a study’s true findings. In fact, bias does not always result in distorted study findings and one can never be certain that bias is present when a study has methodological limitations. However, methodological limitations in study design, conduct, or analysis can be consistently associated with inflated research findings.5 Due to this uncertainty, study outcomes are considered to be at risk of bias rather than ‘biased’.\n\nStudies with ‘some concerns’ or at ‘high’ risk of bias in design, conduct, analysis, or reporting are at greater risk of inflated findings compared with studies at ‘low’ risk of bias, negatively …