Elizabeth Kristjansson

Realist review to understand the efficacy of school feeding programmes

A recent Cochrane review found that school feeding programmes significantly improve the growth and cognitive performance of disadvantaged children. Trisha Greenhalgh,Elizabeth Kristjansson, and Vivian Robinson look more closely at the highly heterogeneous trials to see what works, for whom, and in what circumstances

Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health

A group from the Cochrane Collaboration, Campbell Collaboration, and the World Health Organization Measurement and Evidence Knowledge Network has developed guidance on assessing health equity effects in systematic reviews of healthcare interventions. This guidance is also relevant to primary research

Complex interventions and their implications for systematic reviews: A pragmatic approach

Complex interventions present unique challenges for systematic reviews. Current debates tend to center around describing complexity, rather than providing guidance on what to do about it. At a series of meetings during 2009-2012, we met to review the challenges and practical steps reviewer could take to incorporate a complexity perspective into systematic reviews. Based on this, we outline a pragmatic approach to dealing with complexity, beginning, as for any review, with clearly defining the research question(s). We argue that reviews of complex interventions can themselves be simple or complex, depending on the question to be answered. In systematic reviews and evaluations of complex interventions, it will be helpful to start by identifying the sources of complexity, then mapping aspects of complexity in the intervention onto the appropriate sources of evidence (such as specific types of quantitative or qualitative study). Although we focus on systematic reviews, the general approach is also applicable to primary research that is aimed at evaluating complex interventions. Although the examples are drawn from health care, the approach may also be applied to other sectors (e.g., social policy or international development). We end by concluding that systematic reviews should follow the principle of Occams razor: explanations should be as complex as they need to be and no more.

AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both

The number of published systematic reviews of studies of healthcare interventions has increased rapidly and these are used extensively for clinical and policy decisions. Systematic reviews are subject to a range of biases and increasingly include non-randomised studies of interventions. It is important that users can distinguish high quality reviews. Many instruments have been designed to evaluate different aspects of reviews, but there are few comprehensive critical appraisal instruments. AMSTAR was developed to evaluate systematic reviews of randomised trials. In this paper, we report on the updating of AMSTAR and its adaptation to enable more detailed assessment of systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. With moves to base more decisions on real world observational evidence we believe that AMSTAR 2 will assist decision makers in the identification of high quality systematic reviews, including those based on non-randomised studies of healthcare interventions.

Damned if you do, damned if you don't: subgroup analysis and equity

The final report from the WHO Commission on the social determinants of health recently noted: ‘For policy, however important an ethical imperative, values alone are insufficient. There needs to be evidence on what can be done and what is likely to work in practice to improve health and reduce health inequities.’ This is challenging, because understanding how to reduce health inequities between the poorest and better-off members of society may require a greater use of subgroup analysis to explore the differential effects of public health interventions. However, while this may produce evidence that is more policy relevant, the requisite subgroup analyses are often seen as tantamount to statistical malpractice. This paper considers some of the methodological problems with subgroup analysis, and its applicability to considerations of equity, using both clinical and public health examples. Finally, it suggests how policy needs for information on subgroups can be met while maintaining rigour.

A Comparison of Four Methods for Detecting Differential Item Functioning in Ordered Response Items

Item bias is a major threat to measurement validity. Methods for detecting differential item functioning (DIF) are now commonly used to identify potentially biased items. DIF detection methods for dichotomous items are well developed, but those for ordinal items are less well developed. In this article, the authors compare four methods for detecting DIF in ordinal items: the Mantel, generalized Mantel-Haenszel (GMH), logistic discriminant function analysis (LDFA), and unconstrained cumulative logits ordinal logistic regression (UCLOLR). Factors varied include type of DIF, group ability differences, studied item discrimination, skewness in ability distributions, and sample size ratio. All procedures had good Type I error control as well as high power for detecting uniform DIF. However, the Mantel could not detect nonuniform DIF, and the LDFA also performed poorly in detecting nonuniform DIF, particularly when item discrimination was high. The UCLOLR and GMH performed extremely well under conditions simulated in this study. Implications for research and practice are discussed.

Validation of a Preparation for Decision Making scale

OBJECTIVE The Preparation for Decision Making (PrepDM) scale was developed to evaluate decision processes relating to the preparation of patients for decision making and dialoguing with their practitioners. The objective of this study was to evaluate the scales psychometric properties. METHODS From July 2005 to March 2006, after viewing a decision aid prescribed during routine clinical care, patients completed a questionnaire including: demographic information, treatment intention, decisional conflict, decision aid acceptability, and the PrepDM scale. RESULTS Four hundred orthopaedic patients completed the questionnaire. The PrepDM scale showed significant correlation with the informed (r=-0.21, p<0.01) and support (r=-0.13, p=0.01) subscales (DCS); and discriminated significantly between patients who did and did not find the decision aid helpful (p<0.0001). Alpha coefficients for internal consistency ranged from 0.92 to 0.96. The scale is strongly unidimensional (principal components analysis) and Item Response Theory analyses demonstrated that all ten scale items function very well. CONCLUSION The psychometric properties of the PrepDM scale are very good. PRACTICE IMPLICATIONS The scale could allow more comprehensive evaluation of interventions designed to prepare patients for shared-decision making encounters regarding complex health care decisions.

A multilevel analysis of neighbourhood built and social environments and adult self-reported physical activity and body mass index in Ottawa, Canada

Canadian research examining the combined effects of social and built environments on physical activity (PA) and obesity is limited. The purpose of this study was to determine the relationships among built and social environments and PA and overweight/obesity in 85 Ottawa neighbourhoods. Self-reported PA, height and weight were collected from 3,883 adults using the International PA Questionnaire from the 2003–2007 samples of the Rapid Risk Factor Surveillance System. Data on neighbourhood characteristics were obtained from the Ottawa Neighbourhood Study; a large study of neighbourhoods and health in Ottawa. Two-level binomial logistic regression models stratified by sex were used to examine the relationships of environmental and individual variables with PA and overweight/obesity while using survey weights. Results identified that approximately half of the adults were insufficiently active or overweight/obese. Multilevel models identified that for every additional convenience store, men were two times more likely to be physically active (OR = 2.08, 95% CI: 1.72, 2.43) and with every additional specialty food store women were almost two times more likely to be overweight or obese (OR = 1.77, 95% CI: 1.33, 2.20). Higher green space was associated with a reduced likelihood of PA (OR = 0.93, 95% CI: 0.86, 0.99) and increased odds of overweight and obesity in men (OR = 1.10, 95% CI: 1.01, 1.19), and decreased odds of overweight/obesity in women (OR = 0.66, 95% CI: 0.44, 0.89). In men, neighbourhood socioeconomic scores, voting rates and sense of community belonging were all significantly associated with overweight/obesity. Intraclass coefficients were low, but identified that the majority of neighbourhood variation in outcomes was explained by the models. Findings identified that green space, food landscapes and social cohesiveness may play different roles on PA and overweight/obesity in men and women and future prospective studies are needed.

Relationships Between Neighborhoods, Physical Activity, and Obesity: A Multilevel Analysis of a Large Canadian City

In Canada, there is limited research examining the associations between objectively measured neighborhood environments and physical activity (PA) and obesity. The purpose of this study was to determine the relationships between variables from built and social environments and PA and overweight/obesity across 86 Ottawa, Canada neighborhoods. Individual‐level data including self‐reported leisure‐time PA (LTPA), height, and weight were examined in a sample of 4,727 adults from four combined cycles (years 2001/03/05/07) of the Canadian Community Health Survey (CCHS). Data on neighborhood characteristics were obtained from the Ottawa Neighbourhood Study (ONS); a large study of neighborhoods and health in Ottawa, Canada. Binomial multivariate multilevel models were used to examine the relationships between environmental and individual variables with LTPA and overweight/obesity using survey weights in men and women separately. Within the sample, ∼75% of the adults were inactive (<3.0 kcal/kg/day) while half were overweight/obese. Results of the multilevel models suggested that for females greater park area was associated with increased odds of LTPA and overweight/obesity. Greater neighborhood density of convenience stores and fast food outlets were associated with increased odds of females being overweight/obese. Higher crime rates were associated with greater odds of LTPA in males, and lower odds of male and female overweight/obesity. Season was significantly associated with PA in men and women; the odds of LTPA in winter months were half that of summer months. Findings revealed that park area, crime rates, and neighborhood food outlets may have different roles with LTPA and overweight/obesity in men and women and future prospective studies are needed.

Impact of remuneration and organizational factors on completing preventive manoeuvres in primary care practices

Background: Several jurisdictions attempting to reform primary care have focused on changes in physician remuneration. The goals of this study were to compare the delivery of preventive services by practices in four primary care funding models and to identify organizational factors associated with superior preventive care. Methods: In a cross-sectional study, we included 137 primary care practices in the province of Ontario (35 fee-for-service practices, 35 with salaried physicians [community health centres], 35 practices in the new capitation model [family health networks] and 32 practices in the established capitation model [health services organizations]). We surveyed 288 family physicians. We reviewed 4108 randomly selected patient charts and assigned prevention scores based on the proportion of eligible preventive manoeuvres delivered for each patient. Results: A total of 3284 patients were eligible for at least one of six preventive manoeuvres. After adjusting for patient profile and contextual factors, we found that, compared with prevention scores in practices in the new capitation model, scores were significantly lower in fee-for-service practices (β estimate for effect on prevention score = −6.3, 95% confidence interval [CI] −11.9 to −0.6) and practices in the established capitation model (β = −9.1, 95% CI −14.9 to −3.3) but not for those with salaried remuneration (β = −0.8, 95% CI −6.5 to 4.8). After accounting for physician characteristics and organizational structure, the type of funding model was no longer a statistically significant factor. Compared with reference practices, those with at least one female family physician (β = 8.0, 95% CI 4.2 to 11.8), a panel size of fewer than 1600 patients per full-time equivalent family physician (β = 6.8, 95% CI 3.1 to 10.6) and an electronic reminder system (β = 4.6, 95% CI 0.4 to 8.7) had superior prevention scores. The effect of these three factors was largely but not always consistent across the funding models; it was largely consistent across the preventive manoeuvres. Interpretation: No funding model was clearly associated with superior preventive care. Factors related to physician characteristics and practice structure were stronger predictors of performance. Practices with one or more female physicians, a smaller patient load and an electronic reminder system had superior prevention scores. Our findings raise questions about reform initiatives aimed at increasing patient numbers, but they support the adoption of information technology.