Sasha Shepperd

DISCERN: an instrument for judging the quality of written consumer health information on treatment choices.

OBJECTIVE: To develop a short instrument, called DISCERN, which will enable patients and information providers to judge the quality of written information about treatment choices. DISCERN will also facilitate the production of new, high quality, evidence-based consumer health information. DESIGN: An expert panel, representing a range of expertise in consumer health information, generated criteria from a random sample of information for three medical conditions with varying degrees of evidence: myocardial infarction, endometriosis, and chronic fatigue syndrome. A graft instrument, based on this analysis, was tested by the panel on a random sample of new material for the same three conditions. The panel re-drafted the instrument to take account of the results of the test. The DISCERN instrument was finally tested by a national sample of 15 information providers and 13 self help group members on a random sample of leaflets from 19 major national self help organisations. Participants also completed an 8 item questionnaire concerning the face and content validity of the instrument. RESULTS: Chance corrected agreement (weighted kappa) for the overall quality rating was kappa = 0.53 (95% CI kappa = 0.48 to kappa = 0.59) among the expert panel, kappa = 0.40 (95% CI kappa = 0.36 to kappa = 0.43) among information providers, and kappa = 0.23 (95% CI kappa = 0.19 to kappa = 0.27) among self help group members. Higher agreement levels were associated with experience of using the instrument and with professional knowledge of consumer health information. Levels of agreement varied across individual items on the instrument, reflecting the need for subjectivity in rating certain criteria. The trends in levels of agreement were similar among all groups. The final instrument consisted of 15 questions plus an overall quality rating. Responses to the questionnaire after the final testing revealed the instrument to have good face and content validity and to be generally applicable. CONCLUSIONS: DISCERN is a reliable and valid instrument for judging the quality of written consumer health information. While some subjectivity is required for rating certain criteria, the findings demonstrate that the instrument can be applied by experienced users and providers of health information to discriminate between publications of high and low quality. The instrument will also be of benefit to patients, though its use will be improved by training.

Can we systematically review studies that evaluate complex interventions

In three Viewpoints, Sasha Shepperd and colleagues, Geoff Wong, and Aziz Sheikh explore various approaches to help systematic reviewers who wish to review complex health interventions.

Taking healthcare interventions from trial to practice

The results of thousands of trials are never acted on because their published reports do not describe the interventions in enough detail. How can we improve the reporting?

Is ‘watchful waiting’ a real choice for men with prostate cancer? A qualitative study

Objective To understand what leads men to choose ‘watchful waiting’ rather than active treatment for cancer of the prostate.

Monitoring use of knowledge and evaluating outcomes.

In the knowledge-to-action cycle, after the intervention related to knowledge translation has been implemented, uptake of knowledge should be monitored. [1][1] This step is necessary to determine how and to what extent the knowledge is used by the decision-makers. [1][1] How we measure uptake of

Why men with prostate cancer want wider access to prostate specific antigen testing: qualitative study

Abstract Objectives: To explore the attitudes of men with confirmed or suspected prostate cancer to testing for prostate specific antigen. Design: Qualitative interview study with a purposive sample. Setting: Great Britain. Participants: 52 men with suspected or confirmed prostate cancer, recruited through general practitioners, urologists, patient support groups, and charities. Results: Almost all men remembered their prostate specific antigen test but recalled being given little information beforehand. Arguments in favour of increased access to testing included the belief that early diagnosis would reduce mortality, improve quality of life, and save the NHS money. Men also thought that a national screening programme should be available because symptoms can be ambiguous, screening for cancer is responsible health behaviour, and screening would encourage men to be tested. Four men who opposed a screening programme had gathered information alerting them to uncertainty about the benefits of treatment, and two regretted that they had been tested. Others thought that access to testing is restricted in the United Kingdom because of a lack of government backing, concerns about the accuracy of the test, and a lack of resources. Conclusions: The few men in this study who subscribed to the argument that evidence of the benefits of treatment is a prerequisite for a screening programme did not want to see screening introduced. Men who proposed an alternative set of principles for testing gave reasons that did not all relate to overoptimism about the benefits of early diagnosis. People who plan services and people who respond to requests for testing need to understand mens perspectives and concerns.

Identifying, documenting, and examining heterogeneity in systematic reviews of complex interventions.

This article describes approaches for planning, dealing, and analyzing heterogeneity in a systematic review of complex interventions. Approaches aim to generate a priori hypotheses of the mechanism of action of a complex intervention to identify the key variables that might contribute to variation among studies and guide statistical analysis. In addition to characteristics related to the population, intervention, and outcomes, we describe study-related variables, such as the way the interventions have been implemented and the context and conduct of studies. These approaches will guide reviewers planning a meta-analysis and provide a rationale for not meta-analyzing data if there is too much variability. Potential difficulties in applying meta-analytical techniques to examine statistical association among study results and sources of potential heterogeneity are described; these include the selection of a fixed or random-effects model, the risk of multiple testing and confounding when studies include different aspects of a complex intervention or different subsamples of the intended participant pool.

Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review

BackgroundWe completed a scoping review on the barriers and facilitators to use of systematic reviews by health care managers and policy makers, including consideration of format and content, to develop recommendations for systematic review authors and to inform research efforts to develop and test formats for systematic reviews that may optimise their uptake.MethodsWe used the Arksey and O’Malley approach for our scoping review. Electronic databases (e.g., MEDLINE, EMBASE, PsycInfo) were searched from inception until September 2014. Any study that identified barriers or facilitators (including format and content features) to uptake of systematic reviews by health care managers and policy makers/analysts was eligible for inclusion. Two reviewers independently screened the literature results and abstracted data from the relevant studies. The identified barriers and facilitators were charted using a barriers and facilitators taxonomy for implementing clinical practice guidelines by clinicians.ResultsWe identified useful information for authors of systematic reviews to inform their preparation of reviews including providing one-page summaries with key messages, tailored to the relevant audience. Moreover, partnerships between researchers and policy makers/managers to facilitate the conduct and use of systematic reviews should be considered to enhance relevance of reviews and thereby influence uptake.ConclusionsSystematic review authors can consider our results when publishing their systematic reviews. These strategies should be rigorously evaluated to determine impact on use of reviews in decision-making.

Conceptual and Practical Difficulties with the Economic Evaluation of Health Services Developments

Five recent economic evaluations comparing hospital at home schemes with acute hospital care faced remarkably similar problems. This paper outlines these problems and considers what strategies can be derived from these experiences, which will be relevant to economic evaluations of other aspects of the organisation of care, particularly those crossing the interfaces between primary and secondary health care or the interface between health and social services. The difficulties experienced can be divided into conceptual and practical problems. Conceptual problems were primarily associated with issues of context and related to the choice of comparator, capacity constraints and size of schemes, and the choice between a short or a long run perspective. Practical problems were connected with the time at which schemes were evaluated, the type of clinical study alongside which studies were conducted and the types of data available for use in the analysis. Strategies which can be pursued in conducting economic evaluations of organisational change include giving greater attention to conceptual and hence contextual problems as well as reporting these contextual issues in detail, accepting the need for repeated economic evaluations as organisational changes become more widespread and considering carefully the clinical study design where economic evaluations of organisational change are conducted alongside. These strategies are of importance not just to those conducting economic evaluations but also to those funding appraisals of changes in the organisation of care. Use of different strategies such as those suggested here should be evaluated.

Development and testing of a questionnaire to measure patient satisfaction with intermediate care

Background: Individual trials have suggested high levels of general patient satisfaction with intermediate care, but this topic has not been examined in detail. Aims: To identify the key elements of patient satisfaction with intermediate care, and to see whether these can be validly measured using a questionnaire. Method: A questionnaire was developed on the basis of a literature review and piloting with patients and staff on participating schemes (phase I). In phase II, the questionnaire was tested for validity and reliability in a group of patients recently discharged from two “hospital-at-home” intermediate-care schemes. In phase III, a shortened version of the questionnaire was psychometrically tested in five sites taking part in a national evaluation of intermediate care. Results: 96 patients with an average age of 76.5 years took part in phase II. Test–retest reliability was evaluated by repeating the questionnaire 2 weeks later in a subsample of 42 patients. This was “moderate” (κ 0.4–0.6) for 12 questions, “fair” (κ 0.2–0.4) for 6 questions and “poor” (κ 0.1–0.2) for 5 questions. Scores correlated well with the Client Satisfaction Questionnaire (Spearman’s r = 0.75, p<0.001). 843 patients (57% of those eligible) from five intermediate-care schemes took part in phase III. Principal components analysis suggested six factors or subscales: general satisfaction, affective response, cognitive response, timing of discharge, coordination after discharge, and access to pain relief, although the last three factors comprised only one question each. The intraclass correlation coefficients in the first three subscales varied from 0.82 to 0.89. Scores for all subscales differed by scheme, suggesting construct validity. Only one question (on general satisfaction) was found to be redundant. Conclusion: The questionnaire, with some minor amendments to improve performance, could be used as a validated tool for audit and research in intermediate care. An amended version and scoring programme is available from us on request.