Trudy van der Weijden

Assessing the Quality of Decision Support Technologies Using the International Patient Decision Aid Standards instrument (IPDASi)

Objectives To describe the development, validation and inter-rater reliability of an instrument to measure the quality of patient decision support technologies (decision aids). Design Scale development study, involving construct, item and scale development, validation and reliability testing. Setting There has been increasing use of decision support technologies – adjuncts to the discussions clinicians have with patients about difficult decisions. A global interest in developing these interventions exists among both for-profit and not-for-profit organisations. It is therefore essential to have internationally accepted standards to assess the quality of their development, process, content, potential bias and method of field testing and evaluation. Methods Scale development study, involving construct, item and scale development, validation and reliability testing. Participants Twenty-five researcher-members of the International Patient Decision Aid Standards Collaboration worked together to develop the instrument (IPDASi). In the fourth Stage (reliability study), eight raters assessed thirty randomly selected decision support technologies. Results IPDASi measures quality in 10 dimensions, using 47 items, and provides an overall quality score (scaled from 0 to 100) for each intervention. Overall IPDASi scores ranged from 33 to 82 across the decision support technologies sampled (n = 30), enabling discrimination. The inter-rater intraclass correlation for the overall quality score was 0.80. Correlations of dimension scores with the overall score were all positive (0.31 to 0.68). Cronbachs alpha values for the 8 raters ranged from 0.72 to 0.93. Cronbachs alphas based on the dimension means ranged from 0.50 to 0.81, indicating that the dimensions, although well correlated, measure different aspects of decision support technology quality. A short version (19 items) was also developed that had very similar mean scores to IPDASi and high correlation between short score and overall score 0.87 (CI 0.79 to 0.92). Conclusions This work demonstrates that IPDASi has the ability to assess the quality of decision support technologies. The existing IPDASi provides an assessment of the quality of a DSTs components and will be used as a tool to provide formative advice to DSTs developers and summative assessments for those who want to compare their tools against an existing benchmark.

“Many miles to go …”: a systematic review of the implementation of patient decision support interventions into routine clinical practice

BackgroundTwo decades of research has established the positive effect of using patient-targeted decision support interventions: patients gain knowledge, greater understanding of probabilities and increased confidence in decisions. Yet, despite their efficacy, the effectiveness of these decision support interventions in routine practice has yet to be established; widespread adoption has not occurred. The aim of this review was to search for and analyze the findings of published peer-reviewed studies that investigated the success levels of strategies or methods where attempts were made to implement patient-targeted decision support interventions into routine clinical settings.MethodsAn electronic search strategy was devised and adapted for the following databases: ASSIA, CINAHL, Embase, HMIC, Medline, Medline-in-process, OpenSIGLE, PsycINFO, Scopus, Social Services Abstracts, and the Web of Science. In addition, we used snowballing techniques. Studies were included after dual independent assessment.ResultsAfter assessment, 5322 abstracts yielded 51 articles for consideration. After examining full-texts, 17 studies were included and subjected to data extraction. The approach used in all studies was one where clinicians and their staff used a referral model, asking eligible patients to use decision support. The results point to significant challenges to the implementation of patient decision support using this model, including indifference on the part of health care professionals. This indifference stemmed from a reported lack of confidence in the content of decision support interventions and concern about disruption to established workflows, ultimately contributing to organizational inertia regarding their adoption.ConclusionsIt seems too early to make firm recommendations about how best to implement patient decision support into routine practice because approaches that use a ‘referral model’ consistently report difficulties. We sense that the underlying issues that militate against the use of patient decision support and, more generally, limit the adoption of shared decision making, are under-investigated and under-specified. Future reports from implementation studies could be improved by following guidelines, for example the SQUIRE proposals, and by adopting methods that would be able to go beyond the ‘barriers’ and ‘facilitators’ approach to understand more about the nature of professional and organizational resistance to these tools. The lack of incentives that reward the use of these interventions needs to be considered as a significant impediment.

Gut feelings as a third track in general practitioners' diagnostic reasoning

BackgroundGeneral practitioners (GPs) are often faced with complicated, vague problems in situations of uncertainty that they have to solve at short notice. In such situations, gut feelings seem to play a substantial role in their diagnostic process. Qualitative research distinguished a sense of alarm and a sense of reassurance. However, not every GP trusted their gut feelings, since a scientific explanation is lacking.ObjectiveThis paper explains how gut feelings arise and function in GPs’ diagnostic reasoning.ApproachThe paper reviews literature from medical, psychological and neuroscientific perspectives.ConclusionsGut feelings in general practice are based on the interaction between patient information and a GP’s knowledge and experience. This is visualized in a knowledge-based model of GPs’ diagnostic reasoning emphasizing that this complex task combines analytical and non-analytical cognitive processes. The model integrates the two well-known diagnostic reasoning tracks of medical decision-making and medical problem-solving, and adds gut feelings as a third track. Analytical and non-analytical diagnostic reasoning interacts continuously, and GPs use elements of all three tracks, depending on the task and the situation. In this dual process theory, gut feelings emerge as a consequence of non-analytical processing of the available information and knowledge, either reassuring GPs or alerting them that something is wrong and action is required. The role of affect as a heuristic within the physician’s knowledge network explains how gut feelings may help GPs to navigate in a mostly efficient way in the often complex and uncertain diagnostic situations of general practice. Emotion research and neuroscientific data support the unmistakable role of affect in the process of making decisions and explain the bodily sensation of gut feelings.The implications for health care practice and medical education are discussed.

Patient and public involvement in clinical guidelines: international experiences and future perspectives

Background Clinical practice guidelines (CPG) are important tools for improving patient care. Patient and public involvement is recognised as an essential component of CPG development and implementation. The Guideline International Network Patient and Public Involvement Working Group (G-I-N PUBLIC) aims to support the development, implementation and evaluation of guideline-oriented patient and public involvement programmes (PPIPs). Objective To develop an international practice and research agenda on patient and public involvement in CPG. Method 56 CPG developers, researchers, and patient/public representatives from 14 different countries, were consulted in an international workshop. Recommendations were validated with G-I-N PUBLIC steering committee members. Results Many CPG organisations have set up PPIPs that use a range of participation, consultation and communication methods. Current PPIPs aim to improve the quality and responsiveness of CPGs to public expectations and needs, or to foster individual healthcare decisions. Some organisations use structured involvement methods, including providing training for patient and public representatives. A number of financial, organisational and sociopolitical barriers limit patient and public involvement. The paucity of process and impact evaluations limits our current understanding of the conditions under which patient and public involvement is most likely to be effective. Conclusion Greater international collaboration and research are needed to strengthen existing knowledge, development and evaluation of patient and public involvement in CPG.

‘Rapid Learning health care in oncology’ – An approach towards decision support systems enabling customised radiotherapy’ ☆ ☆☆

PURPOSE An overview of the Rapid Learning methodology, its results, and the potential impact on radiotherapy. MATERIAL AND RESULTS Rapid Learning methodology is divided into four phases. In the data phase, diverse data are collected about past patients, treatments used, and outcomes. Innovative information technologies that support semantic interoperability enable distributed learning and data sharing without additional burden on health care professionals and without the need for data to leave the hospital. In the knowledge phase, prediction models are developed for new data and treatment outcomes by applying machine learning methods to data. In the application phase, this knowledge is applied in clinical practice via novel decision support systems or via extensions of existing models such as Tumour Control Probability models. In the evaluation phase, the predictability of treatment outcomes allows the new knowledge to be evaluated by comparing predicted and actual outcomes. CONCLUSION Personalised or tailored cancer therapy ensures not only that patients receive an optimal treatment, but also that the right resources are being used for the right patients. Rapid Learning approaches combined with evidence based medicine are expected to improve the predictability of outcome and radiotherapy is the ideal field to study the value of Rapid Learning. The next step will be to include patient preferences in the decision making.

A systematic development process for patient decision aids.

BackgroundThe original version of the International Patient Decision Aid Standards (IPDAS) recommended that patient decision aids (PtDAs) should be carefully developed, user-tested and open to scrutiny, with a well-documented and systematically applied development process. We carried out a review to check the relevance and scope of this quality dimension and, if necessary, to update it.MethodsOur review drew on three sources: a) published papers describing PtDAs evaluated in randomised controlled trials and included in the most recent Cochrane Collaboration review; b) linked papers cited in the trial reports that described how the PtDAs had been developed; and c) papers and web reports outlining the development process used by organisations experienced in developing multiple PtDAs. We then developed an extended model of the development process indicating the various steps on which documentation is required, as well as a checklist to assess the frequency with which each of the elements was publicly reported.ResultsKey features common to all patient decision aid (PtDA) development processes include: scoping and design; development of a prototype; ‘alpha’ testing with patients and clinicians in an iterative process; ‘beta’ testing in ‘real life’ conditions (field tests); and production of a final version for use and/or further evaluation. Only about half of the published reports on the development of PtDAs that we reviewed appear to have been field tested with patients, and even fewer had been reviewed or tested by clinicians not involved in the development process. Very few described a distribution strategy, and surprisingly few (17%) described a method for reviewing and synthesizing the clinical evidence. We describe a model development process that includes all the original elements of the original IPDAS criterion, expanded to include consideration of format and distribution plans as well as prototype development.ConclusionsThe case for including each of the elements outlined in our model development process is pragmatic rather than evidence-based. Optimal methods for ensuring that each stage of the process is carried out effectively require further development and testing.

Decision aids that really promote shared decision making: the pace quickens

Decision aids can help shared decision making, but most have been hard to produce, onerous to update, and are not being used widely. Thomas Agoritsas and colleagues explore why and describe a new electronic model that holds promise of being more useful for clinicians and patients to use together at the point of care

Family physicians' diagnostic gut feelings are measurable: construct validation of a questionnaire

BackgroundFamily physicians perceive that gut feelings, i.e. a ‘sense of reassurance’ or a ‘sense of alarm’, play a substantial role in diagnostic reasoning. A measuring instrument is desirable for further research. Our objective is to validate a questionnaire measuring the presence of gut feelings in diagnostic reasoning.MethodsWe constructed 16 case vignettes from real practice situations and used the accompanying ‘sense of reassurance’ or the ‘sense of alarm’ as reference labels. Based on the results of an initial study (26 family physicians), we divided the case vignettes into a group involving a clear role for the sense of reassurance or the sense of alarm and a group involving an ambiguous role. 49 experienced family physicians evaluated each 10 vignettes using the questionnaire. Construct validity was assessed by testing hypotheses and an internal consistency procedure was performed.ResultsAs hypothesized we found that the correlations between the reference labels and corresponding items were high for the clear-case vignettes (0.59 – 0.72) and low for the ambiguous-case vignettes (0.08 – 0.23). The agreement between the classification in clear sense of reassurance, clear sense of alarm and ambiguous case vignettes as derived from the initial study and the study population’s judgments was substantial (Kappa = 0.62). Factor analysis showed one factor with opposites for sense of reassurance and sense of alarm items. The questionnaire’s internal consistency was high (0.91). We provided a linguistic validated English-language text of the questionnaire.ConclusionsThe questionnaire appears to be valid. It enables quantitative research into the role of gut feelings and their diagnostic value in family physicians’ diagnostic reasoning.

Barriers to implementing cardiovascular risk tables in routine general practice.

Design Qualitative study. GPs were interviewed after analysing two audiotaped cardiovascular consultations. Setting Primary health care. Subjects A sample of 15 GPs who audiotaped 22 consultations. Main outcome measures Barriers hampering GPs from following the guideline. Results Data saturation was reached after about 13 interviews. The 25 identified barriers were related to the risk table, the GP or to environmental factors. Lack of knowledge and poor communication skills of the GP, along with pressure of work and demanding patients, cause GPs to deviate from the guideline. GPs regard barriers external to themselves as most important. Conclusion Using the risk table as a key element of the high-risk approach in primary prevention encounters many barriers. Merely incorporating risk tables in guidelines is not sufficient for implementation of the guidelines. Time-efficient implementation strategies dealing in particular with the communication and presentation of cardiovascular risk are needed.

Patient and Public Involvement in Clinical Practice Guidelines: A Knowledge Synthesis of Existing Programs

Background. The role of patient and public involvement programs (PPIPs) in developing and implementing clinical practice guidelines (CPGs) has generated great interest. Purpose. The authors sought to identify key components of PPIPs used in developing and implementing CPGs. Data sources. The authors searched bibliographic databases and contacted relevant organizations. Study selection. In total, 2161 articles and reports were retrieved on PPIPs in the development and implementation of CPGs. Of these, 71 qualified for inclusion in the review. Data extraction. Reviewers independently extracted data on key components of PPIPs and barriers and facilitators to their operation. Data synthesis. Over half of the studies were published after 2002, and more than half originated from the United States, the United Kingdom, Australia, and Germany. CPGs that involved patients and the public addressed a variety of health problems, especially mental health and cancer. The most frequently cited objective for using PPIPs in developing CPGs was to incorporate patients’ values or perspectives in CPG recommendations. Patients and their families and caregivers were the parties most often involved. Methods used to recruit PPIP participants included soliciting through patient/public organizations, sending invitations, and receiving referrals and recruits from clinicians. Patients and the public most often participated by taking part in a CPG working group, workshop, meeting, seminar, literature review, or consultation such as a focus group, individual interview, or survey. Patients and the public principally helped formulate recommendations and revise drafts. Limitations. The authors did not contact the authors of the studies. Conclusion. This literature review provides an extensive knowledge base for making PPIPs more effective when developing and implementing CPGs. More research is needed to assess the impact of PPIPs and resources they require.