Mark J. Dobrow

How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability frameworks:

Objective: To review published frameworks that included criteria for the assessment of external validity, applicability and transferability in their assessment of health research. Methods: Five databases were searched for articles relating to the assessment of external validity or applicability and transferability in health research. A coding framework was developed inductively and used to assess which types of criteria were included in the frameworks. Results: Thirty-eight articles describing 25 frameworks were identified. Eleven focused solely on the assessment of applicability and transferability; 14 presented more general decision-making or evidence appraisal frameworks. The criteria were synthesized into four main categories: setting, intervention, outcomes and evidence. None of the frameworks covered all the criteria identified. A major limitation was the lack of empirical data used to develop many frameworks and the apparent lack of assessment of their perceived utility. Conclusion: A validated framework of applicability and transferability would help those aiming to encourage research use, as well as those conducting research. Greater understanding of applicability and transferability could help to encourage the appropriate use of research and the development of research that is more useful.

How can we improve cancer care? A review of interprofessional collaboration models and their use in clinical management.

BACKGROUND Multimodal cancer care requires collaboration among different professionals in various settings. Practice guidelines provide little direction on how this can best be achieved. Research shows that collaborative cancer management is limited, and challenged by numerous issues. The purpose of this research was to describe conceptual models of collaboration, and analyze how they have been applied in the clinical management of cancer patients. METHODS A review of the literature was performed using a two-phase meta-narrative approach. The first phase involved searching for conceptual models of collaboration. Their components and limitations were summarized. The second phase involved targeted searching for empirical research on evaluation of these concepts in the clinical management of cancer patients. Data on study objective, design, and findings were tabulated, and then summarized according to collaborative model and phase of clinical care to identify topics warranting further research. RESULTS Conceptual models for teamwork, interprofessional collaboration, integrated care delivery, interorganizational collaboration, continuity of care, and case management were described. All concepts involve two or more health care professionals that share patient care goals and interact on a continuum from consultative to integrative, varying according to extent and nature of interaction, degree to which decision making is shared, and the scope of patient management (medical versus holistic). Determinants of positive objective and subjective patient, team and organizational outcomes common across models included system or organizational support, team structure and traits, and team processes. Twenty-two studies conducted in ten countries examining these concepts for cancer care were identified. Two were based on an explicit model of collaboration. Many health professionals function through parallel or consultative models of care and are not well integrated. Few interventions or strategies have been applied to promote models that support collaboration. CONCLUSIONS Ongoing development, implementation and evaluation of collaborative cancer management, in the context of both practice and research, would benefit from systematic planning and operationalization. Such an approach is likely to improve patient, professional and organizational outcomes, and contribute to a collective understanding of collaborative cancer care.

Response Audit of an Internet Survey of Health Care Providers and Administrators: Implications for Determination of Response Rates

Background Internet survey modalities often compare unfavorably with traditional survey modalities, particularly with respect to response rates. Response to Internet surveys can be affected by the distribution options and response/collection features employed as well as the existence of automated (out-of-office) replies, automated forwarding, server rejection, and organizational or personal spam filters. However, Internet surveys also provide unparalleled opportunities to track study subjects and examine many of the factors influencing the determination of response rates. Tracking data available for Internet surveys provide detailed information and immediate feedback on a significant component of response that other survey modalities cannot match. This paper presents a response audit of a large Internet survey of more than 5000 cancer care providers and administrators in Ontario, Canada. Objective Building upon the CHEcklist for Reporting Results of Internet E-Surveys (CHERRIES), the main objectives of the paper are to (a) assess the impact of a range of factors on the determination of response rates for Internet surveys and (b) recommend steps for improving published descriptions of Internet survey methods. Methods We audited the survey response data, analyzing the factors that affected the numerator and denominator in the ultimate determination of response. We also conducted a sensitivity analysis to account for the inherent uncertainty associated with the impact of some of the factors on the response rates. Results The survey was initially sent out to 5636 health care providers and administrators. The determination of the numerator was influenced by duplicate/unattached responses and response completeness. The numerator varied from a maximum of 2031 crude (unadjusted) responses to 1849 unique views, 1769 participants, and 1616 complete responses. The determination of the denominator was influenced by forwarding of the invitation email to unknown individuals, server rejections, automated replies, spam filters, and ‘opt out’ options. Based on these factors, the denominator varied from a minimum of 5106 to a maximum of 5922. Considering the different assumptions for the numerator and the denominator, the sensitivity analysis resulted in a 12.5% variation in the response rate (from minimum of 27.3% to maximum of 39.8%) with a best estimate of 32.8%. Conclusions Depending on how the numerator and denominator are chosen, the resulting response rates can vary widely. The CHERRIES statement was an important advance in identifying key characteristics of Internet surveys that can influence response rates. This response audit suggests the need to further clarify some of these factors when reporting on Internet surveys for health care providers and administrators, particularly when using commercially available Internet survey packages for specified, rather than convenience, samples.

Interviewee Transcript Review: assessing the impact on qualitative research

BackgroundThis paper assesses interviewee transcript review (ITR) as a technique for improving the rigour of interview-based, qualitative research. ITR is a process whereby interviewees are provided with verbatim transcripts of their interviews for the purposes of verifying accuracy, correcting errors or inaccuracies and providing clarifications. ITR, in various forms, is widely used among qualitative researchers, however there is limited methodological guidance on how it should be employed and little is known about its actual impact on the transcript, the data, the interviewee or the researcher.MethodsITR was incorporated into a qualitative research study in which 51 key informant interviews were conducted with a range of senior stakeholders within the Canadian health care system. The changes made by interviewees to their transcripts were systematically tracked and categorized using a set of mutually exclusive categories.ResultsThe study found that ITR added little to the accuracy of the transcript and may create complications if the goal of the researcher is to produce a transcript which reflects precisely what was said at the time of the interview. The advantages of ITR are that it allows interviewees the opportunity to edit or clarify information provided in the original interview, with many interviewees providing corrections, clarifications, and in some cases, adding new material to their transcripts. There are also potential disadvantages, such as a bias created by inconsistent data sources or the loss of data when an interviewee chooses to remove valuable material. The impact of ITR on the interviewee may be both positive and negative, depending on the specific circumstances and the nature of the study. The impact of ITR on the researcher was minimal in this study, but is again subject to specific circumstances of the research context.ConclusionWhile ITR is employed by many researchers across numerous fields, the advantages of its use may be relatively small in terms of verifying the accuracy of qualitative interview transcripts. Researchers are advised to carefully consider both the potential advantages and disadvantages of ITR outlined in this paper before deciding to incorporate the practice within their qualitative study designs.

Paucity of qualitative research in general medical and health services and policy research journals: analysis of publication rates

BackgroundQualitative research has the potential to inform and improve health care decisions but a study based on one year of publications suggests that it is not published in prominent health care journals. A more detailed, longitudinal analysis of its availability is needed. The purpose of this study was to identify, count and compare the number of qualitative and non-qualitative research studies published in high impact health care journals, and explore trends in these data over the last decade.MethodsA bibliometric approach was used to identify and quantify qualitative articles published in 20 top general medical and health services and policy research journals from 1999 to 2008. Eligible journals were selected based on performance in four different ranking systems reported in the 2008 ISI Journal Citation Reports. Qualitative and non-qualitative research published in these journals were identified by searching MEDLINE, and validated by hand-searching tables of contents for four journals.ResultsThe total number of qualitative research articles published during 1999 to 2008 in ten general medical journals ranged from 0 to 41, and in ten health services and policy research journals from 0 to 39. Over this period the percentage of empirical research articles that were qualitative ranged from 0% to 0.6% for the general medical journals, and 0% to 6.4% for the health services and policy research journals.ConclusionsThis analysis suggests that qualitative research it is rarely published in high impact general medical and health services and policy research journals. The factors that contribute to this persistent marginalization need to be better understood.

Cost-Effectiveness Analysis Comparing Conventional Versus Stereotactic Body Radiotherapy for Surgically Ineligible Stage I Non-Small-Cell Lung Cancer

INTRODUCTION In 25% to 35% of patients with early stage I non-small-cell lung cancer (NSCLC), surgery is not feasible, and external-beam radiation becomes their standard treatment. Conventionally fractionated radiotherapy (CFRT) is the traditional radiation treatment standard; however, stereotactic body radiotherapy (SBRT) is increasingly being adopted as an alternate radiation treatment. Our objective was to conduct a cost-effectiveness analysis, comparing SBRT with CFRT for stage I NSCLC in a public payer system. METHODS Consecutive patients were reviewed using 2010 Canadian dollars for direct medical costs from a public payer perspective. A subset of direct radiation treatment delivery costs, excluding physician billings and hospitalization, was also included. Health outcomes as life-years gained (LYGs) were computed using time-to-event methods. Sensitivity analyses identified critical factors influencing costs and benefits. RESULTS From January 2002 to June 2010, 168 patients (CFRT, n = 50; SBRT, n = 118) were included; median follow-up was 24 months. Mean overall survival was 2.83 years (95% CI, 1.8 to 4.1) for CFRT and 3.86 years (95% CI, 3.2 to not reached) for SBRT (P = .06). Mean costs for CFRT were

Multidisciplinary cancer conferences: Exploring the attitudes of cancer care providers and administrators

6,886 overall and

Incremental Cost Analysis of Ambulatory Clinic and Home-Based Intravenous Therapy for Patients with Multiple Myeloma

5,989 for radiation treatment delivery only versus

Referring patients to nurses: Outcomes and evaluation of a nurse flexible sigmoidoscopy training program for colorectal cancer screening

8,042 and

Identifying the conditions needed for integrated knowledge translation (IKT) in health care organizations: qualitative interviews with researchers and research users.

6,962, respectively, for SBRT. Incremental costs (incremental cost-effectiveness ratio [ICER]) per LYG for SBRT versus CFRT were