Gillian Bartlett

Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review

BACKGROUND Systematic literature reviews identify, select, appraise, and synthesize relevant literature on a particular topic. Typically, these reviews examine primary studies based on similar methods, e.g., experimental trials. In contrast, interest in a new form of review, known as mixed studies review (MSR), which includes qualitative, quantitative, and mixed methods studies, is growing. In MSRs, reviewers appraise studies that use different methods allowing them to obtain in-depth answers to complex research questions. However, appraising the quality of studies with different methods remains challenging. To facilitate systematic MSRs, a pilot Mixed Methods Appraisal Tool (MMAT) has been developed at McGill University (a checklist and a tutorial), which can be used to concurrently appraise the methodological quality of qualitative, quantitative, and mixed methods studies. OBJECTIVES The purpose of the present study is to test the reliability and efficiency of a pilot version of the MMAT. METHODS The Center for Participatory Research at McGill conducted a systematic MSR on the benefits of Participatory Research (PR). Thirty-two PR evaluation studies were appraised by two independent reviewers using the pilot MMAT. Among these, 11 (34%) involved nurses as researchers or research partners. Appraisal time was measured to assess efficiency. Inter-rater reliability was assessed by calculating a kappa statistic based on dichotomized responses for each criterion. An appraisal score was determined for each study, which allowed the calculation of an overall intra-class correlation. RESULTS On average, it took 14 min to appraise a study (excluding the initial reading of articles). Agreement between reviewers was moderate to perfect with regards to MMAT criteria, and substantial with respect to the overall quality score of appraised studies. CONCLUSION The MMAT is unique, thus the reliability of the pilot MMAT is promising, and encourages further development.

Impact of patient communication problems on the risk of preventable adverse events in acute care settings

Background: Up to 50% of adverse events that occur in hospitals are preventable. Language barriers and disabilities that affect communication have been shown to decrease quality of care. We sought to assess whether communication problems are associated with an increased risk of preventable adverse events. Methods: We randomly selected 20 general hospitals in the province of Quebec with at least 1500 annual admissions. Of the 145 672 admissions to the selected hospitals in 2000/01, we randomly selected and reviewed 2355 charts of patients aged 18 years or older. Reviewers abstracted patient characteristics, including communication problems, and details of hospital admission, and assessed the cause and preventability of identified adverse events. The primary outcome was adverse events. Results: Of 217 adverse events, 63 (29%) were judged to be preventable, for an overall population rate of 2.7% (95% confidence interval [CI] 2.1%–3.4%). We found that patients with preventable adverse events were significantly more likely than those without such events to have a communication problem (odds ratio [OR] 3.00; 95% CI 1.43–6.27) or a psychiatric disorder (OR 2.35; 95% CI 1.09–5.05). Patients who were admitted urgently were significantly more likely than patients whose admissions were elective to experience an event (OR 1.64, 95% CI 1.07–2.52). Preventable adverse events were mainly due to drug errors (40%) or poor clinical management (32%). We found that patients with communication problems were more likely than patients without these problems to experience multiple preventable adverse events (46% v. 20%; p = 0.05). Interpretation: Patients with communication problems appeared to be at highest risk for preventable adverse events. Interventions to reduce the risk for these patients need to be developed and evaluated.

A 5-year prospective assessment of the risk associated with individual benzodiazepines and doses in new elderly users.

Objectives: To determine the risk of injury associated with the new use of individual benzodiazepines and dosage regimens in the elderly.

A Randomized Trial of the Effectiveness of On-demand versus Computer-triggered Drug Decision Support in Primary Care

OBJECTIVES Prescribing alerts generated by computerized drug decision support (CDDS) may prevent drug-related morbidity. However, the vast majority of alerts are ignored because of clinical irrelevance. The ability to customize commercial alert systems should improve physician acceptance because the physician can select the circumstances and types of drug alerts that are viewed. We tested the effectiveness of two approaches to medication alert customization to reduce prevalence of prescribing problems: on-physician-demand versus computer-triggered decision support. Physicians in each study condition were able to preset levels that triggered alerts. DESIGN This was a cluster trial with 28 primary care physicians randomized to either automated or on-demand CDDS in the MOXXI drug management system for 3,449 of their patients seen over the next 6 months. MEASUREMENTS The CDDS generated alerts for prescribing problems that could be customized by severity level. Prescribing problems included dosing errors, drug-drug, age, allergy, and disease interactions. Physicians randomized to on-demand activated the drug review when they considered it clinically relevant, whereas physicians randomized to computer-triggered decision support viewed all alerts for electronic prescriptions in accordance with the severity level they selected for both prevalent and incident problems. Data from administrative claims and MOXXI were used to measure the difference in the prevalence of prescribing problems at the end of follow-up. RESULTS During follow-up, 50% of the physicians receiving computer-triggered alerts modified the alert threshold (n = 7), and 21% of the physicians in the alert-on-demand group modified the alert level (n = 3). In the on-demand group 4,445 prescribing problems were identified, 41 (0.9%) were seen by requested drug review, and in 31 problems (75.6%) the prescription was revised. In comparison, 668 (10.3%) of the 6,505 prescribing problems in the computer-triggered group were seen, and 81 (12.1%) were revised. The majority of alerts were ignored because the benefit was judged greater than the risk, the interaction was known, or the interaction was considered clinically not important (computer-triggered: 75.8% of 585 ignored alerts; on-demand: 90% of 10 ignored alerts). At the end of follow-up, there was a significant reduction in therapeutic duplication problems in the computer-triggered group (odds ratio 0.55; p = 0.02) but no difference in the overall prevalence of prescribing problems. CONCLUSION Customization of computer-triggered alert systems is more useful in detecting and resolving prescribing problems than on-demand review, but neither approach was effective in reducing prescribing problems. New strategies are needed to maximize the use of drug decision support systems to reduce drug-related morbidity.

Normal distributions of thermal and vibration sensory thresholds

The distributions of sensory thresholds were estimated in a healthy population while controlling for potential covariates. Using the method of levels and the two‐alternative forced choice, thermal and vibration thresholds respectively were measured in the hand and foot of 148 subjects. Age was uniformly distributed between 20 and 86 years. Independent effects of age, gender, height, and skin temperature were estimated using multiple linear regression. Parametric and nonparametric methods were used to estimate the distributions of interest. Significant age‐related increases were observed for all vibration thresholds (P < 0.0001), and for thermal thresholds in the foot (P < 0.0002). Percentiles were estimated for thermal thresholds in the hand and age‐adjusted continuous distributions were calculated for all other thresholds. Height was positively associated with vibration thresholds in the foot (P < 0.003), and appropriate corrections were made. Our results provide reference values for thermal and vibration sensory thresholds in a healthy population, allowing for the accurate diagnosis of disordered sensory function.

Effect of a community oriented problem based learning curriculum on quality of primary care delivered by graduates: historical cohort comparison study

Abstract Objective To assess whether the transition from a traditional curriculum to a community oriented problem based learning curriculum at Sherbrooke University is associated with the expected improvements in preventive care and continuity of care without a decline in diagnosis and management of disease. Design Historical cohort comparison study. Setting Sherbrooke University and three traditional medical schools in Quebec, Canada. Participants 751 doctors from four graduation cohorts (1988-91); three before the transition to community based problem based learning (n = 600) and one after the transition (n = 151). Outcome measures Annual performance in preventive care (mammography screening rate), continuity of care, diagnosis (difference in prescribing rates for specific diseases and relief of symptoms), and management (prescribing rate for contraindicated drugs) assessed using provincial health databases for the first 4-7 years of practice. Results After transition to a community oriented problem based learning curriculum, graduates of Sherbrooke University showed a statistically significant improvement in mammography screening rates (55 more women screened per 1000, 95% confidence interval 10.6 to 99.3) and continuity of care (3.3% more visits coordinated by the doctor, 0.9% to 5.8%) compared with graduates of a traditional medical curriculum. Indicators of diagnostic and management performance did not show the hypothesised decline. Sherbrooke graduates showed a significant fourfold increase in disease specific prescribing rates compared with prescribing for symptom relief after the transition. Conclusion Transition to a community oriented problem based learning curriculum was associated with significant improvements in preventive care and continuity of care and an improvement in indicators of diagnostic performance.

Influence of Postoperative Infectious Complications on Long-Term Survival of Lung Cancer Patients: A Population-Based Cohort Study

Introduction: Surgery is essential to any curative plan for lung cancer, but is associated with a high complication rate. We sought to determine the impact of complications on long-term survival after a curative surgery for lung cancer, independent of the effect on early postoperative mortality. Methods: We studied a population-based cohort of patients with lung cancer who underwent curative-intent surgery in the province of Quebec, Canada, from 2000 to 2005. Kaplan-Meier survival analysis was used to compare unadjusted overall survival (OS) beyond postoperative day 90 for patients with and without complications. Cox regression was used to determine the prognostic impact of 30-day postoperative complications on the OS after adjusting for several confounders. Results: The overall 30-day postoperative complication rate was 58.2% among 4033 eligible patients. A major infectious complication (pneumonia, empyema, or mediastinitis) occurred in 378 patients. The 5-year OS was lower for those with any postoperative complication (62.8%) than those without (73.8%; p < 0.001). Those with major infectious complications had the lowest OS (56.3%; p < 0.001). Postoperative complication was an independent prognostic factor after adjusting for several patient and treatment factors (hazard ratio = 1.37; 95% confidence interval, 1.21–1.54). Adjusted hazard ratio for major infectious complications was 1.67 (95% confidence interval, 1.39–2.01). Conclusions: Postoperative complications, particularly of a major infectious type, are strong negative predictors of long-term survival in lung cancer patients. The strong association between major infectious complications and survival may also open the door to investigational therapies targeting bacterial antigens in the perioperative period in patients who undergo lung cancer surgery.

Four levels of outcomes of information-seeking: A mixed methods study in primary health care

Primary health care practitioners routinely search for information within electronic knowledge resources. We proposed four levels of outcomes of information-seeking: situational relevance, cognitive impact, information use, and patient health outcomes. Our objective was to produce clinical vignettes for describing and testing these levels. We conducted a mixed methods study combining a quantitative longitudinal study and a qualitative multiple case study. Participants were 10 nurses, 10 medical residents, and 10 pharmacists. They had access to an online resource, and did 793 searches for treatment recommendations. Using the Information Assessment Method (IAM), participants rated their searches for each of the four levels. Rated searches were examined in interviews guided by log reports and a think-aloud protocol. Cases were defined as clearly described searches where clinical information was used for a specific patient. For each case, interviewees described the four levels of outcomes. Quantitative and qualitative data were merged into clinical vignettes. We produced 130 clinical vignettes. Specifically, 46 vignettes (35.4%) corresponded to clinical situations where information use was associated with one or more than one type of positive patient health outcome: increased patient knowledge (n = 28), avoidance of unnecessary or inappropriate intervention (n = 25), prevention of disease or health deterioration (n = 9), health improvement (n = 6), and increased patient satisfaction (n = 3). Results suggested information use was associated with perceived benefits for patients. This may encourage clinicians to search for information more often when they feel the need. Results supported the four proposed levels of outcomes, which can be transferable to other information-seeking contexts.

The effect of obesity on antibiotic treatment failure: a historical cohort study.

Obesity, a major health issue, is also an important risk factor for infections. Evidence demonstrates that excess weight affects the disposition of antibiotics but little work has been done to explore if this results in antibiotic treatment failure (ATF). ATF has serious adverse health outcomes and may increase treatment resistance. Given that obese patients often have other health issues, it is important to determine if excess weight independently increases the likelihood of ATF.

Physicians' assessment of the value of clinical information: Operationalization of a theoretical model

Inspired by the acquisition–cognition–application model (T. Saracevic & K.B. Kantor, 1997), we developed a tool called the Information Assessment Method to more clearly understand how physicians use clinical information. In primary healthcare, we conducted a naturalistic and longitudinal study of searches for clinical information. Forty-one family physicians received a handheld computer with the Information Assessment Method linked to one commercial electronic knowledge resource. Over an average of 320 days, 83% of 2,131 searches for clinical information were rated using the Information Assessment Method. Searches to address a clinical question, as well as the retrieval of relevant clinical information, were positively associated with the use of that information for a specific patient. Searches done out of curiosity were negatively associated with the use of clinical information. We found significant associations between specific types of cognitive impact and information use for a specific patient. For example, when the physician reported “My practice was changed and improved” as a result of this clinical information, the odds that information was used for a specific patient increased threefold. Our findings provide empirical data to support the applicability of the acquisition-cognition-application model, as operationalized through the Information Assessment Method, in primary healthcare. Capturing the use of research-based information in medicine opens the door to further study of the relationships between clinical information and health outcomes.