Peter Hibbert

CareTrack: Assessing the appropriateness of health care delivery in Australia

Objective: To determine the percentage of health care encounters at which a sample of adult Australians received appropriate care (ie, care in line with evidence‐based or consensus‐based guidelines).

The application of the Global Trigger Tool: a systematic review

PurposenThis study describes the use of, and modifications and additions made to, the Global Trigger Tool (GTT) since its first release in 2003, and summarizes its findings with respect to counting and characterizing adverse events (AEs).nnnData sourcesnPeer-reviewed literature up to 31st December 2014.nnnStudy selectionnA systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.nnnData extractionnTwo authors extracted and compiled the demographics, methodologies and results of the selected studies.nnnResults of data synthesisnOf the 48 studies meeting the eligibility criteria, 44 collected data from inpatient medical records and four from general practice records. Studies were undertaken in 16 countries. Over half did not follow the standard GTT protocol regarding the number of reviewers used. Acts of omission were included in one quarter of studies. Incident reporting detected between 2% and 8% of AEs that were detected with the GTT. Rates of AEs varied in general inpatient studies between 7% and 40%. Infections, problems with surgical procedures and medication were the most common incident types.nnnConclusionnThe GTT is a flexible tool used in a range of settings with varied applications. Substantial differences in AE rates were evident across studies, most likely associated with methodological differences and disparate reviewer interpretations. AE rates should not be compared between institutions or studies. Recommendations include adding omission AEs, using preventability scores for priority setting, and re-framing the GTTs purpose to understand and characterize AEs rather than just counting them.

Assessing the appropriateness of prevention and management of venous thromboembolism in Australia: a cross-sectional study

Objectives The prevention and management of venous thromboembolism (VTE) is often at variance with guidelines. The CareTrack Australia (CTA) study reported that appropriate care (in line with evidence-based or consensus-based guidelines) is being provided for VTE at just over half of eligible encounters. The aim of this paper is to present and discuss the detailed CTA findings for VTE as a baseline for compliance with guidelines at a population level. Setting The setting was 27 hospitals in 2 states of Australia. Participants A sample of participants designed to be representative of the Australian population was recruited. Participants who had been admitted overnight during 2009 and/or 2010 were eligible. Of the 1154 CTA participants, 481(42%) were admitted overnight to hospital at least once, comprising 751 admissions. There were 279 females (58%), and the mean age was 64u2005years. Primary and secondary outcome measures The primary measure was compliance with indicators of appropriate care for VTE. The indicators were extracted from Australian VTE clinical practice guidelines and ratified by experts. Participants’ medical records from 2009 to 2010 were analysed for compliance with 38 VTE indicators. Results Of the 35u2005145 CTA encounters, 1078 (3%) were eligible for scoring against VTE indicators. There were 2–84 eligible encounters per indicator at 27 hospitals. Overall compliance with indicators for VTE was 51%, and ranged from 34% to 64% for aggregated sets of indicators. Conclusions The prevention and management of VTE was appropriate for only half of the at-risk patients in our sample; this provides a baseline for tracking progress nationally. There is a need for national and, ideally, international agreement on clinical standards, indicators and tools to guide, document and monitor care for VTE, and for measures to increase their uptake, particularly where deficiencies have been identified.

Quality of Health Care for Children in Australia, 2012-2013.

Importance The quality of routine care for children is rarely assessed, and then usually in single settings or for single clinical conditions. Objective To estimate the quality of health care for children in Australia in inpatient and ambulatory health care settings. Design, Setting, and Participants Multistage stratified sample with medical record review to assess adherence with quality indicators extracted from clinical practice guidelines for 17 common, high-burden clinical conditions (noncommunicable [nu2009=u20095], mental health [nu2009=u20094], acute infection [nu2009=u20097], and injury [nu2009=u20091]), such as asthma, attention-deficit/hyperactivity disorder, tonsillitis, and head injury. For these 17 conditions, 479 quality indicators were identified, with the number varying by condition, ranging from 9 for eczema to 54 for head injury. Four hundred medical records were targeted for sampling for each of 15 conditions while 267 records were targeted for anxiety and 133 for depression. Within each selected medical record, all visits for the 17 targeted conditions were identified, and separate quality assessments made for each. Care was evaluated for 6689 children 15 years of age and younger who had 15 240 visits to emergency departments, for inpatient admissions, or to pediatricians and general practitioners in selected urban and rural locations in 3 Australian states. These visits generated 160 202 quality indicator assessments. Exposures Quality indicators were identified through a systematic search of local and international guidelines. Individual indicators were extracted from guidelines and assessed using a 2-stage Delphi process. Main Outcomes and Measures Quality of care for each clinical condition and overall. Results Of 6689 children with surveyed medical records, 53.6% were aged 0 to 4 years and 55.5% were male. Adherence to quality of care indicators was estimated at 59.8% (95% CI, 57.5%-62.0%; nu2009=u2009160 202) across the 17 conditions, ranging from a high of 88.8% (95% CI, 83.0%-93.1%; nu2009=u20092638) for autism to a low of 43.5% (95% CI, 36.8%-50.4%; nu2009=u20092354) for tonsillitis. The mean adherence by condition category was estimated as 60.5% (95% CI, 57.2%-63.8%; nu2009=u200941 265) for noncommunicable conditions (range, 52.8%-75.8%); 82.4% (95% CI, 79.0%-85.5%; nu2009=u200914 622) for mental health conditions (range, 71.5%-88.8%); 56.3% (95% CI, 53.2%-59.4%; nu2009=u200994 037) for acute infections (range, 43.5%-69.8%); and 78.3% (95% CI, 75.1%-81.2%; nu2009=u200910 278) for injury. Conclusions and Relevance Among a sample of children receiving care in Australia in 2012-2013, the overall prevalence of adherence to quality of care indicators for important conditions was not high. For many of these conditions, the quality of care may be inadequate.

CareTrack Kids-part 3. Adverse events in children's healthcare in Australia: study protocol for a retrospective medical record review.

Introduction A high-quality health system should deliver care that is free from harm. Few large-scale studies of adverse events have been undertaken in childrens healthcare internationally, and none in Australia. The aim of this study is to measure the frequency and types of adverse events encountered in Australian paediatric care in a range of healthcare settings. Methods and analysis A form of retrospective medical record review, the Institute of Healthcare Improvements Global Trigger Tool, will be modified to collect data. Records of children aged <16u2005years managed during 2012 and 2013 will be reviewed. We aim to review 6000–8000 records from a sample of healthcare practices (hospitals, general practices and specialists). Ethics and dissemination Human Research Ethics Committee approvals have been received from the Sydney Childrens Hospital Network, Childrens Health Queensland Hospital and Health Service, and the Womens and Childrens Hospital Network in South Australia. An application is under review with the Royal Australian College of General Practitioners. The authors will submit the results of the study to relevant journals and undertake national and international oral presentations to researchers, clinicians and policymakers.

Comparing clinical quality indicators for asthma management in children with outcome measures used in randomised controlled trials: a protocol

Introduction Clinical quality indicators are necessary to monitor the performance of healthcare services. The development of indicators should, wherever possible, be based on research evidence to minimise the risk of bias which may be introduced during their development, because of logistic, ethical or financial constraints alone. The development of automated methods to identify the evidence base for candidate indicators should improve the process of indicator development. The objective of this study is to explore the relationship between clinical quality indicators for asthma management in children with outcome and process measurements extracted from randomised controlled clinical trial reports. Methods and analysis National-level indicators for asthma management in children will be extracted from the National Quality Measures Clearinghouse (NQMC) database and the National Institute for Health and Care Excellence (NICE) quality standards. Outcome measures will be extracted from published English language randomised controlled trial (RCT) reports for asthma management in children aged below 12u2005years. The two sets of measures will be compared to assess any overlap. The study will provide insights into the relationship between clinical quality indicators and measurements in RCTs. This study will also yield a list of measurements used in RCTs for asthma management in children, and will find RCT evidence for indicators used in practice. Ethics and dissemination Ethical approval is not necessary because this study will not include patient data. Findings will be disseminated through peer-reviewed publications.

Are root cause analyses recommendations effective and sustainable? An observational study

ObjectivenTo assess the strength of root cause analysis (RCA) recommendations and their perceived levels of effectiveness and sustainability.nnnDesignnAll RCAs related to sentinel events (SEs) undertaken between the years 2010 and 2015 in the public health system in Victoria, Australia were analysed. The type and strength of each recommendation in the RCA reports were coded by an expert patient safety classifier using the US Department of Veteran Affairs type and strength criteria.nnnParticipants and settingnThirty-six public health services.nnnMain outcome measure(s)nThe proportion of RCA recommendations which were classified as strong (more likely to be effective and sustainable), medium (possibly effective and sustainable) or weak (less likely to be effective and sustainable).nnnResultsnThere were 227 RCAs in the period of study. In these RCAs, 1137 recommendations were made. Of these 8% were strong, 44% medium and 48% were weak. In 31 RCAs, or nearly 15%, only weak recommendations were made. In 24 (11%) RCAs five or more weak recommendations were made. In 165 (72%) RCAs no strong recommendations were made. The most frequent recommendation types were reviewing or enhancing a policy/guideline/documentation, and training and education.nnnConclusionsnOnly a small proportion of recommendations arising from RCAs in Victoria are strong. This suggests that insights from the majority of RCAs are not likely to inform practice or process improvements. Suggested improvements include more human factors expertise and independence in investigations, more extensive application of existing tools that assist teams to prioritize recommendations that are likely to be effective, and greater use of observational and simulation techniques to understand the underlying systems factors. Time spent in repeatedly investigating similar incidents may be better spent aggregating and thematically analysing existing sources of information about patient safety.

Linking quality indicators to clinical trials: an automated approach

Abstract Objective Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations. Design A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov. Intervention Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity. Main Outcome Measure Number of care indicators that mapped to outcome measures in clinical trials. Results While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method. Conclusions We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

Linking clinical quality indicators to research evidence - a case study in asthma management for children

BackgroundClinical quality indicators are used to monitor the performance of healthcare services and should wherever possible be based on research evidence. Little is known however about the extent to which indicators in common use are based on research. The objective of this study is to measure the extent to which clinical quality indicators used in asthma management in children with outcome measurements can be linked to results in randomised controlled clinical trial (RCT) reports. This work is part of a broader research program to trial methods that improve the efficiency and accuracy of indicator development.MethodsNational-level indicators for asthma management in children were extracted from the National Quality Measures Clearinghouse database and the National Institute for Health and Care Excellence quality standards by two independent appraisers. Outcome measures were extracted from all published English language RCT reports for asthma management in children below the age of 12 published between 2005 and 2014. The two sets were then linked by manually mapping both to a common set of Unified Medical Language System (UMLS) concepts.ResultsThe analysis identified 39 indicators and 562 full text RCTs dealing with asthma management in children. About 95% (37/39) of the indicators could be linked to RCT outcome measures.ConclusionsIt is possible to identify relevant RCT reports for the majority of indicators used to assess the quality of asthma management in childhood. The methods reported here could be automated to more generally support assessment of candidate indicators against the research evidence.

STANDING Collaboration: a study protocol for developing clinical standards

Introduction Despite widespread availability of clinical practice guidelines (CPGs), considerable gaps continue between the care that is recommended (‘appropriate care’) and the care provided. Problems with current CPGs are commonly cited as barriers to providing ’appropriate care. Our study aims to develop and test an alternative method to keep CPGs accessible and up to date. This method aims to mitigate existing problems by using a single process to develop clinical standards (embodied in clinical indicators) collaboratively with researchers, healthcare professionals, patients and consumers. A transparent and inclusive online curated (purpose-designed, custom-built, wiki-type) system will use an ongoing and iterative documentation process to facilitate synthesis of up-to-date information and make available its provenance. All participants are required to declare conflicts of interest. This protocol describes three phases: engagement of relevant stakeholders; design of a process to develop clinical standards (embodied in indicators) for ‘appropriate care’ for common medical conditions; and evaluation of our processes, products and feasibility. Methods and analysis A modified e-Delphi process will be used to gain consensus on ‘appropriate care’ for a range of common medical conditions. Clinical standards and indicators will be developed through searches of national and international guidelines, and formulated with explicit criteria for inclusion, exclusion, time frame and setting. Healthcare professionals and consumers will review the indicators via the wiki-based modified e-Delphi process. Reviewers will declare conflicts of interest which will be recorded and managed according to an established protocol. The provenance of all indicators and suggestions included or excluded will be logged from indicator inception to finalisation. A mixed-methods formative evaluation of our research methodology will be undertaken. Ethics and dissemination Human Research Ethics Committee approval has been received from the University of South Australia. We will submit the results of the study to relevant journals and offer national and international presentations.