J.Y. Verbakel

Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs)

Clinical Prediction Rules (CPRs) are tools that quantify the contribution of symptoms, clinical signs and available diagnostic tests, and in doing so stratify patients according to the probability of having a target outcome or need for a specified treatment. Most focus on the derivation stage with only a minority progressing to validation and very few undergoing impact analysis. Impact analysis studies remain the most efficient way of assessing whether incorporating CPRs into a decision making process improves patient care. However there is a lack of clear methodology for the design of high quality impact analysis studies.We have developed a sequential four-phased framework based on the literature and the collective experience of our international working group to help researchers identify and overcome the specific challenges in designing and conducting an impact analysis of a CPR.There is a need to shift emphasis from deriving new CPRs to validating and implementing existing CPRs. The proposed framework provides a structured approach to this topical and complex area of research.

How well do clinical prediction rules perform in identifying serious infections in acutely ill children across an international network of ambulatory care datasets

BackgroundDiagnosing serious infections in children is challenging, because of the low incidence of such infections and their non-specific presentation early in the course of illness. Prediction rules are promoted as a means to improve recognition of serious infections. A recent systematic review identified seven clinical prediction rules, of which only one had been prospectively validated, calling into question their appropriateness for clinical practice. We aimed to examine the diagnostic accuracy of these rules in multiple ambulatory care populations in Europe.MethodsFour clinical prediction rules and two national guidelines, based on signs and symptoms, were validated retrospectively in seven individual patient datasets from primary care and emergency departments, comprising 11,023 children from the UK, the Netherlands, and Belgium. The accuracy of each rule was tested, with pre-test and post-test probabilities displayed using dumbbell plots, with serious infection settings stratified as low prevalence (LP; <5%), intermediate prevalence (IP; 5 to 20%), and high prevalence (HP; >20%) . In LP and IP settings, sensitivity should be >90% for effective ruling out infection.ResultsIn LP settings, a five-stage decision tree and a pneumonia rule had sensitivities of >90% (at a negative likelihood ratio (NLR) of < 0.2) for ruling out serious infections, whereas the sensitivities of a meningitis rule and the Yale Observation Scale (YOS) varied widely, between 33 and 100%. In IP settings, the five-stage decision tree, the pneumonia rule, and YOS had sensitivities between 22 and 88%, with NLR ranging from 0.3 to 0.8. In an HP setting, the five-stage decision tree provided a sensitivity of 23%. In LP or IP settings, the sensitivities of the National Institute for Clinical Excellence guideline for feverish illness and the Dutch College of General Practitioners alarm symptoms ranged from 81 to 100%.ConclusionsNone of the clinical prediction rules examined in this study provided perfect diagnostic accuracy. In LP or IP settings, prediction rules and evidence-based guidelines had high sensitivity, providing promising rule-out value for serious infections in these datasets, although all had a percentage of residual uncertainty. Additional clinical assessment or testing such as point-of-care laboratory tests may be needed to increase clinical certainty. None of the prediction rules identified seemed to be valuable for HP settings such as emergency departments.

The Predictive Value of the NICE "Red Traffic Lights" in Acutely Ill Children

Objective Early recognition and treatment of febrile children with serious infections (SI) improves prognosis, however, early detection can be difficult. We aimed to validate the predictive rule-in value of the National Institute for Health and Clinical Excellence (NICE) most severe alarming signs or symptoms to identify SI in children. Design, Setting and Participants The 16 most severe (“red”) features of the NICE traffic light system were validated in seven different primary care and emergency department settings, including 6,260 children presenting with acute illness. Main Outcome Measures We focussed on the individual predictive value of single red features for SI and their combinations. Results were presented as positive likelihood ratios, sensitivities and specificities. We categorised “general” and “disease-specific” red features. Changes in pre-test probability versus post-test probability for SI were visualised in Fagan nomograms. Results Almost all red features had rule-in value for SI, but only four individual red features substantially raised the probability of SI in more than one dataset: “does not wake/stay awake”, “reduced skin turgor”, “non-blanching rash”, and “focal neurological signs”. The presence of ≥3 red features improved prediction of SI but still lacked strong rule-in value as likelihood ratios were below 5. Conclusions The rule-in value of the most severe alarming signs or symptoms of the NICE traffic light system for identifying children with SI was limited, even when multiple red features were present. Our study highlights the importance of assessing the predictive value of alarming signs in clinical guidelines prior to widespread implementation in routine practice.

Subjective assessment versus ultrasound models to diagnose ovarian cancer: A systematic review and meta-analysis

INTRODUCTION Many national guidelines concerning the management of ovarian cancer currently advocate the risk of malignancy index (RMI) to characterise ovarian pathology. However, other methods, such as subjective assessment, International Ovarian Tumour Analysis (IOTA) simple ultrasound-based rules (simple rules) and IOTA logistic regression model 2 (LR2) seem to be superior to the RMI. Our objective was to compare the diagnostic accuracy of subjective assessment, simple rules, LR2 and RMI for differentiating benign from malignant adnexal masses prior to surgery. MATERIALS AND METHODS MEDLINE, EMBASE and CENTRAL were searched (January 1990-August 2015). Eligibility criteria were prospective diagnostic studies designed to preoperatively predict ovarian cancer in women with an adnexal mass. RESULTS We analysed 47 articles, enrolling 19,674 adnexal tumours; 13,953 (70.9%) benign and 5721 (29.1%) malignant. Subjective assessment by experts performed best with a pooled sensitivity of 0.93 (95% confidence interval [CI] 0.92-0.95) and specificity of 0.89 (95% CI 0.86-0.92). Simple rules (classifying inconclusives as malignant) (sensitivity 0.93 [95% CI 0.91-0.95] and specificity 0.80 [95% CI 0.77-0.82]) and LR2 (sensitivity 0.93 [95% CI 0.89-0.95] and specificity 0.84 [95% CI 0.78-0.89]) outperformed RMI (sensitivity 0.75 [95% CI 0.72-0.79], specificity 0.92 [95% CI 0.88-0.94]). A two-step strategy using simple rules, when inconclusive added by subjective assessment, matched test performance of subjective assessment by expert examiners (sensitivity 0.91 [95% CI 0.89-0.93] and specificity 0.91 [95% CI 0.87-0.94]). CONCLUSIONS A two-step strategy of simple rules with subjective assessment for inconclusive tumours yielded best results and matched test performance of expert ultrasound examiners. The LR2 model can be used as an alternative if an expert is not available.

Analytical accuracy and user-friendliness of the Afinion point-of-care CRP test.

In children it is often essential to recognise serious infections at an early stage to reduce possible life-threatening complications. C reactive protein (CRP) is an acute-phase protein, secreted in response to any infection or inflammation.1 Venous blood sampling can be difficult in children in ambulatory care. A point-of-care (POC) test, provided at the bedside, presents an immediate result from a droplet of blood and is especially useful in children. Previous generations of POC CRP tests have shown good correlation with standard laboratory tests in studies in primary care and emergency departments.1–3 Measuring CRP could contribute to clinical decision-making in diagnosing serious infection.4 We determined the analytical accuracy (closeness of the agreement between the measurement results and a true value) and user-friendliness of the Afinion CRP test (on the Afinion AS100 Analyzer, Alere, USA), in children and adults. To assess analytical accuracy, we performed POC CRP tests in children (aged 1 month–18 years) admitted to an inpatient paediatric unit or attending an outpatient paediatric clinic, and in adults (aged 18–65 years) attending a general practice surgery. User-friendliness was evaluated by the participating general practitioners. This study was approved by the ethical review board of the KU Leuven, under reference ML8239. ### Afinion CRP test The Afinion CRP Test Cartridge consists of a 1.5 µL glass capillary and a reagent container. The result is available within 4 min and the measuring range for CRP is 5–200 mg/L. One physician (JYV) performed all POC CRP tests in children, executing every finger stick in a similar fashion (lateral side of …

The correlation between blood pressure and kidney function decline in older people: a registry-based cohort study

Objectives To examine the relation between static and dynamic blood pressure (BP) measurements and the evolution of kidney function in older people, adjusted for the presence of multimorbidity. Design Retrospective cohort study during a 10-year time interval (2002–2012) in three age strata of patients aged 60 and older. Setting Primary care registration network with 97 general practitioners working in 55 practices regularly submitting collected patient data. Participants All patients with at least one BP measurement in 2002 and at least four serum creatine measurements after 2002 (n=8636). A modified Charlson Comorbidity Index (mCCI) at baseline was registered. Change in systolic and diastolic BP (DBP) and pulse pressure (PP) from 2002 onwards was calculated. The relation between kidney function evolution and baseline BP and change in BP was examined using linear and logistic regression analysis. Main outcome measures The slope of the estimated glomerular filtration rate (eGFR, MDRD, Modification of Diet in Renal Disease equation) was calculated by the ordinal least square method. A rapid annual decline of kidney function was defined as ≥3 mL/min/1.73 m2/year. Results Rapid annual decline of kidney function occurred in 1130 patients (13.1%). High baseline systolic BP (SBP) and PP predicted kidney function decline in participants aged 60–79 years. No correlation between baseline BP and kidney function decline was found in participants aged 80 years and older. An annual decline of ≥1 mm Hg in SBP and PP was a strong risk factor for a rapid annual kidney function decline in all age strata, independent of baseline BP and mCCI. A decline in DBP as also a strong independent predictor in participants aged 60–79 years. Conclusions The present study identified a decline in BP over time as a strong risk factor for kidney function decline in all age strata, adjusted for mCCI and baseline kidney function and BP.

Validating a decision tree for serious infection: diagnostic accuracy in acutely ill children in ambulatory care

Objective Acute infection is the most common presentation of children in primary care with only few having a serious infection (eg, sepsis, meningitis, pneumonia). To avoid complications or death, early recognition and adequate referral are essential. Clinical prediction rules have the potential to improve diagnostic decision-making for rare but serious conditions. In this study, we aimed to validate a recently developed decision tree in a new but similar population. Design Diagnostic accuracy study validating a clinical prediction rule. Setting and participants Acutely ill children presenting to ambulatory care in Flanders, Belgium, consisting of general practice and paediatric assessment in outpatient clinics or the emergency department. Intervention Physicians were asked to score the decision tree in every child. Primary outcome measures The outcome of interest was hospital admission for at least 24 h with a serious infection within 5 days after initial presentation. We report the diagnostic accuracy of the decision tree in sensitivity, specificity, likelihood ratios and predictive values. Results In total, 8962 acute illness episodes were included, of which 283 lead to admission to hospital with a serious infection. Sensitivity of the decision tree was 100% (95% CI 71.5% to 100%) at a specificity of 83.6% (95% CI 82.3% to 84.9%) in the general practitioner setting with 17% of children testing positive. In the paediatric outpatient and emergency department setting, sensitivities were below 92%, with specificities below 44.8%. Conclusions In an independent validation cohort, this clinical prediction rule has shown to be extremely sensitive to identify children at risk of hospital admission for a serious infection in general practice, making it suitable for ruling out. Trial registration number NCT02024282.

Optimizing antibiotic prescribing for acutely ill children in primary care (ERNIE2 study protocol, part B): a cluster randomized, factorial controlled trial evaluating the effect of a point-of-care C-reactive protein test and a brief intervention combined with written safety net advice.

BackgroundDespite huge public campaigns, there is still overconsumption of antibiotics in children with self-limiting diseases. Possible explanations may be the physicians’ and parents’ uncertainty about the gravity of the disease and inadequate communication between physicians and parents leading to lack of reassurance for the parents. In this paper we describe the design and methods of a trial aiming to rationalize antibiotic prescribing by decreasing this uncertainty and parental anxiety.Methods/DesignAcutely ill children without suspected serious disease consulting their family physician will be consecutively included in a four-armed cluster randomized factorial controlled trial. The intervention will consist a Point-of-Care C-reactive protein test and/or a brief intervention with safety net advice. The control group will receive usual care. We intend to include 2560 patients in 88 family practices. Patients will be followed up until cure. The primary outcome measure is the immediate antibiotic prescribing rate. Secondary outcomes are: comparison between groups of speed of clinical recovery, parental concern, parental perception of the quality of the communication, parental satisfaction, use of medication, use of diagnostic tests and medical services during the illness episode, and cost-effectiveness of the interventions. Besides this, we will observationally analyse data of the children included in the large ERNIE2-trial, but excluded in the cluster randomized trial, namely children suspected of serious disease presenting in primary care and children who initially present at the out-patient paediatric clinic or emergency department. We will search for predictors of antibiotic prescribing, speed of clinical recovery, parental concern, parental perception of communication, parental satisfaction, use of medication, diagnostic tests and medical services.DiscussionThis is a unique multifaceted intervention, in that it targets both physicians and parents by aiming specifically at their uncertainty and concerns during the consultation. Both interventions are easy to implement without special training. When proven effective, they could offer a feasible way to decrease inappropriate antibiotic prescribing for children in family practice and thus avoid emergence of bacterial resistance, side effects and unnecessary healthcare costs. Moreover, the observational part of the study will increase our insight in the course, management and parent’s concern of acute illness in children.Trial registrationClinicalTrials.gov Identifier: NCT02024282.

Common evidence gaps in point-of-care diagnostic test evaluation: a review of horizon scan reports

Objective Since 2008, the Oxford Diagnostic Horizon Scan Programme has been identifying and summarising evidence on new and emerging diagnostic technologies relevant to primary care. We used these reports to determine the sequence and timing of evidence for new point-of-care diagnostic tests and to identify common evidence gaps in this process. Design Systematic overview of diagnostic horizon scan reports. Primary outcome measures We obtained the primary studies referenced in each horizon scan report (n=40) and extracted details of the study size, clinical setting and design characteristics. In particular, we assessed whether each study evaluated test accuracy, test impact or cost-effectiveness. The evidence for each point-of-care test was mapped against the Horvath framework for diagnostic test evaluation. Results We extracted data from 500 primary studies. Most diagnostic technologies underwent clinical performance (ie, ability to detect a clinical condition) assessment (71.2%), with very few progressing to comparative clinical effectiveness (10.0%) and a cost-effectiveness evaluation (8.6%), even in the more established and frequently reported clinical domains, such as cardiovascular disease. The median time to complete an evaluation cycle was 9 years (IQR 5.5–12.5 years). The sequence of evidence generation was typically haphazard and some diagnostic tests appear to be implemented in routine care without completing essential evaluation stages such as clinical effectiveness. Conclusions Evidence generation for new point-of-care diagnostic tests is slow and tends to focus on accuracy, and overlooks other test attributes such as impact, implementation and cost-effectiveness. Evaluation of this dynamic cycle and feeding back data from clinical effectiveness to refine analytical and clinical performance are key to improve the efficiency of point-of-care diagnostic test development and impact on clinically relevant outcomes. While the ‘road map’ for the steps needed to generate evidence are reasonably well delineated, we provide evidence on the complexity, length and variability of the actual process that many diagnostic technologies undergo.

Sepsis and Meningitis in Hospitalized Children Performance of Clinical Signs and Their Prediction Rules in a Case-Control Study

Objective Feverish illness is a common presentation to acute pediatric services. Clinical staff faces the challenge of differentiating the few children with meningitis or sepsis from the majority with self-limiting illness. We aimed to determine the diagnostic value of clinical features and their prediction rules (CPR) for identifying children with sepsis or meningitis among those children admitted to a District General Hospital with acute febrile illness. Methods Acutely ill children admitted to a District General Hospital in England were included in this case-control study between 2000 and 2005. We examined the diagnostic accuracy of individual clinical signs and 6 CPRs, including the National Institute for Clinical Excellence “traffic light” system, to determine clinical utility in identifying children with a diagnosis of sepsis or meningitis. Results Loss of consciousness, prolonged capillary refill, decreased alertness, respiratory effort, and the physician’s illness assessment had high positive likelihood ratios (9–114), although with wide confidence intervals, to rule in sepsis or meningitis. The National Institute for Clinical Excellence traffic light system, the modified Yale Observation Scale, and the Pediatric Advanced Warning Score performed poorly with positive likelihood ratios ranging from 1 to 3. Conclusions The pediatrician’s overall illness assessment was the most useful feature to rule in sepsis or meningitis in these hospitalized children. Clinical prediction rules did not effectively rule in sepsis or meningitis. The modified Yale Observation Scale should be used with caution. Single clinical signs could complement these scores to rule in sepsis or meningitis. Further research is needed to validate these CPRs.