Johan van der Lei

Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EU-ADR Project

In this proof‐of‐concept paper we describe the framework, process, and preliminary results of combining data from European electronic healthcare record (EHR) databases for large‐scale monitoring of drug safety.

Data mining on electronic health record databases for signal detection in pharmacovigilance: which events to monitor?

Data mining on electronic health records (EHRs) has emerged as a promising complementary method for post‐marketing drug safety surveillance. The EU‐ADR project, funded by the European Commission, is developing techniques that allow mining of EHRs for adverse drug events across different countries in Europe. Since mining on all possible events was considered to unduly increase the number of spurious signals, we wanted to create a ranked list of high‐priority events.

Manchester triage system in paediatric emergency care: prospective observational study

Objective To validate use of the Manchester triage system in paediatric emergency care. Design Prospective observational study. Setting Emergency departments of a university hospital and a teaching hospital in the Netherlands, 2006-7. Participants 17 600 children (aged <16) visiting an emergency department over 13 months (university hospital) and seven months (teaching hospital). Intervention Nurses triaged 16 735/17 600 patients (95%) using a computerised Manchester triage system, which calculated urgency levels from the selection of discriminators embedded in flowcharts for presenting problems. Nurses over-ruled the urgency level in 1714 (10%) children, who were excluded from analysis. Complete data for the reference standard were unavailable in 1467 (9%) children leaving 13 554 patients for analysis. Main outcome measures Urgency according to the Manchester triage system compared with a predefined and independently assessed reference standard for five urgency levels. This reference standard was based on a combination of vital signs at presentation, potentially life threatening conditions, diagnostic resources, therapeutic interventions, and follow-up. Sensitivity, specificity, and likelihood ratios for high urgency (immediate and very urgent) and 95% confidence intervals for subgroups based on age, use of flowcharts, and discriminators. Results The Manchester urgency level agreed with the reference standard in 4582 of 13 554 (34%) children; 7311 (54%) were over-triaged and 1661 (12%) under-triaged. The likelihood ratio was 3.0 (95% confidence interval 2.8 to 3.2) for high urgency and 0.5 (0.4 to 0.5) for low urgency; though the likelihood ratios were lower for those presenting with a medical problem (2.3 (2.2 to 2.5) v 12.0 (7.8 to 18.0) for trauma) and in younger children (2.4 (1.9 to 2.9) at 0-3 months v 5.4 (4.5 to 6.5) at 8-16 years). Conclusions The Manchester triage system has moderate validity in paediatric emergency care. It errs on the safe side, with much more over-triage than under-triage compared with an independent reference standard for urgency. Triage of patients with a medical problem or in younger children is particularly difficult.

Assessment of Decision Support for Blood Test Ordering in Primary Care: A Randomized Trial

The majority of general practitioners in the Netherlands have replaced traditional paper-based patient records with computer-based records; physicians enter patient data themselves in the computer during patient encounters (1). The use of electronic patient records creates new opportunities to influence physician behavior through implementation of decision support systems (2-7). In recent years, researchers have documented various computer-based decision support systems that have influenced physician behavior (8-17). Other investigators, however, have reported that computer-based decision support has not affected patient care (18). To resolve the issue, investigators have compared the results of studies that were conducted in different settings, used different methods, and involved different populations (19). Studies comparing different methods of providing computer-based decision support in randomized trials are not available. In the Netherlands, 3% to 4% of patient encounters with general practitioners in primary care result in the ordering of blood tests (20). However, ordering of blood tests is not always appropriate (21-29). Researchers argue that excessive ordering of tests causes physicians to pursue evaluation of false-positive results, which in turn leads to additional unnecessary diagnostic examinations (30-35). Two methods have proven effective in reducing the number of tests ordered by Dutch general practitioners. The first method is based on restricting the number of tests that are listed on an order form. Zaat and colleagues (36, 37) developed a restricted paper order form that replaced the existing form. The second method involves introduction of indication-oriented order forms that are based on clinical practice guidelines (38-40). We hypothesized that an indication-oriented order form based on guidelines, which would provide an optimally restricted list of tests that are relevant for a specific indication, would be more effective in decreasing the number of tests ordered compared with an order form that provides an initially limited list of tests. We therefore conducted a randomized trial to compare the effect of two versions of BloodLink, a computer-based clinical decision support system, on blood test ordering among Dutch general practitioners. Methods Participants In August and September 1995, all 64 practices (94 general practitioners) in the region of Delft, the Netherlands, were invited to participate in the study. Only practices that had replaced their paper-based patient records with electronic records and were using the computer during patient encounters were eligible. A total of 46 practices (62 general practitioners) agreed to participate. Randomization To avoid contamination, we performed random allocation at the level of the practice (41, 42). The practices were first stratified by type: solo practices or group practices (two or more general practitioners in the same practice). Each practice was subsequently assigned by simple random allocation to use BloodLink-Restricted or BloodLink-Guideline for the full study period. A researcher who was not involved in the study and was blinded to the identity of the practices performed the randomization by using a random-numbers table. After randomization, 22 practices involving 30 general practitioners were assigned to use BloodLink-Restricted and 24 practices involving 32 general practitioners were assigned to use BloodLink-Guideline. Intervention We developed two versions of BloodLink, a computer-based decision support system. BloodLink-Restricted initially displays a reduced list of tests, whereas BloodLink-Guideline is based on the guidelines of the Dutch College of General Practitioners. Both versions of BloodLink are integrated with the computer-based patient record (43). The option to use BloodLink was added to the screen that the general practitioner uses when entering data in the electronic patient record during patient encounters. The general practitioner can activate BloodLink to order blood tests as an alternative to using paper order forms. Because the total number of tests that can be ordered is too large to display on a computer screen, a set of tests is presented for selection. If the physician requires additional tests that are not currently displayed, he or she can type the first few letters of the names of the required tests, and the system will present all possible matches (including those corresponding to possible typing errors of the general practitioner) for selection. The number of tests that the general practitioners had at their disposal was the same both before and during the intervention (52 clinical chemistry tests and 46 microbiological tests). Options for specific instructions to the laboratory (for example, urgent processing or fasting values) are available. Once the physician has made his or her selections, BloodLink prints a patient-specific test order form and instructions for the laboratory and updates the patient record with the tests that have been ordered. The only difference between the two versions of BloodLink is the method used to present the initial set of tests to the general practitioner. BloodLink-Restricted is based on the idea of a restricted order form. It offers the general practitioner an initial set of 15 tests that have been shown to cover most of the clinical situations seen in primary care (36). BloodLink-Restricted can be viewed as a general electronic order form that presents only 15 tests on the screen, together with a field labeled other tests that allows the physician to request any other blood test (43). The 15 tests are alanine aminotransferase, aspartate aminotransferase, total bilirubin, cholesterol, creatinine, erythrocyte sedimentation rate, free thyroxine, -glutamyltransferase, glucose (and fasting glucose), glycosylated hemoglobin, hemoglobin, mean corpuscular volume, PaulBunnell, potassium, and thyroid-stimulating hormone. At any time, the physician can customize tests for individual patients by adding or deleting tests. BloodLink-Guideline is based on the guidelines of the Dutch College of General Practitioners. By January 1996, the Dutch College of General Practitioners had published 54 guidelines. Some guidelines focus on symptoms that are frequently seen in the primary care setting, such as acute diarrhea, acute sore throat, low back pain, alcohol abuse, fever in children, and sleeping disorders. Other guidelines focus on common diseases in primary care, such as diabetes, asthma, depression, dementia, and eczema. Finally, a set of guidelines covers preventive medicine. We reviewed the most recent version of each guideline, available in January 1996, and noted whether it contained a reference to a blood test (44). We determined the clinical situation in which the test should be performed (indication) and the tests that should be performed in that situation (advised tests). When general practitioners activate the system, BloodLink-Guideline first provides an overview of the available guidelines. The names of these guidelines are familiar to Dutch general practitioners. The general practitioner selects the appropriate guideline. A guideline may describe several different indications for requesting blood tests; for example, the guideline for blood tests and liver disease mentions 10 different indications. After the indication has been identified, the system proposes the relevant tests. The general practitioner then decides whether to adhere to the protocol. At any time, the physician can customize tests for individual patients by adding or removing tests from the proposed list. Although new guidelines are published at regular intervals, the currently available guidelines cover only a limited set of indications for blood tests (44). In the absence of national guidelines, local or regional guidelines may be used. The version of BloodLink-Guideline used during the clinical trial in the Delft region included three regional guidelines for anemia, AIDS, and clotting disorders, in addition to all national guidelines. Even with these additional guidelines, BloodLink-Guideline does not cover all possible indications for blood tests in primary care. To deal with these situations, the general practitioner can select the heading other indication and order any test. Protocol Before the study, the general practitioners were using two paper order forms: one for clinical chemistry and one for microbiology. After BloodLink was installed, one of the authors gave a brief orientation presentation to the participating practitioners. During a 3-month phase-in period, the general practitioners were allowed to use BloodLink in their practices to become acquainted with the system. After this period, the general practitioners were asked whether they were willing to participate in the trial. The study period was March 1996 through February 1997. Physicians always had the choice to use either the BloodLink software or the paper forms to order clinical chemistry and microbiology tests; thus, paper order forms were still available during the entire intervention period. When the general practitioner ordered blood tests during a patient encounter, only one order form was generated regardless of whether the general practitioner used paper forms or BloodLink. The electronic patient record monitored use of BloodLink by the practitioners. To include the requests for blood tests that were made by using traditional paper forms, we retrieved from the regional laboratory all requests for blood tests. Outcomes We counted the number of order forms that the laboratory received from the general practitioners and the number of tests on each form. The main outcome measure was the average number of tests per order form (including paper forms) per practice (summary variable). We defined the most frequently ordered tests as the tests that accounted for 80% of the total number of tests ordered. For these tests, we compu

Electronic Alerts Versus On-Demand Decision Support to Improve Dyslipidemia Treatment: A Cluster Randomized Controlled Trial

Background— Indirect evidence shows that alerting users with clinical decision support systems seems to change behavior more than requiring users to actively initiate the system. However, randomized trials comparing these methods in a clinical setting are lacking. We studied the effect of both alerting and on-demand decision support with respect to screening and treatment of dyslipidemia based on the guidelines of the Dutch College of General Practitioners. Methods and Results— In a clustered randomized trial design, 38 Dutch general practices (77 physicians) and 87 886 of their patients (39 433 men 18 to 70 years of age and 48 453 women 18 to 75 years of age) who used the ELIAS electronic health record participated. Each practice was assigned to receive alerts, on-demand support, or no intervention. We measured the percentage of patients screened and treated after 12 months of follow-up. In the alerting group, 65% of the patients requiring screening were screened (relative risk versus control=1.76; 95% confidence interval, 1.41 to 2.20) compared with 35% of patients in the on-demand group (relative risk versus control=1.28; 95% confidence interval, 0.98 to 1.68) and 25% of patients in the control group. In the alerting group, 66% of patients requiring treatment were treated (relative risk versus control=1.40; 95% confidence interval, 1.15 to 1.70) compared with 40% of patients (relative risk versus control=1.19; 95% confidence interval, 0.94 to 1.50) in the on-demand group and 36% of patients in the control group. Conclusion— The alerting version of the clinical decision support systems significantly improved screening and treatment performance for dyslipidemia by general practitioners.

Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers.

The vision of creating accessible, reliable clinical evidence by accessing the clincial experience of hundreds of millions of patients across the globe is a reality. Observational Health Data Sciences and Informatics (OHDSI) has built on learnings from the Observational Medical Outcomes Partnership to turn methods research and insights into a suite of applications and exploration tools that move the field closer to the ultimate goal of generating evidence about all aspects of healthcare to serve the needs of patients, clinicians and all other decision-makers around the world.

Electronic healthcare databases for active drug safety surveillance: is there enough leverage?

To provide estimates of the number and types of drugs that can be monitored for safety surveillance using electronic healthcare databases.

School-based Internet-tailored fruit and vegetable education combined with brief counselling increases children's awareness of intake levels

OBJECTIVE Childrens fruit/vegetable intake is still below recommended levels. This study applied Internet-tailored advice for schoolchildren and Internet-supported brief dietary counselling (with child and parent) within preventive health care to promote fruit/vegetable intake. SETTING/SUBJECTS The study involved 30 seventh-grade classes (16 in the intervention group and 14 in the control group) with a total of 675 children aged 9-12 years, of whom 495 were allowed to participate. DESIGN A cluster-randomised baseline-post-test experimental design was applied. During school hours, all children completed Internet-administered questionnaires on fruit/vegetable intake and related determinants. Children in the intervention group received immediate online individually tailored nutrition feedback. For each child in the intervention group, a nurse received information concerning the assessment of fruit/vegetable intake via the Internet to support a 5 min counselling protocol to promote fruit/vegetable intake. Children completed a similar post-test questionnaire 3 months after the first assessment. Intention-to-treat analyses were conducted using multilevel regression analyses. RESULTS A total of 486 children (98% of 495) participated (263 in the intervention group, 223 in the control group); 240 child-parent couples in the intervention group attended the counselling. Awareness of inadequate fruit intake (odds ratio (OR) = 3.0; 95% confidence interval (CI) = 1.8-5.3) and knowledge of recommended vegetable intake levels (OR = 2.7; 95% CI = 1.8-4.1) were significantly more likely at post-test in the intervention group than in the control group. No significant effects were found on intake or other determinants. CONCLUSIONS A compact, integrated two-component intervention can induce positive changes in knowledge and awareness of intake levels of fruit/vegetables among schoolchildren. To induce changes in intake levels, more comprehensive interventions may be needed.

Clinical prediction model to aid emergency doctors managing febrile children at risk of serious bacterial infections: diagnostic study.

Objective To derive, cross validate, and externally validate a clinical prediction model that assesses the risks of different serious bacterial infections in children with fever at the emergency department. Design Prospective observational diagnostic study. Setting Three paediatric emergency care units: two in the Netherlands and one in the United Kingdom. Participants Children with fever, aged 1 month to 15 years, at three paediatric emergency care units: Rotterdam (n=1750) and the Hague (n=967), the Netherlands, and Coventry (n=487), United Kingdom. A prediction model was constructed using multivariable polytomous logistic regression analysis and included the predefined predictor variables age, duration of fever, tachycardia, temperature, tachypnoea, ill appearance, chest wall retractions, prolonged capillary refill time (>3 seconds), oxygen saturation <94%, and C reactive protein. Main outcome measures Pneumonia, other serious bacterial infections (SBIs, including septicaemia/meningitis, urinary tract infections, and others), and no SBIs. Results Oxygen saturation <94% and presence of tachypnoea were important predictors of pneumonia. A raised C reactive protein level predicted the presence of both pneumonia and other SBIs, whereas chest wall retractions and oxygen saturation <94% were useful to rule out the presence of other SBIs. Discriminative ability (C statistic) to predict pneumonia was 0.81 (95% confidence interval 0.73 to 0.88); for other SBIs this was even better: 0.86 (0.79 to 0.92). Risk thresholds of 10% or more were useful to identify children with serious bacterial infections; risk thresholds less than 2.5% were useful to rule out the presence of serious bacterial infections. External validation showed good discrimination for the prediction of pneumonia (0.81, 0.69 to 0.93); discriminative ability for the prediction of other SBIs was lower (0.69, 0.53 to 0.86). Conclusion A validated prediction model, including clinical signs, symptoms, and C reactive protein level, was useful for estimating the likelihood of pneumonia and other SBIs in children with fever, such as septicaemia/meningitis and urinary tract infections.

Using electronic health care records for drug safety signal detection: a comparative evaluation of statistical methods.

Background:Drug safety monitoring relies primarily on spontaneous reporting, but electronic health care record databases offer a possible alternative for the detection of adverse drug reactions (ADRs). Objectives:To evaluate the relative performance of different statistical methods for detecting drug-adverse event associations in electronic health care record data representing potential ADRs. Research Design:Data from 7 databases across 3 countries in Europe comprising over 20 million subjects were used to compute the relative risk estimates for drug-event pairs using 10 different methods, including those developed for spontaneous reporting systems, cohort methods such as the longitudinal gamma poisson shrinker, and case-based methods such as case-control. The newly developed method “longitudinal evaluation of observational profiles of adverse events related to drugs” (LEOPARD) was used to remove associations likely caused by protopathic bias. Data from the different databases were combined by pooling of data, and by meta-analysis for random effects. A reference standard of known ADRs and negative controls was created to evaluate the performance of the method. Measures:The area under the curve of the receiver operator characteristic curve was calculated for each method, both with and without LEOPARD filtering. Results:The highest area under the curve (0.83) was achieved by the combination of either longitudinal gamma poisson shrinker or case-control with LEOPARD filtering, but the performance between methods differed little. LEOPARD increased the overall performance, but flagged several known ADRs as caused by protopathic bias. Conclusions:Combinations of methods demonstrate good performance in distinguishing known ADRs from negative controls, and we assume that these could also be used to detect new drug safety signals.