Martijn J. Schuemie

Combining electronic healthcare databases in Europe to allow for large-scale drug safety monitoring: the EU-ADR Project

In this proof‐of‐concept paper we describe the framework, process, and preliminary results of combining data from European electronic healthcare record (EHR) databases for large‐scale monitoring of drug safety.

The incidence of narcolepsy in Europe: Before, during, and after the influenza A(H1N1)pdm09 pandemic and vaccination campaigns

BACKGROUND In August 2010 reports of a possible association between exposure to AS03 adjuvanted pandemic A(H1N1)pdm09 vaccine and occurrence of narcolepsy in children and adolescents emerged in Sweden and Finland. In response to this signal, the background rates of narcolepsy in Europe were assessed to rapidly provide information for signal verification. METHODS We used a dynamic retrospective cohort study to assess the narcolepsy diagnosis rates during the period 2000-2010 using large linked automated health care databases in six countries: Denmark, Finland, Italy, the Netherlands, Sweden and the United Kingdom. RESULTS Overall, 2608 narcolepsy cases were identified in almost 280 million person years (PY) of follow up. The pooled incidence rate was 0.93 (95% CI: 0. 90-0.97) per 100,000 PY. There were peaks between 15 and 30 year of age (women>men) and around 60 years of age. In the age group 5-19 years olds rates were increased after the start of pandemic vaccination compared to the period before the start of campaigns, with rate ratios (RR) of 1.9 (95% CI: 1.1-3.1) in Denmark, 6.4 (95% CI: 4.2-9.7) in Finland and 7.5 (95% CI: 5.2-10.7) in Sweden. Cases verification in the Netherlands had a significant effect on the pattern of incidence over time. CONCLUSIONS The results of this incidence study provided useful information for signal verification on a population level. The safety signal of increased narcolepsy diagnoses following the start of the pandemic vaccination campaign as observed in Sweden and Finland could be observed with this approach. An increase in narcolepsy diagnoses was not observed in other countries, where vaccination coverage was low in the affected age group, or did not follow influenza A(H1N1)pdm09 vaccination. Patient level analyses in these countries are being conducted to verify the signal in more detail.

Anni 2.0: a multipurpose text-mining tool for the life sciences

Anni 2.0 is an online tool (http://biosemantics.org/anni/) to aid the biomedical researcher with a broad range of information needs. Anni provides an ontology-based interface to MEDLINE and retrieves documents and associations for several classes of biomedical concepts, including genes, drugs and diseases, with established text-mining technology. In this article we illustrate Annis usability by applying the tool to two use cases: interpretation of a set of differentially expressed genes, and literature-based knowledge discovery.

A dictionary to identify small molecules and drugs in free text

MOTIVATION From the scientific community, a lot of effort has been spent on the correct identification of gene and protein names in text, while less effort has been spent on the correct identification of chemical names. Dictionary-based term identification has the power to recognize the diverse representation of chemical information in the literature and map the chemicals to their database identifiers. RESULTS We developed a dictionary for the identification of small molecules and drugs in text, combining information from UMLS, MeSH, ChEBI, DrugBank, KEGG, HMDB and ChemIDplus. Rule-based term filtering, manual check of highly frequent terms and disambiguation rules were applied. We tested the combined dictionary and the dictionaries derived from the individual resources on an annotated corpus, and conclude the following: (i) each of the different processing steps increase precision with a minor loss of recall; (ii) the overall performance of the combined dictionary is acceptable (precision 0.67, recall 0.40 (0.80 for trivial names); (iii) the combined dictionary performed better than the dictionary in the chemical recognizer OSCAR3; (iv) the performance of a dictionary based on ChemIDplus alone is comparable to the performance of the combined dictionary. AVAILABILITY The combined dictionary is freely available as an XML file in Simple Knowledge Organization System format on the web site http://www.biosemantics.org/chemlist.

Distribution of information in biomedical abstracts and full-text publications

MOTIVATION Full-text documents potentially hold more information than their abstracts, but require more resources for processing. We investigated the added value of full text over abstracts in terms of information content and occurrences of gene symbol--gene name combinations that can resolve gene-symbol ambiguity. RESULTS We analyzed a set of 3902 biomedical full-text articles. Different keyword measures indicate that information density is highest in abstracts, but that the information coverage in full texts is much greater than in abstracts. Analysis of five different standard sections of articles shows that the highest information coverage is located in the results section. Still, 30-40% of the information mentioned in each section is unique to that section. Only 30% of the gene symbols in the abstract are accompanied by their corresponding names, and a further 8% of the gene names are found in the full text. In the full text, only 18% of the gene symbols are accompanied by their gene names.

Evaluating the Impact of Database Heterogeneity on Observational Study Results

Clinical studies that use observational databases to evaluate the effects of medical products have become commonplace. Such studies begin by selecting a particular database, a decision that published papers invariably report but do not discuss. Studies of the same issue in different databases, however, can and do generate different results, sometimes with strikingly different clinical implications. In this paper, we systematically study heterogeneity among databases, holding other study methods constant, by exploring relative risk estimates for 53 drug-outcome pairs and 2 widely used study designs (cohort studies and self-controlled case series) across 10 observational databases. When holding the study design constant, our analysis shows that estimated relative risks range from a statistically significant decreased risk to a statistically significant increased risk in 11 of 53 (21%) of drug-outcome pairs that use a cohort design and 19 of 53 (36%) of drug-outcome pairs that use a self-controlled case series design. This exceeds the proportion of pairs that were consistent across databases in both direction and statistical significance, which was 9 of 53 (17%) for cohort studies and 5 of 53 (9%) for self-controlled case series. Our findings show that clinical studies that use observational databases can be sensitive to the choice of database. More attention is needed to consider how the choice of data source may be affecting results.

Word Sense Disambiguation in the Biomedical Domain: An Overview

There is a trend towards automatic analysis of large amounts of literature in the biomedical domain. However, this can be effective only if the ambiguity in natural language is resolved. In this paper, the current state of research in word sense disambiguation (WSD) is reviewed. Several methods for WSD have already been proposed, but many systems have been tested only on evaluation sets of limited size. There are currently only very few applications of WSD in the biomedical domain. The current direction of research points towards statistically based algorithms that use existing curated data and can be applied to large sets of biomedical literature. There is a need for manually tagged evaluation sets to test WSD algorithms in the biomedical domain. WSD algorithms should preferably be able to take into account both known and unknown senses of a word. Without WSD, automatic metaanalysis of large corpora of text will be error prone.

Defining a Reference Set to Support Methodological Research in Drug Safety

BackgroundMethodological research to evaluate the performance of methods requires a benchmark to serve as a referent comparison. In drug safety, the performance of analyses of spontaneous adverse event reporting databases and observational healthcare data, such as administrative claims and electronic health records, has been limited by the lack of such standards.ObjectivesTo establish a reference set of test cases that contain both positive and negative controls, which can serve the basis for methodological research in evaluating methods performance in identifying drug safety issues.Research DesignSystematic literature review and natural language processing of structured product labeling was performed to identify evidence to support the classification of drugs as either positive controls or negative controls for four outcomes: acute liver injury, acute kidney injury, acute myocardial infarction, and upper gastrointestinal bleeding.ResultsThree-hundred and ninety-nine test cases comprised of 165 positive controls and 234 negative controls were identified across the four outcomes. The majority of positive controls for acute kidney injury and upper gastrointestinal bleeding were supported by randomized clinical trial evidence, while the majority of positive controls for acute liver injury and acute myocardial infarction were only supported based on published case reports. Literature estimates for the positive controls shows substantial variability that limits the ability to establish a reference set with known effect sizes.ConclusionsA reference set of test cases can be established to facilitate methodological research in drug safety. Creating a sufficient sample of drug-outcome pairs with binary classification of having no effect (negative controls) or having an increased effect (positive controls) is possible and can enable estimation of predictive accuracy through discrimination. Since the magnitude of the positive effects cannot be reliably obtained and the quality of evidence may vary across outcomes, assumptions are required to use the test cases in real data for purposes of measuring bias, mean squared error, or coverage probability.

Methods for drug safety signal detection in longitudinal observational databases: LGPS and LEOPARD

There is a growing interest in using longitudinal observational databases for drug safety signal detection, but most of the existing statistical methods are tailored towards spontaneous reporting. Here a sequential set of methods for detecting and filtering drug safety signals in longitudinal databases is presented.

Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers.

The vision of creating accessible, reliable clinical evidence by accessing the clincial experience of hundreds of millions of patients across the globe is a reality. Observational Health Data Sciences and Informatics (OHDSI) has built on learnings from the Observational Medical Outcomes Partnership to turn methods research and insights into a suite of applications and exploration tools that move the field closer to the ultimate goal of generating evidence about all aspects of healthcare to serve the needs of patients, clinicians and all other decision-makers around the world.