Miew Keen Choong

Systematic review automation technologies.

Systematic reviews, a cornerstone of evidence-based medicine, are not produced quickly enough to support clinical practice. The cost of production, availability of the requisite expertise and timeliness are often quoted as major contributors for the delay. This detailed survey of the state of the art of information systems designed to support or automate individual tasks in the systematic review, and in particular systematic reviews of randomized controlled clinical trials, reveals trends that see the convergence of several parallel research projects.We surveyed literature describing informatics systems that support or automate the processes of systematic review or each of the tasks of the systematic review. Several projects focus on automating, simplifying and/or streamlining specific tasks of the systematic review. Some tasks are already fully automated while others are still largely manual. In this review, we describe each task and the effect that its automation would have on the entire systematic review process, summarize the existing information system support for each task, and highlight where further research is needed for realizing automation for the task. Integration of the systems that automate systematic review tasks may lead to a revised systematic review workflow. We envisage the optimized workflow will lead to system in which each systematic review is described as a computer program that automatically retrieves relevant trials, appraises them, extracts and synthesizes data, evaluates the risk of bias, performs meta-analysis calculations, and produces a report in real time.

Genetic and epigenetic biomarkers of colorectal cancer.

Cancer is a heterogeneous disease caused, in part, by genetic and epigenetic alterations. These changes have been explored in studies of the pathogenesis of colorectal cancer (CRC) and have led to the identification of many biomarkers of disease progression. However, the number of biomarkers that have been incorporated into clinical practice is surprisingly small. We review the genetic and epigenetic mechanisms of colorectal cancer and discuss molecular markers recommended for use in early detection, screening, diagnosis, determination of prognosis, and prediction of treatment outcomes. We also review important areas for future research.

Automatic evidence retrieval for systematic reviews

Background Snowballing involves recursively pursuing relevant references cited in the retrieved literature and adding them to the search results. Snowballing is an alternative approach to discover additional evidence that was not retrieved through conventional search. Snowballing’s effectiveness makes it best practice in systematic reviews despite being time-consuming and tedious. Objective Our goal was to evaluate an automatic method for citation snowballing’s capacity to identify and retrieve the full text and/or abstracts of cited articles. Methods Using 20 review articles that contained 949 citations to journal or conference articles, we manually searched Microsoft Academic Search (MAS) and identified 78.0% (740/949) of the cited articles that were present in the database. We compared the performance of the automatic citation snowballing method against the results of this manual search, measuring precision, recall, and F1 score. Results The automatic method was able to correctly identify 633 (as proportion of included citations: recall=66.7%, F1 score=79.3%; as proportion of citations in MAS: recall=85.5%, F1 score=91.2%) of citations with high precision (97.7%), and retrieved the full text or abstract for 490 (recall=82.9%, precision=92.1%, F1 score=87.3%) of the 633 correctly retrieved citations. Conclusions The proposed method for automatic citation snowballing is accurate and is capable of obtaining the full texts or abstracts for a substantial proportion of the scholarly citations in review articles. By automating the process of citation snowballing, it may be possible to reduce the time and effort of common evidence surveillance tasks such as keeping trial registries up to date and conducting systematic reviews.

Comparing clinical quality indicators for asthma management in children with outcome measures used in randomised controlled trials: a protocol

Introduction Clinical quality indicators are necessary to monitor the performance of healthcare services. The development of indicators should, wherever possible, be based on research evidence to minimise the risk of bias which may be introduced during their development, because of logistic, ethical or financial constraints alone. The development of automated methods to identify the evidence base for candidate indicators should improve the process of indicator development. The objective of this study is to explore the relationship between clinical quality indicators for asthma management in children with outcome and process measurements extracted from randomised controlled clinical trial reports. Methods and analysis National-level indicators for asthma management in children will be extracted from the National Quality Measures Clearinghouse (NQMC) database and the National Institute for Health and Care Excellence (NICE) quality standards. Outcome measures will be extracted from published English language randomised controlled trial (RCT) reports for asthma management in children aged below 12 years. The two sets of measures will be compared to assess any overlap. The study will provide insights into the relationship between clinical quality indicators and measurements in RCTs. This study will also yield a list of measurements used in RCTs for asthma management in children, and will find RCT evidence for indicators used in practice. Ethics and dissemination Ethical approval is not necessary because this study will not include patient data. Findings will be disseminated through peer-reviewed publications.

The implications of biomarker evidence for systematic reviews.

BackgroundIn Evidence-Based Medicine, clinical practice guidelines and systematic reviews are crucial devices for medical practitioners in making clinical decision. Clinical practice guidelines are systematically developed statements to support health care decisions for specific circumstances whereas systematic reviews are summaries of evidence on clearly formulated clinical questions. Biomarkers are biological measurements (primarily molecular) that are used to diagnose, predict treatment outcomes and prognosticate disease and are increasingly used in randomized controlled trials (RCT).MethodsWe search PubMed for systematic reviews, RCTs, case reports and non-systematic reviews with and without mentions of biomarkers between years 1990–2011. We compared the frequency and growth rate of biomarkers and non-biomarkers publications. We also compared the growth of the proportion of biomarker-based RCTs with the growth of the proportion of biomarker-based systematic reviews.ResultsWith 147,774 systematic reviews indexed in PubMed from 1990 to 2011 (accessed on 18/10/2012), only 4,431 (3%) are dedicated to biomarkers. The annual growth rate of biomarkers publications is consistently higher than non-biomarkers publications, showing the growth in biomarkers research. From 20 years of systematic review publications indexed in PubMed, we identified a bias in systematic reviews against the inclusion of biomarker-based RCTs.ConclusionsWith the realisation of genome-based personalised medicine, biomarkers are becoming important for clinical decision making. The bias against the inclusion of biomarkers in systematic reviews leads to medical practitioners deprive of important information they require to address clinical questions. Sparse or weak evidence and lack of genetic training for systematic reviewers may contribute to this trend.

Linking quality indicators to clinical trials: an automated approach

Abstract Objective Quality improvement of health care requires robust measurable indicators to track performance. However identifying which indicators are supported by strong clinical evidence, typically from clinical trials, is often laborious. This study tests a novel method for automatically linking indicators to clinical trial registrations. Design A set of 522 quality of care indicators for 22 common conditions drawn from the CareTrack study were automatically mapped to outcome measures reported in 13 971 trials from ClinicalTrials.gov. Intervention Text mining methods extracted phrases mentioning indicators and outcome phrases, and these were compared using the Levenshtein edit distance ratio to measure similarity. Main Outcome Measure Number of care indicators that mapped to outcome measures in clinical trials. Results While only 13% of the 522 CareTrack indicators were thought to have Level I or II evidence behind them, 353 (68%) could be directly linked to randomized controlled trials. Within these 522, 50 of 70 (71%) Level I and II evidence-based indicators, and 268 of 370 (72%) Level V (consensus-based) indicators could be linked to evidence. Of the indicators known to have evidence behind them, only 5.7% (4 of 70) were mentioned in the trial reports but were missed by our method. Conclusions We automatically linked indicators to clinical trial registrations with high precision. Whilst the majority of quality indicators studied could be directly linked to research evidence, a small portion could not and these require closer scrutiny. It is feasible to support the process of indicator development using automated methods to identify research evidence.

Gene-disease association with literature based enrichment

MOTIVATION Gene set enrichment analysis (GSEA) annotates gene microarray data with functional information from the biomedical literature to improve gene-disease association prediction. We hypothesize that supplementing GSEA with comprehensive gene function catalogs built automatically using information extracted from the scientific literature will significantly enhance GSEA prediction quality. METHODS Gold standard gene sets for breast cancer (BrCa) and colorectal cancer (CRC) were derived from the literature. Two gene function catalogs (CMeSH and CUMLS) were automatically generated. 1. By using Entrez Gene to associate all recorded human genes with PubMed article IDs. 2. Using the genes mentioned in each PubMed article and associating each with the articles MeSH terms (in CMeSH) and extracted UMLS concepts (in CUMLS). Microarray data from the Gene Expression Omnibus for BrCa and CRC was then annotated using CMeSH and CUMLS and for comparison, also with several pre-existing catalogs (C2, C4 and C5 from the Molecular Signatures Database). Ranking was done using, a standard GSEA implementation (GSEA-p). Gene function predictions for enriched array data were evaluated against the gold standard by measuring area under the receiver operating characteristic curve (AUC). RESULTS Comparison of ranking using the literature enrichment catalogs, the pre-existing catalogs as well as five randomly generated catalogs show the literature derived enrichment catalogs are more effective. The AUC for BrCa using the unenriched gene expression dataset was 0.43, increasing to 0.89 after gene set enrichment with CUMLS. The AUC for CRC using the unenriched gene expression dataset was 0.54, increasing to 0.9 after enrichment with CMeSH. C2 increased AUC (BrCa 0.76, CRC 0.71) but C4 and C5 performed poorly (between 0.35 and 0.5). The randomly generated catalogs also performed poorly, equivalent to random guessing. DISCUSSION Gene set enrichment significantly improved prediction of gene-disease association. Selection of enrichment catalog had a substantial effect on prediction accuracy. The literature based catalogs performed better than the MSigDB catalogs, possibly because they are more recent. Catalogs generated automatically from the literature can be kept up to date. CONCLUSION Prediction of gene-disease association is a fundamental task in biomedical research. GSEA provides a promising method when using literature-based enrichment catalogs. AVAILABILITY The literature based catalogs generated and used in this study are available from http://www2.chi.unsw.edu.au/literature-enrichment.

Linking clinical quality indicators to research evidence - a case study in asthma management for children

BackgroundClinical quality indicators are used to monitor the performance of healthcare services and should wherever possible be based on research evidence. Little is known however about the extent to which indicators in common use are based on research. The objective of this study is to measure the extent to which clinical quality indicators used in asthma management in children with outcome measurements can be linked to results in randomised controlled clinical trial (RCT) reports. This work is part of a broader research program to trial methods that improve the efficiency and accuracy of indicator development.MethodsNational-level indicators for asthma management in children were extracted from the National Quality Measures Clearinghouse database and the National Institute for Health and Care Excellence quality standards by two independent appraisers. Outcome measures were extracted from all published English language RCT reports for asthma management in children below the age of 12 published between 2005 and 2014. The two sets were then linked by manually mapping both to a common set of Unified Medical Language System (UMLS) concepts.ResultsThe analysis identified 39 indicators and 562 full text RCTs dealing with asthma management in children. About 95% (37/39) of the indicators could be linked to RCT outcome measures.ConclusionsIt is possible to identify relevant RCT reports for the majority of indicators used to assess the quality of asthma management in childhood. The methods reported here could be automated to more generally support assessment of candidate indicators against the research evidence.

Citation Enrichment Improves Deduplication of Primary Evidence

Objective: To automatically detect duplicate citations in a bibliographical database. Background: Citations retrieved from multiple search databases have different forms making manual and automatic detection of duplicates difficult. Existing methods rely on fuzzy-similarity measures which are error-prone. Methods: We analysed four pairs of original search results from MEDLINE and EMBASE that were used to create systematic reviews. An automatic tool deduplicated citations by first enriching citations with Digital Object Identifiers DOI, and/or other unique identifiers. Duplication of records was then determined by comparing these unique identifiers. We compared our method with the duplicate detection function of a popular citation management desktop application in several configurations. Results: Citation Enrichment identified 93i¾?% range 86i¾?%---100i¾?% of the duplicates indexed online and erroneously marked 3i¾?% range 0i¾?%---6i¾?% documents as duplicates. The citation management application found 68i¾?% range 64i¾?%---72i¾?% without error using default setting. When set for highest deduplication, the citation management application found 94i¾?% of duplicates range 77i¾?%---100i¾?% and 4i¾?% error range 0i¾?%---8i¾?%. Conclusion: Citation enrichment using unique identifiers enhances automatic deduplication. On its own, the approach seems slightly superior to tools that compare citations without enrichment. Methods that combine citation enrichment with existing fuzzy-matching may substantially reduce resource requirements of evidence synthesis.