Bryant T Smith

Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial.

BackgroundData abstraction, a critical systematic review step, is time-consuming and prone to errors. Current standards for approaches to data abstraction rest on a weak evidence base. We developed the Data Abstraction Assistant (DAA), a novel software application designed to facilitate the abstraction process by allowing users to (1) view study article PDFs juxtaposed to electronic data abstraction forms linked to a data abstraction system, (2) highlight (or “pin”) the location of the text in the PDF, and (3) copy relevant text from the PDF into the form. We describe the design of a randomized controlled trial (RCT) that compares the relative effectiveness of (A) DAA-facilitated single abstraction plus verification by a second person, (B) traditional (non-DAA-facilitated) single abstraction plus verification by a second person, and (C) traditional independent dual abstraction plus adjudication to ascertain the accuracy and efficiency of abstraction.MethodsThis is an online, randomized, three-arm, crossover trial. We will enroll 24 pairs of abstractors (i.e., sample size is 48 participants), each pair comprising one less and one more experienced abstractor. Pairs will be randomized to abstract data from six articles, two under each of the three approaches. Abstractors will complete pre-tested data abstraction forms using the Systematic Review Data Repository (SRDR), an online data abstraction system. The primary outcomes are (1) proportion of data items abstracted that constitute an error (compared with an answer key) and (2) total time taken to complete abstraction (by two abstractors in the pair, including verification and/or adjudication).DiscussionThe DAA trial uses a practical design to test a novel software application as a tool to help improve the accuracy and efficiency of the data abstraction process during systematic reviews. Findings from the DAA trial will provide much-needed evidence to strengthen current recommendations for data abstraction approaches.Trial registrationThe trial is registered at National Information Center on Health Services Research and Health Care Technology (NICHSR) under Registration # HSRP20152269: https://wwwcf.nlm.nih.gov/hsr_project/view_hsrproj_record.cfm?NLMUNIQUE_ID=20152269&SEARCH_FOR=Tianjing%20Li. All items from the World Health Organization Trial Registration Data Set are covered at various locations in this protocol. Protocol version and date: This is version 2.0 of the protocol, dated September 6, 2016. As needed, we will communicate any protocol amendments to the Institutional Review Boards (IRBs) of Johns Hopkins Bloomberg School of Public Health (JHBSPH) and Brown University. We also will make appropriate as-needed modifications to the NICHSR website in a timely fashion.

Treatments of Primary Basal Cell Carcinoma of the Skin: A Systematic Review and Network Meta-analysis

Basal cell carcinoma (BCC) is the most common cancer in the United States, with an annual incidence approaching 2 million cases (1). Most cases are not aggressive, but the tumors and their treatment can cause disfigurement or disability, which can adversely affect quality of life (2). The Surgeon Generals recent call to action to prevent skin cancer at the population level emphasizes the public health importance of dealing with BCC (3). Choosing a management strategy for an individual patient with a specific type of BCC from among the many available interventions is complex. Considerations include patient factors (such as age, frailty, immunosuppression, and personal preference), tumor factors (such as histologic subtype, size, and location), and the availability and cost of health care resources. The lack of clarity about the comparative efficacy and safety of the available options overall and in specific circumstances further complicates treatment decision making for both physicians and patients. Surgical removal is widely considered the gold standard and is therefore the most common treatment. However, despite several dozen randomized controlled trials (RCTs) and nonrandomized comparative studies (NRCSs), the relative performance of various surgical techniques and other therapeutic options is unclear. Payers are faced with increased use of costly therapies, such as brachytherapy, without clear evidence about relative benefits to justify increased costs (4). The purpose of this systematic review and network meta-analysis is to evaluate the comparative effectiveness and safety of treatments of primary BCC. Methods This article updates and summarizes the findings on BCC from a comparative effectiveness review for the Agency for Healthcare Research and Quality (AHRQ) on treatments of BCC and cutaneous squamous cell carcinoma (4). We followed the approach outlined in the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews (5). The protocol was developed with input from stakeholders (providers, researchers, payers, patients, and funders) and was prospectively registered with PROSPERO (CRD42016043353). Data Sources and Searches We searched PubMed, the Cochrane Central Register of Controlled Trials, the Cochrane Database of Systematic Reviews, and Embase up to 8 May 2018 to identify eligible studies (the full report [4] gives search strategies, search terms, and search and selection figures). We queried ClinicalTrials.gov (most recently in August 2016) to identify ongoing or completed yet unpublished trials. We limited searches to English-language publications and supplemented them with suggestions from stakeholders and by perusing reference lists of eligible articles and of pertinent systematic reviews and guidelines. Study Selection We screened titles, abstracts, and full texts of eligible studies in duplicate. The population of interest was adult patients with primary BCC. Patient subgroups were specified in the protocol and were defined by tumor location (such as face, hands, trunk, or extremities) and histologic subtype (such as superficial or nodular BCC). We excluded subpopulations with rare genetic disorders associated with increased BCC risk (for example, basal cell nevus syndrome). With input from stakeholders, we identified 22 eligible interventions, which we organized into 9 categories (Table 1). Outcomes of interest included tumor recurrence and lack of histologic tumor clearance (proxies for failure to cure), cosmetic appearance (patient- or observer-reported), quality of life, mortality, and costs. Table 1. Intervention Categories and Specific Interventions We evaluated RCTs and comparative NRCSs. We included NRCSs only if they took steps to control for patient- or lesion-level confounders. We excluded studies that had fewer than 5 lesions per treatment group. Data Extraction and Risk-of-Bias Assessment One reviewer extracted information from eligible studies, and a second reviewer verified the extraction. Any disagreements were resolved by discussion among the team. Data extractions were done on and are publicly accessible through the Systematic Review Data Repository (http://srdr.ahrq.gov) (6). We recorded information on country of origin; population (mean age; sex distribution; and lesion histology, size, and location); interventions; outcomes (at the longest follow-up and at the follow-up closest to but not longer than 2 years); funding source; and attributes of study design, conduct, and analysis addressed in the Cochrane Risk of Bias Tool (7). For RCTs, at least 2 reviewers critically examined sources of bias and their likely effect and finalized their assessments in discussions with the whole team. We examined the effect of missing data by calculating mathematical bounds for effect estimates (8). This was accomplished by examining extreme scenarios in which all missing observations in one group had the outcome of interest and all missing observations in another group did not. For lack of histologic clearance, input from a pathologist informed our assessments of whether each treatment can be successfully masked. For each observational study, we used the Newcastle-Ottawa Scale (9) to guide our deliberations about sources and risk of bias. Data Synthesis, Analysis, and Assessments of Strength of Evidence We summarized all studies qualitatively and described important features of the population (including tumor characteristics), design, intervention, outcomes, and results. In each outcome, the evidence comprises 1 or more connected networks of 2 or more treatments. For each connected network in each outcome, we did frequentist (maximum likelihood) pairwise and network meta-analyses with mixed-effects (random intercepts and fixed intervention slopes) or full random-effects (random intercepts and slopes) multilevel models within the generalized linear and latent mixed-model framework. The network meta-analysis approach models the proportion of events in each trial group and thus can estimate not only the mean treatment effects (odds ratios) between treatments but also the mean proportion of events with each treatment using all of the available data. In Part G of the Supplement, we describe the models and their estimation and provide access to the data and code. Supplement. Supplementary Material We assessed consistency qualitatively and deemed direct and indirect effects to be in agreement when they were in the same direction and the CI of one included the point estimate of the other. We also used a node-split approach to assess for consistency quantitatively (10). In main analyses, we included data from the longest follow-up for each outcome in each trial. We excluded trials that compared versions of the same intervention (for example, different imiquimod treatment schedules). When data were available, we did subgroup analyses by lesion histologic subtype, location, and size (not shown) (4). The unit of analysis was the patient. A minority of studies reported results about lesions (where individual patients had several lesions), which were included as if each lesion belonged to a different patient. We did not correct for clustering of lesions by patient because the requisite estimates of intraclass correlation coefficients were not available. In sensitivity analyses, we included only trial data on outcomes closest to 1 year within the window of 0 to 2 years, and we fitted network meta-analysis models in the Bayesian framework (not shown). Network meta-analyses of intervention categories were congruent with the main analyses that focused on specific interventions (not shown). All analyses were done in R, version 3.4.4, using the igraph, lme4, metafor, and gemtc packages (1115). We report 95% CIs with no corrections to control for type I error. We summarized analyses as key findings and assessed the overall strength of evidence in terms of the risk of bias of the associated evidence base and the directness, precision, and consistency of the evidence, following the AHRQ methods guidance. Role of the Funding Source This report is based on research conducted by the Brown Evidence-based Practice Center under contract to the AHRQ. The funders role was limited to ensuring adherence to administrative requirements, including timelines and fidelity to the written protocol. The AHRQ was not involved in the design, conduct, or interpretation of the analyses. Results The literature searches returned 16154 citations, of which 519 were retrieved and screened in full text. Forty RCTs and 5 NRCSs were eligible (Supplement Figure and Supplement Tables 1 and 2). The randomized evidence base is sparse (Part B of the Supplement). Across all outcomes, the RCTs examined 18 interventions and provided data on 34 head-to-head comparisons, out of 153 possible comparisons. Of the 34 comparisons, 33 were informed by at most 3 trials. The evidence graphs in Figures 1, 2, 3, and 4 depict the head-to-head comparisons in RCTs for recurrence, lack of histologic clearance, and cosmetic outcomes assessed by patients and observers, respectively. They have the same layout to show the partial coverage of the evidence base per outcome. For all outcomes, the evidence base comprises between 2 and 4 connected networks. No comparisons are made between treatments that belong to unconnected networks, and each connected network is sparse. For example, in Figure 1, the largest connected network for recurrence comprises 14 treatments. Of these, 4 were compared with only 1 other treatment and 5 with only 2. Treatments that were compared with the most other treatments were surgical excision (n= 7) and photodynamic therapy (PDT) using methyl-aminolevulinic acid (n= 6). For lack of histologic clearance, aside from PDT using methyl-aminolevulinic acid (compared with 6 other treatments), all treatments were compared with at most 3 other treatments (6 treatments were compared with only 1 other treatment) (Figure 2). Figure 1. Ev

Strategies for improving the lives of US women aged 40 and above living with HIV/AIDS: an evidence map

BackgroundWhile in its early years the HIV epidemic affected primarily the male and the young, nowadays, the population living with HIV/AIDS is approximately 24% women, and its age composition has shifted towards older ages. Many of the older women who live with HIV/AIDS also live with the medical and social conditions that accompany aging. This work aims to identify and characterize empirical studies of strategies for the comprehensive management of women over 40, including transgender women, who live with HIV/AIDS. Forty was chosen as an operational age cutoff to identify premenopausal women who are less likely to bear children, as well as peri- and postmenopausal women.MethodsWe conducted a literature search after discussions with a diverse panel of content experts and other stakeholders and developed an evidence map that identified 890 citations that address questions having to do with programs and barriers to engaging with programs, as well as the role of insurance and comorbidities, and have enrolled older women who live with HIV/AIDS.ResultsOf these, only 37 (4%) reported results of interest for women over 40 who live with HIV/AIDS, or examined interactions between gender and older age that would allow predictions in this subgroup. Few of the 37 eligible studies focused on women facing obvious challenges, such as immigrants, transgender, physically abused, or those recently released from prison. No studies focused on women caring for dependents, including children and grandchildren, or those diagnosed after age 40.ConclusionThe evidence base that is directly applicable to women over 40 who live with HIV/AIDS in the USA is limited, and the research need is broad. We propose research prioritization strategies for this population.

Response to 'Increasing value and reducing waste in data extraction for systematic reviews: Tracking data in data extraction forms'

AbstractᅟThis is a response to a Letter. Data abstraction is a time-consuming and error-prone systematic review task. Shokraneh and Adams categorize available techniques for tracking data during data abstraction into three methods: simple annotation, descriptive addressing, and Cartesian coordinate system. While we agree with the categorization of the techniques, we disagree with the authors’ statement that descriptive addressing is a PDF-independent method, i.e., any sort of descriptive addressing must reference a specific version of PDF file and not just any PDF of said report. Different versions of PDFs of the same report might place text and tables on different locations of the same page and/or on different pages. Consequently, it is our opinion that any kind of source location information should be accompanied by the source or linked by an intermediary service such as the Data Abstraction Assistant (DAA).

Features and functioning of Data Abstraction Assistant, a software application for data abstraction during systematic reviews

INTRODUCTION During systematic reviews, data abstraction is labor- and time-intensive and error-prone. Existing data abstraction systems do not track specific locations and contexts of abstracted information. To address this limitation, we developed a software application, the Data Abstraction Assistant (DAA) and surveyed early users about their experience using DAA. FEATURES OF DAA We designed DAA to encompass three essential features: (1) a platform for indicating the source of abstracted information, (2) compatibility with a variety of data abstraction systems, and (3) user-friendliness. HOW DAA FUNCTIONS DAA (1) converts source documents from PDF to HTML format (to enable tracking of source of abstracted information), (2) transmits the HTML to the data abstraction system, and (3) displays the HTML in an area adjacent to the data abstraction form in the data abstraction system. The data abstractor can mark locations on the HTML that DAA associates with items on the data abstraction form. EXPERIENCES OF EARLY USERS OF DAA When we surveyed 52 early users of DAA, 83% reported that using DAA was either very or somewhat easy; 71% are very or somewhat likely to use DAA in the future; and 87% are very or somewhat likely to recommend that others use DAA in the future. DISCUSSION DAA, a user-friendly software for linking abstracted data with their exact source, is likely to be a very useful tool in the toolbox of systematic reviewers. DAA facilitates verification of abstracted data and provides an audit trail that is crucial for reproducible research.