Daniel M. Hartung

Reporting Discrepancies between the ClinicalTrials.gov Results Database and Peer Reviewed Publications

Medical decision makers who use clinical trial evidence most often rely on findings that are published in peer-reviewed journals. Selective reporting of clinical trial results is a well-documented problem that raises concerns about using journal publications (1). Clinical trial registration is one mechanism aimed at reducing the effect of dissemination biases. Although many clinical trial registries exist, the single largest publicly accessible trial registry, and the only one with a results database, is ClinicalTrials.gov (2). Administered through the National Library of Medicine, ClinicalTrials.gov was developed to provide the public with a Web-based, searchable source of information about trials conducted within the United States. In September 2007, the Food and Drug Administration Amendments Act (FDAAA) was passed, greatly expanding the legal requirements for trial registration and mandating the creation of a publicly accessible clinical trial results database (3). According to FDAAA section 801, as of September 2008, basic summary results must be submitted for certain trials (called applicable clinical trials in the statute). Applicable clinical trials include most phase 2 through 4 trials of drugs, devices, or biologics regulated by the FDA having at least 1 site in the United States or conducted under an investigational new drug application or investigational device exemption (4). Several elements are required to be reported, including number of participants entering and completing the study; number of participants analyzed; demographic data, such as age and sex; summary results for all prespecified primary and secondary outcome measures; and anticipated and unanticipated adverse events by organ system. Results are generally required to be reported within 1 year of study completion, although submission may be delayed if the drug or device is not yet approved or if an application for a new use is to be submitted. The ClinicalTrials.gov results database has the potential to be a great asset for clinicians, patients, and researchers, but the ultimate validity of posted results is unclear. In contrast to the scientific scrutiny trials undergo during peer review for journals, results posted to ClinicalTrials.gov go through a quality assurance process focusing on internal consistency and logic. Although a gold standard repository of clinical trial results does not exist, inconsistencies between the ClinicalTrials.gov results database and other sources of clinical trial data suggest validity problems in 1 or both sources. The goal of this study was to assess the consistency of results reported in the ClinicalTrials.gov results database compared with those summarized in peer-reviewed journal publications. Methods Trial Selection Studies were eligible for inclusion if they posted results to ClinicalTrials.gov, were interventional, and were phase 3 or 4. To allow sufficient time for publication, we limited our search to trials with a primary completion date before 1 January 2009 or a start date before 1 July 2008 if the primary completion date field was not populated. Completed trials with results were sequenced in random order using Excel 2010 (Microsoft, Redmond, Washington) and were screened for the presence of a matching publication until a 10% random sample of trials with results was obtained. Trials were excluded if they did not assign participants to 2 or more interventional groups. We identified matching publications in a sequential process, first examining citations provided within ClinicalTrials.gov and then using a manual search of 2 electronic bibliographic databases. PubMed citations embedded within ClinicalTrials.gov can be provided by the investigator or the National Library of Medicine on the basis of matching National Clinical Trial (NCT) identifiers (5). We considered a publication to be a match if the intervention was the same and 1 or more groups in the trial had an identical number of participants. If relevant studies were not identified using citations provided within ClinicalTrials.gov, an electronic search of MEDLINE and the Cochrane Central Register of Controlled Trials was conducted using the study interventions, condition, principal investigator (if supplied), and date of trial completion as search criteria. Data Abstraction and Comparisons The following elements were abstracted and compared between the ClinicalTrials.gov results record and its corresponding publications: trial design, number of groups, primary outcome measure (POM) descriptions, secondary outcome measure (SOM) descriptions, total enrollment, and primary outcome results. We also abstracted the number of individuals affected by at least 1 adverse event (AE) and the number of individuals at risk, as reported to ClinicalTrials.gov. Comparisons of counts (such as enrollment, participants analyzed for the primary outcome, and number with an AE) were considered discrepant if they were not an exact match. The primary outcome result was required to be consistent to 1 decimal place. In cases of multiple publications, inconsistencies between the ClinicalTrials.gov results record and descriptions in any of the associated publications were considered a discrepancy. We classified POM description inconsistencies by using an existing framework describing the specificity of outcome reporting in ClinicalTrials.gov (6). The POM could deviate entirely in the domain measured or the number reported, the measurement tool used (for example, change in low-density lipoprotein cholesterol level vs. change in total cholesterol level), how the measure was used (for example, percentage change from baseline vs. absolute value), or the method of aggregation (for example, hemoglobin A1c level <7% vs. <8%). We considered SOMs to be consistent if they were mentioned in the results or methods section of the publication and were listed in the ClinicalTrials.gov results record. For trials with multiple publications, we considered the aggregate number of SOMs across all associated publications. When evaluating POM reporting consistency, we first determined whether the descriptions were consistent in both sources. When they were, we looked for discrepancies in the reported value (for example, mean response or count with outcome) or the number of individuals analyzed for the outcome (for example, denominator or number analyzed). For trials with more than 1 POM specified in both sources, any inconsistency in the result numerator or denominator was considered a discrepancy. If discrepancies in trial features resulted in downstream inconsistencies, we compared only the highest-order feature to avoid double counting. Adverse events did not become a mandatory reporting element until September 2009 and are summarized in the ClinicalTrials.gov results record in 2 tables: serious AEs (SAEs) and other (nonserious) AEs (OAEs). The FDA defines an SAE as any event that results in death, is life-threatening, requires or extends hospitalization, results in significant incapacity or interferes with normal life functions, or causes a congenital anomaly or birth defect (7). We compared the total number of SAEs reported in ClinicalTrials.gov with the total reported in the corresponding publications. In cases where the SAE counts differed, we compared the risk difference (experimental group risk minus control group risk) reported in ClinicalTrials.gov with the published estimate. For trials with multiple experimental groups, we selected the group of primary interest stated in the paper; if multiple FDA-approved dosing groups were assessed, we combined these for comparison with the control group. For OAEs, we restricted our comparison to specific AEs that could be matched to the publication without ambiguity and that were not also reported as an SAE in order to eliminate the possibility of double counting participants who may have had both a serious and nonserious AE. We distinguished publications reporting only treatment-related (attributable) AEs because ClinicalTrials.gov requires reporting of AEs regardless of attribution. Finally, we compared the number of deaths reported in each source. In ClinicalTrials.gov, deaths can be reported as an outcome, in the participant flow section, or as an SAE. If death was not a primary or secondary outcome, we compared the number of deaths reported in the participant flow or SAE section of ClinicalTrials.gov with the number reported in the publication. We classified the sources as discrepant only if counts of death differed between them. A second reviewer independently assessed reporting discrepancies between the ClinicalTrials.gov results record and the matched publication in a 20% random sample (22 trials) for all comparisons. Agreement between the primary and secondary abstractors was high, with an average of 0.98 across categories and no single category with a less than 0.91. Role of the Funding Source This work was supported by the National Library of Medicine and the Agency for Healthcare Research and Quality. The funding sources had no role in the design or execution of the study. Results The Figure describes the flow of trials from the initial ClinicalTrials.gov candidate pool to the final study sample. A total of 1669 phase 3 and 4 trials with posted results were initially identified through a query of ClinicalTrials.gov on 15 February 2011. After exclusion of trials with a primary completion date after 1 January 2009 and those not completed or terminated, 1120 trials remained. We randomly screened 357 potentially includable trials until a 10% sample (n= 110) was achieved. Three trials reported results in multiple publications. Table 1 describes the characteristics of the 110 matched trials and the 195 unmatched trials. Most studies were industry-funded, parallel-design trials of drugs. Unmatched trials were more likely to investigate something other than a drug or device and less likely to be cardiovascular trials. Twenty-nine t

Extended-release naltrexone for alcohol and opioid dependence: A meta-analysis of healthcare utilization studies

Through improved adherence, once-monthly injectable extended-release naltrexone (XR-NTX) may provide an advantage over other oral agents approved for alcohol and opioid dependence treatment. The objective of this study was to evaluate cost and utilization outcomes between XR-NTX and other pharmacotherapies for treatment of alcohol and opioid dependence. Published studies were identified through comprehensive search of two electronic databases. Studies were included if they compared XR-NTX to other approved medicines and reported economic and healthcare utilization outcomes in patients with opioid or alcohol dependence. We identified five observational studies comparing 1,565 patients using XR-NTX to other therapies over 6 months. Alcohol dependent XR-NTX patients had longer medication refill persistence versus acamprosate and oral naltrexone. Healthcare utilization and costs was generally lower or as low for XR-NTX-treated patients relative to other alcohol dependence agents. Opioid dependent XR-NTX patients had lower inpatient substance abuse-related utilization versus other agents and

Small area variation and geographic and patient-specific determinants of metabolic testing in antipsychotic users.

8170 lower total cost versus methadone.

Generic Substitution of Lamotrigine Among Medicaid Patients with Diverse Indications: A Cohort-Crossover Study

The American Diabetes Association and American Psychiatric Association recommend metabolic monitoring for all patients using second‐generation antipsychotic (SGA) drugs. We estimated glucose and lipid testing rates among SGA‐users from three state Medicaid programs and investigated small area variation and patient and geographic determinants of testing.

Risk factors associated with linezolid-nonsusceptible enterococcal infections

AbstractBackground: Controversy exists about the safety of substituting generic anti-epileptic drugs (AEDs). Lamotrigine, the prototypical newer AED, is often used for psychiatric and neurological conditions other than epilepsy. The safety of generic substitution of lamotrigine in diverse populations of AED users is unclear. Objective: The objective of this study was to evaluate potential associations between generic substitution of lamotrigine and adverse consequences in a population of diverse users of this drug. Study Design: This study was a retrospective cohort-crossover design using state Medicaid claims data from July 2006 through June 2009. Methods: Subjects were included in the cohort if they converted from brand to generic lamotrigine and had 2 years of lamotrigine use prior to conversion. The frequency of emergency department (ED) visits, hospitalizations and condition-specific ED visits or hospitalizations were recorded in the 60 days immediately following the conversion to generic lamotrigine, then compared with the incidence of the same events during a randomly selected time period indexed to one of the patient’s past refills of branded lamotrigine. Multi-variate conditional logistic regression was used to quantify the association between generic conversion and health services utilization while controlling for changes in lamotrigine dose and concurrent drug use. Results: Of the 616 unique subjects included in this analysis, epilepsy was the most common diagnosis (41%), followed by bipolar disorder (32%), pain (30%) and migraine (18%). Conversion to generic lamotrigine was not associated with a statistically significant increase in the odds of an ED visit (adjusted odds ratio [AOR]=1.35; 95% confidence interval [CI] 0.92, 1.97), hospitalization (AOR=1.21; 95% CI 0.60, 2.50) or condition-specific encounter (AOR 1.75; 95 CI 0.87, 3.51). Conclusions: A statistically significant increase in ED visits, hospitalizations or condition-specific encounters was not observed following the switch from brand to generic lamotrigine, although a type II error cannot be ruled out.

Rural-Urban Differences in Chronic Disease and Drug Utilization in Older Oregonians.

Linezolid is one of few treatment options available for vancomycin-resistant enterococci. The present study investigated risk factors for linezolid-nonsusceptible enterococci using a case-control study of 15 cases and 60 control patients. Previous hospitalization, admission to a medical service, comorbidity, and linezolid and sulfonamide therapy were identified as risk factors.

Characteristics and trends of low-dose quetiapine use in two western state Medicaid programs.

PURPOSE To characterize disease burden and medication usage in rural and urban adults aged ≥85 years. METHODS This is a secondary analysis of 5 years of longitudinal data starting in the year 2000 from 3 brain-aging studies. Cohorts consisted of community-dwelling adults: 1 rural cohort, the Klamath Exceptional Aging Project (KEAP), was compared to 2 urban cohorts, the Oregon Brain Aging Study (OBAS) and the Dementia Prevention study (DPS). In this analysis, 121 participants were included from OBAS/DPS and 175 participants were included from KEAP. Eligibility was determined based on age ≥85 years and having at least 2 follow-up visits after the year 2000. Disease burden was measured by the Modified Cumulative Illness Rating Scale (MCIRS), with higher values representing more disease. Medication usage was measured by the estimated mean number of medications used by each cohort. FINDINGS Rural participants had significantly higher disease burden as measured by MCIRS, 23.0 (95% CI: 22.3-23.6), than urban participants, 21.0 (95% CI: 20.2-21.7), at baseline. The rate of disease accumulation was a 0.2 increase in MCIRS per year (95% CI: 0.05-0.34) in the rural population. Rural participants used a higher mean number of medications, 5.5 (95% CI: 4.8-6.1), than urban participants, 3.7 (95% CI: 3.1-4.2), at baseline (P < .0001). CONCLUSIONS These data suggest that rural and urban Oregonians aged ≥85 years may differ by disease burden and medication usage. Future research should identify opportunities to improve health care for older adults.

A pilot study evaluating alternative approaches of academic detailing in rural family practice clinics

Medicaid programs are concerned about inappropriate, potentially hazardous, and costly off‐label use of second‐generation antipsychotics (SGAs). Several states are exploring policies aimed at managing low‐dose quetiapine, commonly prescribed for off‐label conditions. This study aimed to characterize longitudinal trends and patient characteristics associated with low‐dose quetiapine in two state Medicaid programs. We further aimed to quantify changes in the use of quetiapine associated with a legal settlement that curtailed off‐label promotion of this product.

Trends and Characteristics of US Medicare Spending on Repository Corticotropin

BackgroundAcademic detailing is an interactive, convenient, and user-friendly approach to delivering non-commercial education to healthcare clinicians. While evidence suggests academic detailing is associated with improvements in prescribing behavior, uncertainty exists about generalizability and scalability in diverse settings. Our study evaluates different models of delivering academic detailing in a rural family medicine setting.MethodsWe conducted a pilot project to assess the feasibility, effectiveness, and satisfaction with academic detailing delivered face-to-face as compared to a modified approach using distance-learning technology. The recipients were four family medicine clinics within the Oregon Rural Practice-based Research Network (ORPRN). Two clinics were allocated to receive face-to-face detailing and two received outreach through video conferencing or asynchronous web-based outreach. Surveys at midpoint and completion were used to assess effectiveness and satisfaction.ResultsEach clinic received four outreach visits over an eight month period. Topics included treatment-resistant depression, management of atypical antipsychotics, drugs for insomnia, and benzodiazepine tapering. Overall, 90% of participating clinicians were satisfied with the program. Respondents who received in person detailing reported a higher likelihood of changing their behavior compared to respondents in the distance detailing group for five of seven content areas. While 90%-100% of respondents indicated they would continue to participate if the program were continued, the likelihood of participation declined if only distance approaches were offered.ConclusionsWe found strong support and satisfaction for the program among participating clinicians. Participants favored in-person approaches to distance interactions. Future efforts will be directed at quantitative methods for evaluating the economic and clinical effectiveness of detailing in rural family practice settings.

Using prescription monitoring program data to characterize out‐of‐pocket payments for opioid prescriptions in a state Medicaid program

for existing products,4,5 the modest price reductions for capecitabine are concerning. Analyses of out-of-pocket costs should be interpreted with caution owing to factors like time-varying plan characteristics and copayment assistance not reflected on branded claims. Although list prices do not reflect rebates or discounts to payers, they represent the basis for calculating patient costsharing for the growing number of plans requiring deductibles and coinsurance.6 Generic competition has historically been an important means of cost containment for pharmaceutical products.1,2 While it is unclear whether capecitabine’s generic entry can be generalized to other orally administered anticancer therapies, if similar modest price decreases are observed for other orally administered anticancer drugs following generic entry, generic competition alone may not be sufficient to curb oncology spending. It will be important for future studies to evaluate how generic competition affects spending on other anticancer medications.