Francesco Cozzolino

Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study

Objective To examine whether deviation from the standard intention to treat analysis has an influence on treatment effect estimates of randomised trials. Design Meta-epidemiological study. Data sources Medline, via PubMed, searched between 2006 and 2010; 43 systematic reviews of interventions and 310 randomised trials were included. Eligibility criteria for selecting studies From each year searched, random selection of 5% of intervention reviews with a meta-analysis that included at least one trial that deviated from the standard intention to treat approach. Basic characteristics of the systematic reviews and randomised trials were extracted. Information on the reporting of intention to treat analysis, outcome data, risk of bias items, post-randomisation exclusions, and funding were extracted from each trial. Trials were classified as: ITT (reporting the standard intention to treat approach), mITT (reporting a deviation from the standard approach), and no ITT (reporting no approach). Within each meta-analysis, treatment effects were compared between mITT and ITT trials, and between mITT and no ITT trials. The ratio of odds ratios was calculated (value <1 indicated larger treatment effects in mITT trials than in other trial categories). Results 50 meta-analyses and 322 comparisons of randomised trials (from 84 ITT trials, 118 mITT trials, and 108 no ITT trials; 12 trials contributed twice to the analysis) were examined. Compared with ITT trials, mITT trials showed a larger intervention effect (pooled ratio of odds ratios 0.83 (95% confidence interval 0.71 to 0.96), P=0.01; between meta-analyses variance τ2=0.13). Adjustments for sample size, type of centre, funding, items of risk of bias, post-randomisation exclusions, and variance of log odds ratio yielded consistent results (0.80 (0.69 to 0.94), P=0.005; τ2=0.08). After exclusion of five influential studies, results remained consistent (0.85 (0.75 to 0.98); τ2=0.08). The comparison between mITT trials and no ITT trials showed no statistical difference between the two groups (adjusted ratio of odds ratios 0.92 (0.70 to 1.23); τ2=0.57). Conclusions Trials that deviated from the intention to treat analysis showed larger intervention effects than trials that reported the standard approach. Where an intention to treat analysis is impossible to perform, authors should clearly report who is included in the analysis and attempt to perform multiple imputations.

A systematic review found that deviations from intention-to-treat are common in randomized trials and systematic reviews

OBJECTIVES To describe the characteristics, and estimate the incidence, of trials included in systematic reviews deviating from the intention-to-treat (ITT) principle. STUDY DESIGN AND SETTING A 5% random sample of reviews were selected (Medline 2006-2010). Trials from reviews were classified based on the ITT: (1) ITT trials (trials reporting standard ITT analyses); (2) modified ITT (mITT) trials (modified ITT; trials deviating from standard ITT); or (3) no ITT trials. RESULTS Of 222 reviews, 81 (36%) included at least one mITT trial. Reviews with mITT trials were more likely to contain trials that used placebo, that investigated drugs, and that reported favorable results. The incidence of reviews with mITT trial ranged from 29% (17/58) to 48% (23/48). Of the 2,349 trials, 597 (25.4%) were classified as ITT trials, 323 (13.8%) as mITT trials, and 1,429 (60.8%) as no ITT trials. The mITT trials were more likely to have reported exclusions compared to studies classified as ITT trials and to have received funding. CONCLUSION The reporting of the type of ITT may differ according to the clinical area and the type of intervention. Deviation from ITT in randomized controlled trials is a widespread phenomenon that significantly affects systematic reviews.

The Current State of Validation of Administrative Healthcare Databases in Italy: A Systematic Review

Background : Administrative healthcare databases are widely present in Italy. Our aim was to describe the current state of healthcare databases validity in terms of discharge diagnoses (according to the International Classification of Diseases, ICD-9 code) and their output in terms of research. Methods : A systematic search of electronic databases including Medline and Embase (1995-2013) and of local sources was performed. Inclusion criteria were: healthcare databases in any Italian territory routinely and passively collecting data; medical investigations or procedures at patient level data; the use of a validation process. The quality of studies was evaluated using the STARD criteria. Citations of the included studies were explored using Scopus and Google Scholar. Results : The search strategy allowed the identification of 16 studies of which 3 were in Italian. Thirteen studies used regional administrative databases from Lombardia, Piemonte, Lazio, Friuli-Venezia Giulia and Veneto. The ICD-9 codes of the following diseases were successfully validated: amyotrophic lateral sclerosis (3 studies in four different regional administrative databases), stroke (3 studies), gastrointestinal bleeding (1 study), thrombocytopenia (1 study), epilepsy (1 study), infection (1 study), chronic obstructive pulmonary disease (1 study), Guillain-Barre syndrome (1 study), and cancer diseases (4 studies). The quality of reporting was variable among the studies. Only 6 administrative databases produced further research related to the validated ICD-9 codes. Conclusion : Administrative healthcare databases in Italy need an extensive process of validation for multiple diagnostic codes to perform high quality epidemiological and health services research.

Validation of chronic obstructive pulmonary disease (COPD) diagnoses in healthcare databases: a systematic review protocol

Introduction Healthcare databases are useful sources to investigate the epidemiology of chronic obstructive pulmonary disease (COPD), to assess longitudinal outcomes in patients with COPD, and to develop disease management strategies. However, in order to constitute a reliable source for research, healthcare databases need to be validated. The aim of this protocol is to perform the first systematic review of studies reporting the validation of codes related to COPD diagnoses in healthcare databases. Methods and analysis MEDLINE, EMBASE, Web of Science and the Cochrane Library databases will be searched using appropriate search strategies. Studies that evaluated the validity of COPD codes (such as the International Classification of Diseases 9th Revision and 10th Revision system; the Real codes system or the International Classification of Primary Care) in healthcare databases will be included. Inclusion criteria will be: (1) the presence of a reference standard case definition for COPD; (2) the presence of at least one test measure (eg, sensitivity, positive predictive values, etc); and (3) the use of a healthcare database (including administrative claims databases, electronic healthcare databases or COPD registries) as a data source. Pairs of reviewers will independently abstract data using standardised forms and will assess quality using a checklist based on the Standards for Reporting of Diagnostic accuracy (STARD) criteria. This systematic review protocol has been produced in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA-P) 2015 statement. Ethics and dissemination Ethics approval is not required. Results of this study will be submitted to a peer-reviewed journal for publication. The results from this systematic review will be used for outcome research on COPD and will serve as a guide to identify appropriate case definitions of COPD, and reference standards, for researchers involved in validating healthcare databases. Trial registration number CRD42015029204.

Protocol for validating cardiovascular and cerebrovascular ICD-9-CM codes in healthcare administrative databases: the Umbria Data Value Project

Introduction Administrative healthcare databases can provide a comprehensive assessment of the burden of diseases in terms of major outcomes, such as mortality, hospital readmissions and use of healthcare resources, thus providing answers to a wide spectrum of research questions. However, a crucial issue is the reliability of information gathered. Aim of this protocol is to validate International Classification of Diseases, 9th Revision—Clinical Modification (ICD-9-CM) codes for major cardiovascular diseases, including acute myocardial infarction (AMI), heart failure (HF), atrial fibrillation (AF) and stroke. Methods and analysis Data from the centralised administrative database of the entire Umbria Region (910 000 residents, located in Central Italy) will be considered. Patients with a first hospital discharge for AMI, HF, AF or stroke, between 2012 and 2014, will be identified in the administrative database using the following groups of ICD-9-CM codes located in primary position: (1) 410.x for AMI; (2) 427.31 for AF; (3) 428 for HF; (4) 433.x1, 434 (excluding 434.x0), 436 for ischaemic stroke, 430 and 431 for haemorrhagic stroke (subarachnoid haemorrhage and intracerebral haemorrhage). A random sample of cases, and of non-cases, will be selected, and the corresponding medical charts retrieved and reviewed for validation by pairs of trained, independent reviewers. For each condition considered, case adjudication of disease will be based on symptoms, laboratory and diagnostic tests, as available in medical charts. Divergences will be resolved by consensus. Sensitivity and specificity with 95% CIs will be calculated. Ethics and dissemination Research protocol has been granted approval by the Regional Ethics Committee. Study results will be disseminated widely through peer-reviewed publications and presentations at national and international conferences.

Validating malignant melanoma ICD-9-CM codes in Umbria, ASL Napoli 3 Sud and Friuli Venezia Giulia administrative healthcare databases: a diagnostic accuracy study

Objectives To assess the accuracy of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes in identifying subjects with melanoma. Design A diagnostic accuracy study comparing melanoma ICD-9-CM codes (index test) with medical chart (reference standard). Case ascertainment was based on neoplastic lesion of the skin and a histological diagnosis from a primary or metastatic site positive for melanoma. Setting Administrative databases from Umbria Region, Azienda Sanitaria Locale (ASL) Napoli 3 Sud (NA) and Friuli Venezia Giulia (FVG) Region. Participants 112, 130 and 130 cases (subjects with melanoma) were randomly selected from Umbria, NA and FVG, respectively; 94 non-cases (subjects without melanoma) were randomly selected from each unit. Outcome measures Sensitivity and specificity for ICD-9-CM code 172.x located in primary position. Results The most common melanoma subtype was malignant melanoma of skin of trunk, except scrotum (ICD-9-CM code: 172.5), followed by malignant melanoma of skin of lower limb, including hip (ICD-9-CM code: 172.7). The mean age of the patients ranged from 60 to 61 years. Most of the diagnoses were performed in surgical departments. The sensitivities were 100% (95% CI 96% to 100%) for Umbria, 99% (95% CI 94% to 100%) for NA and 98% (95% CI 93% to 100%) for FVG. The specificities were 88% (95% CI 80% to 93%) for Umbria, 77% (95% CI 69% to 85%) for NA and 79% (95% CI 71% to 86%) for FVG. Conclusions The case definition for melanoma based on clinical or instrumental diagnosis, confirmed by histological examination, showed excellent sensitivities and good specificities in the three operative units. Administrative databases from the three operative units can be used for epidemiological and outcome research of melanoma.

Validity of peptic ulcer disease and upper gastrointestinal bleeding diagnoses in administrative databases: a systematic review protocol.

Introduction Administrative healthcare databases are useful to investigate the epidemiology, health outcomes, quality indicators and healthcare utilisation concerning peptic ulcers and gastrointestinal bleeding, but the databases need to be validated in order to be a reliable source for research. The aim of this protocol is to perform the first systematic review of studies reporting the validation of International Classification of Diseases, 9th Revision and 10th version (ICD-9 and ICD-10) codes for peptic ulcer and upper gastrointestinal bleeding diagnoses. Methods and analysis MEDLINE, EMBASE, Web of Science and the Cochrane Library databases will be searched, using appropriate search strategies. We will include validation studies that used administrative data to identify peptic ulcer disease and upper gastrointestinal bleeding diagnoses or studies that evaluated the validity of peptic ulcer and upper gastrointestinal bleeding codes in administrative data. The following inclusion criteria will be used: (a) the presence of a reference standard case definition for the diseases of interest; (b) the presence of at least one test measure (eg, sensitivity, etc) and (c) the use of an administrative database as a source of data. Pairs of reviewers will independently abstract data using standardised forms and will evaluate quality using the checklist of the Standards for Reporting of Diagnostic Accuracy (STARD) criteria. This systematic review protocol has been produced in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocol (PRISMA-P) 2015 statement. Ethics and dissemination Ethics approval is not required given that this is a protocol for a systematic review. We will submit results of this study to a peer-reviewed journal for publication. The results will serve as a guide for researchers validating administrative healthcare databases to determine appropriate case definitions for peptic ulcer disease and upper gastrointestinal bleeding, as well as to perform outcome research using administrative healthcare databases of these conditions. Trial registration number CRD42015029216.

Accuracy of lung cancer ICD-9-CM codes in Umbria, Napoli 3 Sud and Friuli Venezia Giulia administrative healthcare databases: a diagnostic accuracy study.

Objectives To assess the accuracy of International Classification of Diseases 9th Revision–Clinical Modification (ICD-9-CM) codes in identifying subjects with lung cancer. Design A cross-sectional diagnostic accuracy study comparing ICD-9-CM 162.x code (index test) in primary position with medical chart (reference standard). Case ascertainment was based on the presence of a primary nodular lesion in the lung and cytological or histological documentation of cancer from a primary or metastatic site. Setting Three operative units: administrative databases from Umbria Region (890 000 residents), ASL Napoli 3 Sud (NA) (1 170 000 residents) and Friuli Venezia Giulia (FVG) Region (1 227 000 residents). Participants Incident subjects with lung cancer (n=386) diagnosed in primary position between 2012 and 2014 and a population of non-cases (n=280). Outcome measures Sensitivity, specificity and positive predictive value (PPV) for 162.x code. Results 130 cases and 94 non-cases were randomly selected from each database and the corresponding medical charts were reviewed. Most of the diagnoses for lung cancer were performed in medical departments. True positive rates were high for all the three units. Sensitivity was 99% (95% CI 95% to 100%) for Umbria, 97% (95% CI 91% to 100%) for NA, and 99% (95% CI 95% to 100%) for FVG. The false positive rates were 24%, 37% and 23% for Umbria, NA and FVG, respectively. PPVs were 79% (73% to 83%)%) for Umbria, 58% (53% to 63%)%) for NA and 79% (73% to 84%)%) for FVG. Conclusions Case ascertainment for lung cancer based on imaging or endoscopy associated with histological examination yielded an excellent sensitivity in all the three administrative databases. PPV was moderate for Umbria and FVG but lower for NA.

HEALTH TECHNOLOGY DISINVESTMENT WORLDWIDE: OVERVIEW OF PROGRAMS AND POSSIBLE DETERMINANTS

OBJECTIVES In the past decade, there has been a growing interest in health technology disinvestment. A disinvestment process should involve all relevant stakeholders to identify and deliver the most effective, safe, and cost-effective healthcare interventions. The aim of the present study was to describe the state of the art of health technology disinvestment around the world and to identify parameters that could be associated with the implementation of disinvestment programs. METHODS A systematic review of the literature was performed from database inception to November 2014, together with the collection of original data on socio-economic indicators from forty countries. RESULTS Overall, 1,456 records (1,199 from electronic databases and 257 from other sources) were initially retrieved. After analyzing 172 full text articles, 38 papers describing fifteen disinvestment programs/experiences in eight countries were included. The majority (12/15) of disinvestment programs began after 2006. As expected, these programs were more common in developed countries, 63 percent of which had a Beveridge model healthcare system. The univariate analysis showed that countries with disinvestment programs had a significantly higher level of Human Development Index, Gross Domestic Product per capita, public expenditure on health and social services, life expectancy at birth and a lower level of infant mortality rate, and of perceived corruption. The existence of HTA agencies in the country was a strong predictor (p = .034) for the development of disinvestment programs. CONCLUSIONS The most significant variables in the univariate analysis were connected by a common factor, potentially related to the overall development stage of the country.

Ultrasonography for Endoleak Detection After Endoluminal Abdominal Aortic Aneurysm Repair

Conclusions: This review demonstrates that both ultrasoundmodalities (with or without contrast) showed high specificity. For ruling in endoleaks, contrast-enhanced color duplex ultrasound (CE-CDUS) appears superior to CDUS. In an endoleak surveillance program, CE-CDUS can be introduced as a routine diagnostic modality followed by CT scan only when the ultrasound is positive to establish the type of endoleak and the subsequent therapeutic management. Summary: People with abdominal aortic aneurysm who receive endovascular aneurysm repair (EVAR) need lifetime surveillance to detect potential endoleaks. Computed tomography (CT) angiography is considered the reference standard for endoleak surveillance. Color duplex ultrasound (CDUS) and contrast-enhanced CDUS (CE-CDUS) are less invasive but considered less accurate than CT. To determine the diagnostic accuracy of CDUS and CECDUS in terms of sensitivity and specificity for endoleak detection after endoluminal abdominal aortic aneurysm repair (EVAR), the authors’ searched MEDLINE, Embase, LILACS, ISI Conference Proceedings, Zetoc, and trial registries in June 2016 without language restrictions and without use of filters to maximize sensitivity. Selection criteria included any cross-sectional diagnostic study evaluating participants who received EVAR by both ultrasound (with or without contrast) and CT scan assessed at regular intervals. Two pairs of review authors independently extracted data and assessed quality of included studies using a standard tool (QUADAS) with discrepancies resolved by a third author. The unit of analysis was number of participants for the primary analysis and number of scans performed for the secondary analysis. Meta-analysis was carried out to estimate sensitivity and specificity of CDUS or CE-CDUS using a bivariate model with each index test evaluated separately. As potential sources of heterogeneity, the year of publication, characteristics of included participants (age and gender), direction of the study (retrospective, prospective), country of origin, number of CDUS operators, and ultrasound manufacturer were investigated. Forty-two primary studies with 4220 participants were identified. Twenty studies provided accuracy data based on the number of individual participants (seven of which provided data with and without the use of contrast). Sixteen of these studies evaluated the accuracy of CDUS. These studies were generally of moderate to low quality. Only 3 studies fulfilled all QUADAS items; in 6 (40%) of the studies, the delay between the tests was unclear or longer than 4 weeks; in 8 (50%), the blinding of either the index test or the reference standard was not clearly reported or was not performed; and in 2 studies (12%), the interpretation of the reference standard was not clearly reported. Eleven studies evaluated the accuracy of CE-CDUS. These studies were of better quality than the CDUS studies: 5 (45%) studies fulfilled all the QUADAS items; 4 (36%) did not report clearly the blinding interpretation of the reference standard; and 2 (18%) did not clearly report the delay between the two tests. Based on the bivariate model, the summary estimates for CDUS were 0.82 (95% confidence interval [CI], 0.66-0.91) for sensitivity and 0.93 (95% CI, 0.87-0.96) for specificity whereas for CE-CDUS the estimates were 0.94 (95% CI, 0.85-0.98) for sensitivity and 0.95 (95% CI, 0.90-0.98) for specificity. Regression analysis showed that CE-CDUS was superior to CDUS in terms of sensitivity (LR Chi2 1⁄4 5.08; 1 degree of freedom [df]; P 1⁄4 .0242 for model improvement). Seven studies provided estimates before and after administration of contrast. Sensitivity before contrast was 0.67 (95% CI, 0.47-0.83) and after contrast was 0.97 (95% CI, 0.92-0.99). The improvement in sensitivity with contrast use was statistically significant (LR Chi2 1⁄4 13.47, 1 df; P 1⁄4 .0002 for model improvement). Regression testing showed evidence of statistically significant effect bias related to year of publication and study quality within individual participants based CDUS studies. Sensitivity estimates were higher in the studies published before 2006 than the estimates obtained from studies published in 2006 or later (P < .001); and studies judged as low/unclear quality provided higher estimates in sensitivity. When regression testing was applied to the individual based CE-CDUS studies, none of the items, namely direction of the study design, quality, and age, were identified as a source of heterogeneity. Twenty-two studies provided accuracy data based on number of scans performed (of which four provided data with and without the use of contrast). Analysis of the studies that provided scan based data showed similar results. Summary estimates for CDUS (18 studies) showed 0.72 (95% CI, 0.55-0.85) for sensitivity and 0.95 (95% CI, 0.90-0.96) for specificity whereas summary estimates for CECDUS (eight studies) were 0.91 (95% CI, 0.68-0.98) for sensitivity and 0.89 (95% CI, 0.71-0.96) for specificity. Comments: This review does a nice job evaluating the ability to see endoleaks by duplex imaging but most practicing surgeons are more interested in the effects of an endoleak (sac growth, change in configuration of the stent graft, etc). For most centers, the addition of contrast to the standard duplex EVAR study would likely be cost/time prohibitive without adding significantly to the clinical need of the patient. One might even argue that determining a true negative (specificity) is more important here since certainly in those cases no further testing is needed. In those with a positive study, one needs to determine if there is a negative impact from the endoleak usually determined by sac enlargement before other testing is warranted.