Oriana Ciani

Comparison of treatment effect sizes associated with surrogate and final patient relevant outcomes in randomised controlled trials: meta-epidemiological study

Objective To quantify and compare the treatment effect and risk of bias of trials reporting biomarkers or intermediate outcomes (surrogate outcomes) versus trials using final patient relevant primary outcomes. Design Meta-epidemiological study. Data sources All randomised clinical trials published in 2005 and 2006 in six high impact medical journals: Annals of Internal Medicine, BMJ, Journal of the American Medical Association, Lancet, New England Journal of Medicine, and PLoS Medicine. Study selection Two independent reviewers selected trials. Data extraction Trial characteristics, risk of bias, and outcomes were recorded according to a predefined form. Two reviewers independently checked data extraction. The ratio of odds ratios was used to quantify the degree of difference in treatment effects between the trials using surrogate outcomes and those using patient relevant outcomes, also adjusted for trial characteristics. A ratio of odds ratios >1.0 implies that trials with surrogate outcomes report larger intervention effects than trials with patient relevant outcomes. Results 84 trials using surrogate outcomes and 101 using patient relevant outcomes were considered for analyses. Study characteristics of trials using surrogate outcomes and those using patient relevant outcomes were well balanced, except for median sample size (371 v 741) and single centre status (23% v 9%). Their risk of bias did not differ. Primary analysis showed trials reporting surrogate endpoints to have larger treatment effects (odds ratio 0.51, 95% confidence interval 0.42 to 0.60) than trials reporting patient relevant outcomes (0.76, 0.70 to 0.82), with an unadjusted ratio of odds ratios of 1.47 (1.07 to 2.01) and adjusted ratio of odds ratios of 1.46 (1.05 to 2.04). This result was consistent across sensitivity and secondary analyses. Conclusions Trials reporting surrogate primary outcomes are more likely to report larger treatment effects than trials reporting final patient relevant primary outcomes. This finding was not explained by differences in the risk of bias or characteristics of the two groups of trials.

Determinants of demand for total hip and knee arthroplasty: A systematic literature review

BackgroundDocumented age, gender, race and socio-economic disparities in total joint arthroplasty (TJA), suggest that those who need the surgery may not receive it, and present a challenge to explain the causes of unmet need. It is not clear whether doctors limit treatment opportunities to patients, nor is it known the effect that patient beliefs and expectations about the operation, including their paid work status and retirement plans, have on the decision to undergo TJA. Identifying socio-economic and other determinants of demand would inform the design of effective and efficient health policy. This review was conducted to identify the factors that lead patients in need to undergo TJA.MethodsAn electronic search of the Embase and Medline (Ovid) bibliographic databases conducted in September 2011 identified studies in the English language that reported on factors driving patients in need of hip or knee replacement to undergo surgery. The review included reports of elective surgery rates in eligible patients or, controlling for disease severity, in general subjects, and stated clinical experts’ and patients’ opinions on suitability for or willingness to undergo TJA. Quantitative and qualitative studies were reviewed, but quantitative studies involving fewer than 20 subjects were excluded. The quality of individual studies was assessed on the basis of study design (i.e., prospective versus retrospective), reporting of attrition, adjustment for and report of confounding effects, and reported measures of need (self-reported versus doctor-assessed). Reported estimates of effect on the probability of surgery from analyses adjusting for confounders were summarised in narrative form and synthesised in odds ratio (OR) forest plots for individual determinants.ResultsThe review included 26 quantitative studies−23 on individuals’ decisions or views on having the operation and three about health professionals’ opinions-and 10 qualitative studies. Ethnic and racial disparities in TJA use are associated with socio-economic access factors and expectations about the process and outcomes of surgery. In the United States, health insurance coverage affects demand, including that from the Medicare population, for whom having supplemental Medicaid coverage increases the likelihood of undergoing TJA. Patients with post-secondary education are more likely to demand hip or knee surgery than those without it (range of OR 0.87-2.38). Women are as willing to undergo surgery as men, but they are less likely to be offered surgery by specialists than men with the same need. There is considerable variation in patient demand with age, with distinct patterns for hip and knee. Paid employment appears to increase the chances of undergoing surgery, but no study was found that investigated the relationship between retirement plans and demand for TJA. There is evidence of substantial geographical variation in access to joint replacement within the territory covered by a public national health system, which is unlikely to be explained by differences in preference or unmeasured need alone. The literature tends to focus on associations, rather than testing of causal relationships, and is insufficient to assess the relative importance of determinants.ConclusionsPatients’ use of hip and knee replacement is a function of their socio-economic circumstances, which reinforce disparities by gender and race originating in the doctor-patient interaction. Willingness to undergo surgery declines steeply after the age of retirement, at the time some eligible patients may lower their expectations of health status achievement. There is some evidence that paid employment independently increases the likelihood of operation. The relative contribution of variations in surgical decision making to differential access across regions within countries deserves further research that controls for clinical need and patient lifestyle preferences, including retirement decisions. Evidence on this question will become increasingly relevant for service planning and policy design in societies with ageing populations.

VALIDATION OF SURROGATE ENDPOINTS IN ADVANCED SOLID TUMORS: SYSTEMATIC REVIEW OF STATISTICAL METHODS, RESULTS, AND IMPLICATIONS FOR POLICY MAKERS

OBJECTIVES Licensing of, and coverage decisions on, new therapies should rely on evidence from patient-relevant endpoints such as overall survival (OS). Nevertheless, evidence from surrogate endpoints may also be useful, as it may not only expedite the regulatory approval of new therapies but also inform coverage decisions. It is, therefore, essential that candidate surrogate endpoints be properly validated. However, there is no consensus on statistical methods for such validation and on how the evidence thus derived should be applied by policy makers. METHODS We review current statistical approaches to surrogate-endpoint validation based on meta-analysis in various advanced-tumor settings. We assessed the suitability of two surrogates (progression-free survival [PFS] and time-to-progression [TTP]) using three current validation frameworks: Elston and Taylors framework, the German Institute of Quality and Efficiency in Health Cares (IQWiG) framework and the Biomarker-Surrogacy Evaluation Schema (BSES3). RESULTS A wide variety of statistical methods have been used to assess surrogacy. The strength of the association between the two surrogates and OS was generally low. The level of evidence (observation-level versus treatment-level) available varied considerably by cancer type, by evaluation tools and was not always consistent even within one specific cancer type. CONCLUSIONS Not in all solid tumors the treatment-level association between PFS or TTP and OS has been investigated. According to IQWiGs framework, only PFS achieved acceptable evidence of surrogacy in metastatic colorectal and ovarian cancer treated with cytotoxic agents. Our study emphasizes the challenges of surrogate-endpoint validation and the importance of building consensus on the development of evaluation frameworks.

Meta-analyses of randomized controlled trials show suboptimal validity of surrogate outcomes for overall survival in advanced colorectal cancer

OBJECTIVES To quantify and compare the treatment effects on three surrogate end points, progression-free survival (PFS), time to progression (TTP), and tumor response rate (TR) vs. overall survival (OS) based on a meta-analysis of randomized controlled trials (RCTs) of drug interventions in advanced colorectal cancer (aCRC). STUDY DESIGN AND SETTING We systematically searched for RCTs of pharmacologic therapies in aCRC between 2003 and 2013. Trial characteristics, risk of bias, and outcomes were recorded based on a predefined form. Univariate and multivariate random-effects meta-analyses were used to estimate pooled summary treatment effects. The ratio of hazard ratios (HRs)/odds ratios (ORs) and difference in medians were used to quantify the degree of difference in treatment effects on the surrogate end points and OS. Spearman ρ, surrogate threshold effect (STE), and R(2) were also estimated across predefined trial-level covariates. RESULTS We included 101 RCTs. In univariate and multivariate meta-analyses, we found larger treatment effects for the surrogates than for OS. Compared with OS, treatment effects were on average 13% higher when HRs were measured and 3% to 45% higher when ORs were considered; differences in median PFS/TTP were higher than on OS by an average of 0.5 month. Spearman ρ ranged from 0.39 to 0.80, mean R(2) from 0.06 to 0.65, and STE was 0.8 for HRPFS, 0.64 for HRTTP, or 0.28 for ORTR. The stratified analyses revealed high variability across all strata. CONCLUSION None of the end points in this study were found to achieve the level of evidence (ie, mean R(2)trial > 0.60) that has been set to select high or excellent correlation levels by common surrogate evaluation tools. Previous surrogacy relationships observed between PFS and TTP vs. OS in selected settings may not apply across other classes or lines of therapy.

Diffusion and use of health technology assessment in policy making: What lessons for decentralised healthcare systems?

OBJECTIVE The Italian National Healthcare System (NHS) is one of the most decentralised systems since the devolution reform approved in 2001. HTA is spreading as an important tool for decision-making processes both at central and local levels. The aims of this study were to review the state of the health technology assessment (HTA) programmes in Italy - with a focus on regional and central initiatives - and to discuss consequences of a multi-level structure of HTA agencies in highly regionalised healthcare systems. METHODS Our method combined documentary review with interviews. We reviewed scientific literature about HTAs activities in decentralised systems, legislative and administrative documents from national as well as regional authorities. Semi-structured interviews were conducted with 18 key individuals associated with HTA both at the national and regional levels. Data on HTA programmes implemented or under development in nine regions were collected and analysed according to key principles for the improved conduct of health technology assessments for resource allocation decisions. RESULTS HTA is in the early stage of development in Italy, although with great heterogeneity across regions. The National Agency for Health Services has certainly contributed to HTA diffusion through supporting and training activities. However, the multi-level structure of HTA in Italy has not yet provided full coordination and harmonisation of practices and outcomes across the country, with a consequent exacerbate inequality of access to services and technologies. CONCLUSIONS There is probably need to rethink the multi-layer organizational framework of HTA in Italy by leveraging on current knowledge and efficient redistribution of activities across regions. We would advise for different jurisdictions playing different roles while achieving similar health outcomes for their patients, rather than jurisdictions aiming at doing exactly the same things resulting in unequal access to healthcare service provision.

Time to review the role of surrogate end points in health policy: state of the art and the way forward

The efficacy of medicines, medical devices, and other health technologies should be proved in trials that assess final patient-relevant outcomes such as survival or morbidity. Market access and coverage decisions are, however, often based on surrogate end points, biomarkers, or intermediate end points, which aim to substitute and predict patient-relevant outcomes that are unavailable because of methodological, financial, or practical constraints. We provide a summary of the present use of surrogate end points in health care policy, discussing the case for and against their adoption and reviewing validation methods. We introduce a three-step framework for policymakers to handle surrogates, which involves establishing the level of evidence, assessing the strength of the association, and quantifying relations between surrogates and final outcomes. Although the use of surrogates can be problematic, they can, when selected and validated appropriately, offer important opportunities for more efficient clinical trials and faster access to new health technologies that benefit patients and health care systems.

Intravesical administration of combined hyaluronic acid (HA) and chondroitin sulfate (CS) for the treatment of female recurrent urinary tract infections: A European multicentre nested case-control study

Objectives To compare the clinical effectiveness of the intravesical administration of combined hyaluronic acid and chondroitin sulfate (HA+CS) versus current standard management in adult women with recurrent urinary tract infections (RUTIs). Setting A European Union-based multicentre, retrospective nested case–control study. Participants 276 adult women treated for RUTIs starting from 2009 to 2013. Interventions Patients treated with either intravesical administration of HA+CS or standard of care (antimicrobial/immunoactive prophylaxis/probiotics/cranberry). Primary and secondary outcome measures The primary outcome was occurrence of bacteriologically confirmed recurrence within 12 months. Secondary outcomes were time to recurrence, total number of recurrences, health-related quality of life and healthcare resource consumption. Crude and adjusted results for unbalanced characteristics are presented. Results 181 patients treated with HA+CS and 95 patients treated with standard of care from 7 centres were included. The crude and adjusted ORs (95% CI) for the primary end point were 0.77 (0.46 to 1.28) and 0.51 (0.27 to 0.96), respectively. However, no evidence of improvement in terms of total number of recurrences (incidence rate ratio (95% CI), 0.99 (0.69 to 1.43)) or time to first recurrence was seen (HR (95% CI), 0.99 (0.61 to 1.61)). The benefit of intravesical HA+CS therapy improves when the number of instillations is ≥5. Conclusions Our results show that bladder instillations of combined HA+CS reduce the risk of bacteriologically confirmed recurrences compared with the current standard management of RUTIs. Total incidence rates and hazard rates were instead non-significantly different between the 2 groups after adjusting for unbalanced factors. In contrast to what happens with antibiotic prophylaxis, the effectiveness of the HA+CS reinstatement therapy improves over time. Trial registration number NCT02016118.

Health technology assessment of medical devices: a survey of non-European union agencies.

Objectives: The aim of this study was to review and compare current health technology assessment (HTA) activities for medical devices across non-European Union HTA agencies. Methods: HTA activities for medical devices were evaluated from three perspectives: organizational structure, processes, and methods. Agencies were primarily selected upon membership of existing HTA networks. The data collection was performed in two stages: stage 1–agency Web-site assessment using a standardized questionnaire, followed by review and validation of the collected data by a representative of the agency; and stage 2–semi-structured telephone interviews with key informants of a sub-sample of agencies. Results: In total, thirty-six HTA agencies across twenty non-EU countries assessing medical devices were included. Twenty-seven of thirty-six (75 percent) agencies were judged at stage 1 to have adopted HTA-specific approaches for medical devices (MD-specific agencies) that were largely organizational or procedural. There appeared to be few differences in the organization, process and methods between MD-specific and non–MD-specific agencies. Although the majority (69 percent) of both categories of agency had specific methods guidance or policy for evidence submission, only one MD-specific agency had developed methodological guidelines specific to medical devices. In stage 2, many MD-specific agencies cited insufficient resources (budget, skilled employees), lack of coordination (between regulator and reimbursement bodies), and the inability to generalize findings from evidence synthesis to be key challenges in the HTA of medical devices. Conclusions: The lack of evidence for differentiation in scientific methods for HTA of devices raises the question of whether HTA needs to develop new methods for medical devices but rather adapt existing methodological approaches. In contrast, organizational and/or procedural adaptation of existing HTA agency frameworks to accommodate medical devices appear relatively commonplace.

Method for Movement and Gesture Assessment (MMGA) in Ergonomics

We present a technique for the ergonomic assessment of motor tasks and postures. It is based on movement analysis and it integrates the perceived discomfort scores for joints motions and the time involvement of the different body districts. It was tested on 8 subjects performing reaching movements. The experimental protocol was designed to have an a priori expected comfort ranking, namely, higher values in presence of more uncomfortable tasks. The validation of the Method for Movement and Gesture Assessment (MMGA) in the ergonomic evaluation of a reaching task gave promising results and showed the effectiveness of the index. Possible applications of the method might be the integration into CAD tools and human motion simulation to provide an early comparative evaluation of the ergonomics of the prototyping process and workplace redesign in industry.

Trans-arterial radioembolization in intermediate-advanced hepatocellular carcinoma: systematic review and meta-analyses

Trans-arterial radioembolization (TARE) is a recognized, although not explicitly recommended, experimental therapy for unresectable hepatocellular carcinoma (HCC). A systematic literature review was performed to identify published studies on the use of TARE in intermediate and advanced stages HCC exploring the efficacy and safety of this innovative treatment. Twenty-one studies reporting data on overall survival (OS) and time to progression (TTP), were included in a meta-analysis. The pooled post-TARE OS was 63% (95% CI: 56-70%) and 27% (95% CI: 21-33%) at 1- and 3-years respectively in intermediate stage HCC, whereas OS was 37% (95% CI: 26-50%) and 13% (95% CI: 9-18%) at the same time intervals in patients with sufficient liver function (Child-Pugh A-B7) but with an advanced HCC because of the presence of portal vein thrombosis. When an intermediate and advanced case-mix was considered, OS was 58% (95% CI: 48-67%) and 17% (95% CI: 12-23%) at 1- and 3-years respectively. As for TTP, only four studies reported data: the observed progression probability was 56% (95% CI: 41-70%) and 73% (95% CI: 56-87%) at 1 and 2 years respectively. The safety analysis, focused on the risk of liver decompensation after TARE, revealed a great variability, from 0-1% to more than 36% events, influenced by the number of procedures, patient Child-Pugh stage and treatment duration. Evidence supporting the use of radioembolization in HCC is mainly based on retrospective and prospective cohort studies. Based on this evidence, until the results of the ongoing randomized trials become available, radioembolization appears to be a viable treatment option for intermediate-advanced stage HCC.