Richard D. Riley

Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups

BackgroundConventional systematic review techniques have limitations when the aim of a review is to construct a critical analysis of a complex body of literature. This article offers a reflexive account of an attempt to conduct an interpretive review of the literature on access to healthcare by vulnerable groups in the UKMethodsThis project involved the development and use of the method of Critical Interpretive Synthesis (CIS). This approach is sensitised to the processes of conventional systematic review methodology and draws on recent advances in methods for interpretive synthesis.ResultsMany analyses of equity of access have rested on measures of utilisation of health services, but these are problematic both methodologically and conceptually. A more useful means of understanding access is offered by the synthetic construct of candidacy. Candidacy describes how peoples eligibility for healthcare is determined between themselves and health services. It is a continually negotiated property of individuals, subject to multiple influences arising both from people and their social contexts and from macro-level influences on allocation of resources and configuration of services. Health services are continually constituting and seeking to define the appropriate objects of medical attention and intervention, while at the same time people are engaged in constituting and defining what they understand to be the appropriate objects of medical attention and intervention. Access represents a dynamic interplay between these simultaneous, iterative and mutually reinforcing processes. By attending to how vulnerabilities arise in relation to candidacy, the phenomenon of access can be better understood, and more appropriate recommendations made for policy, practice and future research.DiscussionBy innovating with existing methods for interpretive synthesis, it was possible to produce not only new methods for conducting what we have termed critical interpretive synthesis, but also a new theoretical conceptualisation of access to healthcare. This theoretical account of access is distinct from models already extant in the literature, and is the result of combining diverse constructs and evidence into a coherent whole. Both the method and the model should be evaluated in other contexts.

Bivariate random-effects meta-analysis and the estimation of between-study correlation

BackgroundWhen multiple endpoints are of interest in evidence synthesis, a multivariate meta-analysis can jointly synthesise those endpoints and utilise their correlation. A multivariate random-effects meta-analysis must incorporate and estimate the between-study correlation (ρB).MethodsIn this paper we assess maximum likelihood estimation of a general normal model and a generalised model for bivariate random-effects meta-analysis (BRMA). We consider two applied examples, one involving a diagnostic marker and the other a surrogate outcome. These motivate a simulation study where estimation properties from BRMA are compared with those from two separate univariate random-effects meta-analyses (URMAs), the traditional approach.ResultsThe normal BRMA model estimates ρBas -1 in both applied examples. Analytically we show this is due to the maximum likelihood estimator sensibly truncating the between-study covariance matrix on the boundary of its parameter space. Our simulations reveal this commonly occurs when the number of studies is small or the within-study variation is relatively large; it also causes upwardly biased between-study variance estimates, which are inflated to compensate for the restriction on ρ^MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacH8akY=wiFfYdH8Gipec8Eeeu0xXdbba9frFj0=OqFfea0dXdd9vqai=hGuQ8kuc9pgc9s8qqaq=dirpe0xb9q8qiLsFr0=vr0=vr0dc8meaabaqaciaacaGaaeqabaqabeGadaaakeaaiiGacuWFbpGCgaqcaaaa@2E83@B. Importantly, this does not induce any systematic bias in the pooled estimates and produces conservative standard errors and mean-square errors. Furthermore, the normal BRMA is preferable to two normal URMAs; the mean-square error and standard error of pooled estimates is generally smaller in the BRMA, especially given data missing at random. For meta-analysis of proportions we then show that a generalised BRMA model is better still. This correctly uses a binomial rather than normal distribution, and produces better estimates than the normal BRMA and also two generalised URMAs; however the model may sometimes not converge due to difficulties estimating ρB.ConclusionA BRMA model offers numerous advantages over separate univariate synthesises; this paper highlights some of these benefits in both a normal and generalised modelling framework, and examines the estimation of between-study correlation to aid practitioners.

Reporting of prognostic markers: current problems and development of guidelines for evidence-based practice in the future

Prognostic markers help to stratify patients for treatment by identifying patients with different risks of outcome (e.g. recurrence of disease), and are important tools in the management of cancer and many other diseases. Systematic review and meta-analytical approaches to identifying the most valuable prognostic markers are needed because (sometimes conflicting) evidence relating to markers is often published across a number of studies. To investigate the practicality of this approach, an empirical investigation of a systematic review of tumour markers for neuroblastoma was performed; 260 studies of prognostic markers were identified, which considered 130 different markers.The reporting of these studies was often inadequate, in terms of both statistical analysis and presentation, and there was considerable heterogeneity for many important clinical/statistical factors. These problems restricted both the extraction of data and the meta-analysis of results from the primary studies, limiting feasibility of the evidence-based approach.Guidelines for reporting the results of primary prognostic marker studies in cancer, and other diseases, are given in order to facilitate both the interpretation of individual studies and the undertaking of systematic reviews, meta-analysis and, ultimately, evidence-based practice. General availability of full individual patient data is a necessary step forward and would overcome the majority of problems encountered, including poorly reported summary statistics and variability in cutoff level, outcome assessed and adjustment factors used. It would also limit the problem of reporting bias, although publication bias will remain a concern until studies are prospectively registered. Such changes in practice would help important evidence-based reviews to be conducted in order to establish the most appropriate prognostic markers for clinical use, which should ultimately improve patient care.

Meta‐analysis of diagnostic test studies using individual patient data and aggregate data

A meta-analysis of diagnostic test studies provides evidence-based results regarding the accuracy of a particular test, and usually involves synthesizing aggregate data (AD) from each study, such as the 2 by 2 tables of diagnostic accuracy. A bivariate random-effects meta-analysis (BRMA) can appropriately synthesize these tables, and leads to clinical results, such as the summary sensitivity and specificity across studies. However, translating such results into practice may be limited by between-study heterogeneity and that they relate to some average patient across studies.In this paper we describe how the meta-analysis of individual patient data (IPD) from diagnostic studies can lead to clinical results more tailored to the individual patient. We develop IPD models that extend the BRMA framework to include study-level covariates, which help explain the between-study heterogeneity, and also patient-level covariates, which allow one to assess the effect of patient characteristics on test accuracy. We show how the inclusion of patient-level covariates requires a careful separation of within-study and across-study accuracy-covariate effects, as the latter are particularly prone to confounding. Our models are assessed through simulation and extended to allow IPD studies to be combined with AD studies, as IPD are not always available for all studies. Application is made to 23 studies assessing the accuracy of ear thermometers for diagnosing fever in children, with 16 IPD and 7 AD studies. The models reveal that between-study heterogeneity is partly explained by the use of different measurement devices, but there is no evidence that being an infant modifies diagnostic accuracy.

Primer: an evidence-based approach to prognostic markers

Prognostic markers can help to identify patients at different degrees of risk for specific outcomes, facilitate treatment choice, and aid patient counseling. Compared with other research designs, prognostic studies have been relatively neglected in the broad efforts to improve the quality of medical research, despite their ubiquity. Large protocol-driven, prospective studies are the ideal, with clear, unbiased reporting of the methods used and the results obtained. Unfortunately, published prognostic studies rarely meet such standards, and in this article we discuss their main problems and how they can be improved. In particular, an evidence-based approach to prognostic markers is required, as it is usually difficult to ascertain the benefit of a marker from single studies and a clear view is only likely to emerge from looking across multiple studies. Current systematic reviews and meta-analyses often fail to provide clear evidence-based answers, and rather only draw attention to the paucity of good-quality evidence. Prospectively planned pooled analyses of high-quality studies, along with general availability of individual patient data and adherence to reporting guidelines, would help alleviate many of these problems.

A systematic review of molecular and biological markers in tumours of the Ewing's sarcoma family

The aims of this study were to perform the first systematic review of molecular and biological tumour markers in tumours of the Ewings sarcoma family (ESFT), and evaluate the current evidence for their clinical use. A well-defined, reproducible search strategy was used to identify the relevant literature from 1966 to February 2000. Papers were independently assessed for tumour markers used in the screening, diagnosis, prognosis or monitoring of patients with ESFT. Eighty-four papers studying the use of 70 different tumour markers in ESFTs were identified. Low-quality, inconsistent reporting limited meta-analysis to that of prognostic data for 28 markers. Patients with tumours lacking S-100 protein expression have a better overall survival (OS) (hazard ratio (HR)=0.41, 95% confidence interval (CI) 0.19, 0.89) than those with expression; patients with high levels of serum LDH had a worse OS and disease-free survival (DFS) (OS: HR=2.92, CI 2.16, 3.94, DFS: HR=3.38, 95% CI 2.28, 4.99); patients with localised disease and tumours expressing type 1 EWS-FLI1 fusion transcripts had an improved DFS compared with those with other fusion transcript types (HR=0.17, 95% CI 0.079, 0.37). The knowledge base formed should facilitate more informative future research. Improved statistical reporting and large, multicentre prospective studies are advocated.

The impact of home-based physiotherapy interventions on breathlessness during activities of daily living in severe COPD: A systematic review

OBJECTIVESnTo conduct a systematic review and meta-analysis to determine the impact of home-based physiotherapy interventions on breathlessness during activities of daily living (ADL) in severe chronic obstructive disease (COPD).nnnDATA SOURCESnThe electronic databases AMED, CINAHL, Cochrane Central Register of Controlled Trials, Embase, Medline and Physiotherapy Evidence Database (PEDro) were searched from their inception to Week 20 2008. Bibliographies of all potentially relevant retrieved studies, identified relevant systematic reviews and international guidelines were searched by hand.nnnREVIEW METHODSnInclusion criteria consisted of individuals over 18 years of age with severe COPD (defined as forced expiratory volume in 1 second < or = 50% predicted) without cardiovascular co-morbidities, home-based interventions and valid, reliable breathlessness ADL outcome measures. The PEDro scale assessed methodological quality. Data extraction included baseline characteristics, treatment intervention, frequency of training, level of supervision, breathlessness ADL outcome measure and results. Where possible, a random-effects meta-analysis was applied to appropriate trial data to produce overall quantitative results.nnnRESULTSnSeven studies, providing nine data sets, met the inclusion criteria. Trial PEDro scores ranged from 4 to 7 out of 10. Studies were homogenous at baseline regarding age and COPD severity, although subjects were predominantly male. Five studies investigated inspiratory or expiratory muscle training, and two studies investigated exercises. Statistically significant breathlessness ADL outcome improvements were reported for all interventions except expiratory muscle training. Five studies demonstrated clinical significance (four for inspiratory muscle training and one for exercise). However, due to heterogeneity among study interventions and outcomes, meta-analysis was only considered clinically appropriate on one occasion to pool three inspiratory muscle training studies in relation to breathlessness score. The random-effects meta-analysis indicated that, on average, inspiratory muscle training improved the breathlessness score significantly by 2.36 (95% confidence interval 0.76 to 3.96) compared with controls.nnnCONCLUSIONnInspiratory muscle training and exercise are home-based physiotherapy interventions that may improve breathlessness during ADL in severe COPD. Administration can only be advocated tentatively in outpatient services and primary care at this stage because further higher quality, more homogeneous research with larger sample sizes is required to substantiate the current findings.

Where Next for Evidence Synthesis of Prognostic Marker Studies? Improving the Quality and Reporting of Primary Studies to Facilitate Clinically Relevant Evidence-Based Results

Prognostic markers can help to identify patients with different risks of specific outcomes, facilitate treatment choice, and aid patient counselling. Unfortunately, within any given disease area, the wealth of conflicting and heterogeneous evidence makes it difficult for the clinician to ascertain the overall evidence about specific markers and how to use them in practice. The application of formal methods (e.g., a systematic review and meta-analysis) of obtaining and synthesising evidence is therefore greatly needed in the prognostic marker field. However, in this chapter we illustrate and discuss the reasons why currently poor standards of design, clinical relevance, and reporting in primary studies limit statistically reliable and clinically relevant evidence-based results for prognostic markers. These problems add to those issues for the statistical analysis in primary studies that are discussed in another chapter in this volume. To help overcome the problems we highlight guidelines for conducting and reporting primary prognostic research, and we particularly discuss why the availability of individual patient data would help realise the evidence-based use of prognostic markers in clinical practice.

How individual participant data meta-analyses have influenced trial design, conduct, and analysis

Objectives To demonstrate how individual participant data (IPD) meta-analyses have impacted directly on the design and conduct of trials and highlight other advantages IPD might offer. Study Design and Setting Potential examples of the impact of IPD meta-analyses on trials were identified at an international workshop, attended by individuals with experience in the conduct of IPD meta-analyses and knowledge of trials in their respective clinical areas. Experts in the field who did not attend were asked to provide any further examples. We then examined relevant trial protocols, publications, and Web sites to verify the impacts of the IPD meta-analyses. A subgroup of workshop attendees sought further examples and identified other aspects of trial design and conduct that may inform IPD meta-analyses. Results We identified 52 examples of IPD meta-analyses thought to have had a direct impact on the design or conduct of trials. After screening relevant trial protocols and publications, we identified 28 instances where IPD meta-analyses had clearly impacted on trials. They have influenced the selection of comparators and participants, sample size calculations, analysis and interpretation of subsequent trials, and the conduct and analysis of ongoing trials, sometimes in ways that would not possible with systematic reviews of aggregate data. We identified additional potential ways that IPD meta-analyses could be used to influence trials. Conclusions IPD meta-analysis could be better used to inform the design, conduct, analysis, and interpretation of trials.

متاآنالیز دادههای تکتک شرکتکنندگان در مقایسه با متاآنالیز مبتنی بر تجمیع دادهها

BACKGROUNDnMeta-analyses based on individual participant data (IPD-MAs) allow more powerful and uniformly consistent analyses as well as better characterisation of subgroups and outcomes, compared to those which are based on aggregate data (AD-MAs) extracted from published trial reports. However, IPD-MAs are a larger undertaking requiring greater resources than AD-MAs. Researchers have compared results from IPD-MA against results obtained from AD-MA and reported conflicting findings. We present a methodology review to summarise this empirical evidence .nnnOBJECTIVESnTo review systematically empirical comparisons of meta-analyses of randomised trials based on IPD with those based on AD extracted from published reports, to evaluate the level of agreement between IPD-MA and AD-MA and whether agreement is affected by differences in type of effect measure, trials and participants included within the IPD-MA and AD-MA, and whether analyses were undertaken to explore the main effect of treatment or a treatment effect modifier.nnnSEARCH METHODSnAn electronic search of the Cochrane Library (includes Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effectiveness, CENTRAL, Cochrane Methodology Register, HTA database, NHS Economic Evaluations Database), MEDLINE, and Embase was undertaken up to 7 January 2016. Potentially relevant articles that were known to any of the review authors and reference lists of retrieved articles were also checked.nnnSELECTION CRITERIAnStudies reporting an empirical comparison of the results of meta-analyses of randomised trials using IPD with those using AD. Studies were included if sufficient numerical data, comparing IPD-MA and AD-MA, were available in their reports.nnnDATA COLLECTION AND ANALYSISnTwo review authors screened the title and abstract of identified studies with full-text publications retrieved for those identified as eligible or potentially eligible. A quality assessment was done and data were extracted independently by two review authors with disagreements resolved by involving a third author. Data were summarised descriptively for comparisons where an estimate of effect measure and corresponding precision have been provided both for IPD-MA and for AD-MA in the study report. Comparisons have been classified according to whether identical effect measures, identical trials and patients had been used in the IPD-MA and the AD-MA, and whether the analyses were undertaken to explore the main effect of treatment, or to explore a potential treatment effect modifier.Effect measures were transformed to a standardised scale (z scores) and scatter plots generated to allow visual comparisons. For each comparison, we compared the statistical significance (at the 5% two-sided level) of an IPD-MA compared to the corresponding AD-MA and calculated the number of discrepancies. We examined discrepancies by type of analysis (main effect or modifier) and according to whether identical trials, patients and effect measures had been used by the IPD-MA and AD-MA. We calculated the average of differences between IPD-MA and AD-MA (z scores, ratio effect estimates and standard errors (of ratio effects)) and 95% limits of agreement.nnnMAIN RESULTSnFrom the 9330 reports found by our searches, 39 studies were eligible for this review with effect estimate and measure of precision extracted for 190 comparisons of IPD-MA and AD-MA. We classified the quality of studies as no important flaws (29 (74%) studies) or possibly important flaws (10 (26%) studies).A median of 4 (interquartile range (IQR): 2 to 6) comparisons were made per study, with 6 (IQR 4 to 11) trials and 1225 (542 to 2641) participants in IPD-MAs and 7 (4 to 11) and 1225 (705 to 2541) for the AD-MAs. One hundred and forty-four (76%) comparisons were made on the main treatment effect meta-analysis and 46 (24%) made using results from analyses to explore treatment effect modifiers.There is agreement in statistical significance between the IPD-MA and AD-MA for 152 (80%) comparisons, 23 of which disagreed in direction of effect. There is disagreement in statistical significance for 38 (20%) comparisons with an excess proportion of IPD-MA detecting a statistically significant result that was not confirmed with AD-MA (28 (15%)), compared with 10 (5%) comparisons with a statistically significant AD-MA that was not confirmed by IPD-MA. This pattern of disagreement is consistent for the 144 main effect analyses but not for the 46 comparisons of treatment effect modifier analyses. Conclusions from some IPD-MA and AD-MA differed even when based on identical trials, participants (but not necessarily identical follow-up) and treatment effect measures. The average difference between IPD-MA and AD-MA in z scores, ratio effect estimates and standard errors is small but limits of agreement are wide and include important differences in both directions. Discrepancies between IPD-MA and AD-MA do not appear to increase as the differences between trials and participants increase.nnnAUTHORS CONCLUSIONSnIPD offers the potential to explore additional, more thorough, and potentially more appropriate analyses compared to those possible with AD. But in many cases, similar results and conclusions can be drawn from IPD-MA and AD-MA. Therefore, before embarking on a resource-intensive IPD-MA, an AD-MA should initially be explored and researchers should carefully consider the potential added benefits of IPD.