Ivan Moschetti

Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study

Abstract Objectives To describe how the methodological quality of primary studies is assessed in systematic reviews and whether the quality assessment is taken into account in the interpretation of results. Data sources Cochrane systematic reviews and systematic reviews in paper based journals. Study selection 965 systematic reviews (809 Cochrane reviews and 156 paper based reviews) published between 1995 and 2002. Data synthesis The methodological quality of primary studies was assessed in 854 of the 965 systematic reviews (88.5%). This occurred more often in Cochrane reviews than in paper based reviews (93.9% v 60.3%, P < 0.0001). Overall, only 496 (51.4%) used the quality assessment in the analysis and interpretation of the results or in their discussion, with no significant differences between Cochrane reviews and paper based reviews (52% v 49%, P = 0.58). The tools and methods used for quality assessment varied widely. Conclusions Cochrane reviews fared better than systematic reviews published in paper based journals in terms of assessment of methodological quality of primary studies, although they both largely failed to take it into account in the interpretation of results. Methods for assessment of methodological quality by systematic reviews are still in their infancy and there is substantial room for improvement.

A Review of Online Evidence-based Practice Point-of-Care Information Summary Providers

Background Busy clinicians need easy access to evidence-based information to inform their clinical practice. Publishers and organizations have designed specific tools to meet doctors’ needs at the point of care. Objective The aim of this study was to describe online point-of-care summaries and evaluate their breadth, content development, and editorial policy against their claims of being “evidence-based.” Methods We searched Medline, Google, librarian association websites, and information conference proceedings from January to December 2008. We included English Web-based point-of-care summaries designed to deliver predigested, rapidly accessible, comprehensive, periodically updated, evidence-based information to clinicians. Two investigators independently extracted data on the general characteristics and content presentation of summaries. We assessed and ranked point-of-care products according to: (1) coverage (volume) of medical conditions, (2) editorial quality, and (3) evidence-based methodology. We explored how these factors were associated. Results We retrieved 30 eligible summaries. Of these products, 18 met our inclusion criteria and were qualitatively described, and 16 provided sufficient data for quantitative evaluation. The median volume of medical conditions covered was 80.6% (interquartile range, 68.9% - 84.2%) and varied for the different products. Similarly, differences emerged for editorial policy (median 8.0, interquartile range 5.8 - 10.3) and evidence-based methodology scores (median 10.0, interquartile range 1.0 - 12.8) on a 15-point scale. None of these dimensions turned out to be significantly associated with the other dimensions (editorial quality and volume, Spearman rank correlation r = -0.001, P = .99; evidence-based methodology and volume, r = -0.19, P = .48; editorial and evidence-based methodology, r = 0.43, P =.09). Conclusions Publishers are moving to develop point-of-care summary products. Some of these have better profiles than others, and there is room for improved reporting of the strengths and weaknesses of these products.

Speed of updating online evidence based point of care summaries: Prospective cohort analysis

Objective To evaluate the ability of international point of care information summaries to update evidence relevant to medical practice. Design Prospective cohort bibliometric analysis. Setting Top five point of care information summaries (Clinical Evidence, EBMGuidelines, eMedicine, Dynamed, UpToDate) ranked for coverage of medical conditions, editorial quality, and evidence based methodology. Main outcome measures From June 2009 to May 2010 we measured the incidence of research findings relating to potentially eligible newsworthy evidence. As samples, we chose systematic reviews rated as relevant by international research networks (such as, Evidence-Based Medicine, ACP Journal Club, and the Cochrane Collaboration). Every month we assessed whether each sampled review was cited in at least one chapter of the five summaries. The cumulative updating rate was analysed with Kaplan-Meier curves. Results From April to December 2009, 128 reviews were retrieved; 53% (68) from the literature surveillance journals and 47% (60) from the Cochrane Library. At nine months, Dynamed had cited 87% of the sampled reviews, while the other summaries had cited less than 50%. The updating speed of Dynamed clearly led the others. For instance, the hazard ratios for citations in EBM Guidelines and Clinical Evidence versus the top performer were 0.22 (95% confidence interval 0.17 to 0.29) and 0.03 (0.01 to 0.05). Conclusions Point of care information summaries include evidence relevant to practice at different speeds. A qualitative analysis of updating mechanisms is needed to determine whether greater speed corresponds to more appropriate incorporation of new information.

Human albumin solution for resuscitation and volume expansion in critically ill patients.

BackgroundHuman albumin solutions are used in a range of medical and surgical problems. Licensed indications are the emergency treatment of shock and other conditions where restoration of blood volume is urgent, burns, and hypoproteinaemia. Human albumin solutions are more expensive than other colloids and crystalloids.ObjectivesTo quantify the effect on mortality of human albumin and plasma protein fraction (PPF) administration in the management of critically ill patients.Search strategyWe searched the Cochrane Injuries Group trials register, Cochrane Central Register of Controlled Trials, Medline, Embase and BIDS Index to Scientific and Technical Proceedings. Reference lists of trials and review articles were checked, and authors of identified trials were, contacted. The search was last updated in August 2004.Selection criteriaRandomised controlled trials comparing albumin/PPF with no albumin/PPF, or with a crystalloid solution, in critically ill patients with hypovolaemia, burns or hypoalbuminaemia.Data collection and analysisWe collected data on the participants, albumin solution used, mortality at the end of follow up, and quality of allocation concealment. Analysis was stratified according to patient type.Main resultsWe found 32 trials meeting the inclusion criteria and reporting death as an outcome. There were 1632 deaths among 8452 trial participants. For hypovolaemia, the relative risk of death following albumin administration was 1.01 (95% confidence interval 0.92–1.10). This estimate was heavily influenced by the results of the SAFE trial, which contributed 91% of the information (based on the weights in the meta-analysis). For burns, the relative risk was 2.40 (1.11–5.19) and for hypoalbuminaemia the relative risk was 1.38 (0.94–2.03). There was no substantial heterogeneity between the trials in the various categories (X2=21.86, df=25, p=0.64). The pooled relative risk of death with albumin administration was 1.04 (0.95–1.13).ConclusionsFor patients with hypovolaemia there is no evidence that albumin reduces mortality when compared with cheaper alternatives such as saline. There is no evidence that albumin reduces mortality in critically ill patients with burns and hypoalbuminaemia. The possibility that there may be highly selected populations of critically ill patients in which albumin may be indicated remains open to question. However, in view of the absence of evidence of a mortality benefit from albumin and the increased cost of albumin compared to alternatives such as saline, it would seem reasonable that albumin should only be used within the context of well concealed and adequately powered randomised controlled trial.Plain language summaryThere is no evidence that giving human albumin to replace lost blood in critically ill or injured people improves survival when compared to giving saline.Trauma, burns or surgery can cause people to lose large amounts of blood. Fluid replacement, giving fluids intravenously (into a vein), is used to help restore blood volume and hopefully reduce the risk of dying. Blood products (including human albumin), non-blood products or combinations can be used. The review of trials found no evidence that albumin reduces the risk of dying. Albumin is very expensive in which case it may be better to use cheaper alternatives such as saline for fluid resuscitation.

Clinical and subclinical cardiac late effects in pediatric Hodgkin's lymphoma survivors

Purpose Cardiac late effects are responsible for a significant burden of mortality and morbidity among pediatric Hodgkins lymphoma (HL) survivors (HLS). The aim of our study was to assess clinical and subclinical cardiac sequelae in a cohort of childhood HLS treated in the 1980s with doxorubicin, bleomycin, vinblastine, and dacarbazine (the ABVD regimen) and limited-field radiotherapy (RT). Methods We retrospectively examined a series of HLS treated from 1979 to 1989. We searched for subtle cardiac abnormalities in a subgroup of asymptomatic individuals, who underwent rest and exercise echocardiography at least 20 years after completing their therapies. Their cardiac assessment included physical examination, electrocardiogram (ECG), and resting and postexercise echocardiograms. Results On thorough cardiac assessment a mean of 21 years after their diagnosis, none of the 53 unselected asymptomatic HLS showed physical signs or significant ECG abnormalities during or after the stress echo test. Twenty-two (41%) of the 53 patients revealed valvular abnormalities, with mitral regurgitation in 28%, aortic regurgitation in 9%, and both in 4%. No significant myocardial dysfunction as a result of previous combined doxorubicin treatment and chest RT was identified. Only 2 individuals had mild pericardial alterations. Conclusions The present study shows that long-term cardiac effects are common in HLS treated with the ABVD regimen and RT. The most frequent complications observed in this sample were essentially coronary artery disease and valvular abnormalities. None of the survivors in this sample showed overt congestive heart failure, a finding in contrast with larger studies.

Internet-Based Education for Health Professionals

who discontinued their MD/PhD program participation but completed the MD degree program requirements (as indicated on the GQ). Two other categories of MD/PhD program noncompleters—students who discontinued the MD portion of the program but completed the PhD degree and students who completely discontinued participation in both MD and PhD degree programs—were not included in our sample, as they would not have had the opportunity to complete the GQ, which is completed only by medical school graduates. So it cannot be inferred from our study that most students who discontinued their MD/PhD program participation finished their MD degree. As medical school attrition does occur for both academic and nonacademic reasons, albeit at a relatively low rate, the level of attrition that we reported probably underestimated the overall extent of MD/PhD program attrition. Miller’s comments about T2 translational research career paths should be considered in the context of the results of the MD/PhD Students’ Attitudes, Goals, and Education (SAGE) Survey, which was completed by nearly 500 MD/PhD program enrollees. Of those respondents who expressed a desire to engage in research in their future careers, 27% chose basic science, 53% chose disease-oriented research, and 15% chose clinical or epidemiologic research to describe their future primary research type. Thus, among MD/PhD program participants, there appears to be interest in engaging in types of research other than basic science and T1 translational research. The range of PhD programs available to MD/PhD students enrolled in US allopathic medical schools is now very broad, extending well beyond basic science disciplines. As suggested by Rosenberg in his Editorial, a comprehensive assessment of MD/PhD program outcomes could be highly informative in many regards, including documentation of the career paths chosen by MD/PhD program graduates with PhDs in a range of disciplines.

Rimonabant for overweight and “metabolic syndrome”: the attempt to supersize disease and risk by pharmaceutical marketing

A Cochrane systematic review explored the potential role of rimonabant for overweight and obesity [1]. On the basis of surrogate outcomes reported in the included RCTs, rimonabant has been a candidate (by its producer) to act as a pleiotropic agent for the entire cardiovascular risk spectrum. Is this compelling evidence to consider rimonabant the new panacea for the ‘‘metabolic syndrome’’ or are we facing another attempt of disease mongering by a new market frontier? Rimonabant has been shown to reduce food intake, appetite and body weight in overweight or obese people. Four randomized controlled trials (RIO-Europe [2], RIONorth America [3], RIO-diabetes [4], RIO-lipids [5]) evaluated rimonabant 20 mg versus rimonabant 5 mg versus placebo. All interventions were given over 1 year and included the addition of a hypocaloric diet. All trials were designed, conducted and reported with the contribution of Sanofi-Aventis drug company. The Cochrane review (‘‘Rimonabant for overweight and obesity’’) [1] includes evidence from these four trials.

Systematic reviews highlight the complex balance between good and harm from screening studies

Background Mammographic screening for breast cancer is controversial, as reflected in greatly varying national policies. p Objectives The objective was to assess the effect of screening for breast cancer with mammography on mortality and morbidity.

Outcome reporting bias in government-funded RCTs.

An-Wen Chan and associates,[1][1] in their evaluation of outcome reporting bias in 48 randomized controlled trials funded by the Canadian Institutes of Health Research (CIHR), found that a high number (median 26) of outcomes were declared in each protocol, but not all of these outcomes were reported

Renaissance of ares publica clinical research: global access to trial registers

The pivotal importance of registering clinical trials is increasingly accepted, and several governmental and non-governmental bodies, including the Italian Ministry of Health, have developed guidelines or set up registries in recent years and months1. Three initiatives at the international level seem to us to foster the global awareness of the ethical and scientific obligation to register clinical trials. First, the International Committee of Medical Journal Editors (ICMJE) initiative2. Several decades since the idea of registering clinical trials was first proposed, the ICMJE took a significant step to urge academics and researchers to use the registers: after July 1, 2005, trials that are not registered will not be considered for publication in the medical journals that adhere to the ICMJE initiative. Which studies are eligible then? All trials in which volunteers are prospectively assigned to a medical intervention or a control, and in which a causal-relationship between the intervention and an outcome is the focus2. Medical interventions include drugs as well as surgical procedures, behavioural treatments and quality improvement processes. Only trials looking at pharmacokinetics and early toxicity are clearly excluded. To maintain an accessible and informative process, the ICMJE is supporting freely accessible and not for profit registers, and the 20 item World Health Organisation (WHO) minimum data set that requires key clinical and methodological fields3. The WHO minimal data set is part of the second landmark initiative to build up the International Clinical Trial Registry Platform4. This WHO entity coordinates an international network of primary registers to harmonise registration across the world. The national and international registers ensure the collection of all required items, data entry validation and other aspects of quality assurance. Several registers expanded their list of items in response to this initiative. We particularly recommend registering trials in two international registers: the US National Library of Medicine’s ClinicalTrials.gov5 and the UK’s Current Controlled Trials International Standard Randomised Controlled Trials Number Register (ISRCTN)6. Both are widely known, and ensure a process to update data and reduce the possibility of duplicate registration. The third initiative, known as the Ottawa Statement, is a consensus document from an international group of researchers with a close connection to and support by The Cochrane Collaboration7. The aim of this group of researchers is to provide a set of guiding principles for the development of trial registers. The first document lists principles for development, while the second document makes the principles operational8. For example, the first principle states that every trial should have a unique ID assigned by a single international source. The proposed mechanism to make this operational is that WHO assigns a Universal Reference Trial Number for a global unique ID. The Italian translation of the Ottawa Statement, Part One (Principles for international registration of protocol information and results from human trials of health-related interventions) will be published in the supplement “Medicina Italia” of the first 2007 issue of Internal and Emergency Medicine as well in the Ottawa Statement website (http:// ottawagroup.ohri.ca). What’s the situation in Italy? Italy has been one of the first countries to legislate a universal compulsory trials’ registration. Through Local Ethic Committees (LECs) all trials are registered at the National Monitoring Centre for Clinical Trials (OsSC)9. Unfortunately not all the information is in the public domain; only funding bodies and the LECs have unrestricted access, while the public has access to only very selected information. This status quo does not comply with the principles and the policies of the above three initiatives. We urge the Ministry of Health to seriously consider revisiting the limitations in the amount of information that is available to the public to foster full transparency. In the meantime Italian researchers should duplicate their efforts, registering their trials within the OsSC and in an international open access register. We are aware that in the past two years many important goals have been achieved toward a global registration of and access to clinical trials. We are even more conscious that there is much room for improvement. Trial registers will only be useful if they increase the accessibility and transparency of evidence. The real test for trial registers is Cochrane’s corner