H. M. Vesterinen

Meta-analysis of data from animal studies: A practical guide

Meta-analyses of data from human studies are invaluable resources in the life sciences and the methods to conduct these are well documented. Similarly there are a number of benefits in conducting meta-analyses on data from animal studies; they can be used to inform clinical trial design, or to try and explain discrepancies between preclinical and clinical trial results. However there are inherit differences between animal and human studies and so applying the same techniques for the meta-analysis of preclinical data is not straightforward. For example preclinical studies are frequently small and there is often substantial heterogeneity between studies. This may have an impact on both the method of calculating an effect size and the method of pooling data. Here we describe a practical guide for the meta-analysis of data from animal studies including methods used to explore sources of heterogeneity.

Improving the translational hit of experimental treatments in multiple sclerosis.

Background: In other neurological diseases, the failure to translate pre-clinical findings to effective clinical treatments has been partially attributed to bias introduced by shortcomings in the design of animal experiments. Objectives: Here we evaluate published studies of interventions in animal models of multiple sclerosis for methodological design and quality and to identify candidate interventions with the best evidence of efficacy. Methods: A systematic review of the literature describing experiments testing the effectiveness of interventions in animal models of multiple sclerosis was carried out. Data were extracted for reported study quality and design and for neurobehavioural outcome. Weighted mean difference meta-analysis was used to provide summary estimates of the efficacy for drugs where this was reported in five or more publications. Results: The use of a drug in a pre-clinical multiple sclerosis model was reported in 1152 publications, of which 1117 were experimental autoimmune encephalomyelitis (EAE). For 36 interventions analysed in greater detail, neurobehavioural score was improved by 39.6% (95% CI 34.9—44.2%, p < 0.001). However, few studies reported measures to reduce bias, and those reporting randomization or blinding found significantly smaller effect sizes. Conclusions: EAE has proven to be a valuable model in elucidating pathogenesis as well as identifying candidate therapies for multiple sclerosis. However, there is an inconsistent application of measures to limit bias that could be addressed by adopting methodological best practice in study design. Our analysis provides an estimate of sample size required for different levels of power in future studies and suggests a number of interventions for which there are substantial animal data supporting efficacy.

Risk of Bias in Reports of In Vivo Research: A Focus for Improvement.

The reliability of experimental findings depends on the rigour of experimental design. Here we show limited reporting of measures to reduce the risk of bias in a random sample of life sciences publications, significantly lower reporting of randomisation in work published in journals of high impact, and very limited reporting of measures to reduce the risk of bias in publications from leading United Kingdom institutions. Ascertainment of differences between institutions might serve both as a measure of research quality and as a tool for institutional efforts to improve research quality.

Prevalence and natural history of pain in adults with multiple sclerosis: Systematic review and meta-analysis

The prevalence, associations, and natural history of pain in multiple sclerosis (MS) are poorly understood. The objective of this work was to study the prevalence of pain syndromes in MS both cross‐sectionally, and longitudinally during the MS disease course. We systematically identified prospective studies detailing pain prevalence in definite MS. We used pooled prevalence estimates, explored heterogeneity using meta‐regression, and analysed prevalence during the disease course using both estimates at disease milestones and longitudinal studies. Twenty‐eight articles (7101 subjects) describing overall pain, or pain syndromes, met inclusion criteria. Pooled overall pain prevalence (17 studies, 5319 subjects) was 63% (95% confidence interval [CI] 55–70%). Marked heterogeneity in this estimate was not significantly explained by selected study design variables (use of outpatient sample, timeframe prior to study over which pain was assessed) or sample demographic variables (mean Expanded Disability Status Scale, mean disease duration, proportion of female sex, and proportion with progressive MS). We quantified prevalence of headache (43%; 95% CI 33–52%), neuropathic extremity pain (26%; 95% CI 7–53%), back pain (20%; 95% CI 13–28%), painful spasms (15%; 95% CI 8.5–23%), Lhermitte sign (16%; 95% CI 10–25%), and trigeminal neuralgia (3.8%; 95% CI 2–6%) in included studies. Prevalence of pain at MS disease milestones (prior to onset, at onset, and at relapse) and during longitudinal follow‐up was poorly described. Pain is common in MS, as are specific pain syndromes. The clinical associations and natural history of pain in MS require clarification. Future study could be enhanced by standardised study design.

Animal models of bone cancer pain: Systematic review and meta-analyses

&NA; Systematic review identified 38 animal models of bone cancer pain. Reported methodological quality was low; improving this may enhance translation to the clinic. &NA; Pain can significantly decrease the quality of life of patients with advanced cancer. Current treatment strategies often provide inadequate analgesia and unacceptable side effects. Animal models of bone cancer pain are used in the development of novel pharmacological approaches. Here we conducted a systematic review and meta‐analysis of publications describing in vivo modelling of bone cancer pain in which behavioural, general health, macroscopic, histological, biochemical, or electrophysiological outcomes were reported and compared to appropriate controls. In all, 150 publications met our inclusion criteria, describing 38 different models of bone cancer pain. Reported methodological quality was low; only 31% of publications reported blinded assessment of outcome, and 11% reported random allocation to group. No publication reported a sample size calculation. Studies that reported measures to reduce bias reported smaller differences in behavioural outcomes between tumour‐bearing and control animals, and studies that presented a statement regarding a conflict of interest reported larger differences in behavioural outcomes. Larger differences in behavioural outcomes were reported in female animals, when cancer cells were injected into either the tibia or femur, and when MatLyLu prostate or Lewis Lung cancer cells were used. Mechanical‐evoked pain behaviours were most commonly reported; however, the largest difference was observed in spontaneous pain behaviours. In the spinal cord astrocyte activation and increased levels of Substance P receptor internalisation, c‐Fos, dynorphin, tumor necrosis factor‐&agr; and interleukin‐1&bgr; have been reported in bone cancer pain models, suggesting several potential therapeutic targets. However, the translational impact of animal models on clinical pain research could be enhanced by improving methodological quality.

Similar effect of autologous and allogeneic cell therapy for ischemic heart disease : Systematic review and meta-analysis of large animal studies

Rationale: In regenerative therapy for ischemic heart disease, use of both autologous and allogeneic stem cells has been investigated. Autologous cell can be applied without immunosuppression, but availability is restricted, and cells have been exposed to risk factors and aging. Allogeneic cell therapy enables preoperative production of potent cell lines and immediate availability of cell products, allowing off-the-shelf therapy. It is unknown which cell source is preferred with regard to improving cardiac function. Objective: We performed a meta-analysis of preclinical data of cell therapy for ischemic heart disease. Methods and Results: We conducted a systematic literature search to identify publications describing controlled preclinical trials of unmodified stem cell therapy in large animal models of myocardial ischemia. Data from 82 studies involving 1415 animals showed a significant improvement in mean left ventricular ejection fraction in treated compared with control animals (8.3%, 95% confidence interval, 7.1–9.5; P <0.001). Meta-regression revealed a similar difference in left ventricular ejection fraction in autologous (8.8%, 95% confidence interval, 7.3–10.3; n=981) and allogeneic (7.3%, 95% confidence interval, 4.4–10.2, n=331; P =0.3) cell therapies. Conclusions: Autologous and allogeneic cell therapy for ischemic heart disease show a similar improvement in left ventricular ejection fraction in large animal models of myocardial ischemia, compared with placebo. These results are important for the design of future clinical trials. # Novelty and Significance {#article-title-48}Rationale: In regenerative therapy for ischemic heart disease, use of both autologous and allogeneic stem cells has been investigated. Autologous cell can be applied without immunosuppression, but availability is restricted, and cells have been exposed to risk factors and aging. Allogeneic cell therapy enables preoperative production of potent cell lines and immediate availability of cell products, allowing off-the-shelf therapy. It is unknown which cell source is preferred with regard to improving cardiac function. Objective: We performed a meta-analysis of preclinical data of cell therapy for ischemic heart disease. Methods and Results: We conducted a systematic literature search to identify publications describing controlled preclinical trials of unmodified stem cell therapy in large animal models of myocardial ischemia. Data from 82 studies involving 1415 animals showed a significant improvement in mean left ventricular ejection fraction in treated compared with control animals (8.3%, 95% confidence interval, 7.1–9.5; P<0.001). Meta-regression revealed a similar difference in left ventricular ejection fraction in autologous (8.8%, 95% confidence interval, 7.3–10.3; n=981) and allogeneic (7.3%, 95% confidence interval, 4.4–10.2, n=331; P=0.3) cell therapies. Conclusions: Autologous and allogeneic cell therapy for ischemic heart disease show a similar improvement in left ventricular ejection fraction in large animal models of myocardial ischemia, compared with placebo. These results are important for the design of future clinical trials.

Dopamine agonists in animal models of Parkinson’s disease: A systematic review and meta-analysis ☆

BACKGROUND Parkinsons disease (PD) can be a severely disabling condition in spite of therapies currently available. Systematic review and meta-analysis can provide an overview of a field of research and identify potential sources of bias and limits to efficacy. In this study we use these tools to describe the reported efficacy of dopamine agonists in animal models of PD. METHODS Publications were identified by electronic searching of three online databases. Data were extracted for neurobehavioural outcome, for study design and for the reporting of measures to avoid bias. Standardised mean difference meta-analysis was used to provide summary estimates of efficacy, with the effects of study quality and study design explored using stratified meta-analysis. RESULTS 253 publications reported the use of a dopamine agonist in an animal model of PD; of these 121 reported data suitable for inclusion in meta-analysis. 47 interventions were tested in 601 experiments using 4181 animals. Overall, neurobehavioural outcome was improved by 1.08 standard deviations (SD; 95% Confidence Interval (CI) 0.97-1.19). Reporting of measures to reduce bias was low and publications which reported the blinded assessment of outcome had significantly smaller effect sizes (0.85, 95% CI 0.64 to 1.07) than those which did not (1.18, 95% CI 1.05 to 1.31, p < 0.005). CONCLUSIONS While dopamine agonists do appear to have efficacy in animal models of PD the low prevalence of reporting of measures to avoid bias is of concern. Systematic review of individual interventions may be helpful in the design of future preclinical and clinical trials.

Systematic review and meta-analysis of temozolomide in animal models of glioma: was clinical efficacy predicted?

Background:Malignant glioma is an aggressive tumour commonly associated with a dismal outcome despite optimal surgical and radio-chemotherapy. Since 2005 temozolomide has been established as first-line chemotherapy. We investigate the role of in vivo glioma models in predicting clinical efficacy.Methods:We searched three online databases to systematically identify publications testing temozolomide in animal models of glioma. Median survival and number of animals treated were extracted and quality was assessed using a 12-point scale; random effects meta-analysis was used to estimate efficacy. We analysed the impact of study design and quality and looked for evidence of publication bias.Results:We identified 60 publications using temozolomide in models of glioma, comprising 2443 animals. Temozolomide prolonged survival by a factor of 1.88 (95% CI 1.74–2.03) and reduced tumour volume by 50.4% (41.8–58.9) compared with untreated controls. Study design characteristics accounted for a significant proportion of between-study heterogeneity, and there was evidence of a significant publication bias.Conclusion:These data reflect those from clinical trials in that temozolomide improves survival and reduces tumour volume, even after accounting for publication bias. Experimental in vivo glioma studies of temozolomide differ from those of other glioma therapies in their consistent efficacy and successful translation into clinical medicine.

Systematic Survey of the Design, Statistical Analysis, and Reporting of Studies Published in the 2008 Volume of the Journal of Cerebral Blood Flow and Metabolism

Translating experimental findings into clinically effective therapies is one of the major bottlenecks of modern medicine. As this has been particularly true for cerebrovascular research, attention has turned to the quality and validity of experimental cerebrovascular studies. We set out to assess the study design, statistical analyses, and reporting of cerebrovascular research. We assessed all original articles published in the Journal of Cerebral Blood Flow and Metabolism during the year 2008 against a checklist designed to capture the key attributes relating to study design, statistical analyses, and reporting. A total of 156 original publications were included (animal, in vitro, human). Few studies reported a primary research hypothesis, statement of purpose, or measures to safeguard internal validity (such as randomization, blinding, exclusion or inclusion criteria). Many studies lacked sufficient information regarding methods and results to form a reasonable judgment about their validity. In nearly 20% of studies, statistical tests were either not appropriate or information to allow assessment of appropriateness was lacking. This study identifies a number of factors that should be addressed if the quality of research in basic and translational biomedicine is to be improved. We support the widespread implementation of the ARRIVE (Animal Research Reporting In Vivo Experiments) statement for the reporting of experimental studies in biomedicine, for improving training in proper study design and analysis, and that reviewers and editors adopt a more constructively critical approach in the assessment of manuscripts for publication.

Systematic review and stratified meta-analysis of the efficacy of RhoA and Rho kinase inhibitors in animal models of ischaemic stroke

BackgroundThere is currently only one clinically approved drug, tissue plasminogen activator (tPA), for the treatment of acute ischaemic stroke. The RhoA pathway, including RhoA and its downstream effector Rho kinase (ROCK), has been identified as a possible therapeutic target. Our aim was to assess the impact of study design characteristics and study quality on reported measures of efficacy and to assess for the presence and impact of publication bias.MethodsWe conducted a systematic review and meta-analysis on publications describing the efficacy of RhoA and ROCK inhibitors in animal models of focal cerebral ischaemia where outcome was assessed as a change in lesion size or neurobehavioural score, or both.ResultsWe identified 25 published papers which met our inclusion criteria. RhoA and ROCK inhibitors reduced lesion size by 37.3% in models of focal cerebral ischaemia (95% CI, 28.6% to 46.0%, 41 comparisons), and reduced neurobehavioural data by 40.5% (33.4% to 47.7%, 30 comparisons). Overall study quality was low (median=4, interquartile range 3–5) and measures to reduce bias were seldom reported. Publication bias was prevalent and associated with a substantial overstatement of efficacy for lesion size.ConclusionsRhoA and ROCK inhibitors appear to be effective in animal models of stroke. However the low quality score, publication bias and limited number of studies are areas which need attention prior to conducting clinical trials.