Marlies Leenaars

SYRCLE’s risk of bias tool for animal studies

BackgroundSystematic Reviews (SRs) of experimental animal studies are not yet common practice, but awareness of the merits of conducting such SRs is steadily increasing. As animal intervention studies differ from randomized clinical trials (RCT) in many aspects, the methodology for SRs of clinical trials needs to be adapted and optimized for animal intervention studies. The Cochrane Collaboration developed a Risk of Bias (RoB) tool to establish consistency and avoid discrepancies in assessing the methodological quality of RCTs. A similar initiative is warranted in the field of animal experimentation.MethodsWe provide an RoB tool for animal intervention studies (SYRCLE’s RoB tool). This tool is based on the Cochrane RoB tool and has been adjusted for aspects of bias that play a specific role in animal intervention studies. To enhance transparency and applicability, we formulated signalling questions to facilitate judgment.ResultsThe resulting RoB tool for animal studies contains 10 entries. These entries are related to selection bias, performance bias, detection bias, attrition bias, reporting bias and other biases. Half these items are in agreement with the items in the Cochrane RoB tool. Most of the variations between the two tools are due to differences in design between RCTs and animal studies. Shortcomings in, or unfamiliarity with, specific aspects of experimental design of animal studies compared to clinical studies also play a role.ConclusionsSYRCLE’s RoB tool is an adapted version of the Cochrane RoB tool. Widespread adoption and implementation of this tool will facilitate and improve critical appraisal of evidence from animal studies. This may subsequently enhance the efficiency of translating animal research into clinical practice and increase awareness of the necessity of improving the methodological quality of animal studies.

Enhancing search efficiency by means of a search filter for finding all studies on animal experimentation in PubMed

Collecting and analysing all available literature before starting an animal experiment is important and it is indispensable when writing a systematic review (SR) of animal research. Writing such review prevents unnecessary duplication of animal studies and thus unnecessary animal use (Reduction). One of the factors currently impeding the production of ‘high-quality’ SRs in laboratory animal science is the fact that searching for all available literature concerning animal experimentation is rather difficult. In order to diminish these difficulties, we developed a search filter for PubMed to detect all publications concerning animal studies. This filter was compared with the method most frequently used, the PubMed Limit: Animals, and validated further by performing two PubMed topic searches. Our filter performs much better than the PubMed limit: it retrieves, on average, 7% more records. Other important advantages of our filter are that it also finds the most recent records and that it is easy to use. All in all, by using our search filter in PubMed, all available literature concerning animal studies on a specific topic can easily be found and assessed, which will help in increasing the scientific quality and thereby the ethical validity of animal experiments.

A step-by-step guide to systematically identify all relevant animal studies.

Before starting a new animal experiment, thorough analysis of previously performed experiments is essential from a scientific as well as from an ethical point of view. The method that is most suitable to carry out such a thorough analysis of the literature is a systematic review (SR). An essential first step in an SR is to search and find all potentially relevant studies. It is important to include all available evidence in an SR to minimize bias and reduce hampered interpretation of experimental outcomes. Despite the recent development of search filters to find animal studies in PubMed and EMBASE, searching for all available animal studies remains a challenge. Available guidelines from the clinical field cannot be copied directly to the situation within animal research, and although there are plenty of books and courses on searching the literature, there is no compact guide available to search and find relevant animal studies. Therefore, in order to facilitate a structured, thorough and transparent search for animal studies (in both preclinical and fundamental science), an easy-to-use, step-by-step guide was prepared and optimized using feedback from scientists in the field of animal experimentation. The step-by-step guide will assist scientists in performing a comprehensive literature search and, consequently, improve the scientific quality of the resulting review and prevent unnecessary animal use in the future.

Publication Bias in Laboratory Animal Research: A Survey on Magnitude, Drivers, Consequences and Potential Solutions

Context Publication bias jeopardizes evidence-based medicine, mainly through biased literature syntheses. Publication bias may also affect laboratory animal research, but evidence is scarce. Objectives To assess the opinion of laboratory animal researchers on the magnitude, drivers, consequences and potential solutions for publication bias. And to explore the impact of size of the animals used, seniority of the respondent, working in a for-profit organization and type of research (fundamental, pre-clinical, or both) on those opinions. Design Internet-based survey. Setting All animal laboratories in The Netherlands. Participants Laboratory animal researchers. Main Outcome Measure(s) Median (interquartile ranges) strengths of beliefs on 5 and 10-point scales (1: totally unimportant to 5 or 10: extremely important). Results Overall, 454 researchers participated. They considered publication bias a problem in animal research (7 (5 to 8)) and thought that about 50% (32–70) of animal experiments are published. Employees (n = 21) of for-profit organizations estimated that 10% (5 to 50) are published. Lack of statistical significance (4 (4 to 5)), technical problems (4 (3 to 4)), supervisors (4 (3 to 5)) and peer reviewers (4 (3 to 5)) were considered important reasons for non-publication (all on 5-point scales). Respondents thought that mandatory publication of study protocols and results, or the reasons why no results were obtained, may increase scientific progress but expected increased bureaucracy. These opinions did not depend on size of the animal used, seniority of the respondent or type of research. Conclusions Non-publication of “negative” results appears to be prevalent in laboratory animal research. If statistical significance is indeed a main driver of publication, the collective literature on animal experimentation will be biased. This will impede the performance of valid literature syntheses. Effective, yet efficient systems should be explored to counteract selective reporting of laboratory animal research.

A search filter for increasing the retrieval of animal studies in Embase.

Collecting and analysing all available literature before starting a new animal experiment is important and it is indispensable when writing systematic reviews of animal research. In practice, finding all animal studies relevant to a specific research question turns out to be anything but simple. In order to facilitate this search process, we previously developed a search filter for retrieving animal studies in the most often used biomedical database, PubMed. It is a general requirement for systematic reviews, however, that at least two databases are searched. In this report, we therefore present a similar search filter for a second important database, namely Embase. We show that our filter retrieves more animal studies than (a combination of) the options currently available in Embase. Our search filters for PubMed and Embase therefore represent valuable tools for improving the quality of (systematic) reviews and thereby of new animal experiments.

The Usefulness of Systematic Reviews of Animal Experiments for the Design of Preclinical and Clinical Studies

The question of how animal studies should be designed, conducted, and analyzed remains underexposed in societal debates on animal experimentation. This is not only a scientific but also a moral question. After all, if animal experiments are not appropriately designed, conducted, and analyzed, the results produced are unlikely to be reliable and the animals have in effect been wasted. In this article, we focus on one particular method to address this moral question, namely systematic reviews of previously performed animal experiments. We discuss how the design, conduct, and analysis of future (animal and human) experiments may be optimized through such systematic reviews. In particular, we illustrate how these reviews can help improve the methodological quality of animal experiments, make the choice of an animal model and the translation of animal data to the clinic more evidence-based, and implement the 3Rs. Moreover, we discuss which measures are being taken and which need to be taken in the future to ensure that systematic reviews will actually contribute to optimizing experimental design and thereby to meeting a necessary condition for making the use of animals in these experiments justified.

Improving planning, design, reporting and scientific quality of animal experiments by using the Gold Standard Publication Checklist, in addition to the ARRIVE guidelines

Several studies have demonstrated serious omissions in the way research that use animals is reported. In order to improve the quality of reporting of animal experiments, the Animals in research: reporting in vivo experiments (ARRIVE) Guidelines were published in the British Journal of Pharmacology in August 2010.

Immune responses and side effects of five different oil-based adjuvants in mice

In this study, five different oil based adjuvants were compared to assess efficacy and side effects. Mice were injected subcutaneously (s.c.) or intraperitoneally (i.p.) with a weak immunogen (synthetic peptide) emulsified in Freunds adjuvant (FA), Specol, RIBI, TiterMax or Montanide ISA50. Efficacy of adjuvants was evaluated based on their properties to induce peptide specific IgG1, IgG2a and total IgG antibodies, native protein cross-reactive antibodies and cytokine production. Side effects were evaluated based on clinical and behavioural abnormalities, and (histo)pathological changes. Although marked differences in isotype profile and height of titre are observed among the different adjuvants used, we found that FA, Montanide ISA50 and Specol worked equally well in the s.c. and i.p. route, TiterMax functioned only when given i.p. and RIBI also did not perform up to par. The number of cytokine (interferon-gamma and interleukin-4) producing spleen cells was significantly higher after injection of RIBI compared with other adjuvants. Injection of FA or TiterMax resulted in severe pathological changes while after RIBI injection minimal changes were observed. In conclusion, high peptide specific antibody levels with limited side effects can be obtained by s.c. injection of peptide combined with Montanide ISA50 or Specol as alternatives to FA.

Updated version of the Embase search filter for animal studies.

In a short report published in Laboratory Animals in October 2011, we presented a search filter for the maximum retrieval of animal studies from the bibliographic database Embase. This search filter greatly facilitates the search for animal experiments in this large biomedical database, just as a similar search filter does for PubMed. Collecting and analysing all relevant animal research data are important before starting a new animal experiment and are indispensable when writing systematic reviews of animal research. In this letter to the editor, an updated version of the Embase search filter is presented. The former version of the filter does not work adequately anymore, because of alterations in ‘Emtree’, the Elsevier life science thesaurus. A thesaurus is a hierarchically structured, controlled vocabulary for efficiently searching the database in question. The functioning of the search filter critically depends on the way in which the thesaurus terms are related to one another. As of January 2013, the Emtree term human/ is included in the Emtree term Chordata/, which in itself is part of the Emtree term animal/ (http://www.elsevier. com/online-tools/embase/emtree). Previously, the Emtree terms human/ and animal/ were in separate branches of the tree. As a consequence of the altered relation between human/ and animal/, the former version of the filter started to include all studies that used humans (and no other animals) as subjects, whereas the filter was intended to exclude exactly this type of studies. We have adapted the ‘mode’ in which the Emtree terms animal/ and Chordata/ are present in the search filter, so that the filter no longer automatically includes the purely human studies. The total number of records the updated search filter retrieves on its own (5,884,276) and in combination with a search strategy specific for the topic pancreatitis and probiotics (83) are similar to those in the original version of the filter (search date 3 May 2013). The new version of the search filter (version 2; 31 January 2013) can be found in Supplement 1 (see http://lan.sagepub.com/content/48/ 1/88/suppl/DC1). The Embase thesaurus is updated three times a year. It is likely that some of these updates will affect the functioning of our search filter. For that reason, we will check each time the thesaurus is updated to see whether our search filter still works properly. If not, we will make the necessary changes to the filter and replace the version now presented in Supplement 1 (see above link). In this way, the most updated version of the search filter will always be available online and it will remain easy to identify all the animal studies on a particular topic in Embase.

Systematic reviews of animal studies; missing link in translational research?

Background The methodological quality of animal studies is an important factor hampering the translation of results from animal studies to a clinical setting. Systematic reviews of animal studies may provide a suitable method to assess and thereby improve their methodological quality. Objectives The aims of this study were: 1) to evaluate the risk of bias assessment in animal-based systematic reviews, and 2) to study the internal validity of the primary animal studies included in these systematic reviews. Data Sources We systematically searched Pubmed and Embase for SRs of preclinical animal studies published between 2005 and 2012. Results A total of 91 systematic reviews met our inclusion criteria. The risk of bias was assessed in 48 (52.7%) of these 91 systematic reviews. Thirty-three (36.3%) SRs provided sufficient information to evaluate the internal validity of the included studies. Of the evaluated primary studies, 24.6% was randomized, 14.6% reported blinding of the investigator/caretaker, 23.9% blinded the outcome assessment, and 23.1% reported drop-outs. Conclusions To improve the translation of animal data to clinical practice, systematic reviews of animal studies are worthwhile, but the internal validity of primary animal studies needs to be improved. Furthermore, risk of bias should be assessed by systematic reviews of animal studies to provide insight into the reliability of the available evidence.