Journal of Veterinary Medicine and Health
Open Access

Animals; Reproducibility; Practical research; Discrepancies

Introduction

A key component of the scientific process is reproducibility, which sets it apart from anecdotal evidence. Reproducibility is important for the advancement of both basic and practical research, and it can be severely inhibited by poor reproducibility. Nevertheless, mounting data suggests that repeatability is subpar across several life sciences domains. Prinz and colleagues found significant discrepancies (65%) between published and in-house data in the fields of oncology, women’s health, and cardiovascular diseases. Oncologists from Amgen could only confirm 6 out of 53 published findings. Of more than 100 compounds that showed promising effects on amyotrophic lateral sclerosis, only 8 out of 18 studies could be replicated (ALS), None of the preclinical experiments’ results held true when they were repeated by the Cambridge-based ALS Therapy Development Institute. Poor reproducibility, however, comes with major ethical issues in addition to being a loss of time and resources for fruitless study. While in basic and preclinical animal research, it may result in unjustified injury to experimental animals, irreproducibility of preclinical research may expose patients to unwarranted risks in clinical research. The internal and external validity of experimental results are both accounted [1-5] for by the experimental design and conduct, which are significantly dependent on reproducibility. External validity is the extent to which findings are transferable to different environments, experimenters, study populations, and even different animal strains or species (including humans). As a result, it also establishes if the findings are repeatable among replication investigations (i.e., across various labs, experimenters, study populations, etc.). Internal validity relates to how much a causal relationship between an experimental treatment and outcome is justified, and it vitally depends on scientific rigour, or how much systematic bias is minimised in the experimental design and conduct. Poor internal validity resulting from a lack of scientific rigour has been proposed as another important factor in the low repeatability of animal studies. There are many different types of bias (such as selection bias, performance bias, and detection bias), and there are specific ways to reduce them (such as randomization, blinding, and sample-size calculation). Publications must include adequate material on experimental design and conduct, including steps taken against bias risks, to enable replication of findings and to assess the internal validity of studies, for example, in the peer review process. Systematic evaluations, however, often discovered a low prevalence of reporting of safeguards against bias hazards (sometimes referred to as reporting) in papers including animal [6-8] research. As a result, reporting for allocation concealment ranged from 8% to 55.6%, for blinded outcome assessment from 3% to 61%, for randomization from 7% to 55%, and for sample size calculation from 0% to 3%. Low reporting rates have been used as justification for a lack of scientific rigour. In fact, a number of systematic evaluations discovered links between inaccurate reporting and exaggerated treatment effects. Thus, reporting requirements have developed into a key tool in the battle against the possibility of bias in animal research. Although more than 1,000 journals endorsed the ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments) developed by the UK-based NC3Rs (National Centre for the Replacement, Refinement & Reduction of Animals in Research), this did not result in a significant improvement of reporting in animal studies. Although there is still much room for improvement, Macleod and colleagues have discovered that reporting has increased over the past decades, suggesting that awareness is indeed rising. The majority of the research on the internal validity of animal studies has been centred on reporting in academic journals. However, the majority of published research has undergone peer review when funding applications are made, and in some nations (such as Switzerland and Germany), specific animal experiments are authorised by local, state, or federal authorities. The approval of animal experiments, for instance, is based on a clear harm-benefit analysis in Switzerland, where any harm done to the animals is weighed against the anticipated benefit (knowledge gained) of the experiment. Risks of bias may influence the weight given to the predicted benefit of a study in the harm-benefit analysis because the advancement of knowledge is significantly dependent on the scientific validity of the findings. A thorough harm-benefit analysis therefore requires knowledge of bias hazards and the countermeasures employed to counteract them. In the current study, we compared the rates at which these measures were described in applications (for purposes of simplicity hereafter also referred to as reporting) with the rates of reporting of the same measures in a representative sub-sample of publications (n = 50) resulting from animal experiments. We did this by screening applications for animal experiments submitted to the cantonal authorities in Switzerland (n = 1,277) for evidence of the use of measures to avoid risks of bias. This allowed us to compare the evidence presented in scholarly journals with the evidence of scientific rigour provided to the authorities when approving animal research for the first time, and to determine whether inadequate reporting in experiment applications predicts inadequate reporting in the scientific literature.

Materials and Method

Sampling process

An anonymized database of all applications submitted in Switzerland since 1983 was obtained from the FSVO and contained applications for animal research (Form A, S1 Text). A contract between the FSVO and the authors of this study, which guaranteed secrecy to the applicants, was the basis for access to applications stored by the FSVO. Based on previously established inclusion and exclusion criteria, applications were chosen. Thus, only new applications submitted in the years 2008, 2010, and 2012 were considered, and applications relating to disease diagnosis, education and training, and the protection of people, animals, and the environment through toxicological or other safety tests mandated by law were automatically disregarded. These requirements were completed by 1590 applications, which underwent a rigorous screening process.

Checklist

A checklist was developed (S2 Text) based on checklists used in earlier studies examining the use of methods to reduce risks of biases as reported in the published literature in order to assess risks of bias in the experiments mentioned in the applications. We limited the items on our checklist to those that are generally relevant to all types of experimental studies and can be evaluated objectively without specialised knowledge of the research topic. We also included the seven items that we found in the [9] literature the most frequently: allocation concealment, blinded outcome assessment, randomization, formal sample size calculation, inclusion and exclusion criteria, a primary outcome variable, and a statistical analysis plan.

Discussion

This study examined whether poor reporting in applications for animal experiments, that is, before the studies have actually been conducted, is a predictor of poor reporting in the scientific literature. This was done in light of the low reporting rates in publications of animal research and evidence suggesting that poor reporting may reflect a lack of scientific rigour. For two reasons, the study was limited to animal studies that were permitted in Switzerland. First, Switzerland has a mechanism for authorising animal experiments that calls for a thorough explanation of the study methods for each one that is anticipated. These study procedures serve as the foundation for the harm-benefit analysis that determines whether or not to approve specific studies. Second, The Swiss Federal Food Safety and Veterinary Office (FSVO) provided access to all applications for animal studies via their online platform (e-tierversuche), which allows scientists to submit their applications for animal experiments and engage with the authorities. It is commendable that the FSVO approved this metaresearch given the unprecedented open access to application forms for animal studies. According to Chan, obtaining this kind of support for clinical trial protocols for meta-research has historically been challenging for reasons of secrecy. Access to the application forms was accessible without breaching confidentiality, as stated in the Materials and Methods.

Results

A final sample of 1,277 applications for animal experiments that were accepted by Swiss cantonal authorities in the years 2008, 2010, and 2012 were included in our database. A statistical analysis strategy, inclusion and exclusion criteria, allocation concealment, blinded outcome assessment, sample size calculation, inclusion and exclusion criteria, primary outcome, and blinded outcome assessment were used to evaluate the scientific rigour of the study. The internal validity score, which was the main outcome variable for the statistical analysis of the impacts of different study descriptors on reporting rates, was produced in addition to individually examining each item.

Acknowledgments

For assistance and support in gaining access to the applications for animal trials, the authors gratefully acknowledge Heinrich Binder, Sven Süptitz, and Michel Lehmann from the Swiss Food Safety and Veterinary Office (FSVO). The authorities from the veterinary offices of the cantons of Aargau, Luzern, Freiburg, and Basel Stadt as well as the authorities of the veterinary office of the canton of Zürich as well as access to their archives are also gratefully acknowledged. We would especially like to thank Emily S. Sena for her insightful comments on the evaluation of the study protocol and the scoring of measures against bias, as well as Beatriz Vidondo for her assistance with the statistics.

1. Cloquell A,Mateo I (2019)Surgical management of a brain abscess due to plant foreign body in a dog.Open Vet J 9:216–21.

Indexed at, Crossref, Google Scholar

2. Cottam EJ,Gannon K (2015)Migration of a sewing needle foreign body into the brainstem of a cat.JFMS Open Rep1:1-10.

Indexed at, Crossref, Google Scholar

3. Hao D,Yang Z,Li FA (2017) 61 year old man with intracranial sewing needle.J Neurol Neurophysiol8:1-10.

Google Scholar

4. Fischer BR,Yasin Y,Holling M,Hesselmann V (2012) Good clinical practice in dubious head trauma - the problem of retained intracranial foreign bodies.Int J Gen Med 5:899–902.

Indexed at, Crossref, Google Scholar

5. Maghsoudi M,Shahbazzadegan B,Pezeshki A (2016)Asymptomatic intracranial foreign body: an incidental finding on radiography.Trauma Mon21:22206.

Indexed at, Google Scholar

6. Pelin Z,Kaner T (2012)Intracranial metallic foreign bodies in a man with a headache.Neurol Int.4:18.

Indexed at, Crossref, Google Scholar

7. Sturiale CL,Massimi L,Mangiola A,Pompucci A,Roselli R,et al. (2010)Sewing needles in the brain: infanticide attempts or accidental insertion?Neurosurgery67:E1170–9.

Indexed at, Crossref, Google Scholar

8. Abbassioun K,Ameli NO,Morshed AA (1979)Intracranial sewing needles: review of 13 cases.J Neurol Neurosurg Psychiatry42:1046–9.

Indexed at, Crossref, Google Scholar

9. Bozkurt H,Arac DA (2019) Late onset adult seizure due to intracerebral needle: case-based update.Childs Nerv Syst35:593–600.

Crossref, Google Scholar