In August 2023, the Center for Truth in Science (Center) wrote about the importance of registering scientific protocols at the outset of work on a study (read here), utilizing publicly available sites such as the Center for Open Science and PROSPERO.
At that time, the Center announced it would require the registration of protocols for all research grant awardees on an appropriate site and the commitment to do so must be declared in funding applications. We believe this is especially important in the case of systematic reviews, which, until recently, have not been as common as other research methods, (i.e. clinical trials).
As we pointed out last year, a 2022 article in Critical Reviews in Toxicology outlines some of the problems found in many published systematic reviews, including a failure to follow good research practices such as pre-defined search strategies, optimal methods for determining and reporting eligibility criteria, and reporting of study quality using checklists. They also pointed out that an essential element in conducting a state of the science systematic review is registering or publishing the study protocol as soon as it is developed, as the work begins. This makes the field aware of the study and allows for comments from outside experts to help reduce bias and confounding factors. There is also value in the field knowing the study is coming soon, which has the potential to reduce the duplication of efforts and hold the authors accountable to finish the analysis and attempt to get published.
It also adds further elements of transparency and strength:
- Accountability comes from the expectation that if there are any changes as the study evolves, they will be added to the publicly registered protocol.
- Making the study protocol public allows for the dissemination of current ideas about study design and data analysis on a research question or area.
- Publishing the protocol in a scientific journal, when possible, is an additional level of peer review important for improving the science. (We acknowledge this is often not feasible with real world need for evidence and funder timelines.)
Beyond the protocol, the Center also requires its funded research to meet additional levels of transparency regarding data and methods so that other scientists can replicate the reviews:
- A clear description of the methods of identifying, selecting, and validating information.
- How the searching and screening process would be accomplished.
- How quality assessments would be done, with which systems, and why.
- A plan for meta-analyses, if appropriate.
- A plan for publication of results in a peer-reviewed scientific journal, including the transparency necessary for others to run the same analyses, to determine replicability of results.
As we reported in June (read here), the Center awarded a research grant to Professor Anick Bérard PhD FCAHS FISPE (Canada Research Chair Tier 1 on Medications and Pregnancy, Faculty of Pharmacy, University of Montreal) and a strong team of co-investigators and statisticians she brought together at the Research Unit on Medications and Pregnancy, CHU Sainte-Justine, Montreal, Canada. The work will focus on a state-of-the-science systematic review of the literature on Acetaminophen Use During Pregnancy and the Risk of Neurodevelopment Disorders in Childhood.
We are excited to announce that as the work begins the protocol has been registered on both the Center for Open Science (https://osf.io/7rcqe) and PROSPERO (ID: 566964). The investigators look forward to seeing insightful comments from qualified scientific and clinical experts in the field, to support the best science possible on this important question.