As the Center for Truth in Science embarks on its initial round of meta-research projects, I am frequently asked if it is possible for science to determine “absolute truths.” The Covid-19 pandemic has reminded us that scientific truths can evolve over time—what we thought was true about the virus in February changed drastically over the following five months as new data was gathered and analyzed.
The bedrock of scientific truth is simple: Get the evidence right. The results of studies that are not based on accurate measurements, sound procedures, and robust protocols cannot be called scientific. As Michela Massimi, professor of philosophy of science at the University of Edinburgh, wrote recently, “No perspective worthy of being called ‘scientific’ survives fudging the evidence, massaging or altering data, or discarding evidence.”
Unfortunately, such errors in scientific research are more common than we would like to believe.
In 2005, Dr. John Ioannidis, the father of meta-research, published a paper demonstrating that most published research findings in the medical field were false. Contributing factors for the erroneous results included the use of small study sizes, a reliance on a small number of studies, a willingness to publish studies reporting small effects, the prevalence of “fishing expeditions” to generate new hypotheses or explore unlikely correlations, intellectual prejudices and conflicts of interest, and competition among researchers to produce positive results.
More recently, an April 2018 report published by the National Association of Scholars, The Irreproducibility Crisis of Modern Science, highlighted an “epistemic problem, which is most visible in the large numbers of articles in reputable peer reviewed journals in the sciences that have turned out to be invalid or highly questionable. Irreproducibility can stem from several causes, chief among them fraud and incompetence.”
This should frighten, or at least serve as cause of concern, for all of us. What are the consequences of incorrect, flawed, or fraudulent scientific research? The NAS report offers insight into the cost—both literally and figuratively—of fraudulent research, citing that in 2015, U.S. researchers spent nearly $28 billion on irreproducible preclinical research for drug treatments. The consequences are escalated by the number of potentially harmful decisions that citizens, lawmakers, and scientists make on the basis of such research, including faulty public policies in the areas of public health, climate science, and more. The greatest cost cannot be quantified: Each time scientists release incorrect research, the public loses trust in the research they produce, in expertise as a whole, and eventually, in science altogether.
Even in instances where “absolute truth” cannot yet be achieved in science due to rapidly emerging evidence and data, the public deserves to be aware of—and understand—the accuracy of results of scientific studies, particularly when these studies are the subjects of current or future litigation. Objective, independent review of such studies is critical for maintaining a healthy and balanced judicial system. That is precisely what the Center for Truth in Science seeks to accomplish. Through analysis of the methodologies and protocols that formed the basis of existing studies, we evaluate the accuracy of said studies and, most importantly, inform the public so that consumers can better sort fact from fiction and judges and juries can make unbiased, informed decisions.
The cost of fraudulent science in the judicial system is too high—and the risk of losing the public’s trust in expertise or science as a whole too great—to continue down this path. It is time to restore scientific integrity.