Last month we examined the problem of “replication” in scientific research and that, when reviewing the accuracy of the results of a study, scientists often find it impossible to replicate the results of the original researchers. In fact, the scientists who conducted the original studies are often unable to replicate their own results.

The consequences of such flawed science are enormous: faulty scientific experiments can lead to harmful decisions on public policy and personal health that undermine the public’s trust in science itself.

Center for Truth in Science board member David Randall has written extensively on this topic, including “The Irreproducibility Crisis of Modern Science,” a 2018 commentary published by the National Association of Scholars (NAS) and co-authored by Christopher Welser.

In the report, Randall and Welser found that a variety of common forms of improper scientific practice contribute to the reproducibility crisis. “Some researchers look for correlations until they find a spurious ‘statistically significant’ relationship,” they state. “Many more have a poor understanding of statistical methodology, and thus routinely employ statistics improperly in their research. Researchers may consciously or unconsciously bias their data to produce desired outcomes or combine data sets in such a way as to invalidate their conclusions. Uncontrolled researcher freedom makes it easy for researchers to err in all the ways described above.”

What else contributes to the crisis? First, a lack of transparency, with researchers rarely sharing data and methodology once completing their studies. Second, a professional culture that highlights positive results over negative results, even placing positive results over any attempts to reproduce earlier research, leading scientists to avoid replication studies and hide any with negative results.

Finally, they discussed groupthink and the inhibitions that come with checking results for accuracy. “… replication studies can undermine comfortable beliefs. An entire academic discipline can succumb to groupthink and create a professional consensus with a strong tendency to dismiss results that question its foundations,” they conclude.

As a result of the concerns raised by their findings, the NAS proposed 40 reforms that address all levels of the reproducibility crisis, ranging from statistical standards, data handling, research practices, university policies, professional journals, private philanthropy, government funding, government regulation, state and federal legislation, to judicial reforms.

The NAS report presents a thorough and easily digestible analysis of the reproducibility crisis and its impact on public policy debates, judicial rulings, and our personal decisions. Equally important, it presents common sense solutions to correcting these problems and restoring our faith in the findings of objective and unbiased scientific research.

The Center’s focus on issues at the intersection of science, justice, and the economy make this exceptionally relevant to our work and we are proud to highlight one of our Board members’ efforts to bring integrity and solutions to the table.