Should kids load up on candy during Halloween? One doctor says no. “It’s definitely important that you don’t eat too much,” Dr. Jason Ng, an endocrinologist and clinical assistant professor at the University of Pittsburgh School of Medicine, told TODAY. “Your body essentially is on a roller coaster, and that roller coaster effect really takes a lot out of people.”

However, one study found the opposite. Child consumption of massive amounts of candy on and after Halloween did not result in a significant  increase  of emergency room visits for hyperglycemia, just as the study’s investigator had initially hypothesized. This type of finding is called a “null” result and it has been, until recently, rarely published.

It is widely known that researchers and journal editors prefer studies that provide evidence of a significant relationship, such as finding that exposure to a chemical is harmful or that a new drug effectively treats a disease. Those kinds of results are exciting, perceived as  stronger, help establish the reputations of the scientists, and sell more journals. 

This preferential treatment has created a notable publication disparity between studies that find a preferred  positive relationship,  and those that find no association (or possibly even a helpful  relationship). Robert Rosenthal coined the term “file drawer problem” to describe this dilemma, writing, “journals are filled with the 5% of the studies that show Type I errors, while the file drawers are filled with the 95% of the studies that show nonsignificant results.”

When you combine mostly positive papers you can get increases in “Type 1” errors, otherwise known as  false positives.  For example, a summary of studies may falsely conclude that a chemical is dangerous. That false positive may find its way into courtrooms or regulations, where it is used to penalize and overregulate safe chemicals.

There is hope, however. The Center for Truth in Science is conducting systematic analyses of existing research on glyphosate, PFAS, ethylene oxide, and talc—four chemical and mineral compounds at the heart of the intersection among science, justice, and the economy—in an effort to bring increased clarity regarding existing research into the current regulatory and legal debates over their uses. 

In addition, the Center for Open Science, a nonprofit whose mission is to “increase openness, integrity and reproducibility of scientific research” has created a publishing format that requires pre-registration and peer review of research protocols before any data is collected, with journals that agree to publish the results even if they are null. More than 250 journals have adopted these practices.  

In this Registered Reports publishing format, researchers submit their hypothesis, proposed methodology, protocols, and analysis plans to a journal for peer review. What will likely deter  investigators from p-hacking (looking for any statistical positives in a data set) or HARKing (hypothesizing after the results are known) is that peer reviewers almost always suggest changes that lead to an increase  in credibility of the results.

This is  getting us closer to a balanced look at what is dangerous and what is not, but it is only a start. While it is good that 250 journals have signed up to Registered Reports, there are more than 20,000 science journals in total. We may not know the exact Halloween answer yet, but publishing more null results is inevitable and may help us get there sooner.