Of all the choices we make in the grocery store (Whole milk or skim? What brand of cereal? Can I afford this fish?) we rarely ask: Is the product in this box what it claims to be? Largely, because we don’t have to. 

Thanks to federal agencies such as the Food and Drug Administration (FDA), American consumers shop with reasonable confidence that our food, cosmetics, and medicines are safe as advertised. Similarly, consumers of information read stories in established news outlets with reasonable confidence that a team of editors and fact-checkers are safeguarding the quality of the content. 

These are some of the processes that we’ve created to feel safe—and be safe—in society. And they work. We feel so safe sometimes that we forget there are exceptions to these processes. 

Dietary supplements, for example, fall outside the purview of the FDA. And it’s becoming harder in the age of the internet and social media to discern between good sources of information and biased ones. Perhaps it’s time for people to channel their inner scientist, and to get more skeptical about accepting claims at face value.

This is a lesson some consumers are learning the hard way, after a new study published in JAMA found that 22 out of 25 of the melatonin gummy products they analyzed were mislabeled. 

According to The New York Times: 

[O]ne contained only 74 percent of the advertised amount of melatonin, while another had 347 percent of the labeled amount. Yet another product contained no detectable melatonin at all. 

Researchers tested gummies from only a single bottle of each product, so it’s possible that the amount of melatonin varied from batch to batch. But the findings point to a staggering discrepancy between the amount of melatonin consumers think they’re ingesting and how much they might actually take, said Dr. Pieter Cohen, an associate professor at Harvard Medical School and the lead author of the paper. ‘You are at the mercy of the dietary supplement industry,’ Dr. Cohen said.

This supports the findings of a 2017 study in the Journal of Clinical Sleep Medicine which found that more than 71% of the 30 types of melatonin supplements analyzed failed to have a dosage within 10% of what the label claimed, with some containing up to 478% more than the amount listed on the label. Twenty-six percent contained serotonin, which requires a prescription. 

Melatonin is a hormone that is produced naturally in the body to regulate circadian rhythm, and people have started taking it as a supplement to convince their body it’s time to sleep. However, these products contain amounts far higher than what the human body makes. Three milligrams of melatonin, which falls in the middle range of melatonin taken by adults, is about 1,000-times more than what the body makes naturally. 

High levels of melatonin are linked with some side effects and questions remain surrounding the safety of its long-term use, as well as its efficacy as a sleep aid, and jetlag or anxiety reducer. Despite the efforts of wellness websites and social media influencers to normalize the use of melatonin as perfectly safe, even in children, more research is needed. 

The melatonin scandal gained press coverage on the heels of a report from a new longitudinal study that pushes back against the popular keto diet trend. Following a keto diet requires cutting carbs and eating at least 75% of calories from fats. Keto enthusiasts report weight loss and other health benefits but like most diets, the devil is in the details. 

The results were presented at the American College of Cardiology’s 2023 Scientific Session, and stated that the keto diet can increase levels of LDL cholesterol, leading to higher risk of heart events such as heart attacks, strokes, and blocked arteries. 

According to CNN: 

‘After an average of 11.8 years of follow-up–and after adjustment for other risk factors for heart disease, such as diabetes, high blood pressure, obesity and smoking–people on an LCHF diet had more than two-times higher risk of having several major cardiovascular events, such as blockages in the arteries that needed to be opened with stenting procedures, heart attack, stroke and peripheral arterial disease,’ researchers found, according to the news release.

Since the study was observational and relied on self-report, it can indicate an association, but not conclude a causal relationship. It also had a small sample size. More research will be needed.

Notably, the findings were echoed shortly thereafter by an American Heart Association report in April 2023, published in the journal Circulation, which ranked popular diets on heart health. The report noted that low and no-carb diets can encourage high intakes of saturated fat, which may lead to increases in LDL cholesterol levels.

According to the Washington Post: 

The authors said one of the purposes of their report was to counter widespread misinformation about nutrition promoted by diet books, blogs and people on TikTok, Instagram and Twitter — where posts promoting keto and paleo eating plans have surged in recent years. 

The amount of misinformation that has flourished on social media sites has reached ‘critical levels,’ said Christopher D. Gardner, the Director of Nutrition Studies at the Stanford Prevention Research Center and chair of the committee that wrote the report.

There are countless other examples of health claims made about artificial sweeteners, food dye, red meat, coffee, parabens, exercise, and more. These claims get attention, they become trendy and popular—but being a popular idea doesn’t make it accurate.

Below are some questions consumers should be asking before accepting claims and trends at face value: 

• Who is making the claim? Has it been verified by a neutral authority? If it’s a study, was it peer-reviewed and published in a high-quality journal?
• If it’s a news article, does the headline accurately represent the facts of the story? Or is it clickbait? What is the story actually claiming? 
• Have you actually seen evidence that the claim is true? Or have you heard the claim so many times, and in so many places that you are assuming it to be true? 
• Who is the original source of the claim? Is it an advocacy group? A research team? Does the source stand to benefit from the claim being made? 
• Did the study state how exposure or dosage was determined? Were participants tested? Did it rely on self-report? (Testing is more reliable.)
• What was the duration of the study? Are the amounts and durations of the exposure what we would reasonably see in real life? Or are they extreme? 
• Was the study conducted on humans or animals? Does the sample represent the population being studied? Is the sample size large enough? If an animal study, was a human equivalent exposure adequately determined?
• If it was an experiment, were individuals randomly assigned to groups? Were they aware of what group they were assigned to? (If so, it could bias their experience.)
• Are the conclusions drawn appropriately reasonable, or did the investigator or reporter inflate their findings?
• Could any other variables have influenced the findings? Did the author/reporter account for these?
• Is there really enough evidence to prove the claim? Have any other researchers been able to replicate the results? Or is more research needed? 

Appropriately skeptical consumers look at the information available and consider, “how do we know this?” and upon learning the source, asking, “how do they know this?” But that is easier said than done.

Consumers are exhausted from too many choices and too much information. When we need to make several decisions in a row, we tend to make lazy choices. At least without some help.

We hope these questions serve as a helpful toolkit for skeptical consumers who want to make wise, science-based choices.