Late night host Stephen Colbert coined the word “truthiness” to describe something that is believed to be true because it feels true, rather than because it is rooted in facts and evidence.

While Colbert was using the word primarily to lampoon politics and current events, it also applies to the human tendency to accept findings of scientific studies and their accounts reported by the media at face value without scrutinizing the ability of the particular research to produce the results reported.

A recent example of “scientific truthiness” can be found in an article in The New York Times with the headline: Walking Just 10 Minutes a Day May Lead to a Longer Life. The article shares that a new analysis discovered that just 10 minutes of moderate exercise daily could prevent more than 111,000 premature deaths each year.

Given the study was published by an epidemiologist from the National Cancer Institute in a decent peer-reviewed journal, JAMA Internal Medicine, it would be tempting to skip scrutiny and accept what The New York Times alluded to in its headline.

The claim feels true (although almost too good to be). We have been told, with good evidence, that exercise is good for us. We have been told also with good evidence that cardio workouts are good for heart health. And, surely, a little bit of exercise would be better than nothing, right?

Unfortunately, for many members of the public and the media, the questioning of study reports stops there.

However, as Peter Attia, M.D. a health and longevity specialist, points out on his blog, there are serious limitations in this research and the way it was reported. He writes, “The problem is that without critical understanding of the limitations of observational studies, the author of the news article didn’t recognize major flaws to the study.”

We would add, the author of the news article also did not know where to draw the line on what kind of conclusions can be drawn from the way the observational study was done.

Dr. Attia does an excellent job of describing the study’s three main drawbacks, which he calls flaws:

1. Only 7 Days of Data

The data was only collected for seven days. This is nowhere near a range of time acceptable to draw any meaningful conclusions about relationships between one variable and another.

Consider how many things can change over the course of a lifetime: diets, injuries, living environments, stressors, workout routines, jobs, sleep habits, sudden illness, etc. Seven days does not paint an accurate representation of a person’s life.

2. Healthy User Bias

It would be reasonable to assume the people who are more physically active are also more health conscious, and would participate in other healthy decisions across their lives that would contribute to a longer life span. Who is to say walking was the differentiating factor?

Dr. Attia points out the study authors factored many co-variates into their analysis, but it would be very difficult to capture them all. There are also the issues of age and illness, noting that older and sick individuals were more likely to be less active, but also more likely to die during the study period from things not related to activity level.

3. Correlation vs. Causation

While The New York Times article stated that walking would lead to fewer deaths, the study found only a correlation between higher moderate to vigorous physical activity (MVPA) and fewer deaths. To establish a direct cause-and-effect relationship between walking-equivalent physical activity and longevity, the researchers would have had to design a controlled trial in which participants were randomly assigned to an intervention or a control group. Ideally, the participants would not know if they were in the control or experimental group (and this would be difficult given the nature of this particular research question).

In this case, a clinical trial would have been nearly impossible, and possibly unethical to design and implement. Researchers instead observed participants and were not able to control their exposure to a variable (in this case, physical activity), and subsequently ran a statistical model based off of those observations, which were often estimates.

The Center agrees with Dr. Attia that while the findings may give us some valuable insights, they do not establish a causal relationship of any kind—there are too many estimates and confounding variables to be sure that one thing is the cause of the other.

Unfortunately, none of these limitations were stated clearly in The New York Times article.

Perhaps people could sneak their daily 10 minutes of movement in by making the leap from these inputs to the conclusions drawn?

Kidding aside, while there are several studies that indicate physical movement is good for health and can help increase longevity, this article is making a specific claim: 10 minutes of moderate exercise a day “…would, collectively prevent 111,000 deaths annually…”

Based on the limitations of the design and the data used in the analysis, that claim simply cannot be supported.

The Center for Truth in Science has always urged the public to hone their critical thinking skills and ask pertinent questions while reading media accounts of health studies. We also urge reporters to do a better job of understanding the studies they write about, and to take responsibility in accurately reporting reasonable conclusions. Save the “catchy” headlines for something less important to public health.