No one’s even going to notice, I figured. This isn’t even a blip.
Then it happened.
There was an email. Then a client asked me about it. Then a post appeared on my Facebook page. And then the floodgates opened.
“Meat and cheese. It’s as bad as smoking?”
For years I’ve told my clients not to avoid saturated fats, red meat, cheese, eggs, or salt. I’ve told most of them to get in more protein. I’ve told them that the negative impressions of these foods come from poorly conducted research and poorly conducted journalism.
The emails started coming in.
My mom called.
Meat and Cheese as Bad as Smoking is the headline on dozens of publications. What’s going on here?
A journal, Cell Metabolism, published the following study: Low Protein Intake Is Associated with a Major Reduction in IGF-1, Cancer, and Overall Mortality in the 65 and Younger but Not Older Population.
Not an overwhelming title, but then this:
“We provide convincing evidence that a high-protein diet — particularly if the proteins are derived from animals — is nearly as bad as smoking for your health.”
-Dr. Valter Longo, of the University of Southern California
So, I did what any responsible person would do and read the study. I also read what the news said and what prominent rebuttals to the conclusions of the paper said.
And here’s my conclusion:
Ummm, I’m pretty sure most people are not reading the actual paper. I’m shocked by how far off the mark much of the journalism and rebuttals are.
That said, here are the reasons why using this paper to recommend a reduction in animal proteins is irresponsible stupid.
Here’s a quick rundown on what the paper purports to show:
- High protein intake from animal sources increases the risk of overall mortality (death from any cause, from cancer to space aliens) by a whopping 75% and increases cancer death risk 4-fold. 4-fold!
- These changes are seen only between the ages of 50 and 65, at which time high protein is protective and low protein is harmful.
- Regardless of age, high protein intake is associated with a 5-fold increase in diabetes mortality.
- Insulin-like Growth Factor-1 (IGF-1) causes cancer cells to grow, and, after injecting mice with cancer cells, higher IGF-1 causes greater rates of growth. (This, by the way, is the only redeeming quality of the paper, and is also not at all shocking. IGF-1 makes everything from bones to muscles to cancer cells grow.)
And here are the issues:
1) Questionable data collection:
The data on food intake was obtained by asking people what they thought they ate for the previous 24 hours. Then they followed up with the participants 18 years later and decided that the meat they said they ate 18 years and 1 day ago was the determining factor in their demise (or lack there of). Sure. That seems like….. science?
Anyone who has EVER worked with anyone on nutrition, or even logged his or her own food intake, knows that a 24-hour recall survey is not a terribly accurate depiction of actual food consumption. An individual’s ability to accurately guess at and recall actual portion sizes is, shall we say, highly variable.
The authors of the study indicate that the methods used are better than other kinds of food surveys. Which is like saying that trying to blow up a balloon from 2 feet away is better than trying to blow it up from 10 feet away. True, but still pretty meaningless.
The data they used was from the National Health and Nutrition Examination Survey (NHANES) III, which collected these food intake surveys between 1988 and 1994. There is no indication as to why this particular NHANES was used instead of a more recent one, or why 18 years was used as a follow up period instead of, say, 15. This is not necessarily cause for concern since this kind of data collection for this kind of study is pretty shoddy in the first place, but it does raise the question of why this particular sample population was chosen.
The NHANES III collected food intake surveys from approximately 40,000 people. This study whittled that initial sample down to a puny 6,281 people. By way of comparison, a similar type of study conducted in Europe used a sample of nearly 500,000 people. Needless to say, it came to very different conclusions.
Strangely, the authors, after noting that the small sample size is a weakness of the study, they then use it as a defense of the quality of their work, saying that:
“…one would expect a small sample size to decrease statistical power and make it harder to detect associations. Therefore, our ability to detect significance indicates that the associations between protein and mortality are robust.”
Actually, one would expect a small sample size to not be representative of the population at large. Beth and I both read this paper. Because of our small sample size we have a high statistical significance of this paper causing eye rolling.
2) More sample size problems:
Of the paltry 6,831 person sample size, only 437 were in the low protein group. Of those 437, only 363 said that their already questionable food intake surveys were indicative of their typical consumption. 363.
We’ve gone from paltry to pathetic.
2) No differentiation between kinds of food intake:
Did the high protein group eat different amounts of fruits and veggies than the low protein group? What were the sources of animal protein? Pork rinds? Salami? Filet mignon? Rib eyes? Salmon?
No indication.
How about fat sources? Was it mostly margarine? EFAs? Vegetable oils? Animal fats?
No indication.
Only the recollection of average overall macronutrient intake in a given 24 hour period was considered. Micronutrient intake information was available to the researchers, but not considered.
3) Lack of control for known risk factors:
In defense of the research, smoking was accounted for (so don’t believe it when people say it wasn’t). However…
We know there is an association between processed meat intake and cancer risk. As noted above, this was not controlled for. We also know there are other risk factors for overall mortality and cancer, like alcohol consumption. Again, not controlled for, even though that data was available to the researchers, and even though we know that those who eat more meat tend to drink more alcohol.
The average waist circumference in the data collected was over 38”. This was the average amongst both men and women. And it was only the mean, so, in essence, for every svelte participant with a 28” waist there was someone with a 48” waist. These individual data points do not appear to have been taken into account. Obesity is also a risk factor for overall mortality and cancer, and even though the researchers could easily have calculated BMI they did not.
Only controlling for a tiny subset of known risk factors and then saying that eating meat and cheese is like smoking a pack of cigarettes makes me question someone’s bias, agenda, or reasoning skills.
4) Other anomalies:
The high protein group reported eating approximately 20% fewer calories than the low protein group. Is there a growth hormone modulated IGF-1 response to fewer calories in this population? This was not considered.
According to the researchers, the 66+ year old group did better on high protein. Why the sudden change? Could it be that those who made it past 65 did so because their animal protein intake was from better sources? Maybe. Could it be something else entirely? Sure. Does that data raise questions about the researchers’ conclusions? You bet.
5) Statistics vs. Data:
Here’s my favorite part. If you look at the raw data, compared to the low protein group, the high protein group had:
- A lower risk of dying from cardiovascular disease
- The same overall mortality, not a 75% increase
- A lower risk of dying from cancer, not a 4-fold increase
The raw data actually shows the opposite of what the researchers conclude, with the exception of diabetes mortality.
And what about that 5-fold increase in diabetes mortality risk in the high protein group? This stems from stats built off having only 1 death in the low-protein group. Yup, 1 of the 437 people in the low protein group died of diabetes and THAT was used to extrapolate risk to the other groups.
Running a statistical model on a sample this small, with data this questionable, and then pretending that those stats mean anything is dumb at best. At best.
6) Questionable affiliation:
In addition to being a researcher on this study, Dr. Valter Longo is the founder of L-Nutra, a company that sells vegetable protein based products to cancer patients. This in and of itself is not necessarily cause for concern or conspiracy theories.
However, that paired with the fact that Valter Longo made the irresponsible (and factually incorrect, even if his flawed data were accurate) statement that meat and cheese is the equivalent of smoking is pretty damning.
If not for spurious conclusions and inflammatory public statements no one would care. Because of the combination, Longo’s statements and research should be looked at with caution.
Can we put this to bed now?
This study is garbage. Anyone using it to sell you a headline, a supplement, or anything else is, at best, irresponsible. At best.
One comment
Comments are closed.