Normally I’d be continuing my ongoing series on the evolutionary history of the human brain. However, there is yet another red meat scare story making the rounds—and many readers have asked me to analyze it. Should we really be eating less red meat?
I don’t like to spend my time debunking specific studies—because as I said in a previous article about bad science, it’s like trying to hold back the ocean with a blue tarp and some rebar. However, I’ve wanted to write an article about the limitations and potential abuses of observational studies for some time, and “Red Meat Consumption And Mortality” is as good a starting point as any.
What Kind Of Study Is This, Anyway? Randomized Controlled Trials Vs. Observational Studies
The first and most important question we must ask is “What actual scientific data is this article based on?” It’s often tricky to find out, because most “news” articles don’t even mention the title of the original scientific paper, let alone link to it. (I usually start with a Pubmed search on the authors and narrow it down by journal.) In the overwhelming majority of cases, we’ll find that the data in question comes from what’s known as a “retrospective cohort study”.
In some cases, there isn’t any data: it’s just a lightly-camouflaged press release from a supplement peddler or drug manufacturer, designed to sell you pills, powders, or extracts. We’ll ignore those for now.
When most of us think of a scientific study, we’re thinking of a randomized controlled trial (RCT). The participants are divided randomly into two groups, matched as well as possible for age, sex, health history, smoking status, and any other factor that might affect the outcome. One group is given the treatment, the other is given no treatment.
In the more rigorous forms of RCT, the “no treatment” group is given a sham treatment (known as “placebo”) so that the subjects don’t know whether they’ve received treatment or not. This is sometimes called a “single-blinded” trial. Since the simple act of being attended to has a positive and significant effect on health (the “placebo effect”), unblinded trials (also known as “open label”) are usually not taken very seriously.
To add additional rigor, some trials are structured so that the clinicians administering the treatment don’t know who’s receiving the real treatment or not. This is sometimes called a “double-blinded” trial. And if the clinicians assessing outcomes don’t know who received the real treatment, it’s sometimes called “triple-blinded”. (These terms are now being discouraged in favor of simply calling a study “blinded” and specifying which groups have been blinded.)
Double-blinded, randomized controlled trials are the gold standard of research, because they’re the only type of trial that can prove the statement “X causes Y”. Unfortunately, RCTs are expensive—especially nutrition studies, which require feeding large groups over extended periods, and to be completely rigorous, isolating the subjects so they can’t consume foods that aren’t part of the experiment. (These are usually called “metabolic ward studies”.)
Result: RCTs are infrequently done, especially in the nutrition field.
What Is An Observational Study? Cohort Studies and Cross-Sectional Studies
Since nutrition RCTs are so rare, almost all nutrition headlines are based on observational studies.
In an observational study, the investigators don’t attempt to control the behavior of the subjects: they simply collect data about what the subjects are doing on their own. There are two main types of observational studies: cohort studies identify a specific group and track it over a period of time, whereas population studies measure characteristics of an entire population at one single point in time.
Cohort studies can be further divided into prospective cohort studies, in which the groups and study criteria are defined before the study begins, and retrospective cohort studies, in which existing data is “mined” after the fact for possible associations. (More.)
As the terminology starts getting intense (e.g. case-control studies vs. nested case-control studies), I’ll stop here.
The overwhelming majority of nutrition headlines are from cohort studies, in which health data has been collected for years (or decades) from a fixed group of people, often with no specific goal in mind. Expressed in the simplest possible language:
“Let’s watch the same group of people for decades, measure some things every once in a while, and see what happens to them. Then we can go back through the data and see if the people with a specific health issue had anything else in common.”
It’s easy to see that looking for statistical associations in data that already exists is far easier and cheaper than performing a randomized clinical trial. Unfortunately, there are several problems with observational studies. The first, and most damning, is that observational studies cannot prove that anything is the cause of anything else! They can only show an association between two or more factors—
—and that association may not mean what we think it means. In fact, it may not mean anything at all!
There are more potential pitfalls of the retrospective observational studies which underlie almost every nutrition headline. Let’s explore some of them.
Problem: Sampling Bias
Here’s the classic example of sampling bias:
Going into the 1948 presidential election, polls consistently predicted a Dewey victory, by a substantial margin of 5-15%. Of course, Harry S Truman won by 4.4%. The reason the poll results differed so much from the actual outcome was that the polling was done by telephone—and in 1948, private telephone lines were very expensive. Therefore, the pollsters were unwittingly selecting only the relatively wealthy—who tended to vote Republican—for their survey. (More: DEWEY DEFEATS TRUMAN and Cancer Statistics, J Natl Cancer Inst (2009) 101 (16): 1157.)
In other words, the entire group we’re studying may have inadvertently been selected for certain characteristics that skew our results, making them inapplicable to the population at large.
Selection Bias, or The Healthy Volunteer Problem
“Selection bias” occurs because, unlike an RCT in which the participants are randomly assigned to groups that are matched as well as possible, the people in an observational study choose their own behavior.
Most women will be familiar with the classic story of selection bias: the saga of hormone replacement therapy, or HRT.
1991: “Every woman should get on HRT immediately, because it prevents heart attacks!”
2002: “Every woman should get off HRT immediately, because it causes heart attacks!”
What happened?
Int. J. Epidemiol. (2004) 33 (3): 464-467.
The hormone replacement–coronary heart disease conundrum: is this the death of observational epidemiology?
Debbie A Lawlor, George Davey Smith and Shah Ebrahim“…the pooled estimate of effect from the best quality observational studies (internally controlled prospective and angiographic studies) inferred a relative reduction of 50% with ever [sic] use of HRT and stated that ‘overall, the bulk of the evidence strongly supports a protective effect of estrogens that is unlikely to be explained by confounding factors’.4
By contrast, recent randomized trials among both women with established CHD and healthy women have found HRT to be associated with slightly increased risk of CHD or null effects.1,2 For example, the large Women’s Health Initiative (WHI) randomized trial found that the hazards ratio for CHD associated with being allocated to combined HRT was 1.29 (95% CI: 1.02, 1.63), after 5.2 years of follow-up.1″
How did a 50% reduction in CHD (coronary heart disease) turn into a 30% increase in CHD?
It’s because the initial data from 1991 was from the Nurses’ Health Study, an associative cohort study which could only answer the question “What are the health characteristics of nurses who choose to undergo HRT versus nurses who don’t?” The followup data from 2002 was from a randomized clinical trial, which answered the much more relevant question “What happens to two matched groups of women when one undergoes HRT and the other doesn’t?”
It turns out that the effect of selection bias—women voluntarily choosing to be early adopters of a then-experimental procedure—completely overwhelmed the actual health effects of HRT. In other words, nurses who were willing to undergo cutting-edge medical treatment were far healthier than nurses who weren’t.
Is The Data Any Good? Garbage In = Garbage Out
This huge pitfall of observational studies is often neglected: in large cohort studies, data is often self-reported, and self-reported data is often wildly inaccurate.
Since we’re already discussing the Nurses’ Health Study, let’s take a closer look at its food consumption data. This study attempted to rigorously evaluate the accuracy of the FFQs (Food Frequency Questionaire) filled out by study participants:
Int J Epidemiol. 1989 Dec;18(4):858-67.
Food-based validation of a dietary questionnaire: the effects of week-to-week variation in food consumption.
Salvini S, Hunter DJ, Sampson L, Stampfer MJ, Colditz GA, Rosner B, Willett WC.“The reproducibility and validity of responses for 55 specific foods and beverages on a self-administered food frequency questionnaire were evaluated. One hundred and seventy three women from the Nurses’ Health Study completed the questionnaire twice approximately 12 months apart and also recorded their food consumption for seven consecutive days, four times during the one-year interval.”
In other words, the standard FFQ for the Nurses’ Health Study consists of “Try to remember what you ate last year, on average.” We might expect this not to be terribly accurate…
…and we’d be right.
“They found that the FFQ predicted true intake of some foods very well and true intake of other foods very poorly. True intake of coffee could explain 55 percent of the variation in answers on the FFQ, while true intake of beer could explain almost 70 percent. True intake of skim milk and butter both explained about 45 percent, while eggs followed closely behind at 41 percent.
But the ability of the FFQ to predict true intake of meats was horrible. It was only 19 percent for bacon, 14 percent for skinless chicken, 12 percent for fish and meat, 11 percent for processed meats, 5 percent for chicken with skin, 4 percent for hot dogs, and 1.4 percent for hamburgers.
If your jaw just dropped, let me assure you that you read that right and it is not a typo. The true intake of hamburgers explained only 1.4 percent of the variation in people’s claims on the FFQ about how often they ate hamburgers!”
–“Will Eating Meat Make Us Die Younger?”, Chris Masterjohn, March 27, 2009
Stop for a moment and wrap your mind around this fact: the intake of meat reported by the hundreds of studies which use data mined from the Nurses’ Health Study is almost completely unrelated to how much meat the study participants actually ate.
Here’s a graph of the ugly truth, again from Chris Masterjohn:
The left-hand bars are the first questionaire, which we'd expect to be closer to the reported data in the NHS than the second questionaire (right-hand bars).
Why might this be the case?
“Focusing on the second questionnaire, we found that butter, whole milk, eggs, processed meat, and cold breakfast cereal were underestimated by 10 to 30% on the questionnaire. In contrast, a number of fruits and vegetables, yoghurt and fish were overestimated by at least 50%. These findings for specific foods suggest that participants over-reported consumption of foods often considered desirable or healthy, such as fruit and vegetables, and underestimated foods considered less desirable.” –Salvini et.al., via Chris Masterjohn
In support, I note that reported intake of yellow squash and spinach was also correlated by less than 10% with actual intake. Additionally, I’ll point you towards this article, which begins with a startling statistic: 64% of self-reported ‘vegetarians’ in the USA ate meat on at least one of the two days on which their dietary intake was surveyed.
In other words, the observational studies that cite meat intake data from the Nurses’ Health Study are not telling you about the health of nurses who actually eat meat: they’re telling you about the health of nurses who are willing to admit to eating meat on a written questionaire—and the two are almost completely unrelated. Furthermore, I see no basis to claim that any other data set based on occasional self-reported dietary intake will be substantially more accurate.
Thanks again to Chris Masterjohn for his work: “Will Eating Meat Make Us Die Younger?” and the classic “New Study Shows that Lying About Your Hamburger Intake Prevents Disease and Death When You Eat a Low-Carb Diet High in Carbohydrates.”
“Correlation Does Not Imply Causation”: What Does That Mean?
The logical fallacy of “correlation proves causation” is extremely common—because it’s very easy to slide into.
It’s called “cum hoc ergo propter hoc” in Latin, if you want to impress people at the risk of being pedantic. Literally translated, it means “with this, therefore because of this.”
You can entertain yourself for hours with this long list of logical fallacies, both formal and informal.
In plain language, “correlation does not imply causation” means “Just because two things vary in a similar way over time doesn’t mean one is causing the other.” Since observational studies can only prove correlation, not causation, almost every nutrition article which claims “X Causes Y” is factually wrong. The only statements we can make from an observational study are “X Associated With Y” or “X Linked With Y”.
We’ve already covered the cases in which sampling bias and selection bias skew the results, and the cases in which the data is inaccurate: let’s look at the purely logical pitfalls.
First, we could be dealing with a case of reverse causation. (“I always see lots of firemen at fires: therefore, firemen cause fires and we should outlaw firemen.”)
Second, we could be dealing with a third factor. “Sleeping with one’s shoes on is strongly correlated with waking up with a headache. Therefore, sleeping with one’s shoes on causes headaches.” Obviously, in this case, being drunk causes both…but when we’re looking at huge associative data sets and trying to learn more about diseases we don’t understand, the truth isn’t so obvious.
The third factor is often one of the pitfalls we’ve previously discussed: sampling bias, selection bias, or inaccurate data. Another example of selection bias: “Playing basketball is strongly correlated with being tall. Therefore, everyone should play basketball so they grow taller.” (Hat tip to Tom Naughton for the analogy.)
Or, the relationship could be a spurious relationship—pure coincidence.
Personally, I blame M. Night Shyamalan.
Complete Bunk: It Happens
There is also the possibility that the truth is being stretched or broken…that the data is being misrepresented. This isn’t as common with peer-reviewed science as it is with books and popular media (see Denise Minger’s debunking of “The China Study” and “Forks Over Knives”), but it can and has occurred.
“Red Meat Blamed For 1 In 10 Early Deaths”: Where’s The Science?
Now that we understand the limitations and potential pitfalls of observational studies, we can rationally evaluate the claims of the news articles based on them. For example, here’s the actual study on which the latest round of “Red Meat Will Kill You” stories is based:
Arch Intern Med. doi:10.1001/archinternmed.2011.2287
Red Meat Consumption and Mortality: Results From 2 Prospective Cohort Studies
An Pan, PhD; Qi Sun, MD, ScD; Adam M. Bernstein, MD, ScD; Matthias B. Schulze, DrPH; JoAnn E. Manson, MD, DrPH; Meir J. Stampfer, MD, DrPH; Walter C. Willett, MD, DrPH; Frank B. Hu, MD, PhD
“Prospective cohort study?” Apparently this is yet another observational study—and therefore, it cannot be used to prove anything or claim that anything “causes” anything else. Correlation is not causation.
Unfortunately, while the study authors maintain this distinction, it’s quickly lost when it comes time to write newspaper articles. Here’s a typical representative:
Headline: “Red meat is blamed for one in 10 early deaths” (The Daily Telegraph)
[False. Since Pan et.al. is an observational study, we can’t assign blame.]“Eating steak increases the risk of early death by 12%.”
[Another false statement: associational studies cannot prove causation.]“The study found that cutting the amount of red meat in peoples’ diets to 1.5 ounces (42 grams) a day, equivalent to one large steak a week, could prevent almost one in 10 early deaths in men and one in 13 in women.”
[Note the weasel words “could prevent”. Just like playing basketball could make you taller, but it won’t. And just like HRT could have prevented heart attacks: instead, it caused them.]“Replacing red meat with poultry, fish or vegetables, whole grains and other healthy foods cut the risk of dying by up to one fifth, the study found.”
[No, it didn’t. The risk of dying was associated with self-reported intake of red meat and “healthy foods”.]
“But that’s just definitional nitpicking,” you say. “What about that 12% association?” It’s not nitpicking at all—because we’ve just opened the door to explaining that association in many other ways.
What Does “Red Meat Consumption and Mortality” (Pan et.al.) Really Tell Us?
“We prospectively observed 37 698 men from the Health Professionals Follow-up Study (1986-2008) and 83 644 women from the Nurses’ Health Study (1980-2008) who were free of cardiovascular disease (CVD) and cancer at baseline. Diet was assessed by validated food frequency questionnaires and updated every 4 years.”
–Pan et.al.
Remember the Nurses’ Health Study?
The same study we talked about above—which was used to claim that HRT decreased heart disease by 50%, while a controlled trial showed that HRT actually increased heart disease by 30%?
The same study we talked about above—for which we’ve already proven, using peer-reviewed research, that the self-reported meat consumption data from the “food frequency questionaires” was unrelated to how much meat the nurses actually ate? And that the nurses, like most of us, exaggerated their intake of foods they thought were healthy by over 50%, and decreased their intake of foods they thought were unhealthy (like red meat) by up to 30%?
Yes, we’ve just kicked the legs out from under this entire study. It’s pinning a 12% variation in death rate on data we’ve already proven to be off by -30% to +50%—and more importantly, to be unrelated to the nurses’ actual consumption of red meat. (Or of meat in general…even chicken was only recalled with 5-14% accuracy.)
So much for the headlines! Here’s an accurate statement, based on the actual data from Pan et.al.:
And just to nail this down, here’s another study—also from the Harvard School Of Public Health—which comes to the opposite conclusion:
Circulation. 2010 Jun 1;121(21):2271-83. Epub 2010 May 17.
Red and processed meat consumption and risk of incident coronary heart disease, stroke, and diabetes mellitus: a systematic review and meta-analysis.
Micha R, Wallace SK, Mozaffarian D.
Department of Epidemiology, Harvard School of Public Health, Boston, MA, USA.Red meat intake was not associated with CHD (n=4 studies; relative risk per 100-g serving per day=1.00; 95% confidence interval, 0.81 to 1.23; P for heterogeneity=0.36) or diabetes mellitus (n=5; relative risk=1.16; 95% confidence interval, 0.92 to 1.46;
But Wait, There’s More
We’re done, and I could easily stop here—but there’s more to talk about! Note this surprising statement from the “Results” section:
“Additional adjustment for saturated fat and cholesterol moderately attenuated the association between red meat intake and risk of CVD death, and the pooled HR (95% CI) dropped from 1.16 (1.12-1.20) to 1.12 (1.07-1.18).”
–Pan et.al. (Credit to “wildwabbit” at Paleohacks for catching this one.)
And the data from Table 1 clearly shows that the people who admitted to eating the most red meat had, by far, the lowest cholesterol levels.
Wait, what? Aren’t saturated fat and cholesterol supposed to cause heart disease? This is another clue that the story, and the data, isn’t quite as advertised.
Here’s another trick that’s been played with the data: contrary to the statement “replacing 1 serving of total red meat with 1 serving of fish, poultry, nuts, legumes, low-fat dairy products, or whole grains daily was associated with a lower risk of total mortality”, the curve they draw in Figure 1 has been dramatically, er, “smoothed.” The source data, in Table 2, shows that the age-adjusted quintiles of reported unprocessed red meat intake from the Nurses’ Health Study (remember, we’ve already proven these numbers aren’t real) have hazard ratios of 1.00, 1.05, 0.98, 1.09, and 1.48.
In other words, the data isn’t a smooth curve…it’s a hockey stick, and the relative risk is basically 1.0 except for the top quintile. (Credit to Roger C at Paleohacks for catching this one.)
This is important because it helps us to explain the 12% increase based on reported red meat consumption. We already know that the subjects of the study weren’t truthfully reporting their meat intake of any kind—and that foods perceived unhealthy were underreported on average, while foods reported healthy were overreported on average.
Table 1 shows that the highest quintile of reported red meat consumption was strongly associated with other behaviors and characteristics known to be associated with poor health: smoking, drinking, high BMI. Most impressively, it was associated with a 69% increase (NHS data) or 44% increase (HPF data) in reported total calories per day, which lends weight to the idea that the lower quintiles were simply underreporting their intake of foods they considered “unhealthy”, including red meat…
…unless we accept that 1/5 of nurses live on 1200 calories per day (and coincidentally report the lowest red meat intake) while 1/5 eat over 2000 calories per day (and coincidentally report the highest red meat intake).
Calorie consumption is our smoking gun. The average American female aged 20-59 consumes approximately 1900 calories/day, and not all nurses are female. (Source: NHANES 1999-2000, through the CDC.)
Therefore, a reported average consumption of 1200 calories/day is extremely implausible. It’s even less plausible that nurses who reported the lowest intake of red meat just happened to be on a 1200-calorie semi-starvation diet; that total reported calorie intake just happened to rise dramatically with reported red meat intake; and that only the nurses who reported eating the most red meat consumed a statistically average number of total calories.
Since we already know from Salvini et.al. that actual consumption is unrelated to reported consumption, underreporting of red meat and other foods perceived as “unhealthy” by the lower quintiles is a far more reasonable explanation.
So What’s The Real Story?
While we’ll probably never know the truth, I believe the most parsimonious explanation is this:
Nurses and other health professionals know intimately the mainstream advice on health, and cannot fail to have given it to thousands of patients over the decades: “eat less, stop smoking, drink less alcohol, avoid cholesterol, avoid saturated fat, avoid red meat.” Therefore, any health professional willing to admit in writing to smoking, drinking, and eating over three servings of red meat per day (see the NHS data in Table 1) most likely doesn’t care very much about their own state of health.
And just as we saw with the HRT data—where a theoretical 50% decrease in heart disease was later proven to mask a real 30% increase, due to the selection bias inherent in the very same dataset (Nurses’ Health Study) used here—I think that we’ll someday find out through controlled, randomized trials that eating plenty of red meat, eggs, and other whole, natural foods high in cholesterol and saturated fat is the real “heart-healthy diet.”
Live in freedom, live in beauty.
JS
Welcome Daily Crux readers! If you enjoyed this article, you might want to sign up for my RSS feed or my mostly-weekly newsletter. You can also bookmark the index to my previous articles, organized by topic.
What an epic this turned out to be! Please use the buttons below to forward this article around, and please send this link to anyone who sends you one of the innumerable scare stories. And if you learn of other solid debunking articles I can link, contact me or leave a comment!
You can support my continued efforts to bring you these dense, informative articles by buying a copy of my “Bold, fresh and interesting,” “Elegantly terse,” “Scathing yet beautiful,” “Funny, provocative, entertaining, fun, insightful” book, The Gnoll Credo—or even a T-shirt.
Can I just say…TIMELY and THANKS! After the latest story about how meat kills, some of my veg friends were calling for a, what say you Paleo friends – ha ha, you’re gonna die?
Ah gnolls, the voice of reason.
This kerfuffle truly saddened me, why am I constantly having to defend or justify myself and what I choose to eat/feed my daughter? More ammo for the sat fat tax here in the UK (that said I think they don’t want the fight and to be shown to have been leading the British public a merry dance for the past 30 years).
Why was theire no big international health splash about the second study?
The veggies have totally siezed the highground in the science, the PR and the ethics despite the fact that it is unethical and unscientific to exclude animal products.
Many in my family are increasing the helathiness of thir diets this press bollocks has really worried them, which of course was the point.
[…] […]
Fantastic article. I had written a short analysis of the data, but I didn’t even consider the reliability of the data itself.
This might be a little too nitpicky, but even a triple-blind controlled study can’t prove causality. It’s still, at best, a very proximate temporal association. Proving causality is worthy of an article of it’s own.
Awesome, J. Hopefully the people who would be best served by reading your debunking won’t lack the intelligence to appreciate it’s significance…hopefully.
On another subject, Peter Attia, the gentleman who I saw give a presentation with Gary Taubes last year and who has some rather astonishing numbers from his own personal nutritional experiment (albeit n=1, but, hey, if he can do it, then it at least proves that it is possible to eat more calories, exercise less and shed body fat as long as one severely restricts carbohydrates) finally has a web presence (blog and then some) over at waroninsulin.com just as I recommend his readers to you I recommend his blog to you. In addition to being an M.D. He’s also a ketoadapted endurance athlete who crosstrains with crossfit type exercises at a very high level!
Nice summary. There was some excellent commentary along these lines on mainstream radio in the UK (notably BBC Radio 4 and Channel 4 news), where similar points were made. These are national news outlets.
Radio 4 featured Joanna Blythman who broadly outlined a paleo diet in favour over dietary advice based upon CW, and generally disparaged the tide of bad nutritional advice we get fed. TBH I think we in the West are getting fatigued by the conflicting dietary advice we get.
Zoe Harcombe also makes a great point on her blog – “The numbers are very small. The overall risk of dying was not even one person in a hundred over a 28 year study. If the death rate is very small, a possible slightly higher death rate in certain circumstances is still very small. It does not warrant a scare-tactic, 13% greater risk of dying headline – this is ‘science’ at its worst”
I avoid foods that have the same name the world over. I also avoid foods that change their nutritional stripes to reflect dietary advice of the day. In a similar way I am pretty sure I have my diet dialled in because I am both health conscious AND such headlines have not and do not make me change my diet one bit.
Even if they were correct, would you give up a nice porterhouse every week for another bowl of tofu on seaweed?
Not me…….
[…] There were a bunch of sensationalist headlines and much hand-wringing, which fortunately sprung some well-written reviews (1,2,3,4). […]
FYI, Mark’s Daily Apple has a startlingly similar piece by Denise Minger, that by all appearances was posted after yours…
masterful, sir. thank you!
On the general subject, what is your opinion of “crossover” studies (assuming adequate washout periods, etc.)? Regards, Wayne
[…] who write about the “paleo” diet, such as Robb Wolf, Mark Sisson/Denise Minger, and J. Stanton, do a good job of pointing out the study’s flaws. My full time job is reviewing medical […]
[…] http://www.gnolls.org/2893/always-be-skeptical-of-nutrition-headlines-or-what-red-meat-consumption-a… Share […]
Sound and fun, J.
I must say, when I heard the story (first thing … News on the TV while I get ready for work), my ears perked up. But when the word “processed” was placed in front of red meat, I just ignored the rest.
Taking the long view, if this was actually true we would have died out millennia ago … or it is “ideal” 😀
The worst thing about this is that it creates a kind of self-supporting cycle. As the message about a given food gets stronger, people tend to “compensate” their actual intakes more and more, which in turns makes for even more numerous and scary headlines.
[…] Read full article… […]
Truth will out. Or we’ll outlive the SAD eaters because our dietary choices are based on truth and wisdom–a powerful cocktail for success and thriving. I propose we invite the authors of this study to a panel discussion at AHS12 at Harvard this August to respond to your post, Denis Minger’s, and Zoe Harcombe’s deconstruction of their study. I’d really like to hear their reactions at justifying their own existence.
My dad worked in a steel mill when he was young and ate red meat. He lived in Los Angeles the last 50 years of his life and ate red meat, he drank beer and wind with an occasional shot of whiskey. He lived to be 97 years and 2 months. Ergo eating red meat, drinking and breathing polluted air is good for your health.
[…] Read full article… icBeacon('investmentwatchblog'); Be Sociable, Share! Tweet(function() {var s = document.createElement('SCRIPT'), s1 = document.getElementsByTagName('SCRIPT')[0];s.type = 'text/javascript';s.async = true;s.src = 'http://widgets.digg.com/buttons.js';s1.parentNode.insertBefore(s, s1);})();Related news, headlines and opinion:Absurd doctor says feeding babies raw milk is ‘child endangerment’RED MEAT BLAMED FOR 1 IN 10 EARLY DEATHSFactory grown meat seen as more sustainable source of foodKeenian EconomicsVIDEO: Coulter Serves Up Jokes, ‘Red Meat’ in CPAC speech March 14th, 2012 | […]
Thirty plus years ago I read a small book by Roger Williams (not sure of his name but he had many scientific honors in his resume). “Nutrition in a nutshell”. His advice is still valid: Variety is the best nutrition of all. He even reminded us that protein was not just muscle tissue, as in steak. Kidneys, pancreas, brains too, in moderation. In another early study on cholesterol a doctor from Canada (I think the book was The Cholesterol Controversy, pointed out that animals on extremely low cholesterol diet aged faster and on autopsy had spleens with “abnormal ” histological findings typical of old age.
JS, a quick note: the correlation/causation fallacy would be, I believe, more correctly translated: “with this, therefore because of this”. You left out “ergo”. 🙂
Keep up the good work!
JS your final comment is bunkum
It’s already been proven. The foods we’ve been told are healthy are not. The foods we’ve been told to avoid are the healthy ones. See “Sleeper” by Woody Allen!
Chandra:
Though it's not always easy, I try to remain the voice of reason.
Neal:
I predicted this long ago: see You Are A Radical, And So Am I. Expect the onslaught to continue.
ddn:
Fortunately I remembered Chris Masterjohn had already done the hard work on that topic!
And yes, you're right: nothing can really “prove” causality, and there are plenty of shenanigans in the RCTs of, say, statin drugs. But we can get closer and closer to the truth.
Chuckie B:
I've seen Dr. Attia's site, and I'm very interested in his performance data, because the studies to date are almost never on people eating anything approaching an ancestral diet — let alone people in ketosis.
Asclepius:
That's a good point about the relative vs. absolute risk: even if we took the claims at face value they wouldn't matter much. However, I don't like the rest of Harcombe's article, because it goes on and on about the bad health characteristics of the high reported meat quintile — but Pan et.al. claim to have already controlled for those factors in order to come up with the 12% figure. That's why I don't use that particular argument except as a gateway to pointing out the howling disparity in calorie intake.
I agree that it doesn't change my behavior, either, because I'm comfortable with my knowledge of human biology and evolutionary history — but I want to provide information that helps others to understand why these sorts of headlines are bunk.
Paul:
I wouldn't go on a diet that prolonged my life to age 150 if it made me look like Dean Ornish.
guest:
Denise's article was posted just a few hours after mine, so I'm sure we were working on them in parallel. I'm encouraged by the fact that I didn't miss anything Denise found!
More soon!
JS
jake:
I appreciate the support. Please forward it on to anyone you think might find it helpful or interesting!
Wayne:
I'm suspicious of crossover studies, but it depends on what's being measured.
The classic example of when to NOT use a crossover is the Finnish Mental Hospital Study. There's no such thing as a “washout period” for heart disease! On the other hand, if you're measuring short-term metabolic parameters like heart rate, serum uric acid, etc., you can often get away with it.
Paul:
This all goes back to my article Live Now, Live Later: “I reject the bizarre concept that millions of years of evolution has selected us to enjoy only things that kill us, and to avoid everything that keeps us alive and healthy.”
Fmgd:
You're correct. 1) “Wearing fedoras causes heart disease.” 2) Thirty years of anti-fedora advertising campaigns later, only people who don't care about their own health still wear fedoras. 3) “See? People who wear fedoras die younger. Fedoras really are bad for you!”
More coming…
JS
Ok I agree with you that many polls are manipulated and give incorrect information but to the question of eating meat and health, you gave your personal view that you think it is good for the heart without any evidence to back up your statement.
So I want to give my view based on personal experience. I used to be a heavy meat eater and I was recommended by some friends to try cutting the meat and eating more of a vegetarian diet. The results for me were great. I noticed i got sick less, and I noticed i could run longer without getting tired(i was jogger at the time). I do not feel the necessity of being a complete vegetarian but I feel that a light meat diet that mainly is composed of vegetarian items is a healthy diet.
[…] Gnolls does a fantastic breakdown of the issues with the observational science here […]
@Tony Suckla,
I believe the aim of this article was not to show that eating meat is good, but to show how the study in question and similar studies don’t really support the headlines they generate.
This article isn’t even trying to prove that Pan et.al.’s data actually shows meat not to be a problem, it’s just arguing that the study doesn’t justify, or even couldn’t justify, saying it is.
[…] is pretty much all Big Dog’s eat. Give your brain a workout and read this for the truth: http://www.gnolls.org/2893/always-be-skeptical-of-nutrition-headlines-or-what-red-meat-consumption-a… Share this:TwitterFacebookRedditDiggStumbleUponPinterestEmailPrintLike this:LikeBe the first to […]
Aaron:
The thought “I may have to talk to some of these people at AHS 2012” crossed my mind. I've endeavored to keep my discussion civil and science-based.
Yes, I'm absolutely willing to participate in such a panel discussion — and I'd love to take it to the next level, to wit: “Apparently we've known for over twenty years (since Salvini et.al.) that the food consumption data from the Nurses' Health Study ranges from “wildly inaccurate” to “complete bunk”. Why are we still basing research on it — and, even worse, why are we basing public policy recommendations on it?”
Zak:
I could make a similar argument about my grandfather: smoked two packs a day for over 50 years, died of a non-smoking-related illness in his 80s. Smoking is good for you!
Ed:
Roger Williams was an eminent scientist (he discovered pantothenic acid — vitamin B5 — among other things), but I'm unable to find any of his nutrition writing online.
I don't think variety for its own sake is a good idea: “a little bit of everything” is only a good strategy when you don't know what's good and what's bad. I think it's best to eat more of the good stuff and less of the bad stuff! But since I have no way to know what Williams' actual words or intent are, I won't press the issue.
And yes, cholesterol is a vitamin, not a poison: it's absolutely necessary to life.
Uncephalized:
Good catch. I added it back in.
peter aris:
Which comment do you mean?
Tom:
Most “health food” is indeed bad for us, but I don't believe the converse is true: everyone agrees that Twinkies and Mountain Dew are unhealthy, but the Twinkie/Dew Diet won't leave you in good health.
Also note that “health food” is indeed better than the SAD (Standard American Diet). My favorite analogy is low-tar cigarettes: they're indeed less harmful than regular cigarettes…but you're still smoking cigarettes.
Tony:
It's certainly possible to eat an relatively unhealthy meat-heavy diet (e.g. KFC and corn dogs), and it's also possible to eat a relatively healthy low-meat diet (e.g. the Kitavans).
As I point out in this article, roughly 2/3 of self-described “vegetarians” in the USA actually eat meat! The real difference between them and admitted meat-eaters is that their diet is healthier in many other ways. For instance, “vegetarians” consume far more fish, vegetables, and fruit, and far less beer and french fries, than admitted meat-eaters. So it's not surprising that they're healthier…our contention is that they'd do even better if they'd make the same dietary changes but keep unprocessed, sustainably raised meat on the menu!
Fmgd:
Exactly. I'll go further and say that this article is using Pan et.al. as an example of why most nutrition headlines are either misleading or factually incorrect.
I'm caught up! (On this thread…tomorrow I'll start responding to other threads.) Thanks, everyone, for your support and corrections. Keep them coming — and please don't be shy. Spread this article on Facebook, Twitter, via email, or anywhere else you've seen the scare stories pop up!
JS
Have had the headlines been quoted to me at work, and yes they enjoy the ammo but are not really interested in looking at the data..in my N=1 study of the subject I find eating red meat every day boring…so prefer to vary it with liver, brains, sweetbreads, fish, poultry, all free range of course and mixed in with as many vegies I can find ( organic as they taste better ) if I live longer I will take my secret to the grave as no one wants to hear from a “gnoll/paleo” nut job like me.
[…] what is wrong with the study. Some really great articles blowing apart the study can be found on Gnolls.org,The Free Range Human, RobbWolf.com and perhaps the best article of all is on Mark’s Daily […]
For years I worked as a statistician and research assistant. Then I taught statistics in a public high school. I have been in mathematics education practically all my adult life (I am 64). Thanks to J. Stanton for once again taking the time and trouble to explain real scientific inquiry verses survey/opinion compilation. I will keep eating red meat and enjoy it.
i have seen surprisingly little of this, mostly because when i am forwarded things like this i reply along these sorts of lines.
will put this on my facebook page though as people should have complete knowledge of stuff like this.
also the pics are really funny 😀
It’s good to see a lot of well-written articles showing up in the paleo blogging community debunking the science behind these trials. I’m an intelligent, consciences eater. and a few minutes going through a food survey makes my head swim. It’s just such a lousy way to gather data, and to base anything scientific on that data is an invitation for disaster.
The “health-conscious bias” is very hard for the average person to grasp, too. I remember during my vegetarian days I would often get a response from people that “Oh, that’s why you’re so healthy.” I would then have to sigh and explain that I don’t smoke, I exercise regularly, I eat plenty of whole food, and have genes that predispose me to be thin. And that potato chips and soda are vegetarian as well.
Now as a paleo eater I still have the same health-conscious bias, which means that my copious red meat consumption isn’t associated with hot-dog buns, french fries, and soda. I somehow think that puts me in a different category of data…assuming the data is even good enough to be categorized.
[…] Re: Red Meat is blamed for 1 in 10 early deaths Thanks Delta I read another critique that I really enjoyed Always Be Skeptical Of Nutrition Headlines: Or, What “Red Meat Consumption and Mortality”… […]
Thanks for answering my “crossover” question. Regarding this red meat study, is there any chance that you might someday do a layman’s explanation of “multivariate analysis” as used in such studies. E.g., does one need to assume dose-dependency and examine all the individual data points? Is it unreliable to use quintile averages (if they did so) in such analysis? Etc. Thanks and Regards, Wayne
The vast majority of people will just see/hear the fearful title put forth by the medias then scan the text reassured in their low fat/high carb comfort zones. It’s only us “converts” who question the science and want more validation.
Guess we all need to add balance and objectivity to the debate by getting the truth out there so posts like this really, really help! Thank you.
Geo
[…] kuvaa siitä. Paleoblogosfäärissä on kritisoitu sitä hyvin perustein (kuten Denise Minger ja J. Stanton). Data on ollut hyvin huonolaatuista, eikä korrelaatio tarkoita […]
BT:
Most people aren't interested in hearing anything that contradicts their existing viewpoint…paleo eaters included.
However, I'd be cautious about taking your secret to the grave…we could easily end up like Denmark, with a tax on saturated fat, if we don't continue to represent ourselves proudly. People are less likely to push for something like that if they believe that there's a reasonable alternative viewpoint, even if they don't subscribe to it themselves.
Oliver:
Thank you. It means a lot to have my work approved by a professional.
eddie:
I've been hoping for an excuse to post an xkcd cartoon for some time.
Thanks for sharing it on Facebook! (“Likes” are nice, but “shares” are what get the word out to people who aren't already fans of mine.)
Beowulf:
Exactly! “Vegetarian” has become synonymous with “healthy eating”, due to tireless vegetarian propaganda…we need to change that.
Wayne:
I'm not confident enough in my knowledge of statistics to teach the multivariate analysis class: perhaps Oliver can step in?
George:
I appreciate the support! Don't lose hope: judging from the number of page views this article is getting, I'd say that there are quite a few people questioning the science.
No, I'm not up there with the Daily Telegraph or the LA Times — but there are a lot more of us than you might think.
JS
[…] Always be Skeptical of Nutrition Headlines – J. Stanton […]
One good reason to eat LESS meat (less than what the North American eats, that is), apart to fight rampant obesity (a real societal problem), is to SAVE THE PLANET! Check what it takes of water and crop to produce just one kilo of red meat ! Not to mention their obnoxious emissions…
[…] Stanton wrote a good explanation of observational studies and their faults. He also points out that the Hormone Replacement Therapy debacle of the 1990s started with […]
Roger:
Industrial agriculture is destructive and unsustainable whether the end product is fed to people or to cattle. I've written about this topic at length.
In contrast, grass-fed ruminants have negative carbon impact if properly managed — something that cannot be said for any form of agriculture. (It must be so: otherwise the Serengeti and the Great Plains would have blown away to dust millions of years ago from the impact of all those wildebeest and bison, and the Pleistocene wouldn't have been a continual global cooling trend, complete with Ice Ages.)
Also, consider that most of the figures for carbon impact of even the worst industrial feedlot beef are sheer baloney — they assume that cattle are grain-fed their entire life, which is impossible. (They die if you feed them on grain for more than a few months.)
JS
[…] Gnolls.org – J. Stanton […]
The physicist Heisenberg received the Nobel prize for showing that there is really no such thing as cause and effect. He called it the theory of indeterminacy. It seems that cause and effect are just illusions.
The above statement comes from Dr. Thomas Hora’s book, Existential Metaphychiatry, p. 141.
I have not read Heisenberg’s work myself, but it seems to me that there must be something to it. I would be highly suspect of anyone claiming a cause and effect relationship between anything two or more random phenomena, such as the eating of meat and the incidence of illness. In my opinion, it’s mad and ludicrous.
Ric:
It might be impossible to “prove” cause and effect, but RCTs can get us to a reasonable functional facsimile…
…and those who decide to test whether the train will “really” run them over generally get run over.
JS
[…] who take supplements are healthier are observational studies. The same types of studies that, say, tell us that hormone replacement therapy is a good idea [CTRL + F for "selection bias" to see the part that talks about that study]. Or tell us that people […]
[…] “Always be skeptical of nutrition headlines” from Gnolls.org This entry was posted in Workout of the Day by Todd. Bookmark the permalink. […]
Hi JS,
Thank you for the giggles the spurious relationship graphs gave me…priceless!
Susan:
Personally, I blame M. Night Shyamalan.
JS
Fantastic article JS, both informative and witty. Brilliant.
SP:
These articles seem to come out every year or two: yet another mining of the same known-bad data sets to produce another spurious association between reported meat consumption and something negative.
I think the latest one purports to show that meat-eaters beat their wives more often (not kidding). Of course, the Third Factor in play is that vegetarians are many times more likely to be college-educated and of higher socioeconomic strata, and therefore much less likely to engage in domestic violence.
JS
“I think the latest one purports to show that meat-eaters beat their wives more often (not kidding). Of course, the Third Factor in play is that vegetarians are many times more likely to be college-educated and of higher socioeconomic strata, and therefore much less likely to engage in domestic violence.”
WOW…just wow. That’s just unbelievable. I have two friends, one of which is vegan and the other is vegetarian and I remember cooking red mince meat and my vegan friend made a sly remark saying, ‘Heart disease…’ and I had to hold my tongue, unfortunately. I’m glad I stumbled on your great site; it’s not only informative but witty too, keep up the great work! P.S. Can you link me in the direction of a site and/or video that casts a critical eye on veganism specifically? Cheers.
I may not be JS, but rawfoodsos comes to mind as a site that's critical of veganism. It's a site run by a former raw vegan I believe.
Ah, thank you Jen! Much appreciated.
SP:
Raw Food SOS is brilliant, and it's where I direct anyone who tells me to read The China Study or Forks Over Knives.
There's also some good information at Beyond Vegetarianism, and Let Them Eat Meat explores the ethical dimensions of veg*anism vs. omnivorism.
JS
[…] the good the bad and the fucking stupid in science and how to read it ……heres a good post: Always Be Skeptical Of Nutrition Headlines: Or, What “Red Meat Consumption and Mortality”… Reply With […]
[…] Gnolls does a fantastic breakdown of the issues with the observational science here […]
[…] actually are eating. There is this and other confounding factors….you should give this a read: Always Be Skeptical Of Nutrition Headlines: Or, What “Red Meat Consumption and Mortality”… I think he walks people through the basics of dissecting studies very well in this piece. So […]
Denmarks fat Tax has being scrapped!!!!
Yippeee
That is good news … there were similar rumblings here in the UK.
JayJay, Paul:
Any excuse to raise taxes is generally met with support by certain elements of the government. Hopefully the repeal of the saturated fat tax denotes the high water mark of the diet-heart hypothesis, and we can actually start making forward progress.
JS
Hopefully you're right and that as bad as gets. Did anyone seriously think that putting a tax on real basic food was a good idea? Butter? Meat? FFS like Farmers don't have enough to worry about.
Funnily enough everyone just went over the border to buy their fat foods cheaper and the small business lost out, of course.
The more I read about it, the madder I get, so I better stop now and go eat some bacon.
Even RCTs can be (and are) successfully gamed, by using non-inert placebos: http://asserttrue.blogspot.com/2013/01/are-placebos-really-sugar-pills.html
JayJay:
Who knew butter could be an act of civil disobedience?
Alex M:
That's a great article. When so much money is at stake (especially sunk costs), it's quite profitable to cheat.
JS
I understand that the data showing red meat consumption mortality risk is flawed, on the other hand, is there some study that would show that eating 3 steaks a day (which is what most paleo dieters do) is safe? Could our evolutionary requirement for meat be more like couple of bugs a day and an animal once a week?
Martin,
I imagine our ancestors ate an animal when they could catch it. Also, I think it depends on how much of that steak someone is eating and how lean that steak is (a tenderloin is much leaner than a ribeye). Most steaks/animal meat aren't all protein, they come with a certain amount of naturally occurring fat.
Martin:
I'm partway through a series recounting the physical evidence we actually have about the evolution of our ancestors' diet…the index is here.
Meanwhile, note that there is a major difference between “how much meat we require to stay alive in a modern world with free access to highly processed grain products” and “how much meat we require to stay alive in the Paleolithic world in which we evolved, in which we had access only to living foods, and in which we had only minimal and portable technology to process them.” Paleolithic people didn't have access to highly concentrated sources of refined carbohydrates (note: even “whole grain bread” is a highly refined food), and calories must come from somewhere.
JS
[…] make nutrition complicated so that you’ll keep buying books and going to meetings. Remember that observational studies don’t necessarily tell you whether something is healthy to eat: they tell you whether the […]
[…] to Rule Them All.” No single study magically proves anything. Every day it seems there is a new study saying “X is the new cancer cure!” then a while later you might see “X causes […]
[…] There were a bunch of sensationalist headlines and much hand-wringing, which fortunately sprung some well-written reviews (1,2,3,4). […]
[…] There were a bunch of sensationalist headlines and much hand-wringing, which fortunately sprung some well-written reviews (1,2,3,4). […]
[…] Gnolls does a fantastic breakdown of the issues with the observational science here […]
[…] chocolate study really tells us next to nothing (smarter people than me, such as J. Stanton, have said the same of such studies many times). And I’m not even interested in picking on the chocolate study. There are plenty […]