Disclaimer
• Your life and health are your own responsibility.
• Your decisions to act (or not act) based on information or advice anyone provides you—including me—are your own responsibility.

Categories

You Can't Debunk Everything: How To Avoid Being Baffled By Baloney

Caution: contains SCIENCE!

Anyone who makes a serious effort to understand the science behind nutrition will understand immediately that news items—most of which simply reprint the press release—are usually pure baloney. In order to learn anything interesting, we require access to the papers themselves.

Unfortunately, that’s not the end of the shenanigans. Abstracts and conclusions often misrepresent the data. Data is selectively reported to omit negatives (for example, statin trials trumpet a decrease in heart disease while intentionally failing to report all-cause mortality). And experiments are often designed in such a way as to guarantee the desired result.

Is there any way to deal rationally with the unending onslaught?

This approach, though satisfying, is discouraged by our legal system.


How To Get The Results You Want

First, I’ll walk through a few examples of studies carefully designed to produce a result opposite to what happens to all of us in the real world. Please note that I am not accusing anyone of scientific fraud! What I’m showing is that you can ‘prove’ anything you want if you set up your conditions and tests correctly, and choose the right data from your results.

Example 1: Quit While You’re Ahead

Public Health Nutrition: 7(1A), 123–146
Diet, nutrition and the prevention of excess weight gain and obesity
BA Swinburn, I Caterson, JC Seidell and WPT James

This paper, which purports to be an objective review of the evidence, claims with a straight face that “Foods high in fat are less satiating than foods high in carbohydrates.”

Wait, what?

The authors cite two sources. One is a book I don’t have access to. The other is:

Eur J Clin Nutr. 1995 Sep;49(9):675-90.
A satiety index of common foods.
Holt SH, Miller JC, Petocz P, Farmakalidis E.

“Isoenergetic 1000 kJ (240 kcal) servings of 38 foods separated into six food categories (fruits, bakery products, snack foods, carbohydrate-rich foods, protein-rich foods, breakfast cereals) were fed to groups of 11-13 subjects. Satiety ratings were obtained every 15 min over 120 min after which subjects were free to eat ad libitum from a standard range of foods and drinks. A satiety index (SI) score was calculated by dividing the area under the satiety response curve (AUC)…”

Only the abstract is available online, but here’s the list of foods they tested, each with its measured ‘satiety index’—from which we find the surprising ‘facts’ that oranges and apples are more satiating than beef, oatmeal is more satiating than eggs, and boiled potatoes are 50% more satiating than any other food in the world!

This is obviously nonsense…but what’s going on here?

Upon reading the abstract, we can see the problem right away: they only measured satiety for two hours! Last I checked, it was more than two hours between breakfast and lunch, or between lunch and dinner. In fact, if you have any sort of commute, two hours doesn’t even get you to your morning coffee break! And given that a mixed meal of protein, carbohydrate, and fat hasn’t even left your stomach in two hours, we can see that this data does not support the breathtakingly bizarre conclusion drawn by Swinburn et. al.

In fact, it’s hard to see what conclusion this study supports beyond “when you don’t let people drink any water with their food, foods that are mostly water take up much more room in their stomach.” This is why oatmeal makes you feel so full…

…for about two hours, until the glucose is all absorbed and your sugar high wears off.

To see what happens when you track oatmeal vs. eggs for more than two hours, you can read the exhaustively-instrumented study in How “Heart-Healthy Whole Grains” Make Us Fat.

Example 2: Construct an Artificial Scenario

American Journal of Clinical Nutrition, Vol 61, 960S-967S
Carbohydrates, fats, and satiety.
BJ Rolls

Fat, not carbohydrate, is the macronutrient associated with overeating and obesity…Although more data are required, currently the best dietary advice for weight maintenance and for controlling hunger is to consume a low-fat, high-carbohydrate diet with a high fiber content.

Really? And what evidence are we supporting this with?

“The most direct way to assess how these differences in post-ingestive processing affect hunger, satiety, and food intake is to deliver the nutrients either intravenously on intragastrically. Such infusions ensure that taste and learned responses to foods will not influence the results. Thus, to examine in rnore detail the mechanisms involved in the effects of carbohydrate and fat on food intake, we infused pure nutrients either through intravenous or intragastric routes.”

And the author goes on to cite her own study to that effect, found here:

American Journal of Clinical Nutrition, Vol 61, 754-764
Accurate energy compensation for intragastric and oral nutrients in lean males.
DJ Shide, B Caballero, R Reidelberger and BJ Rolls

There’s only one problem with this theory: I don’t eat via intravenous or intragastric infusion, and neither do you.

Physiol Behav. 1999 Aug;67(2):299-306.
Comparison of the effects of a high-fat and high-carbohydrate soup delivered orally and intragastrically on gastric emptying, appetite, and eating behaviour.
Cecil JE, Francis J, Read NW.

“When soup was administered intragastrically (Experiment 1) both the high-fat and high-carbohydrate soup preloads suppressed appetite ratings from baseline, but there were no differences in ratings of hunger and fullness, food intake from the test meal, or rate of gastric emptying between the two soup preloads.”

That’s what the Rolls study above also found. Yet…

“When the same soups were ingested (Experiment 2), the high-fat soup suppressed hunger, induced fullness, and slowed gastric emptying more than the high-carbohydrate soup and also tended to be more effective at reducing energy intake from the test meal.

Oops! Apparently when you eat fat (as opposed to injecting it into your veins or stomach), it is indeed more satiating than carbohydrate. Raise your hand if you’re surprised.

Example 3: Confound Your Variables

Am J Clin Nutr December 1987 vol. 46 no. 6 886-892
Dietary fat and the regulation of energy intake in human subjects.
Lauren Lissner, PhD; David A Levitsky, PhD; Barbara J Strupp, PhD;

“Twenty-four women each consumed a sequence of three 2-wk dietary treatments in which 15-20%, 30-35%, or 45-50% of the energy was derived from fat. These diets consisted of foods that were similar in appearance and palatability but differed in the amount of high-fat ingredients used. Relative to their energy consumption on the medium- fat diet, the subjects spontaneously consumed an 11.3% deficit on the low-fat diet and a 15.4% surfeit on the high-fat diet (p less than 0.0001), resulting in significant changes in body weight (p less than 0.001). A small amount of caloric compensation did occur (p less than 0.02), which was greatest in the leanest subjects (p less than 0.03). These results suggest that habitual, unrestricted consumption of low- fat diets may be an effective approach to weight control.

This study looks much more solid at first glance: test subjects were given prepared foods which theoretically differed only in fat content, and which were theoretically tested to have equal palatability. Let’s take that at face value for the moment, and ask ourselves: why were they eating more on the high-fat diet?

First, let’s look at what made the diet “high-fat”. They helpfully list the ingredients added to each meal to make it higher in fat in Table 1.

DFR Figure 1

Notice what’s been added in the right column? I see a lot of n-6 laden “vegetable oil” (store-bought mayonnaise is made with soybean oil) and margarine. Since this study was done back in 1987, the margarine would be absolutely loaded with trans fats, which we now know are strongly associated with obesity, heart disease, inflammation, and…disrupted insulin sensitivity. (Research review.)

So they’re testing what happens when carbohydrates are replaced primarily with seed oil and trans fats—which makes this study irrelevant to anyone considering a healthy diet.

Second, we have the problem that both breakfast and lunch were served in unitary portions.

“All foods, including those served as units (eg, muffins, sandwiches), could be consumed entirely or in part. … Sandwiches were available in whole or half units.”

Though the study doesn’t say explicitly, it’s probably a good assumption that the higher-fat versions contained far more calories, since the researchers tried to make them as similar as possible in appearance. It’s well-known that offering people larger portions causes them to eat more…especially since unlike breakfast and dinner (which were eaten in the laboratory), lunch was taken out. (If you take a sandwich with you, what are the odds you won’t finish it that afternoon, regardless of size?)

That’s not the worst part, though. The worst part is that their data is completely worthless because of interaction between the different diets!

Normally, controlled trials are done on separate, statistically matched groups of subjects, in order to make sure that effects from one treatment don’t bleed over into another.

However, in some cases, “crossover trials” are conducted, where each group is given each treatment in sequence—separated by a “washout period” that is supposed to let any effects of the previous treatment dissipate. This is less desirable (how do you know all the effects have ‘washed out’?), and is usually done because it allows the experimenters to screen and follow a smaller group.

However, Lissner et. al. didn’t even follow a crossover protocol. Instead, they employed a complicated “latin square” design, in which each test subject consumed a different meal type each day! A typical subject would consume a low-fat meal one day, a high-fat meal the next, a medium-fat meal the third day, and the sequence would repeat.

Can you imagine a drug trial where patients took one drug on odd days, and another drug on even days? How could you possibly disentangle the effects?

All of us have eaten a huge dinner and not been hungry the next morning…or gone to bed hungry and been ravenous when we awakened. In this insane design, each high-fat meal was guaranteed to be surrounded by two days of lower-fat meals. Yet in Figure 2, they graph energy intake for each day as if it were the same people eating each diet for two weeks!

In conclusion, this study is triply useless: first, due to using known industrial toxins for the “high-fat” diet, second, due to unequal portion size, and third, due to an intentionally broken design that commingles the effects of the three diets.

Holding Back The Ocean

The purpose of this article—and of gnolls.org—isn’t to debunk silly press releases, misleading websites, or even misleading scientific papers. I’ve given you some useful debunking weapons for your arsenal, but they’re not enough—because trying to dodge every slice of baloney thrown at us on a daily basis is like trying to hold back the ocean with a blue tarp and some rebar.

The usual strategy is to find a belief we like and stick with it, regardless of the evidence—but that way lies zealotry. What we need is a higher level of understanding. We need knowledge that lets us rationally dismiss the baloney and junk science, while conserving our time and attention for the few nuggets of real, new, important information.

We need to understand how human bodies work.

In other words, we need to understand ourselves.

We need to understand the basics of human biology and chemistry, how it was shaped from ape and mammal biology and chemistry, and how much it shares with all Earth life. We need to understand our multi-million year evolutionary history as hunters and foragers, how we were selected for survival on the African savanna, and how that selection pressure turned little 80-pound apes into modern humans.

And once we’ve used this understanding to answer basic questions like "How is food digested?" and "How are nutrients converted into energy?", we can use those answers to dismiss the baloney and junk science, allowing us to spend our valuable time and attention on real information. Because while it’s blindingly obvious to anyone who’s tried both that eating eggs for breakfast is more satiating than eating a bagel, it’s important to know why.

Conclusion: There Is No Easy Way, But There Is A Better Way

“Here’s a study that says so” isn’t a reason: it’s just a set of observations. We need to know how our bodies work. Only then can we rationally judge the meaning of these observations.

There is no shortcut to this knowledge. If there were one, and I knew it, I would already have told you. The best I can do is to continue to hone my own understanding—and I’ll continue to share what I know (or think I know) with you.

Live in freedom, live in beauty.

JS


As this upcoming weekend is a holiday, I may not have time to write an article for next week.

In the meantime, you can occupy yourself with my “Elegantly terse”, “Utterly amazing, mind opening, and fantastically beautiful”, “Funny, provocative, entertaining, fun, insightful” novel, The Gnoll Credo. (More effusive praise here, and here.) It’s a trifling $10.95 US, it’s available worldwide, and you can read the first 20 pages here.

The world is a different place after you’ve read The Gnoll Credo. It will change your life. This is not hyperbole. Read the reviews.

Did you find this article useful or inspiring? Don’t forget to use the buttons below to share it!