NINE   BAD SCIENCE

To understand how nutrition science could have been so spectacularly wrong about dietary fat and health, it’s important to understand that doing nutrition science isn’t easy. In fact, it’s a lot harder than most of the scientists who do it for a living realize or at least are willing to admit. For one thing, the scientific tools at their disposal are in many ways ill suited to the task of understanding systems as complex as food and diet. The assumptions of nutritionism—such as the idea that a food is not a system but rather the sum of its nutrient parts—pose another set of problems. We like to think of scientists as being free from ideological taint, but of course they are as much the product of their ideological environment as the rest of us. In the same way nutritionism can lead to a false consciousness in the mind of the eater, it can just as easily mislead the scientist.

The problem starts with the nutrient. Most nutritional science involves studying one nutrient at a time, a seemingly unavoidable approach that even nutritionists who do it will tell you is deeply flawed. “The problem with nutrient-by-nutrient nutrition science,” points out Marion Nestle, a New York University nutritionist, “is that it takes the nutrient out of the context of the food, the food out of the context of the diet, and the diet out of the context of the lifestyle.”

If nutrition scientists know this, why do they do it anyway? Because a nutrient bias is built into the way science is done. Scientists study variables they can isolate; if they can’t isolate a variable, they won’t be able to tell whether its presence or absence is meaningful. Yet even the simplest food is a hopelessly complicated thing to analyze, a virtual wilderness of chemical compounds, many of which exist in intricate and dynamic relation to one another, and all of which together are in the process of changing from one state to another. So if you’re a nutrition scientist you do the only thing you can do, given the tools at your disposal: Break the thing down into its component parts and study those one by one, even if that means ignoring subtle interactions and contexts and the fact that the whole may well be more than, or maybe just different from, the sum of its parts. This is what we mean by reductionist science.

Scientific reductionism is an undeniably powerful tool, but it can mislead us too, especially when applied to something as complex, on the one side, as a food and on the other a human eater. It encourages us to take a simple mechanistic view of that transaction: Put in this nutrient, get out that physiological result. Yet people differ in important ways. We all know that lucky soul who can eat prodigious quantities of fattening food without ever gaining weight. Some populations can metabolize sugars better than others. Depending on your evolutionary heritage, you may or may not be able to digest the lactose in milk. Depending on your genetic makeup, reducing the saturated fat in your diet may or may not move your cholesterol numbers. The specific ecology of your intestines helps determine how efficiently you digest what you eat, so that the same 100 calories of food may yield more or less food energy depending on the proportion of Firmicutes and Bacteroides resident in your gut. In turn, that balance of bacterial species could owe to your genes or to something in your environment. So there is nothing very machinelike about the human eater, and to think of food as simply fuel is to completely misconstrue it. It’s worth keeping in mind too that, curiously, the human digestive tract has roughly as many neurons as the spinal column. We don’t yet know exactly what they’re up to, but their existence suggests that much more is going on in digestion than simply the breakdown of foods into chemicals.

Also, people don’t eat nutrients; they eat foods, and foods can behave very differently from the nutrients they contain. Based on epidemiological comparisons of different populations, researchers have long believed that a diet containing lots of fruits and vegetables confers some protection against cancer. So naturally they ask, What nutrient in those plant foods is responsible for that effect? One hypothesis is that the antioxidants in fresh produce—compounds like beta-carotene, lycopene, vitamin E, and so on—are the X factor. It makes good theoretical sense: These molecules (which plants produce to protect themselves from the highly reactive forms of oxygen they produce during photosynthesis) soak up the free radicals in our bodies, which can damage DNA and initiate cancers. At least that’s how it seems to work in a test tube. Yet as soon as you remove these crucial molecules from the context of the whole foods they’re found in, as we’ve done in creating antioxidant supplements, they don’t seem to work at all. Indeed, in the case of beta-carotene ingested as a supplement, one study has suggested that in some people it may actually increase the risk of certain cancers. Big oops.

What’s going on here? We don’t know. It could be the vagaries of human digestion. Maybe the fiber (or some other component) in a carrot protects the antioxidant molecule from destruction by stomach acids early in the digestive process. Or it could be we isolated the wrong antioxidant. Beta is just one of a whole slew of carotenes found in common vegetables; maybe we focused on the wrong one. Or maybe beta-carotene works as an antioxidant only in concert with some other plant chemical or process; under other circumstances it may behave as a pro-oxidant.

Indeed, to look at the chemical composition of any common food plant is to realize just how much complexity lurks within it. Here’s a list of just the antioxidants that have been identified in a leaf of garden-variety thyme:

This is what you ingest when you eat food flavored with thyme. Some of these chemicals are broken down by your digestion, but others go on to do various as-yet-undetermined things to your body: turning some gene’s expression on or off, perhaps, or intercepting a free radical before it disturbs a strand of DNA deep in some cell. It would be great to know how this all works, but in the meantime we can enjoy thyme in the knowledge that it probably doesn’t do any harm (since people have been eating it forever) and that it might actually do some good (since people have been eating it forever), and even if it does nothing at all, we like the way it tastes.

It’s important also to remind ourselves that what reductive science can manage to perceive well enough to isolate and study is subject to almost continual change, and that we have a tendency to assume that what we can see is the important thing to look at. The vast attention paid to cholesterol since the 1950s is largely the result of the fact that for a long time cholesterol was the only factor linked to heart disease that we had the tools to measure. (This is sometimes called parking-lot science, after the legendary fellow who loses his keys in a parking lot and goes looking for them under the streetlight—not because that’s where he lost them but because that’s where it’s easiest to see.) When we learned how to measure different types of cholesterol, and then triglycerides and C-reactive protein, those became the important components to study. There will no doubt be other factors as yet unidentified. It’s an old story: When Prout and Liebig nailed down the macronutrients, scientists figured that they now understood the nature of food and what the body needed from it. Then when the vitamins were isolated a few decades later, scientists thought, okay, now we really understand food and what the body needs for its health; and today it’s the polyphenols and carotenoids that seem to have completed the picture. But who knows what else is going on deep in the soul of a carrot?

The good news is that, to the carrot eater, it doesn’t matter. That’s the great thing about eating foods as compared with nutrients: You don’t need to fathom a carrot’s complexity in order to reap its benefits.

The mystery of the antioxidants points up the danger in taking a nutrient out of the context of food; scientists make a second, related error when they attempt to study the food out of the context of the diet. We eat foods in combinations and in orders that can affect how they’re metabolized. The carbohydrates in a bagel will be absorbed more slowly if the bagel is spread with peanut butter; the fiber, fat, and protein in the peanut butter cushion the insulin response, thereby blunting the impact of the carbohydrates. (This is why eating dessert at the end of the meal rather than the beginning is probably a good idea.) Drink coffee with your steak, and your body won’t be able to fully absorb the iron in the meat. The olive oil with which I eat tomatoes makes the lycopene they contain more available to my body. Some of those compounds in the sprig of thyme may affect my digestion of the dish I add it to, helping to break down one compound or stimulate production of an enzyme needed to detoxify another. We have barely begun to understand the relationships among foods in a cuisine.

But we do understand some of the simplest relationships among foods, like the zero-sum relationship: If you eat a lot of one thing, you’re probably not eating a lot of something else. This fact alone may have helped lead the diet-heart researchers astray. Like most of us, they assumed that a bad outcome like heart disease must have a bad cause, like saturated fat or cholesterol, so they focused their investigative energies on how these bad nutrients might cause disease rather than on how the absence of something else, like plant foods or fish, might figure in the etiology of the disease. Nutrition science has usually put more of its energies into the idea that the problems it studies are the result of too much of a bad thing instead of too little of a good thing. Is this good science or nutritionist prejudice? The epidemiologist John Powles has suggested this predilection is little more than a Puritan bias: Bad things happen to people who eat bad things.

But what people don’t eat may matter as much as what they do. This fact could explain why populations that eat diets containing lots of animal food generally have higher rates of coronary heart disease and cancer than those that don’t. But nutritionism encouraged researchers to look beyond the possibly culpable food itself—meat—to the culpable nutrient in the meat, which scientists have long assumed to be the saturated fat. So they are baffled indeed when large dietary trials like the Women’s Health Initiative and the Nurses’ Health Study fail to find evidence that reducing fat intake significantly reduces the incidence of heart disease or cancer.

Of course thanks to the low-fat-diet fad (inspired by the same reductionist hypothesis about fat), it is entirely possible to slash your intake of saturated fat without greatly reducing your consumption of animal protein: Just drink the low-fat milk, buy the low-fat cheese, and order the chicken breast or the turkey bacon instead of the burger. So did the big dietary trials exonerate meat or just fat? Unfortunately, the focus on nutrients didn’t tell us much about foods. Perhaps the culprit nutrient in meat and dairy is the animal protein itself, as some researchers hypothesize. (The Cornell nutritionist T. Colin Campbell argues as much in his recent book, The China Study.) Others think it could be the particular kind of iron in red meat (called heme iron) or the nitrosamines produced when meat is cooked. Perhaps it is the steroid growth hormones typically present in the milk and meat; these hormones (which occur naturally in meat and milk but are often augmented in industrial production) are known to promote certain kinds of cancer.

Or, as I mentioned, the problem with a meat-heavy diet might not even be the meat itself but the plants that all that meat has pushed off the plate. We just don’t know. But eaters worried about their health needn’t wait for science to settle this question before deciding that it might be wise to eat more plants and less meat. This of course is precisely what the McGovern committee was trying to tell us.

The zero-sum fallacy of nutrition science poses another obstacle to nailing down the effect of a single nutrient. As Gary Taubes points out, it’s difficult to design a dietary trial of something like saturated fat because as soon as you remove it from the trial diet, either you have dramatically reduced the calories in that diet or you have replaced the saturated fat with something else: other fats (but which ones?), or carbohydrates (but what kind?), or protein. Whatever you do, you’ve introduced a second variable into the experiment, so you will not be able to attribute any observed effect strictly to the absence of saturated fat. It could just as easily be due to the reduction in calories or the addition of carbohydrates or polyunsaturated fats. For every diet hypothesis you test, you can construct an alternative hypothesis based on the presence or absence of the substitute nutrient. It gets messy.

And then there is the placebo effect, which has always bedeviled nutrition research. About a third of Americans are what researchers call responders—people who will respond to a treatment or intervention regardless of whether they’ve actually received it. When testing a drug you can correct for this by using a placebo in your trial, but how do you correct for the placebo effect in the case of a dietary trial? You can’t: Low-fat foods seldom taste like the real thing, and no person is ever going to confuse a meat entrée for a vegetarian substitute.

Marion Nestle also cautions against taking the diet out of the context of the lifestyle, a particular hazard when comparing the diets of different populations. The Mediterranean diet is widely believed to be one of the most healthful traditional diets, yet much of what we know about it is based on studies of people living in the 1950s on the island of Crete—people who in many respects led lives very different from our own. Yes, they ate lots of olive oil and more fish than meat. But they also did more physical labor. As followers of the Greek Orthodox church, they fasted frequently. They ate lots of wild greens—weeds. And, perhaps most significant, they ate far fewer total calories than we do. Similarly, much of what we know about the health benefits of a vegetarian diet is based on studies of Seventh-Day Adventists, who muddy the nutritional picture by abstaining from alcohol and tobacco as well as meat. These extraneous but unavoidable factors are called, aptly, confounders.

One last example: People who take supplements are healthier than the population at large, yet their health probably has nothing whatsoever to do with the supplements they take—most of which recent studies have suggested are worthless. Supplement takers tend to be better educated, more affluent people who, almost by definition, take a greater than usual interest in personal health—confounders that probably account for their superior health.

But if confounding factors of lifestyle bedevil epidemiological comparisons of different populations, the supposedly more rigorous studies of large American populations suffer from their own arguably even more disabling flaws. In ascending order of supposed reliability, nutrition researchers have three main methods for studying the impact of diet on health: the case-control study, the cohort study, and the intervention trial. All three are seriously flawed in different ways.

In the case-control study, researchers attempt to determine the diet of a subject who has been diagnosed with a chronic disease in order to uncover its cause. One problem is that when people get sick they may change the way they eat, so the diet they report may not be the diet responsible for their illness. Another problem is that these patients will typically report eating large amounts of whatever the evil nutrient of the moment is. These people read the newspaper too; it’s only natural to search for the causes of one’s misfortune and, perhaps, to link one’s illness to one’s behavior. One of the more pernicious aspects of nutritionism is that it encourages us to blame our health problems on lifestyle choices, implying that the individual bears ultimate responsibility for whatever illnesses befall him. It’s worth keeping in mind that a far more powerful predictor of heart disease than either diet or exercise is social class.

Long-term observational studies of cohort groups such as the Nurses’ Health Study represent a big step up in reliability from the case-control study. For one thing, the studies are prospective rather than retrospective: They begin tracking subjects before they become ill. The Nurses’ Study, which has collected data on the eating habits and health outcomes of more than one hundred thousand women over several decades (at a cost of more than one hundred million dollars), is considered the best study of its kind, yet it too has limitations. One is its reliance on food-frequency questionnaires (about which more in a moment). Another is the population of nurses it has chosen to study. Critics (notably Colin Campbell) point out that the sample is relatively uniform and is even more carnivorous than the U.S. population as a whole. Pretty much everyone in the group eats a Western diet. This means that when researchers divide the subject population into groups (typically fifths) to study the impact of, say, a low-fat diet, the quintile eating the lowest-fat diet is not all that low—or so dramatically different from the quintile consuming the highest-fat diet. “Virtually this entire cohort of nurses is consuming a high-risk diet,” according to Campbell. That might explain why the Nurses’ Study has failed to detect significant benefits for many of the dietary interventions it’s looked at. In a subject population that is eating a fairly standard Western diet, as this one is, you’re never going to capture the effects, good or bad, of more radically different ways of eating. (In his book, Campbell reports Walter Willett’s personal response to this criticism: “You may be right, Colin, but people don’t want to go there.”)

The so-called gold standard in nutrition research is the large-scale intervention study. In these studies, of which the Women’s Health Initiative is the biggest and best known, a large population is divided into two groups. The intervention group changes its diet in some prescribed way while the control group (one hopes) does not. The two groups are then tracked over many years to learn whether the intervention affects relative rates of chronic disease. In the case of the Women’s Health Initiative study of dietary fat, a $415 million undertaking sponsored by the National Institutes of Health, the eating habits and health outcomes of nearly forty-nine thousand women (aged fifty to seventy-nine) were tracked for eight years to assess the impact of a low-fat diet on a woman’s risk of breast and colorectal cancer and cardiovascular disease. Forty percent of the women were told to reduce their consumption of fat to 20 percent of total calories. When the results were announced in 2006, it made front-page news (The New York Times headline said LOW-FAT DIET DOES NOT CUT HEALTH RISKS, STUDY FINDS) and the cloud of nutritional confusion beneath which Americans endeavor to eat darkened further.

Even a cursory examination of the study’s methods makes you wonder what, if anything, it proved, either about dietary fat or meat eating. You could argue that, like the Nurses’ Healthy Study, all any such trials prove is that changing one component in the diet at a time, and not by much, does not confer a significant health benefit. But perhaps the strongest conclusion that can be drawn from an analysis of the Women’s Health Initiative is about the inherent limitations of this kind of nutrient-by-nutrient nutrition research.

Even the beginning student of nutritionism will immediately spot several flaws: The focus was on dietary fat rather than on any particular food, such as meat or dairy. So women could reach their goal simply by switching to lower-fat animal products. Also, no distinctions were made between different types of fat: Women getting their allowable portion of fat from olive oil or fish were lumped together with women getting their fat from low-fat cheese or chicken breasts or margarine. Why? Because when the study was designed sixteen years ago, the whole notion of “good fats” was not yet on the mainstream scientific scope. Scientists study what scientists can see.

Another problem with the trial was that the low-fat group failed to hit the target of reducing their fat intake to 20 percent of total calories. The best they could manage was 24 percent in the first year, but by the end of the study they’d drifted back to 29 percent, only a few percentage points lower than the control group’s fat intake. Which was itself drifting downward as the women allowed to eat as much fat as they wanted presumably read the newspapers and the food product labels and absorbed the culture’s enthusiasm for all things low fat. (This corruption of a control group by popular dietary advice is called the treatment effect.) So it’s hardly surprising that the health outcomes of the two groups would not greatly differ—by the end, they might have been consuming pretty much the same diet.

I say “might have been” because we actually have little idea what these women were really eating. Like most people asked about their diet, they lied about it—which brings us to what is perhaps the single biggest problem in doing nutrition science. Even the scientists who conduct this sort of research conduct it in the knowledge that people underestimate (let’s be generous) their food intake all the time. They have even developed scientific figures for the magnitude of the error. “Validation studies” of dietary trials like the Women’s Health Initiative or the Nurses’ Study, which rely on “food-frequency questionnaires” filled out by subjects several times a year, indicate that people on average eat between a fifth and a third more than they say they do on questionnaires.* How do the researchers know that? By comparing what people report on their food-frequency questionnaires with interviews about their dietary intake over the previous twenty-four hours, thought to be somewhat more reliable. Somewhat. Because as you might expect, these “twenty-four-hour recall” data have their own accuracy problems: How typical of your overall diet is what you ate during any single twenty-four-hour period?

To try to fill out the food-frequency questionnaire used by the Women’s Health Initiative, as I recently did, is to realize just how shaky the data on which all such dietary studies rely really are. The survey, which takes about forty-five minutes to complete, starts off with some relatively easy questions. “Did you eat chicken or turkey during the last three months?” Having answered yes, I then was asked, “When you ate chicken or turkey, how often did you eat the skin?” And, “Did you usually choose light meat, dark meat, both?” But the survey soon became harder, as when it asked me to think back over the past three months to recall whether when I ate okra, squash, or yams were they fried, and if so, were they fried in stick margarine, tub margarine, butter, shortening (in which category they inexplicably lumped together hydrogenated vegetable oil and lard), olive or canola oil, or nonstick spray? I would hope they’d take my answers with a grain of salt because I honestly didn’t remember and in the case of any okra eaten in a restaurant, even a hypnotist or CIA interrogator could not extract from me what sort of fat it was fried in. Now that we spend half of our food dollars on meals prepared outside of the home, how can respondents possibly know what type of fats they’re consuming?

Matters got even sketchier in the second section of the survey, when I was asked to specify how many times in the last three months I’d eaten a half-cup serving of broccoli, among a dizzying array of other fruits and vegetables I was asked to tally for the dietary quarter. I’m not sure Marcel Proust himself could recall his dietary intake over the last ninety days with the sort of precision demanded by the FFQ.

When you get to the meat section, the portion sizes specified haven’t been seen in America since the Hoover administration. If a four-ounce portion of steak is considered “medium,” was I really going to admit that the steak I enjoyed on an unrecallable number of occasions during the past three months was probably the equivalent of two or three (or in the case of a steak house steak, no fewer than four) of these portions? I think not. In fact, most of the “medium serving sizes” to which I was asked to compare my own consumption made me feel like such a pig that I badly wanted to shave a few ounces here, a few there. (I mean, I wasn’t under oath or anything.)

These are is the sort of data on which the largest questions of diet and health are being decided today. “The most intellectually demanding challenge in the field of nutrition,” as Marion Nestle writes in Food Politics, “is to determine dietary intake.” The uncomfortable fact is that the entire field of nutritional science rests on a foundation of ignorance and lies about the most basic question of nutrition: What are people eating? Over lunch, I asked Nestle if I was perhaps being too harsh. She smiled.

“To really know what a person is eating you’d have to have a second invisible person following them around, taking photographs, looking at ingredients, and consulting accurate food composition tables, which we don’t have.” When you report on an FFQ that you ate a carrot, the tabulator consults a U.S. Department of Agriculture database to determine exactly how much calcium or beta-carotene that carrot contained. But because all carrots are not created equal, their nutrient content varying with everything from the variety planted and type of soil it was planted in to the agriculture system used (organic? conventional?) and the carrot’s freshness, these tables suffer from their own inaccuracies.

I was beginning to realize just how much suspension of disbelief it takes to be a nutrition scientist.

“It’s impossible,” Nestle continued. “Are people unconsciously underestimating consumption of things they think the researcher thinks are bad or overestimating consumption of things they think the researcher thinks are good? We don’t know. Probably both. The issue of reporting is extraordinarily serious. We have to ask, How accurate are the data?”

It’s not as though the epidemiologists who develop and deploy FFQs are unaware of their limitations. Some of them, like Walter Willett, strive heroically to repair the faulty data, developing “energy adjustment” factors to correct for the fact that the calories reported on surveys are invariably wrong and complicated “measurement error” algorithms to fix the errors in the twenty-four-hour recall surveys used to fix the errors in the FFQ.

I tracked down Gladys Block, the prominent epidemiologist who developed the FFQ on which the Women’s Health Initiative based its own questionnaire. We met for coffee in Berkeley, where she is a professor in the School of Public Health. Nearing retirement, Block is unusually thoughtful about the limits of her field and disarmingly candid. “It’s a mess,” she said, speaking not of the FFQ itself but of the various formulae and algorithms being used to correct errors in the data. “Because if the energy [i.e., the reported calorie consumption] is off, then the nutrients are off too. So if you’re going to correct for calories, do you then also correct for…” She paused and then sighed. “No, it’s a mess.”

Block thinks the problem with nutrition science, which she feels “has led us astray,” is not the FFQ itself but mis-and overinterpretation of the data derived from the FFQ, a tool for which she makes realistic but strikingly modest claims: “The real purpose of the FFQ is to rank people” on their relative consumption of, say, fruits and vegetables or total calories. “If someone reports consuming five hundred calories a day, that’s not true, obviously, but you can say they’re probably at the low end of the spectrum. People overworry about accuracy.”

This was not the sort of thing I expected to hear from an epidemiologist. But then neither was this: “I don’t believe anything I read in nutritional epidemiology anymore. I’m so skeptical at this point.”