Making worthless data confess
A recent, well-financed study shows the glycemic index (GI) to be a less-than-optimal way of managing diabetes with diet. Meanwhile, a major name in the world of mainstream nutrition comments on this study and shows his own bias. Oh dear. Let’s take a look.
Before we launch into this study, which we’re going to just briefly review because I want to spend more time on the commentary, I want to propose to you a thought experiment. Suppose I ask you to design a study to see what happens when subjects with diabetes eat low-GI carbs as compared to what happens when they eat high-GI carbs. It seems pretty simple. If you’ve got half a brain, you would recruit subjects with diabetes, go through all of the randomizing rigmarole to ensure that both groups of subjects were as alike as possible, i.e., subjects in both groups were about the same size, same ratio of sexes, same degree of blood sugar elevation, etc. Then you would start the subjects in one group on an amount of carbohydrate, let’s say 220 gm per day, that were mainly low-GI carbs and the other group on about the same amount of carbohydrate composed of high-GI carbs. You would teach each of these groups how to follow their specific GI diets and would have a way of monitoring for compliance. Then you would set them to it and recheck them in 3 months or 6 months or a year or whatever you decided your study length to be. Pretty simple stuff, right?
Just for grins, let’s throw in a twist.
Let’s say I’m funding your experiment, and I come to you and tell you that I want you to add another group to your study. I tell you: Look, I’m tired of Eades, Atkins, Taubes and the rest of these jerks babbling on about how wonderful the low-carb diet is. We all know – wink, wink – that the low-GI diet is going to kick some tail in this study, so let’s add a group of subjects who will follow a low-carb diet over this same study period. Then we can show that a low-GI diet is not only better for diabetics than a high-GI diet of the same number of carbs, but is better than the much-vaunted low-carb diet as well. Or at the very worst, it’s the same as the low-carb diet. If we can show the low-GI diet to be equal or better than the low-carb diet for the dietary treatment of diabetes, then we can shut those low-carbers up for good.
Sounds reasonable for a study. But, remember, you are designing this study, and you are a reader of this blog. You know that a low-carb diet will hammer a 220 gm low-GI diet in terms of weight loss, lipid improvement, blood sugar stabilization, etc. It’s obvious that the money behind this study doesn’t want the outcome to be what you know it’s going to be. What to do? What to do? You wring your hands for a bit, then you hit on a brilliant idea.
You decide to put the subjects in the low-carb group on 200 gm of carbohydrate of their choice. Hey, now. 200 gm of carbohydrate is lower than 220 gm of carbohydrate, so it’s perfectly legitimate to call the 200 gm of carb per day diet a low-carb diet because it is lower. Right?
Problem solved. You’ve set up your study to show just what you want it to show.
Which is pretty much what the folks did who performed the study we’re going to discuss. Problem is, their study didn’t turn out exactly as they had planned.
Last year a paper with the ungainly title The Canadian Trial of Carbohydrates in Diabetes (CCD), a 1-y controlled trial of low-glycemic index dietary carbohydrate in type 2 diabtetes: no effect on glycated hemoglobin but reduction in C-reactive protein was published in the American Journal of Clinical Nutrition (AJCN). This paper describes a study pretty much like the imaginary one I described above.
Three groups of subjects were put on three diets: one group on a low-GI diet, another on a high-GI diet and the third on a low-carb diet. Those subjects on the low-GI diet consumed an average of 233 gm of low-GI carbs while those on the high-GI diet consumed 220 grams of high-GI carbs and those on the low-carb diet ate 196 gm of all kinds of carbs. As Dave Barry would write: I AM NOT MAKING THIS UP.
The subjects remained on their respective diets for a year and were then evaluated for differences in outcome. In almost every condition tested for – body weight, HgbA1c, lipids, blood sugar, blood pressure – ended up not different among the three groups. The only significant change was that the low-GI group ended up with lower C-reactive protein levels than the high-GI group with the so-called low-carb group right in between. But, though there was a statistically significant difference between the high- and low-GI groups in terms of C-reactive protein, the measurement was in the normal range for all three groups. So, the study doesn’t really militate in favor of a low-GI diet in any major way.
But, in my opinion, the study is so flawed that no conclusions should be drawn from it. Why? Because the data is so screwed up. And you know what they say about garbage in, garbage out…
When the initial dietary history was taken on the study subjects to determine their baseline diets in an effort to design the diets for the study period, the researchers determined that these subjects were consuming, on average, about 1900 calories per day. These were overweight people – both male and female – who were an average age of 60. All the data needed to make the metabolic rate calculations wasn’t available in the paper, but I was able to extract enough valid data to run the numbers I needed. And although my calculations aren’t exactly on the button accurate, they are, as they say, close enough for government work. (Which BTW is one of my favorite sayings, one that is positively pregnant with meaning.)
I’ll spare you the brain damage of how I did the calculations, but I applied the Mifflin-St Jeor equations (which are those deemed the most accurate) to the data at hand to determine the actual resting metabolic rates (RMR) of the subjects. It turns out that the RMR is about 1500-1600 calories per day. This is the RMR, which is the metabolic rate for lying in a bed barely breathing in a dark room. To get total metabolic rate, it’s typical to multiply the RMR by a factor depending upon activity level. For “seated work with no option of moving around and little or no strenuous leisure activity,” it is common to multiply the RMR by a factor of 1.4-1.5. Let’s assume that these subjects are doing a little more than sitting in chairs all day and go ahead and multiply by the 1.5. Doing so gives us a total daily energy consumption of from 2250 to 2400 calories, and that doesn’t include the thermic effect of food, which adds about another 10 percent.
Let’s go with 2300 calories, which is almost assuredly low. If the subjects of this study were claiming to consume only 1900 calories per day (and they weren’t dieting), anyone with good sense would say, ‘hmmm, something isn’t quite right here.’ These subjects could not be weight stable on the regular diets they were on at 1900 calories per day. They would have to be losing weight. Over the course of the study – which was a year long – they maintained their weight, they didn’t lose an ounce. In fact, they actually gained a little. If you multiply 400 calories per day (the amount of the deficit between what they claimed to be eating and their total daily energy expenditure) over 365 days, you would find that these subjects had an average caloric deficit of 146,000, which at 3500 calories per pound of fat should mean that they lost about 42 pounds each, but they didn’t lose any weight whatsoever; they gained. Something strange going on here.
Recall the people in Ancel Keys starvation experiment (which remember lasted only 24 weeks) got an average of almost 1600 calories per day and they became walking skeletons in just a few months. How could these subjects consume a mere 300 calories per day more for an entire year and not lose anything? The answer is that the data was incorrect. And should have been fixed right at the start.
So, basically, the study is worthless. It doesn’t tell us squat other than that people lie about what they eat and some researchers are gullible enough to believe them. Despite it’s worthlessness, however, this study inspired one of the big-names in mainstream obesity research – Xavier Pi-Sunyer – to use it to demonstrate his own bias or idiocy. You make the call.
In the same issue of AJCN that the above study appears there was also a commentary by Dr. Pi-Sunyer. In reading his commentary, I find it almost difficult to believe that so much mis- and disinformation can be accommodated in just a mere page and a half.
I had read the paper before I read Dr. Pi-Sunyer’s commentary, so I was pleased that right up front he got to the heart of the issue, which is the lousy data. If you’re going to have a study looking at the differences between diets, it seems pretty obvious that food intake is your most important data. This issue was not lost on Dr. Pi-Sunyer. After starting off with the obligatory summary of the paper under review, Dr. Pi-Sunyer cuts to the chase:
[The paper] shows the difficulty of accurately measuring food intake in overweight persons. The average body mass index (BMI; in kg/m2) of the group was 31, and the range was 24–40. The average caloric intake reported with the 3 diets at baseline ranged from 1810 to 1930 with an average weight of [approx] 84 kg and at the end of the study ranged from 1800 to 2020 with a slightly higher average weight. Patients did not lose weight; they actually gained. But, even if he or she is very sedentary, a person weighing 84 kg requires more calories than are reported in the study to maintain weight. Thus, the reporting of caloric intake by diaries is shown clearly to be inaccurate, and there is significant underestimation of energy intake. Such underestimation has been reported previously from this laboratory [Pi-Sunyer's] and by many others. The tools for measuring food intake in humans are very imprecise, as documented here. Yet investigators (and journals) persist in publishing such data as if they were accurate and persist in presenting percentages of macronutrients to one decimal place (see Table 3 in reference 1) as if there were any confidence in such decimals.
Dr. Pi-Sunyer clearly grasps the notion that the data is totally flawed. And I was heartened. But then he goes on as if a) the data were valid and b) that the tiny differences in carbohydrate intake are meaningful.
After noting that there was no difference found in the levels of HgbA1c between any of the study groups, he writes:
…compared with baseline data, these mildly diabetic type 2 patients actually did worse with regard to HbA1c and weight while following each of the 3 experimental diets. This finding suggests that we must be careful about disrupting subjects’ or patients’ diets with radical, doctrinaire changes that may actually be counterproductive. Furthermore, the diets had carbohydrate contents that varied from 39% to 52% of energy intake, and yet this variability had no effect on the subjects’ HbA1c. This finding confirms previous reports that the proportion of carbohydrate in the diet is not very important in determining the concentration of fasting blood glucose and that variations of 10% to 15% of total calories make little difference to overall control in patients with early type 2 diabetes
Here we have our first two pieces of dis- or mis-information. He says these data (that he has already deemed as useless) suggest “that we must be careful about disrupting subjects’ or patients’ diets with radical” changes that could make them worse. Say what? What’s radial about a diet that goes up a few calories and doesn’t do much of anything and obviously wasn’t followed anyway? This is subtle anti-low-carb talk. Then he comes out with the much less subtle anti-low-carb propaganda that this study (the one filled with meaningless data) “confirms previous reports that the proportion of carbohydrates in the diet is not very important…”
Indeed. Readers of this blog know what would have happened had the third group gone on a real low-carb diet instead of the 196 gm “low-carb” diet the study subjects actually (supposedly) followed. “Confirms previous reports that the proportion of carbohydrates in the diet is not very important..” Are these the same reports that have appeared in most of the major scientific journals showing the low-carb diet to be superior for weight loss, blood sugar stabilization, reducing blood pressure and normalizing lipids? Are these the reports he’s talking about?
Then writes he:
This report is unique in having followed subjects for 1 y and in using careful monitoring of the subjects’ diet and providing continued professional nutritional advice. It is interesting that the long-term results show that the 3 diets had little ultimate effect on either triacylglycerol or HDL-cholesterol concentrations. Thus, the arguments of the champions of a low-GI or a low-carbohydrate diet—that these 2 types of diets will result in lower triacylglycerol and higher HDL concentrations—have not been upheld in this careful, year-long dietary study.
Huh? “Careful, year-long dietary study.” Dr. Pi-Sunyer, are you talking about the crappy year-long dietary study that you just trashed for having lousy data? Or will lousy data do to promote your anti-low-carb bias.
He goes on and on dissecting out various tidbits of this finding and that, all of which are, of course, meaningless, but that doesn’t stop our friend from trying nevertheless.
At last Dr. Pi-Sunyer ends on a hopeless note:
Finally, for the proponents of a low-GI diet, the fact that these investigators, who are well known for their nutritional expertise, were able to provide a sustained difference in GI of only 8 units over 1 y attests to the difficulty of maintaining a low-GI diet over the long term. A realistic lower-GI diet that could be sustained in these patients with mild type 2 diabetes had no significant effect. Given the data from Wolever et al and the previous equivocal data with respect to this issue, it seems unwise at this point to burden type 2 diabetes patients with trying to pick and choose among different high- and low-GI foods.
“…these investigators, who are well known for their nutritional expertise…” They are so adept at doing nutritional research that Dr. Pi-Sunyer spends the entire fist third of his commentary trashing their work. Unbelievable.
Methinks Dr. Pi-Sunyer could almost be a contender for the Blackburn award.
But, sadly, such is the state of much published nutritional research these days.
Oh, and did I mention that Dr. Pi-Sunyer is on the scientific committee to determine the 2010 nutritional guidelines?














Red meat doesn’t give you cancer. A study told that people who eat more meat got more cancer. They could have eat more sausages with a lot of shit ingredients and they could have folloing the advice to use instable polyunsaturated oils in their cooking.
Red meat and butter is probably only good for your health.
A low carb high fat diet is the ultimate diet for all humans. That means less than 60gram carbohydrates a day. Look at Weston A Price to see what I’m talking about.
Guts and grease!!
I have a question though and I hope you will be able to answer it soon, though it is not entirely related to this topic … On a low carb diet – what do you eat when you have a stomach virus? There has been a bug sweeping through our community and it got me the other day. Basically, I fasted until the symptoms began to subside. When that happened, though, I did NOT feel like eating eggs, meat, or cottage cheese!!! I ate crackers, bananas, and some chicken noodle soup … I’m just wondering what you would recommend during illness that would be palatable to the ill stomach but still on the list of good food choices.
The best thing is to sip clear liquids at first (Gatorade light isn’t a bad choice), then I like to move on to bouillon.