I’ve read the paper that is the topic of today’s post from beginning to end five times. Not because it is a brilliant, enlightening paper, but because I found it so worthless I kept thinking there was something I was missing. If this paper had been published in the Journal of the American Dietetic Association or some other third tier journal I wouldn’t have thought so much about it. Were it published in a second tier journal such as Metabolism, I would have wondered a little more. But it was published in the October issue of the venerable American Journal of Clinical Nutrition, a first tier journal for sure, and, arguably, the most prestigious nutritional journal in the world.
I’ve decided to use this paper, Arterial endothelial dysfunction in baboons fed a high-cholesterol, high-fat diet to demonstrate how totally meaningless research can find its way to the best of journals thanks to a built-in bias among the “peers”? who are reviewing such research and to show how to interpret a scientific/medical article.
Critically reading a scientific paper is a piece of detective work. One has to discover motives, obfuscations, biases, and sloppy work and put it all together to get the real picture, not just the picture the author of the paper wants to be seen. Just like a good detective who assumes everyone is lying until stories are corroborated, so it is with the scientific literature. One must always corroborate, probe, compare and dig deeply because almost nothing is as it appears on the surface. As Sherlock Holmes says, “These are very deep waters.”? In the case of the study we will in due course explore, the waters are very deep indeed.
But before we start dissecting our study, let’s take a look at how almost all scientific articles are laid out.
Anatomy of a Scientific Article
Scientific articles are composed of the following seven parts: title, abstract, introduction, methods, results, discussion, and references.
Title
The first thing one sees when looking at a scientific article is the title, which is really a mini abstract. It is also the first place the biases of the author(s) can be manifest. When a researcher does a PubMed search the only information supplied are the titles of the various articles the search has identified. Since there are typically anywhere from a hundred to several thousand articles such a search retrieves, the title is the only idea the researcher has as to whether a particular article is of interest. Unfortunately there is no guarantee that the title is an accurate summary of the paper. It can be or it can be totally misleading. One I will always remember was from a paper published in the February 1996 issue of the American Journal of Clinical Nutrition entitled: “Similar weight loss with low- or high-carbohydrate diets.” This was a famous low-carb study done in a metabolic ward in a hospital, so the data are much more reliable than the more-often-used recall diaries. Subjects either followed a low-carb, high-fat 1000 Calorie diet or a high-carb, low-fat 1000 Calorie diet. After six weeks the subjects on the low-carb diet had dropped their blood glucose and triglyceride levels significantly and had reduced their fasting insulin levels by about 50 percent. Every parameter evaluated was significantly better on the low-carb diet except for weight loss. Although all the subjects lost weight, those on the low-carb diet lost a little more, but it didn’t reach the level of statistical significance. This article could have been entitled: “Vast improvement of triglycerides, glucose, and insulin levels on a low-carb diet.” The title would have been more accurate than the one they actually used and much less misleading. But it wouldn’t have fit the authors’ biases. (This is a wonderful article that can be downloaded in full from the above link; it should be read by all.)
Abstract
The abstract is a brief summary of the paper giving the methods, results and conclusions in abbreviated form. Since no one expects the abstracts to be lengthy, the authors can more or less pick and choose the parts of the research paper they want to emphasize while minimizing (or even failing to mention at all) those they would prefer to minimize. Unfortunately, many people read only the abstracts, and so get an incomplete–or even incorrect–impression of the substance of the research report.
Introduction
The introduction to a scientific paper is where the scope of the study is presented. Authors discuss why they chose to do the particular bit of research they did and what they expected to find. They identify other research on the same or similar subjects and identify gaps in the data that they hope their paper will fill. Reading the introduction to a paper gives one the first sense of what the authors are trying to do and a feeling for any biases the authors may have. For instance, if the introduction of a study says something along the lines of “LDL cholesterol is known to be a major risk factor in the development of cardiovascular disease”, it’s pretty obvious that the authors have bought into the lipid hypothesis, a factor that’s important to know in evaluating their study.
Methods and Materials
The methods section of a scientific article is the fine print. Sometimes literally. Many journals set the type for the methods section in a smaller font. It is also figuratively the fine print in that it is the section of the paper that almost no one reads, but that everyone should in order to really understand the paper. The methods section lays out in often tedious detail the protocols of the study and how the researchers actually went about doing the research. By reading the methods section one should have all the information one needs to replicate the study. Unfortunately, authors will often leave out of this section important information that should be listed and that would help immeasurably in understanding the study. In the paper we will discuss shortly there are major pieces of information MIA from the methods section.
Results
This section is where the researchers present the findings of their study. It is usually filled with charts and tables laying out all the data. It is important to study this section carefully in order to compare the actual data with how the researchers interpret this data.
Discussion
In the discussion section the researchers tell us what it all means. From their perspective. It is in this section that the researchers can lay the heavy hand of their bias on the reader. Often the researchers will want a result so strongly that they will resort to nonsensical, meaningless statements. I can’t tell you how many times I’ve read in discussion sections about data that was “trending towards significance.” Reading those words would lead one to believe that the data was meaningful, which is what the writer of those words desperately wants to be the case, but the data aren’t meaningful. A result is either statistically significant or it isn’t. Data can’t “trend toward” significance anymore than a women can “trend toward” being pregnant.
References
This section comes at the end of the article and contains all the citations for other papers referred to. This listing of studies would seem to be a pretty innocuous compilation of papers, but, unfortunately, it is often used to mislead. Many times papers are cited to prove a point, but when the actual paper is studied that point is nowhere to be found. Or sometimes the paper makes the opposite point. It is always good to drill a little deeper into the study by reading some of the supporting papers just to make sure they really do support whatever claims are being made for them.
Now let’s consider the study in question.
Starting with the title–“Arterial endothelial dysfunction in baboons fed a high-cholesterol, high fat diet”–we would assume that this study is going to show us that a high-cholesterol, high-fat diet causes endothelial dysfunction in baboons eating such a diet.
Let me digress a little and make sure we’re all on the same page with endothelial dysfunction. The endothelium is the single layer of cells that lines all the arteries in the body. The endothelium is highly metabolically active tissue constantly producing many substances that when released allow arteries to function properly to distribute blood throughout the body. The endothelium can send signals that make the arteries relax and reduce blood pressure or constrict and increase it; these cells can send signals to increase or decrease the accumulation of platelets, can send signals that attract inflammatory cells, and produce nitric oxide, a powerful vascular regulator. Anything that causes the endothelium to not perform its many tasks properly causes endothelial dysfunction. Endothelial dysfunction is thought to be the first step in the progression to overt vascular disease.
The title leads us to believe that a high-cholesterol, high-fat diet leads to endothelial dysfunction, at least in baboons. And, most people will connect the dots and conclude that baboons are primates and humans are primates, so therefore a high-cholesterol, high-fat diet will cause endothelial dysfunction in humans.
Moving along, what does the abstract tell us?
First, it tells us that
Endothelial dysfunction signals the initiation and progression of atherosclerosis.
I’ll buy that; that’s what we discussed above.
Then the abstract contains this strange statement:
Elevated LDL-cholesterol concentrations have been suggested to induce endothelial dysfunction, but direct in vivo evidence for the relation is still lacking.
In other words, we (the authors) believe in the lipid hypothesis, but realize that there really hasn’t ever been any direct evidence showing that it’s true. The implication is that this paper is going to remedy that.
The objective of the study:
We examined the hypothesis that a high-cholesterol, high-fat (HCHF) diet can directly cause endothelial dysfunction in vivo.
A summary of the results:
We found that the HCHF diet induced a high inflammatory status [my italics], as indicated by increased concentrations of interleukin 6, tumor necrosis factor (TNF-alpha), and monocyte chemoattractant protein 1.
And the authors’ conclusions:
Our results show that an HCHF diet can directly induce inflammation and endothelial dysfunction.
A quick read through of the abstract pretty much confirms what we’ve figured out just from the title alone. These folks are going to show us that a high-cholesterol, high-fat diet fed to baboons causes endothelial dysfunction, which is the first step towards the development of serious cardiovascular disease.
If you were like the many people and read only the abstract, you might be going around telling others that you just read a study that proves fat and cholesterol cause heart disease.
The introduction starts off with a pretty strange statement.
It is well known that not every individual with known atherogenic risk factors, such as hypercholesterolemia, develops atherosclerosis. Moreover, these known risk factors explain only 50% of the difference between healthy and clinically affected persons.
And
No studies have documented the interactions between circulating environmental risk factors [LDL-cholesterol and other lipids] for atherosclerosis and the arterial wall in living human subjects.
These researchers are throwing down the gauntlet. They are saying, the lipid hypothesis has never been proven, but with this paper, we may change all that. We’re gonna by God show that eating a high-cholesterol, high-fat diet will cause endothelial dysfunction.
When we get into the Methods and Materials Section we start to get our first hint that a little fudgaroo is going to take place.
Although the intent of the in vivo experiments was to identify functional changes in ECs [endothelial cells] after an atherogenic diet [presupposes here that a high-fat, high-cholesterol diet is atherogenic, alerting us to their bias], such experiments cannot determine the specific components that are responsible for these changes. Therefore, we also designed an in vitro challenge experiment.
Translation: We didn’t get the results we were looking for in the original study, so we decided to torture the data a little until it gave us the confession we wanted. In this case they didn’t really torture the data, they decided to fiddle with the baboon endothelial cells until they got a result they could live with, one that fit their bias. I’ve elected to ignore the “in vitro challenge” from the scope of this blog because I think it was a desperation measure done to have something to report despite the fact that it was meaningless.
Here is another admission buried in a large paragraph:
The baboons selected for this study were previously classified as high responders because their serum LDL-cholesterol concentrations increased after an HCHF diet.
The baboons chosen for the experiment were those that had previously been shown to respond to a high-fat, high-cholesterol diet by increasing their levels of LDL-cholesterol. Hmmm.
Here is the most stunning piece of information of all. It is stunning by its omission of probably the single most important piece of information needed to intelligently analyze this study.
The baboons had been fed a normal unpurified stock diet for monkeys (SWF Primate Diet; Harlan Teklad, Madison, WI) for >40 wk before the 7-wk HCHF dietary challenge. Both diets were fed ad libitum.
There it is. This is a study about how a diet affects the endothelium, and, with the exception of one table, you have just read the entirety of the information provided on the diet provided. I searched the Harlan Teklad website in vain for a description of the SWF Primate Diet. The table (which if I were a more skilled blogger could include for you here) shows the macronutrient differences between the two diets, but doesn’t tell anything about the composition. Was the added cholesterol oxidized? Were there trans fatty acids. Was there high-fructose corn syrup? Sucrose? We don’t know because the authors decided not to include this monumentally important information in a nutritional/dietary study.
The table does show one interesting fact that isn’t commented upon in the paper. The high-cholesterol, high-fat diet contains 23% more calories per gram than does the baseline diet. And since the animals consumed the diet ad libitum, does this mean that on the HCHF diet they ate 23% more? We don’t know. But if they did consume more, how do we know that whatever abnormalities are found aren’t a consequence of excess calories and not of the cholesterol and/or fat. Don’t know; can’t tell.
In my mind this one giant omission invalidates the entire study.
To continue. After having been fed the SWF Primate Diet (whatever that is) for 40+ weeks the baboons are switched to a high-cholesterol, high-fat diet. The day of the dietary changeover the researchers take blood from the baboons and surgically remove a segment of their left femoral arteries (the femoral artery is the large artery running through the thigh) to provide a baseline for any changes that might take place. Three weeks after starting the high-fat, high-cholesterol diet researchers once again draw blood, then after a total of seven weeks on the high-fat diet a final blood sample is taken and a segment of the right femoral artery is harvested to compare with the left one removed seven weeks earlier.
What did the researchers find?
The HCHF diet substantially changed the lipoprotein profile in the baboons.
How?
Total cholesterol concentrations were dramatically increased (85.6%; P < 0.0001) after 3 wk of the HCHF diet.
Okay, but is this a surprise? These baboons were chosen for this study because they responded to such a diet by increasing their cholesterol levels. But how do we even know that this cholesterol elevation wasn’t a caloric phenomenon and had nothing to do with the added fat and cholesterol in the diet?
The authors said the HCHF diet substantially changed the lipoprotein profile. What else changed besides total cholesterol?
We observed a significant increase in serum oxLDL-cholesterol [oxidised LDL-cholesterol] concentrations at the 3rd week of the HCHF diet compared with basal concentrations (54.9%, P = 0.03), but the concentration increase at the 7th week of the dietary challenge (19.6%) was not significant (P = 0.11). No statistically significant changes in the concentration of triacylglycerols [triglycerides] or the ratio of total cholesterol to HDL cholesterol were observed during the HCHF dietary challenge.
So we had a little blip of oxLDL-cholesterol at the three week mark, which could easily be attributed to an acute change in the diet, but nothing after the seven weeks? No changes in triglycerides, no change in the cholesterol/HDL ratio? I thought these guys said the HCHF diet substantially changed the lipoprotein profile. Other than a change in total cholesterol, which really isn’t part of a lipoprotein profile, it doesn’t look to me like it changed it at all.
Did the diet cause any inflammatory changes? Any changes in antioxidant status? C-reactive protein?
Our results showed that the following 4 patterns of proinflammatory changes occurred when baboons were fed the HCHF diet: 1) MCP-1 concentrations significantly increased and remained high until the end of the 7-wk dietary challenge; 2) IL-6 and TNF-alpha concentrations were dramatically increased after 3 wk of the dietary challenge but decreased to below baseline concentrations by the end of the challenge; 3) soluble TNF-alpha receptor II concentrations were significantly lower than baseline concentrations after 7 wk of the dietary challenge but not after 3 wk; and 4) the total antioxidant status and the concentrations of IFN-gamma , IL-1β¸, and CRP did not change significantly during the entire dietary challenge period. The concentrations of other endothelial dysfunctional markers, ie, soluble VCAM-1 and soluble ICAM-1, did not change significantly during the full 7-wk HCHF challenge.
So, other than a doubling of MCP-1 nothing happened. No inflammatory changes whatsoever. No changes in antioxidant status. No changes in CRP (C reactive protein).
What is MCP-1? MCP-1 (monocyte chemoattratant protein 1) is a cytokine that attracts monocytes (a type of white blood cell) to an area in response to an infection other immunologic insult. What does it mean in this study? Who knows since we don’t know what the caloric content of the high-cholesterol, high-fat diet was. It’s unlikely to be much of a factor, however, since virtually no other inflammatory parameters were changed. Generally during inflammation a whole host of markers change, not simply a single one while all the others remain normal. Since a couple of other cytokines (IL-6 and TNF-alpha) increased at the three week mark but returned to baseline or below after seven weeks it is possible that MCP-1 has a longer trajectory back to normal and if evaluated after, say, ten weeks, would also have returned to baseline. This, of course, is speculation on my part, but certainly reasonable given that no other measured cytokines were out of line after the seven weeks.
How about changes in the arteries themselves?
We examined the femoral artery macroscopically and microscopically; no atherosclerotic lesions, including early fatty streaks, were observed after the 7-wk HCHF challenge.
So, let’s see what we’ve got. After 7 weeks on an indeterminate high-cholesterol, high-fat diet the baboon subjects that were chosen because they had already proven to be hyper responders to fat and cholesterol in the diet increased their cholesterol levels and had a doubling of MCP-1. That’s it. No other lipid changes, no changes in other inflammatory parameters, and no visible (by either eyeball or microscope) changes in the arteries. We can attribute the elevated cholesterol levels to the fact that these baboons were chosen specifically because of their tendency to increase their cholesterol levels when following a high-fat, high-cholesterol diet.
So, what it all boils down to is nothing much at all.
But, you wouldn’t know that by reading the Discussion Section of the paper.
Consistent with our previous observations [when they determined which baboons were hyper responders], the HCHF diet can effectively increase total blood cholesterol and non-HDL cholesterol concentrations in baboons.
Our results show that the HCHF diet also results in gross systemic inflammatory responses, as indicated by markedly elevated cytokine concentrations.
Oh really?
After several paragraphs explaining why there were no elevations of all the cytokines and other inflammatory markers they were looking for, the authors conclude with
In summary, our results clearly show that the HCHF diet resulted not only in increased serum cholesterol concentrations but also in inflammation and endothelial dysfunction.
So they would have us believe.
As Huang Po, a Buddhist teacher from long, long ago put it:
The foolish reject what they see, not what they think. The wise reject what they think, not what they see.
The authors of this paper saw very little conclusive data in this study. It’s pretty obvious what they thought.