I have a great friend who was a Pulitzer Prize nominated Wall Street Journal investigative reporter for over a dozen years. Based on his experience there, he occasionally gives two-day seminars on how to deal with the press. He always starts out on day one by asking the question, What is news?
What is news? Think about it and try to come up with your own answer before reading on.
Beef_cuts
Participants in these seminars come up with all kinds of answers. Just about all are variants of one theme: News is the reporting of current events that are of great importance or interest to citizens.
Wrong answer.
According to my friend, news is whatever the news reporters determine is news and decide to report.
I agree with him. Which brings me to the recent study fingering the consumption of animal protein as an accelerated trip to the graveyard.
Thousand of scientific articles are published each month. I always wonder what it is that drives journalists to reach into this haystack and pluck out one particular straw of an article to write about. Surely there are numerous papers of importance, so why do they all seem to grab the same one? Invariably, whichever one journalists deem newsworthy appears in every news report throughout mainstream media land. The recent protein paper was so honored. It ended up getting reported on everywhere.
What I don’t wonder about is what is going to happen to me when journalists decide these kinds of papers are newsworthy. My Twitter account and the comments section on this blog are going to get blown up by people wondering what I think it all means.
In the early days of my blogging career (it’s been almost ten years now), I relished tearing into these sorts of papers and presenting them as the rubbish most really are. Or at least pointing out the major shortcomings.
Now that I’ve ripped apart a ton of them, I view them as a major pain in the rear.  Why?
Robb Wolf said it best in his response to this anti-protein paper:

I know that part of my job is to act as an interface between the science/medical scene and the folks who do not have a medical background. I take that role seriously but I’m not a fan of Groundhog Day. Well, the movie was kick-ass, but living it is not a party.

Robb is exactly right. Each time a paper like this one comes out, it is like Groundhog Day* for those of us who do the scientific translation of these kinds of papers from medicalese to regular English and tell why they aren’t all the media cracks them up to be. It’s like Groundhog Day because in a week or a month or two, there will be another paper just like it, and all the pleas for us to opine will start again.
Even with my rapid reading capabilities, it takes me at least three hours** to read one of these papers critically. I have to read, make notes and pull at least a half dozen – often more – articles (and read those) to see if they are really confirming whatever point the author is trying to make. Then I have to cogitate on it a bit before I start writing my rebuttal. My written response takes, depending upon the complexity of the article involved, anywhere from three to six hours. So, the time commitment to do a comprehensive review of a paper such as the one in question, and do it right, involves a serious commitment of time.
I don’t mind doing these reviews — and I’m sure Robb doesn’t either — if the paper I’m dealing with is half way decent. Problem is, all the papers the media seems to seize on a publicize are the same kinds. Red meat causes heart disease; red meat causes cancer, saturated fat clogs arteries, dietary fat causes obesity, yada yada yada. Same song, second verse. It’s like Hercules fighting the frigging Hydra. Lop off one head, and two more appear. All these kinds of studies have been refuted so many times, that it’s almost pointless to do it again. Yet they still keep popping up like, well, Groundhog Day.
But those in the media seem to think each time one of these turkeys is published that it’s a red letter day and some brand new scientific truth has been released to the masses. (Sadly, even the staid Wall Street Journal succumbed to this one. At least they had sense enough to pick up that it was really two studies.)
Anyway, you get the picture.
With all that said, there are a few out there who aren’t as jaded as I on this sort of thing, who have a lot more energy, and who have taken the time to appropriately dissect this paper.
I linked above to the assessment of the almost-as-jaded-as-I Robb Wolf.
Denise Minger did her typical thorough job of it as well.
I’m particularly glad she wrote the following about the NHANES data used in this study:

And it gets worse. While it’d be nice to suspend disbelief and pretend the NHANES III recall data still manages to be solid, that’s apparently not the case. A 2013 study took NHANES to task and tested how accurate its “caloric intake” data was, as calculated from those 24-hour recall surveys. The results? Across the board, NHANES participants did a remarkable job of being wrong. Nearly everyone under-reported how many calories they were consuming—with obese folks underestimating their intake by an average of 716 calories per day for men and 856 calories for women. That’s kind of a lot. The study’s researchers concluded that throughout the NHANES’ 40-year existence, “energy intake data on the majority of respondents … was not physiologically plausible.” D’oh. If such a thing is possible, the 24-hour recall rests at an even higher tier of suckitude than does its cousin, the loathesome food frequency questionnaire.

Most of us in the nutrition biz have known the government run and funded NHANES data are pretty worthless, but the recent paper Denise links to, Validity of U.S. Nutritional Surveillance: National Health and Nutrition Examination Survey Caloric Energy Intake Data, 1971–2010, shows just how worthless.
Weighing in from across the pond, Zoë Harcombe wrote not just one, but two posts about this study.
Animal protein as bad as smoking?!
Headlines based on 6 deaths!
She points out one of the most common tricks in the book used in studies like these. If you can’t find an overall correlation between whatever the risk factor is your testing and an overall outcome, start breaking up your data by age or some other factor until you can show a correlation for some subset. It’s called torturing the data until it confesses. Then use the confession extracted to get your headlines.

After finding no overall association, the researchers spotted a pattern with age and split the information into participants aged 50-65 and participants over 65. They then found (direct quotation again): “Among those ages 50–65, higher protein levels were linked to significantly increased risks of all-cause and cancer mortality. In this age range, subjects in the high protein group had a 74% increase in their relative risk of all-cause mortality (HR: 1.74; 95% CI: 1.02–2.97) and were more than four times as likely to die of cancer (HR: 4.33; 95% CI: 1.96–9.56) when compared to those in the low protein group.”

In her second post, she homes in on the fact that the authors used as a baseline the small database of 6 deaths in one group over an 18 year time period.

Here we find the real headline. What the researchers didn’t want us to find out. The “four times more likely to die” global headline grabber was based on a reference group of six deaths. Yes six deaths. And not just six deaths –  but six deaths over an 18 year study. And the ‘researchers’ tried to claim that animal protein is as bad as smoking based on this?

She goes on to discuss the folly of making large claims based on small datasets.
The folks at Examine.com, whom I don’t know from Adam, did an excellent review of the study.
High protein diets linked to cancer: Should you be concerned?
The author points out that this study is really two studies, not one.

First, it should be mentioned that to fully appreciate this study we must view it as two studies. There is an epidemiological study and there is a mouse intervention study; anytime tumor growth is mentioned, it refers to the mouse study, and causation can only be applied to the mouse study. It cannot be applied to the human study (as it is an epidemiological study).

This, as you might remember, is a technique used by T. Colin Campbell in his book The China Study. Mix and match data about humans and rodents, use the pronouns as if it all applies to humans, and confuse the heck out of your readers. Except the readers don’t think they’re confused. They think they’re reading about human studies.
It’s important to note in this study on animal protein that since all the data about humans comes from observational or epidemiological studies, it shows only correlations. Not causality.
And the actual experimental part of the study was done on rodents and applies to rodents, not humans. And the tumor studies were done not with tumors the rodents developed during the course of their little natural lives, but were done on tumors implanted by the researchers. The data gathered is interesting but far from being applicable to humans.
But the casual reader of this and similar studies confuses the rodent data with the human data. Most of the main stream media certainly did.
Even if this study were done by experimentation on humans (which would be unethical), the results are meaningless unless they can be repeated by other groups of scientists.
Typically when studies showing highly significant results are repeated, the findings aren’t nearly as robust in the follow up studies, and, in many cases, fall off with repeated studying leading to the conclusion that the first study was really an outlier and the findings came in as they did by chance.
Never, ever rely on just one study to prove anything.
I’m going to keep this post at the ready, so that the next time one of these studies gets plastered all over the mainstream media, and a hundred people email and tweet me about what it all means, I’ll send them a link.
As always, if you disagree with my take or if you want to put your own spin on it, please do so through the comments section.


*Apropos of absolutely nothing having anything to do with this post, I feel compelled to tell you that Danny Rubin, the screenwriter who wrote Groundhog Day, is a friend of mine. We met years ago when MD and I still lived in Arkansas but traveled often to Santa Fe. When we finally moved to Santa Fe, we lived close to Danny, his wife Louise and their two kids, who all lived in a big adobe on Garcia Street hard by the Downtown Subscription (where we always walked for our daily coffee fix) and the Garcia Street Books (still my favorite bookstore – hope it’s still there). Danny has a new book out about the writing of Groundhog Day, which I’ve pulled down on my Kindle but haven’t read yet. He is one hilarious and creative guy, so I’m sure his book is the same. He now teaches screenwriting at Harvard and lives in MA.
** Often more than three hours. I am a reviewer for a scientific journal, and recently I spent about 12 hours trying to make sense out of a submission. I finally threw up my hands and recommended rejection. If I can’t understand what the authors aren’t getting at in 12 hours, it shouldn’t be published. But it probably will get picked up by another journal. The point is, many of these articles take a lot of time to study to try to figure out what’s going on. Three hours is probably the minimum.

58 Comments

  1. Here are a couple Comments from Robb’s QandA:
    Someone called “Tim” says
    I’m so confused on how your Tribe can down play and disagree with all of these studies. Have you done any research on any of this? Just wondering. I know it affects business so Maybe that’s the issue.
    Reply
    Robb Wolf
    March 4, 2014 at 8:13 pm
    Tim, have YOU read the whole paper? I did. It’s the same bollocks as always. http://robbwolf.com/2012/03/14/red-meat-part-healthy-diet/
    Sorry that twists your panties.
    Reply
    Robb Wolf
    March 5, 2014 at 9:52 am
    Tim!! Glad you could make an ass of yourself on this. The study author has a plant based protein company: http://www.zoeharcombe.com/2014/03/animal-protein-as-bad-as-smoking/
    Any additional pithy comments? Maybe you should take some time and read this blog, listen to the podcasts, hm?
    Reply
    Permalink

  2. A point that occurs to me is that the study, such as it was, did not relate mortality to the amount of protein consumed on that fateful day 18 years ago.
    It was about the % of energy from protein.
    This means that there is no dose, and no dose response.
    We don’t know that someone eating a higher % protein actually ate more protein.
    Dose-response curve is one of the ways that we can test the credibility of observational data. But that is not possible when the actual dose isn’t any part of the data.

  3. Yet another terrific post.
    Peter, as usual, has an interesting layer to add on the breathtaking cynicism of these researchers:
    http://high-fat-nutrition.blogspot.com/2014/03/would-you-like-soya-oil-poured-over.html
    As a trained vet, one thing Peter can elaborate on is the fact that not all mice are alike – and for these experiments, these researchers are specifically selecting genetically broken mice that die on low-carb diets.
    (And that’s amazing that you know Danny Rubin. There’s no better screenwriter.)

    1. I read most of Danny’s book last night after putting up this post. Really good.
      Peter always has something interesting to say.

  4. You asked us to post if we disagree. Well, I strongly disagree with your use of the word “d’oh”. While from a purely etymological perspective, there are precursors that could be spelled that way, I think we all know that the true modern usage is “d’oh!”, with an exclamation point.
    I dare you to attempt to disprove the following overview of the subject:
    https://www.youtube.com/watch?v=khSIYmTzt6U
    Well, I know you meat-eaters think nothing of distorting the truth, and sneaking this “d’oh” into your blog article, surrounded by so much otherwise irrefutable logic, is particularly shameful and disingenuous.
    If you are really honest, do you hear even one mere “d’oh” in the above citation? Every single one of them qualifies as a full “d’oh!” — and a good faith argument could easily even be made for “D’oh!”.
    I rest my case.

  5. David Perlmutter seems to find some value in the study, writing on his blog (http://www.drperlmutter.com/dangerous-meat-dairy/), “this study basically uncovers the danger of eating animal products now purveyed in America. Grain-fed animals produce meat and dairy products dramatically higher in dangerous, inflammation producing omega-6 fats, while they are deficient in healthful, heart-protective, immune-enhancing, brain-friendly omega-3s.” Is this compatible with your analysis of the study, or is he being generous in his interpretation?

    1. I think he’s being very generous. Especially since it’s a myth that grain-fed animals produce meat loaded with omega-6 fats. You get more omega- in a handful of almonds than you do in a large steak, and no one decries you if you eat the almonds.

      1. During WW2 British cattle were grain-fed on imported grain to increase milk production, and as far as I can tell grain-feeding has been part of cattle farming in Canada and the US for 100 years or more.
        No-one complained about this (and the experts knew a lot more about nutrition during WW2 than they do today).
        It is probably more relevant that cattle are now fed soy, palm kernel expeller, and other junk as well as the grains, and have access to grass and silage less often.

      2. Perlmutter confuses me, he wrote some great stuff in Grain Brain, but in interviews and stuff he seems to back-track and talks about how he’s not advocating a meat/dairy heavy diet, and seems to think vegetarianism is a good thing. Dunno.
        As to the grain vs grass feeding of cows, I’ve got a ton of resources folk can sift through about that here:
        http://highsteaks.com/forum/health-nutrition-and-science/beef-grass-vs-grain-fed-factory-farming-cafo-sustainability-126.0.html

  6. I don’t think anyone has ever studied this, but I think Scandanavians eat more smorgasbord than Kenyans. Thus eating smorgasbord causes blond hair.

  7. Curious. A mouse intervention study mingled with a human epidemiological study. Must be quite a trick to get those mice to smoke a pack a day.

  8. “According to my friend, news is whatever the news reporters determine is news and decide to report.”
    Having participated in events that I later saw covered in the “news,” I have often been struck by the realization that had it not been for the time, date, and location, I would not have recognized the event as being the one I witnessed just from description. Same for articles on which I have actual expertise.
    I have drawn the conclusion that most reporters choose that profession simply because they can’t possibly make a living any other way.

  9. I think that regarding an observational “study” as being a real study is a fundamental logical error.
    As a friend of mine once said, “If you have a correlation without a a clearly demonstrated underlying mechanism, you don’t have a study. At best, you have a grant proposal.”

  10. For me, the usefulness of these seemingly tedious assaults on animal protein is to remind me that there are countless practicing physicians (my colleagues) who are still swilling the kool-aid on fat, protein, and their effects on human health.
    The chair of an upcoming local medical conference on Treatment of Pre-Diabetes incredibly asked me if this study challenged my low-carb approach at all. Reminding me that a hefty proportion of the audience for my talk will be persistently delusional.
    The Groundhog Day analogy helps me take in all of this with a smile, telling myself to take a Beginner’s Mind approach to each new audience.

  11. Is the entire paragraph beginning, “And it gets worse” a quote from Denise Minger? If so, maybe that could be made clearer.

  12. Does the guy who wrote Groundhog Day talk much about the timespan and themes and stuff?
    On Reddit on a weekly basis there’s another Today I Learned about the movie which is usually to do with it supposedly lasting 10,000 years and that it has a ton of stuff to do with Buddhism.
    Plenty of examples of what I’m talking about here:
    http://www.reddit.com/r/keto/search?q=groundhog+day+til

    1. These people on Reddit need to read Danny’s book so they can quit speculating. He cranked the first draft out in four days, Harold Ramis got the studio to pick it up, then Danny, Harold Ramis and Bill Murray fiddled with it until it became what it is. There is a copy of the original screenplay in the book along with many copies of notes and changes along the way.
      BTW, what does the TIL mean in the front of each of the Reddit entries? I’m a Reddit virgin.

      1. According to a website devoted to Reddit it means “Today I learned” –
        I feel totally overwhelmed by the amount of info available on the web – maybe it’s not a bad thing to be “a virgin” about some of it – if not a downright luddite!

        1. Based on what I read in the screenwriter’s book, the majority of these people hadn’t learned anything.

  13. According to Lord Northcliffe, a British publisher active in the early years of the 20th century, “News is what somebody somewhere wants to suppress; all the rest is advertising.”
    Would seem applicable to this particularly harebrained study. . .

  14. Dear Dr Eades,
    I don’t know weather you caught it, but the extra data in Zoe’s second mail lets us calculate TotMortality/studyYear. The rates given are a bit misleading, since the LoProt group was not only the smallest, but the observation time was also the shortest. Enough to make a difference.
    Group Total mortality /y Cancer mortality /y
    ———————————————————————————–
    LoPot, younger: 1. 54% 0.23% (misdiagnosis?)
    MedProt, younger: 1.53% 0.60%
    HiProt, younger: 2.07% 0.79%
    ———————————————————————————–
    LoProt, older: 6.00% 1.51%
    MedProt, older: 4.48% 0.99%
    HiProt, older: 5.07% 0.63%
    The one figure not fitting with anything else in the table above is the one based on 6 casualties. Nothing explains the almost 10-fold increase after 65 when everything else looks smooth and similar. Could there be misdiagnosis in the cause of death (maybe unintentional). Death may be caused by several issues, and the cause may not be diagnosed at all by medicals…
    Cheers and thanks again for your blog!
    LeenaS

    1. Who knows? When it’s all this unclear, it’s not worth it to me to try to ferret out what the author’s are trying to say. Thanks for the effort.

    2. If you scrutinize this data carefully, you see that the younger group with 1.54% mortality were “low pot.” Maybe that’s the key. We need a new analysis that controls for weed.
      This is beginning to remind me of the search for the Malaysian plane. Much analysis but little real data.

  15. Kudos for even bothering to address this topic (again).
    What surprises me even more than your willingness (for the time being) to indulge the requests for your opinion every time one of these scare tactic headlines comes out, is that people continue *asking* for your commentary. As if the latest “animal foods = instant death” blurb on the 6 o’clock news might persuade you to call it quits on 3 decades of saving lives with a low-carb diet that embraces red meat.
    Not that I don’t value and enjoy your commentary, because I very much do. Just have to shake my head when people like you, Robb Wolf, and a host of others get pinged to comment…as if people don’t already know what your likely response would be.

  16. This is why those of us who bust their heads into science every day need to keep doing our homeworks so that bad science or bad purposes would not affect the general unexperienced reader. It’s sad that many people still fall for these so called “studies” everyday.
    Good response Mike. I appreciate you!

  17. Colin Campbell sounds like a real study. And he’s less than an hour away (Ithaca, NY). Say the word and I’ll drive to his house and nail him on the back of the head with a wet snowball. Yeah we had a snowstorm yesterday but it’s rapidly melting. 🙂

    1. Be careful or you might be accused of a hate crime. For all I know, vegans are a protected minority. 🙂

  18. Observational and ecological studies are generally used to determine the presence of effect of cancer risk-modifying factors. Researchers generally agree that environmental factors such as smoking, alcohol consumption, poor diet, lack of physical activity, and low serum 25-hdyroxyvitamin D levels are important cancer risk factors. This ecological study used age-adjusted incidence rates for 21 cancers for 157 countries (87 with high-quality data) in 2008 with respect to dietary supply and other factors, including per capita gross domestic product, life expectancy, lung cancer incidence rate (an index for smoking), and latitude (an index for solar ultraviolet-B doses). The factors found to correlate strongly with multiple types of cancer were lung cancer (direct correlation with 12 types of cancer), energy derived from animal products (direct correlation with 12 types of cancer, inverse with two), latitude (direct correlation with six types, inverse correlation with three), and per capita gross national product (five types). Life expectancy and sweeteners directly correlated with three cancers, animal fat with two, and alcohol with one. Consumption of animal products correlated with cancer incidence with a lag time of 15-25 years. Types of cancer which correlated strongly with animal product consumption, tended to correlate weakly with latitude; this occurred for 11 cancers for the entire set of countries. Regression results were somewhat different for the 87 high-quality country data set and the 157-country set. Single-country ecological studies have inversely correlated nearly all of these cancers with solar ultraviolet-B doses. These results can provide guidance for prevention of cancer.
    Grant WB. A multicountry ecological study of cancer incidence rates in 2008 with respect to various risk-modifying factors, Nutrients. 2014;6(1):163-189. http://www.mdpi.com/2072-6643/6/1/163

    1. Thanks for the citation and abstract.
      As I’m sure you know, correlation is not causation. I just don’t want others to be confused.

  19. What makes the news — the best “press release ” headline grabber. Mainstream Media (MSM) seems to rely on creative press releases and International wire services. I don’t like this type of “journalism”. Damn the real science -what is the best headline. It was obvious that this headline came from a well crafted press release as it appeared several days before I could find it on PubMed.
    Here is one that appeared during the same time frame, that received zero press in MSM,http://www.sciencedaily.com/releases/2014/03/140311163101.htm
    My concern is with the dilution of credibility for “peer review literature”. I may be old, but “back in the day”, this study woruld have had trouble getting out of my department, let alone getting published. I also doubt they attempted to present it at a National Association Meeting. Where colleagues could tear it apart. I get concerned with, who is picking the “peers” to review submissions? I blame the Journal for shoddy practices.
    I always appreciate your critical reviews and perspective.

  20. Since empirical data in the hands of disreputable folks can be massaged to say almost anything they want it to say when it is taken out of context or lifted out of an improperly designed study, I always try to find out more about those who are responsible, especially in the case of studies that are quoted in such sensational ways as the blatantly anti-meat fiction being discussed here. The most likely suspects (a priori) in my mind would have to be the politically active groups that have made their anti-meat objectives public, but the money trails that have been able to implicate such groups in the past are not as easy to find these days. The political climate has become extremely difficult for meat producers, some of whom are in my family, and is an unjust and even criminal situation given the incorrect science used to justify the persecuting propaganda raining down on producers and the general public alike. This injustice needs to be combatted and stopped as a matter of public health as well as a matter of curbing a real impediment to the advancement of scientific knowledge. Thank you for your rational rebuttals to these irrational attacks on science and our food.

  21. Thanks as always Mike! I don’t know what we’ll do if you burn out. Since I read your blog on Metabolism and Ketosis, I have lost 120+ lbs (from 278), lowered my CRP from a deadly 3.55 to a mere 0.55, and watched as slowly things are returning to normal. I rolled my eyes when I read this news piece, and was expecting the usual BS, and so never even bothered to check your blog to see if you weighed in, but am real glad to see you did! I just finished reading the Slow Burn book you did, and hoping to give it a try because I still feel the insulin resistance at work. Do you see how HUGE your efforts have been for me? I clicked with you ever since I first saw you in Tom’s movie “Fat Head”. At that time, I remember thinking, who the heck is he? Well, I certainly know who you are NOW!

    1. I’m glad to hear you’ve done so well. Hope you enjoy the Slow Burn regimen. I do it all the time.
      Thanks for writing.

    1. I have seen it and other articles like it. I love coconut oil, but I always take these kinds of pieces with a major grain of salt.

  22. Mike, I think your analogy to fighting the hydra is excellent, but, to riff off of the Groundhog day theme, I think the more fitting analogy would be like playing whac-a-mole.

    1. Hey Aaron
      Good to hear from you. I agree re the whac-a-mole. The only reason I brought up Groundhog Day was because Robb used it, and it is sort of appropriate because every time I turn around, there is another one of these to refute. Sort of like Bill Murray waking up to the Sonny and Cher song every morning.

  23. I tend to sometimes take these supposed “scientific studies” with a grain of salt. essentially because there’s so many of them that to really take them all seriously would pretty much leave you cooped-up in your room eating nothing but avocados. Life is too short. Let’s get real!

  24. I like your friend’s definition. Why are people going to say something in the newspaper is news? Because it’s in the newspaper!
    It reminds of George Costanza on “Seinfeld,” when he was pitching a TV show to executives. The executives wanted to know why anyone would watch his show, to which he replied, “Because it’s on television!”
    Make sense!

  25. With the amount of studies like this that pop up, it’s hard to know what to believe.
    A good read none the less!

  26. Good writeup. To add to your opening lines, not diet specific, but isn’t there a “bigger picture” here with regards to media distortion in the news? Specifically, *why* is the media repeatedly presenting this distorted view? I can understand that the science skills of journalists can be very lacking, but is it just coincidence that they err on the side of the “low fat orthodoxy”?
    What concerns me is that surely it isn’t just nutrition reporting that is this shoddy? For example, I wonder about the reporting of economics studies. There too, is a field, like nutrition, based on dogma and ideology where the academics writings are little understood and accepted uncritically. So long as they are in line with the existing orthodoxy, which, similar to the food industry, often appears to be supporting the power structures, rather than seeking impartial truth.
    What *can* we believe?

    1. I’m with you. Once I found out how off base medical reporting is on diet and how the mainstream has it so wrong, I’ve begun to worry about the theories held sacred by all branches of medicine.

Leave a Reply

Your email address will not be published. Required fields are marked *