When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

PLOS BLOGS Absolutely Maybe

Saturated Biases: Where the AHA Advice on Coconut Oil Went Wrong

 

Cartoon in a restaurant

 

Even for an evidence food fight – and those get plenty heated – the hue and cry about the new American Heart Association (AHA) advice on coconut oil has been quite something. “We advise against the use of coconut oil”, they wrote, and boy, it was ON.

A quick recap in case you don’t know what the fuss is about. In 1961, the AHA called for us all to swallow less saturated fat to reduce our risk of cardiovascular disease (CVD – heart/blood vessel disease and stroke). By the 1970s, whole milk consumption in the US was plummeting.

The diet and economies of developing nations big on palm and coconut oil – both saturated vegetable fats – changed course. The backlash against fats helped fuel the increase of carbs that in turn contributed to rises in obesity and diabetes – both of which increase CVD risk.

In 2013, the AHA took a stand again: in favor of “a dietary pattern that….includes…low-fat dairy products,…nontropical vegetable oils…” and a belated warning about carbohydrate increase as a potential adverse effect – with no specific shots fired at coconut-eaters. But along the way there were some giant plot twists: coconut-everything became a trendy “superfood” – and the “butter’s not so bad after all” camp grew and got public traction.

So why the new statement? Because, they write, “discordant conclusions” from meta-analyses have “created confusion among patients, their physicians and the public”. (Explainers I’ve written about meta-analysis are here and here.)

I’m going to list 4 reasons I believe the AHA gets it wrong on coconut oil. They are really dimensions of a single problem, though: not applying careful and transparent scientific methods to get on top of bias. In a hot zone of conflict, that’s a recipe for more conflict and confusion, not less.

 

 

Of its 24 pages, the AHA report devotes more than 1 to disclosing conflicts of interest of the members of the writing panel (but not others), and 6 pages to references. But there’s only half a page reporting on their methodology of choosing, analyzing, and drawing conclusions from those references, addressing very little of what we need to know. Impeccable credentials can’t substitute for strong scientific methods and reporting. I think we’re all particularly biased when it comes to what we eat and drink, and it’s a notoriously difficult topic to get reliable research results on, too. We all need guardrails when we barrel down these roads.

Systematic reviewing is a painstaking, methodical process you go through to try to minimize your biases in assessing evidence, including meta-analysis. The US National Academy of Medicine has standards for developing trustworthy guidelines that apply those principles. The GRADE Working Group does too, including how to minimize bias in going from evidence to recommendations. (*Disclosure: I was a member of the GRADE group when these were developed.) Following a rigorous process along these lines would at least have led to a more transparent report.

 

1. Inadequate search and method of choosing studies

 

Cartoon about considering all the evidence

 

The abstract of the AHA report states that they discuss “the scientific evidence, including the most recent studies” and make their conclusions taking into consideration “the totality of the scientific evidence, satisfying rigorous criteria for causality”.

They are asking us to take their word for it here, because they don’t tell us how they tried to determine the totality of the evidence and find recent studies. This means the adequacy and potential bias in finding and choosing studies can’t be fully assessed. We can check the adequacy of their results, though.

For their coconut oil section, they don’t cite any study outside a single review, which includes evidence up to the end of 2015 (Eyres, 2016). The Eyres review has a very specific remit, so it only covers a patch of the questions related to coconut oil and CVD, not all of them, and not all health issues. By virtue of its exclusion criteria, this review cannot support these claims in the AHA report:

Clinical trials that compared direct effects on CVD of coconut oil and other dietary oils have not been reported.

..coconut oil…has no known offsetting favorable effects…

I haven’t attempted a systematic search for relevant trials related to coconut oil and health, but I did have a quick and dirty look for ones on CVD. It’s easy to find some that aren’t included. In fact, there are more participants in the randomized studies I found that aren’t included in the Eyres review than there are randomized participants in it. These were:

  • Cardoso (2015) studies people with coronary artery disease
  • de Paula Franco (2015) tests coconut flour in overweight women
  • McKenney (1995) studies coconut oil in people taking statins
  • Enns (2015) studies health outcome measures in people with peripheral arterial disease
  • Ganji (1994 and 1996), Assunção (2015), and Shedden (2017) are trials with coconut oil supplements

That last one was published after the AHA panel was finished its work, which was sometime in 2016. [Update: a commenter on this post notified another recent one, Irawati, 2017.] There are at least 2 more on the way – in Canada and England. And then there’s another category of study I didn’t look for, that’s often raised as relevant: comparing the effect of medium chain triglycerides on metabolic measures, including blood pressure, like St. Onge (2008).

I checked to see if there had been an adequate search and method behind the choice of systematic reviews and meta-analyses in the main part of the report. For those reviewing trials, they refer to 4:

  • Mozaffarian (2010, date of last search June 2009)
  • Chowdhury (2014, date of last search July 2013)
  • Hooper (Cochrane Collaboration) (2015, date of last search March 2014)
  • Mensink (WHO) (2016, date of last search December 2013)

They cite the re-evaluation of a primary trial in Ramsden (2016) – but not the systematic review with meta-analysis that’s in that paper as well (date of last search, March 2015). [Update: See a critique of this in comment below.]  I found another 7 in a quick search with a search date in 2009 or later, one from 2009 that didn’t report the date of search, and another published after AHA was finished. A systematic search could find more, but again, it seems there are more eligible studies in this category outside the AHA report than in it.

 

(Skeaff, 2009; Smart, 2011 which found no trials on its question; Stroster, 2013; NICE, 2014;  Schwingschackl, 2014; Bueno 2015 and Mumme 2015 on medium chain triglycerides and weight loss; Pimpin, 2016; Hamley, 2017). 

 

 

2. Not being equally critical of all the studies

 

Cartoon of dueling meta-analysts

 

It’s not easy to disentangle biases in trials, or get a grip on why meta-analyses are discordant. (An explainer from me here.) But at the very least, you need to subject each study to the same rules and standards.

That didn’t happen here, in 3 determinative ways:

  • No systematic assessment of methodological quality of reviews and meta-analyses was reported and we can’t be sure it happened;
  • The quality of only 2 of the 8 studies in the review of coconut oil were discussed, and only 7 of the 8 were mentioned at all;
  • The same standards of methodological quality applied to trials on other questions weren’t applied to studies on coconut oil.

The AHA report calls the Eyres coconut oil review a systematic review – but the authors do not. It has some of the elements of a systematic review. If you call it a systematic review, though, then it’s a very poor quality one: pretty unfair, since they don’t claim to have done one. (I used AMSTAR and came to a 1/11 score, though you could apply the criteria loosely to get to 3 – still bad.)

One of the things the coconut oil review doesn’t do is systematically assess the methodological quality of the studies included, and then take that into account when coming to individual conclusions. And the AHA panel doesn’t either – although they exercised a lot of critical precision on studies responsible for others’ conclusions that conflict with the AHA.

In addressing other studies, the AHA sets this bar:

…a randomized trial of a food or nutrient must achieve a biologically meaningful difference in intake between treatment and control groups and sustain it for a long enough time to deliver a valid result.

Large randomized trials that went for years and are accepted by others were rejected as a basis for AHA decision-making in the main part of the report. Whereas, of the 7 coconut oil trials they accepted for decision-making, only 3 were classified by Eyres as randomized. The 7 studies ranged from tiny to small, and went for only days to a maximum of 8 weeks. (The eighth, to reduce the amount of coconut oil in a high coconut diet, went for longer.) As Eyres writes:

Much of the research has important limitations that warrant caution when interpreting results, such as small sample size, biased samples, inadequate dietary assessment, and a strong likelihood of confounding.

 

3. Incorrect representation of the results of the coconut oil review

Road sign cartoon

According to AHA,

The authors [Eyres et al] also noted that the 7 trials did not find a difference in raising LDL cholesterol between coconut oil and other oils high in saturated fat such as butter, beef fat, or palm oil.

That one has showed up in a lot of coverage, and even headlines like “Nutrition experts warn coconut oil is on par with beef fat, butter”.

But that’s not what Eyres says. Firstly, 7 trials didn’t compare coconut oil to other saturated fats: only 5 did (3 randomized, 2 non-randomized). And there were differences: this wasn’t a meta-analysis with a single, summative and definitive result. (Just before this statement about the 7 trials, the AHA reports that one of them found butter raised LDL more than coconut oil: another did too.)

Eyres concluded that, on the question of the comparative effect on LDL cholesterol of coconut oil and other saturated fats, “the results are largely inconsistent”. I agree – they’re not consistent at all. But then, these studies are so small and biased, the real question is far more basic: do we know enough about the effects of coconut oil on CVD risk factors and other important issues to conclude anything at all?

 

 

4. Making strong conclusions based on weak evidence

 

The AHA’s main conclusions are based on 4 trials with the tail end of the latest one in 1971. Diet has changed so dramatically since then, that their analysis isn’t compelling. (See Nina Teicholz and Eric ThornGary Taubes and Larry Husten for very critical discussion on this.)

On coconut oil, the AHA has taken a stand on very shaky ground with some major claims – as though they had a very strong systematic review of reliable research on all possible health consequences of dietary coconut oil. They don’t.

The people arguing the opposite – that coconut oil is so healthy you should try to use it every day – are also on shaky ground. The first time a “nutrient as disease prevention” idea took off in a major way was probably dietary fiber to prevent colorectal cancer. I wrote a post a few years ago about the history, why that didn’t work out as expected, and why that’s typical of these kinds of nutritional claims.

Preventing actual disease or reducing risk factors via dietary change is tough. Unbalancing a diet can go wrong quite easily, though. If you wanted to increase your coconut oil and did it by having a couple of piña coladas every day, for example, you’d have a whole new set of problems! Dietary changes usually come with unintended baggage.

 

Cartoon pina cola

 

On the other hand, the occasional piña colada or coconut cream curry in a healthy diet isn’t going to cause a heart attack or stroke.

It’s not at all clear why AHA singled out coconut oil for a specific analysis, but no other non-dairy saturated fats. I didn’t even know for sure what dietary coconut oil meant when I read this report. Do I eat a lot of it? Is it a common ingredient in foods that I don’t think of as coconut-y? I had no idea.

It really doesn’t look as though coconut oil is a big dietary factor on average in the US – and if this data is right, the amount consumed hasn’t changed much. That might not include coconut milk, but all non-dairy milk isn’t a large proportion of milk either (and it’s mostly soy and almond).

Maybe the coconut “craze” is more hype than behavior – and socioeconomically specific in a way that’s unlikely to match CVD risk. On the other hand, palm oil use has apparently soared and it would be hard to avoid: it’s about the food industry using it. And then there’s the bacon boom, and super sizing…

If we’re concerned about America’s big public health problems with saturated fats, the AHA’s coconut oil salvo is distracting us from what matters most.

 

Continued in Circling the Wagons, Science Style: AHA Saturated Biases Redux.

~~~~ 

On 29 June, I posted a comment on the AHA Advisory at PubMed Commons. I submitted the section of my comment on coconut oil to the AHA, together with a link to the PubMed Commons comment and this blog post. And I emailed the journal with a link to the comment and post. The journal replied that there was a 500-word limit to what the AHA would consider, and I should re-submit.

On 24 July, the lead author and chair of the AHA Advisory replied to my comment at PubMed Commons, acknowledging no error, and firing shots at me for statements I did not make. I replied at PubMed Commons on 2 August, with a blog post detailing the issues in dispute. (Shortened formal letter to the AHA pending.)

 

* Disclosures: I have no income or livelihood bias about saturated fats. But I’m saturated in plenty of bias nonetheless, even though I’ve never given much thought to coconut oil before this last week. Some of my family are deeply committed to veganism and environmental activism, and a friend is involved in the manufacture of vegan foods using coconut oil. I was a member of the GRADE Working Group when it was developing guidance on rating evidence and recommendations, and methods for going from evidence to recommendations. I was a member of the Australian National Health and Medical Research Council working party on guidelines for developing, evaluating and implementing clinical practice guidelines (1996-1998). I participated in developing the Cochrane Collaboration, and participated in summarizing the original Cochrane review on dietary fat for consumers that is discordant with the AHA. (My day job includes working on curating and disseminating systematic reviews and guidelines at PubMed Health.)

 

[Updates 29 June]: Working on a comment for PubMed Commons I realized I had left Assunção (2015) out of the original post: added. I changed the National Academy of Medicine (formerly the Institute of Medicine) to a summary explaining the key elements of their recommendations. In response to a comment from Erik Arnesen and a tweet from John Campbell, I realized my wording around the St. Onge example was misleading: I changed it from “category of relevant study…this category of fat” to “category of study I didn’t look for, that’s often raised as relevant…medium chain triglycerides (MCT)…”. Thanks, Erik and John!

[Update 30 June 2017] George Henderson notified another relevant trial in a comment on this post: I added it to the post. Thanks, George! Note: In order to be consistent with which studies I include on this post, I decided on including any report of a controlled trial specifically related to CVD, that has cholesterol levels or more direct health measures as study outcome, regardless of the trial results – and to note here any reports of trials that I exclude. That meant that I added Ganji (1994), and note here one trial I did not include because of its outcome measures: LaBarrie (2017 [PDF]). I did not examine trials on non-CVD related health issues to check if they included CVD outcome measures, and I did not keep a list of them.

[Update 31 July 2017] I corrected the search date for studies in the Eyres review: I had overlooked that they updated the search at the end of the 2015. This post originally said it was 2013, which was likely to be out of date.

 

The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)

 

* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.

 

Discussion
  1. Thank you, this is brilliant.
    As far as I know, Unilever is an AHA sponsor, also works with Harvard, and is the world’s biggest supplier of palm oil products.
    Go figure.
    If any saturated fat has a plausible link to CHD it’s palmitic acid, because this is the SFA that’s also elevated in the blood, semi independently of dietary intake, by diets high in refined carbohydrates.
    Lauric acid largely escapes this effect due to its chain length and polarity.

  2. I think there are enough reasons to explain it, Barth: the problem of unsystematic evidence assessment is, unfortunately, very widespread.

  3. Thank you, George! I don’t know enough about that, but it seemed to me a critical issue to systematically assess. For me, that the report lacks a strong section on pinpointing which are the most critical contributors to health problems, was problematic. Hard to know why.

  4. Great post Hilda.
    Couldn’t agree more with the problem of unsystematic evidence assessment across the board here. I’d also add who is assessing the evidence is important.

    Example in case being one of the sources you cite in your post [Ramsden (2016) and subsequent meta-analysis]. The study and the MA/review classed the data upon which it was based as robust and at low risk of bias. However, our recent appraisal of the original data upon which it is based found serious flaws which limit the findings and conclusions http://www.bmj.com/content/357/bmj.j2108

    It seems on all sides the truth is in the eye of the beholder – at the expense of science, evidence-based practice and public understanding. A saving grace being that evidence-based methods at least allow us to point out these flaws as you so brilliantly have here. Thanks again.

  5. Thanks, David! Wish I’d seen your paper earlier! I’ve added a link in the post to your comment. (Note: a reminder that the Minnesota trial is not one of the 4 core trials the AHA relied on.)

    Yes, I don’t know where we’d be if there weren’t techniques to help keep us somewhere close to a straight and narrow!

  6. Questions and queries demand a clear response as the matter is of significance to tne scientists and patient community.

  7. Thanks, Erik! Made me realize “this category of fat” was completely misleading – I’d had the study open in another window and it was clear to me, but of no use to readers at all! I’ve updated it (and thanked you for pointing it out). I hope it’s clearer now. (I didn’t change the part about MCT in the meta-analysis section, because I think that was fine.)

  8. Here is a very recent study which would probably have been missed in a search as coconut oil is just a control fat.
    https://www.cambridge.org/core/services/aop-cambridge-core/content/view/AE968030D10419CA2F8EA2929FBF936D/S0007114517000150a.pdf/dietary_fat_and_physiological_determinants_of_plasma_chylomicron_remnant_homoeostasis_in_normolipidaemic_subjects_insight_into_atherogenic_risk.pdf
    To me ir appears that the reduced post-prandial lipemia after coconut oil feeding 1) would offset the risk associated with its effect on LDL if such exists to an unknown extent, 2) be consistent with lauric acid and other medium chain fats in coconut oil having some properties in common with MCTs (at least for practical purposes)

  9. Thanks, George! I didn’t include every comparative study I found, but I didn’t find that one. I expect there are more – I didn’t do anything close to full systematic search. I also didn’t include the studies I saw on health issues other than CVD. I’m not sure what a careful, systematic assessment would show.

  10. George – I added the study you sent in an update, and decided on criteria for which studies should be included. Details in an update (30 June) on the post. Thanks, again!

  11. Hi Hilda,

    You’re block IMO is at it’s strongest when critically appraising the methods used in terms the review. However, the background narrative appears to be overly influenced by one particular point of view (carbs = cause of increase in diabetes and obesity). Please see http://www.stephanguyenet.com/bad-sugar-or-bad-journalism-an-expert-review-of-the-case-against-sugar/ and here https://www.cato-unbound.org/2017/01/11/stephan-guyenet/americans-eat-too-much-cake-government-isnt-blame for a nice overview of the current debate.

  12. Dr. Emil Q Javier:

    “The AHA errs in its uncritical bundling of coconut oil with animal fats in its health advisory against saturated fats. The AHA should take into account that there are two families of saturated fats — those with MCFAs like coconut, and the rest of the vegetable oils which contain LCFAs. In the same manner that it distinguishes the healthier omega-3 unsaturated fats from the not so healthy omega-6 unsaturated fats.”

    “Tongue in cheek, since soybean oil constitutes 93 percent of vegetable oil consumption in the US, and the cardiovascular disease, cancer, diabetes and obesity problems have reached epidemic proportions, the AHA may want to explore the virtues of virgin coconut oil in its next health advisory. Perchance the health fortune of US citizens may change for the better. It cannot get worse.”

  13. You’re welcome – this one is easy to miss because coconut oil was the control oil. I wonder how many more there are like that?
    Very good work here, Hilda.

  14. In my opinion they’ve made a mistaken assumption, which is that the elevation of LDL by dietary fat has the same impact on health as the elevation of LDL by genetic factors (which is definitely associated with CHD in the modern environment, though this was probably not the case 200 years ago). There are good reasons to think this isn’t so. The saturated fats that raise LDL most also raise HDL most, and the substitution of fat for carbohydrate also lowers the TG/HDL ratio. The latter is a much better predictor of risk than LDL alone. Whereas the LDL genetic risk scores do not increase HDL or improve TG/HDL.
    Furthermore, lowering LDL with a statin only reduces coronary events in subgroups with high TG/HDL ratios. That is, in cases where reverse cholesterol transport is inhibited by excessive insulin. Not a word that appears in an explanatory sense in the AHA report. Those with low TG/HDL ratios have lower event rates and get no additional benefit from statins, even in secondary prevention trials.
    There’s been no systematic exploration of the relationship between fat oxidation (what seems to make some fats raise LDL and HDL more than others is their faster rate of oxidation), the TG/HDL ratio, reverse cholesterol transport, and cardiovascular risk. But it’s hard to see how fats can build up in arteries at a greater rate if they’re being oxidised to generate ATP at a faster rate. Especially considering that glucagon, the fat-burning hormone, inhibits the synthesis of cholesterol, whereas insulin stimulates it.

Leave a Reply

Your email address will not be published. Required fields are marked *


Add your ORCID here. (e.g. 0000-0002-7299-680X)

Back to top