Skip to content

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

PLOS BLOGS Absolutely Maybe

Free Speech and Journals’ Responsibility in Vaccine Debates



“Learn what they don’t want you to know.” That pretty much captures the conspiracy-laced journey into data you often see at the heart of campaigning against a vaccine, doesn’t it?

It’s the sub-title an activist used* to encourage people to read a new book billed as “a groundbreaking exposé”. It’s called The HPV Vaccine on Trial: Seeking Justice for a Generation Betrayed, by Mary Holland and colleagues. Published at the beginning of October, Amazon listed it in its top 10 selling books in both preventive medicine and vaccinations by the end of the month.

In the case they mount against the HPV vaccine, Holland and her co-authors include the critique of the Cochrane review published in July by BMJ Evidence-Based Medicine (BMJ EBM). That’s emblematic of how quickly a medical journal publication can be cemented into anti-vaccine lore. Regardless of what happens next, to many, the Cochrane review will always have been discredited – and to some, the Cochrane Collaboration as a source of reviews. I wrote about that in depth here and here as the events unfolded, but here’s a quick catchup.

The critique was by a group under the banner of the Nordic Cochrane Centre: Lars Jørgensen, Peter Gøtzsche, and Tom Jefferson (Jørgensen 2018). It was, I pointed out in my first post, marked by errors and spin. The authors had done a systematic review with different methods, submitted for publication months prior, but not published.

Cochrane’s editors-in-chief, David Tovey and Karla Soares-Weiser, published their audit of the critique’s claims on 3 September. They concluded that although the authors had identified additional data, it wouldn’t change the conclusions of the review. They called the criticisms “overstated” and wrote:

Scientific debate is to be welcomed, and differences of opinion between different Cochrane ‘voices’ is not unexpected. However, public confidence may be undermined, unnecessary anxiety caused, and public health put at risk, if that debate is not undertaken in an appropriate way. This is especially true when such debates take place in public. There is already a formidable and growing anti-vaccination lobby. If the result of this controversy is reduced uptake of the vaccine among young women, this has the potential to lead to women suffering and dying unnecessarily from cervical cancer.

The journal editors’ response on 12 September included this:

Academic freedom means communicating ideas, facts and criticism without being censored, targeted or reprimanded. We believe that the article by Jørgensen et al provokes healthy debate and poses important questions about the need to ensure that all available evidence is included in systematic reviews to properly inform healthcare decisions.

I argued that this particular debate hadn’t been healthy at all, and that statements are only facts if they are true. We’ll come back to this.

On 17 September, the critique authors responded to Cochrane’s response, acknowledging some errors, and revising down some of their original claims – while adding some new ones.

Throughout, I asked the BMJ EBM editors if they were going to issue corrections, and add the citations needed to justify a key claim of the critique. There was no response. So I was pleasantly surprised when they published their own review on 17 October, seeking “further public comment” on areas where they suggest formal correction/clarification could be necessary. I am responding. [Update: I posted my response to the editors here.]

Where does this leave us now? From what the BMJ EBM editors have written, at least some level of formal correction is on the cards. But a great deal of damage has already been done, and major problems in this critique should have been obvious to the peer reviewers and editors. I agree with what Doug Altman argued back in 1980:

[I]t is unethical to publish results that are incorrect or misleading…Once published, a piece of research achieves both respectability and credibility so that it is important for journals to make strenuous efforts to detect substandard research.

A publication that casts doubt on the safety of a vaccine demands a great deal of diligence from a journal. Safety concerns about the HPV vaccine have to be addressed, and out in the open: critically, though, journals as well as authors are responsible for how that’s done. That goes to both rigor and perspective – and with a concern for how it is likely to be interpreted by a lay audience.

What are these safety concerns, and how would they stand in relation to the potential benefits of the vaccine?  The same group of authors, with the addition of Peter Doshi, published a paper in The BMJ on 24 September. In that, they acknowledge that their (still unpublished) systematic review has major data gaps. When I dug through their references trying to find the basis for the key claim that serious adverse events (that may or may not be caused by the vaccine) are under-reported, this is what they cited:

Gøtzsche PC, Jørgensen KJ, Jefferson T, Auken M, Brinth L. Our comment on the decision by the European Ombudsman about our complaint over maladministration at the European Medicines Agency related to safety of the HPV vaccines. 2 Nov 2017. https://nordic. the_ombudsmans_decision_2_nov_2017.pdf.

Rapporteurs’ day 150 joint response assessment report of Gardasil 9. 25 Nov 2014. EMEA/H/C/3852.

That requires a lot of further digging. The key study relied on is one from the WHO Uppsala Monitoring Centre, described by its authors as a “novel” approach to searching for signals in its international database of suspected drug reactions. They looked, not for diagnoses, but for collections of symptoms which might (or might not be) the conditions causing HPV vaccine scares in several countries (POTS, CRPS, or ME/CFS). The study couldn’t show whether these were actual adverse effects of the vaccine. They were more common with young girls/women having HPV rather than other vaccines – but the difference wasn’t significant for serious cases. Worth more study, the authors wrote. The number of these cases was very low, though.

If this were to turn out to be an adverse effect of the vaccine – and that is  still a very big “if” – it would be a rare one. Weighed against that? In the US before the vaccine could have had an impact, “approximately 0.6 percent of women will be diagnosed with cervical cancer at some point during their lifetime” – or just over 1 in 200 women. In the US, about a third of them will die. And far more women would be diagnosed with precancerous lesions, with the distress and treatments associated with their own adverse effects that goes with that. (More on that in this post.) Globally, about half a million women each year are diagnosed with cervical cancer.

Given the rarity and uncertainty at the base of this, the rhetoric and claims seem to me to be “overstated”, too. Heidi Larson from the Vaccine Confidence Project recently warned of the public health danger of reducing trust in vaccines and authorities, writing:

We have found that it is useful to categorize misinformation into several levels. Among the most damaging is bad science: people with medical credentials stoking overblown or unfounded fears.

One of the critique’s authors, Gøtzsche, argued on an Irish TV program in 2016 [PDF] that:

If this is being abused by anti-vaccine groups, that’s their fault. It isn’t mine. I have not said anything about this vaccine, whether it is good or bad. Nothing.

I don’t think anti-vaccine groups are misusing his work. You can pick out responsibly-worded statements about uncertainty in his voluminous papers and statements since he became active on this vaccine in 2016 – but there are unambiguous statements about a safety cloud over the vaccine, too, coupled with fuel for conspiracist arguments – you can’t trust the doctors, the drug companies, most researchers, or the regulatory authorities to find and tell the truth.

The drug industry and their clinical trials on drugs, Gøtzsche argues in his books and public discourse, is fraudulent, corrupt, criminal. The logical takeaway for a reader is that harmful events related to vaccine ingredients are being ignored and knowledge of them is being suppressed, no one who says the vaccine is safe can be trusted, and therefore the safety cloud can’t ever really be dispelled.

In April, Gøtzsche read a paper to a conference of one of the organizations set up to advocate for people concerned about HPV vaccine harm (IFICA in Ireland). It exemplifies why he’s seen as a champion in anti-HPV vaccine circles. Mixed in with statements that, out of context, are hedged with reasonable scientific uncertainty, is “red meat” like this:

The story about the vaccines is a very interesting one. [I]t is one example among many others that we cannot trust our drug regulators…

My research group has found interesting results this year which we have submitted for publication. I cannot tell you the results now, only that we did find more serious neurological harms with the vaccines despite all the efforts by the drug industry and the EMA [European Medicines Agency] to make it almost impossible to find anything…

I find it pretty interesting, when clinical investigators, likely with conflicts of interest in relation to the sponsoring drug company, decide that none of the serious adverse events are vaccine-related. There are also reports of deaths that cannot possibly be said not to be related to the vaccine, at least not definitively so…

[T]he propaganda about the safety of the HPV vaccines has had the effect that many doctors do not report their suspicions of serious harms, either because they ignore them due to the public reassurances about safety; or because they are worried they might get into trouble if they report them…

It is likely that serious neurological harms are caused by an autoimmune reaction, where the body reacts against its own nervous tissue.

People could, he argued, wait a couple of years to use the vaccine till the issues are resolved – just have Pap smears – ignoring the exposure to HPV infection in those years, and that Pap smears miss many cancers and are associated with a lot of harm, too.

With books, videos, and speaking tours, all about the dangers of drugs and research on drugs, this is a long-haul campaign with a one-note agenda: it’s not just disseminating and discussing scientific ideas.

Recently, I was invited to peer review a chapter of the latest version of the Cochrane Collaboration’s methods handbook for conducting and reporting Cochrane systematic reviews. On the 2-page potential conflict of interest disclosure form you had to complete for this, was this question:

Have you published work in the area covered by this chapter and might this be perceived as a non-financial conflict of interest (such as being known to have published research or having strong views in favour or contrary to the work in this chapter)?

Good to know, right? In an area as super-heated by ideology and public campaigning as vaccines, I think journals have a responsibility to be even more careful when a campaigner seeks to cast a cloud over vaccine evidence. Being a scientist is unfortunately no guarantee that you can manage your own biases.


Cartoon of an audience member thinking "He would say that, wouldn't he?"



This is the 5th in a series of posts about the unfolding events that began with the publication of a critique of the Cochrane review of clinical trials of the HPV vaccine to prevent cervical cancer. The first critiqued that critique. The second looked at what we know about whether it’s working as would be expected from the trials. The third goes into the crisis that unfolded at the Cochrane Collaboration, and the responses to the critique. The fourth discussed extremism and anti-industry bias. The sixth is my response to BMJ EBM’s proposed corrections.

Disclosures: I led the development of a fact sheet and evaluation of evidence on HPV vaccine for consumers in 2009 for Germany’s national evidence agency, the Institute for Quality and Efficiency in Healthcare (IQWiG), where I was the head of the health information department. We based our advice on this 2007 systematic review including 6 trials with 40,323 women, and an assessment of those trials. The findings were similar to those of the 2018 Cochrane review. I have no financial or other professional conflicts of interest in relation to the HPV vaccine. My personal interest in understanding the evidence about the HPV vaccine is as a grandmother (of a boy and a girl).

I am one of the members of the founding group of the Cochrane Collaboration and was the coordinating editor of a Cochrane review group for 7 years, and coordinator of its Consumer Network for many years. I am no longer a member, although I occasionally contribute peer review on methods. I often butt heads with the Cochrane Collaboration (most recently as a co-signatory to this letter in the BMJ). I have butted heads on the subject of bias with authors of the Copenhagen critique.

I was invited to speak at Evidence Live, and my participation was supported by the organizers, a partnership between the BMJ and the Centre for Evidence-Based Medicine (CEBM) at the University of Oxford’s Nuffield Department of Primary Care Health Sciences – the director of the CEBM is the editor of BMJ EBM. Between 2011 and 2018, I worked on PubMed projects at the National Center of Biotechnology Information (NCBI), which is part of the US National Institutes of Health. I am currently working towards a PhD on some factors affecting the validity of systematic reviews.


* The quote that starts this post is attributed to John Gilmore of the Autism Action Network, and was posted here:


The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)




  1. Brilliant, as always.
    You quote from Gøtzsche’s IFICA video, what struck me about that video was that it is presented as:
    “Prof. Peter Gotzsche, Director of the Nordic Cochrane Centre,
    speaking via Skype on “Complaint to the European Ombudsman and maladministration at EMA” in relation to the EMA HPV vaccine safety review.
    IFICA conference, Dublin, Ireland, 21/04/2018.”
    So the video is supposedly about the complaint to the Ombudsman? But somehow, in a 30 minute video, he neglects to mention the conclusion of the Ombudsman resulting from that complaint.
    It’s not a long and complex conclusion:

    Based on the inquiry, the Ombudsman closes this case with the following conclusion:

    There was no maladministration by the European Medicines Agency.

    The complainant and EMA will be informed of this decision.”
    That conclusion was given in November 2017, almost a year ago. The IFICA video is from April this year.
    I also noticed that the Nordic Cochrane Annua Report mentions the complaint to the Ombudsman, and likewise omits the Ombudsman’s conclusion.

    Thank you for an excellent summary, and I look forward to your next installment.

  2. Knowledge will regularly be built on past knowledge. If someone has worked out some drug companies dress up the benefits and hide the problems they will be better placed to dig up hidden harms and accurately review data and info.
    As to Doug Altman’s quote, there’s results and there’s interpretation. Subset results may need interpretation so that they aren’t misleading.38 years on, the issue is also not publishing research, hiding inconvenient trials, selective publishing. This includes the ema or any body hiding problematic data under the guise of people will lose faith in their doctors.
    As to the ombudsman’s response, rosewind can take it as the final or the divine word if he wishes or we can look at gøtzsches.response and keep adding over time..? In essence? .the regulatory mechanism is broken,, there’s no available process to fix it, , the people need to know…
    Seems right to me.

  3. Isn’t it easier to explain the source of health problems of those girls in Ireland?

    Anyone who dealt with quantitative methods knows that the only one thing you can be 100% sure (in a biologically diverse materiality) is that you can’t be sure of… anything.

Leave a Reply

Your email address will not be published. Required fields are marked *

Add your ORCID here. (e.g. 0000-0002-7299-680X)

Back to top