Skip to content

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

PLOS BLOGS Absolutely Maybe

Boilover: The Cochrane HPV Vaccine Fire Isn’t Really About the Evidence – but it’s Critical to Science


It was always a volatile situation. If it blew, it was going to be even harder to keep perspective through all the fire, noise, and smoke. And it just did boil over – very, very, loudly. Now, getting clarity is urgent, and critical far beyond Cochrane.


Photos of cooking oil explosion demonstration
Cooking oil boilover demonstration, photo sequence by Joergens.mi/Wikipedia

First of all, what on earth just happened, and where do we go from here? Why is there so much talk about a Cochrane Collaboration crisis?

“It is about data”, one of the central antagonists said before the public explosion, according to a report in The Lancet. But while, yes, people are fighting about data and methodology, that doesn’t even begin to explain this explosion, its intensity, or its importance to science and society. If it were just about data, we would know what to do, and just knuckle down to it.

We have to get better at handling these incendiary situations that rapidly mobilize increasingly wide circles of people, though. The early days are vitally important. This might be our biggest chance to limit the damage, speed the recovery, and get something positive out of all this.

How did we go from a vaccine evidence dispute into a sudden crisis over expelling the Nordic Cochrane Centre’s director, Peter Gøtzsche, from the Cochrane Collaboration? Let’s try to untangle some of this.

I wrote about the dispute over the recent Cochrane HPV vaccine review here recently. I’ll discuss how that’s been unfolding first.

I’ll pick up where the previous post left off: with the Cochrane chief editors’ response, posted on their website on 3 September. Systematic reviewers from the central team who weren’t authors of the Cochrane review pitched in, to do an analysis of whether the review’s conclusions were dangerously flawed. These systematic reviewers and editors are separate from the ones who authored and edited the review.

They asked Jørgensen and Gøtzsche, who are both from the Nordic Cochrane Centre and authors of the critique, to identify the 20 trials this Copenhagen group had said were eligible and missed. It wasn’t provided.

So over at Cochrane HQ, they slogged through the list for the full index the Copenhagen group had previously published and sent to the Cochrane group who were finalizing the review.

The Cochrane chief editors’ response has a flowchart breaking down how many were unique studies, how many were completed, how many they considered eligible for the review and so on. In all, 137 potentially relevant trials were independently assessed by 2 systematic reviewers, who found 5 eligible trials that the Cochrane reviewers had not found in their own searches. There were also some that would be eligible, but the necessary data weren’t yet publicly available. Trying to get unpublished data had been planned by the authors as their next task: Cochrane reviews can be updated at any time after they are published.

The Cochrane response reads “we do not underestimate the importance of the missing data”, but it wouldn’t, they believe, change the review’s conclusions once it’s added. Only 1 trial had data on the primary cervical cancer-related outcomes – moderate or worse cervical lesions or invasive cancer. (See my original post for an explanation about this.)

There’s new information in the response on the conflict of interest issue:

Jørgensen et al also stated that the lead author of the review leads the European Medicine Agency’s post-marketing surveillance and linked this to funding from a manufacturer. In fact, Professor Arbyn took the initiative to introduce a surveillance study in his country after having been informed that the European Medicine Agency had approved the Gardasil vaccine, remarking that the post-marketing surveillance conducted in Northern Europe was relevant but should include also non-Nordic countries. Professor Arbyn is not funded by the European Medicine Agency nor by any vaccine manufacturer.

While the Cochrane editors concluded that the criticisms were “over-stated” and won’t change conclusions,

Some of the criticisms will inform the next version of this Cochrane Review and the planned review of comparative studies of HPV vaccines.

They will, they said, review any list of trials provided, and try to account for discrepancies.


Meanwhile, the editors of the journal that published the critique, BMJ EBM, responded to the Cochrane editors’ response on the 12th. The conflict of interest statement notes that the editor-in-chief (Carl Heneghan) is a close colleague of Tom Jefferson, one of the Nordic Cochrane Center authors of the Copenhagen critique, and an advocate of the same approach to systematic review methods. He’s a Cochrane editor/reviewer, too.

While the Cochrane editors involved people independent of the original work to weigh in, reporting the shortcomings they found there, the BMJ EBM editors see no problem in what they published, or their process. Cochrane criticized the adequacy of their peer review: their response is essentially, How so? We see nothing wrong.

I’ve criticized the adequacy of the editorial process too, so here’s my answer to that question.

Publishing a claim that half the eligible trials and participants are missing required references: how else could peer reviewers, editors, and later readers, reasonably assess the validity of the claim?

When I started fact-checking the critique, it took only minutes of fact-checking to see that some of the criticisms were errors. It took only the normal effort level of peer review to see the serious error of saying Cochrane and the trialists had under-reported the number of serious adverse events in a trial, when what they reported was in fact the number of women experiencing adverse events – which is of course smaller.

I had pointed this out to them too, and asked if they were going to issue a correction. No response, and no correction.

The BMJ EBM editors published the peer reviewer comments and authors’ responses. It doesn’t look as though the peer reviewers considered validity assessment or fact-checking part of their remit, either.

The editors conclude their response with this:

…articles in our journal will seek to hold organisations to account and will and should not shrink from offering criticisms that may be considered inconvenient. Academic freedom means communicating ideas, facts and criticism without being censored, targeted or reprimanded. We believe that the article by Jørgensen et al provokes healthy debate and poses important questions about the need to ensure that all available evidence is included in systematic reviews to properly inform healthcare decisions.

This is useful to parse, I think, because it’s so clearly articulating several issues that are keys to why responses the last few days are so polarized. I’ll come back to the question of accountability for Cochrane later, because that’s also key.

This statement puts academics and their freedom front and center. Academic freedom is, to many academics and others, a protected value. Here’s how I’ve explained this before:

Most of us have some kind of “protected values”. They are ones that have a moral force for us. We don’t like to trade them off against anything. A protected value in the mix is a common source of rigidity and anger that ramps up to outrage.

Invoke the academic freedom value, and enough’s said, to people who hold it that way too. It’s the core of an identity and the ability to pursue the truth, wherever it might lead. To work, as a scholar, in accordance with your own principles. Its opposite is exemplified by academics in prisons. “Academic freedom” is, therefore, an extremely emotive term.

But I’m not an academic, and I don’t see this is as an academic matter. I don’t think academics’ values and interests should be the main point in public debate. There are limits to all freedoms – right where they butt up into responsibilities.

You’re only communicating a “fact” if it’s actually true. That requires serious diligence. Some ideas are extraordinarily damaging, to individuals and society, and have no redeeming value. Lots of ideas academics had/have about race, for example, demonstrate that. And there are prerequisites for healthy debates.

Throwing flammable fuels around open flames isn’t “healthy” – not even for academics. When I read that closing paragraph, it crashed into my protected values, and I was instantly disturbed. It felt familiar and that nagged at me for a while. Then I realized what it reminded me of. And I think the analogy helps explain my protected values here. Bear with me on this.

It reminded of Richard Horton, editor of The Lancet, in 2004, justifying first publishing and then not retracting, the Wakefield paper that launched the “MMR causes autism” societal crisis. Except that instead of not shrinking from being “inconvenient”, the language was about not being “conservative” about “sometimes unpopular thinking”. But the argument was the same one.

It wasn’t until measles deaths returned after a long absence, and the General Medical Council stepped in, that The Lancet retracted the paper… another 6 years. Now Europe is tracking measles epidemics and deaths again, and in the US, a 2015 poll found only 41% of people were pretty sure vaccines don’t cause autism.

Here we now stand on what seems to be the brink of a drop in cervical cancer and other HPV-related cancers for cohorts of women from high HPV vaccination countries. And safety panics are being stoked: so much, that in Denmark, vaccination rates plummeted from around 90% of 12-year-old girls in 2014 to less than 40%. A similar phenomenon happened in Japan.

Ideologically or commercially motivated doubt-sowing are causing serious damage, and we’re not good at dealing with it yet. Journalist Ray Moynihan published an opinion about the Cochrane crisis focused on this, which includes:

What’s at stake in the current bloody fight unfolding within Cochrane’s Governing Board, is not just the credibility of individuals or organisations, it’s the future of reliable trustworthy evidence in a world of increasing falsity and fake news. To see this future threatened foreshadows a disaster for all of us.

The concept of a healthy debate has to take the reality of these consequences into account, and we have to find a balance that has room for the different principles. The stakes are high inside and outside of the ivory tower. That means accountability and responsibility are critical for Cochrane – but they’re just as critical for other scientific journals as well.

When people find critical data they should put it out there. I don’t agree with the Cochrane implication that it shouldn’t be done publicly. But if you’re going to publish evidence about vaccine effectiveness and safety in 2018, or launch major allegations about it, you have a responsibility to exercise a lot of diligence, with openness to ensure verifiability of claims.

Which brings us to the Copenhagen group’s response to the Cochrane response to their critique. BMJ EBM posted it on the 17th. That was right in the middle of the boilover, but let’s finish this strand.

The Copenhagen group report that they have now submitted their own systematic review for publication somewhere, and are now willing to share unpublished data with the Cochrane authors. That means to me, it’s not worth going too deeply into this – I’ll wait for the publication to try to come to grips with much of the detail. Because now, the effort needs to shift to the Cochrane review updating – that’s one of the advantages of a Cochrane review. Whatever its flaws now, they can be fixed.

The first big issue is the discrepancy in judgments about eligible trials. Here’s my analysis, based on the table where they provide details about the trials. [PDF]


  • Cochrane included 26 trials, with 73,428 girls/women.
  • The Copenhagen group said half were missing, and there were 46 eligible trials included in their database, for 42 of which there was available data on 121,704 participants altogether. So that’s 16 to 20 trials they said were missing from the Cochrane review.


  • Cochrane judged 5 trials with available data to be eligible, along with 3 without data available: for a total of 34 eligible trials.
  • The Copenhagen group now reports that there were an additional 17 trials with data, of which 11 are eligible and 6 that were only potentially eligible: for a total of 37 eligible trials.

It turns out, the Copenhagen group only looked at the list of included studies – not the list of excluded studies (studies considered and rejected) or the list of the rest of the references. (You can see how that’s organized here – it’s  standard for all Cochrane reviews.) They wrote:

When we checked again, we found some of the studies in the review’s reference list (3). The Cochrane HPV reviewers chose to use idiosyncratic referencing with study IDs such as “Phase 2 trial (ph2,2v)”, “Immunobridging (ph3,2v)” and “CVT (ph3,2v), which made the study assessment complicated. For numbers of participants, we did not subtract the male participants that were included in three of the studies, as we should have done.

What about the extra 6 trials they deem eligible, in comparison to the Cochrane audit?

  • 3 were in the list of excluded studies in the original Cochrane review. because data wasn’t available, either for the whole trial or just female participants. The Copenhagen group has available unpublished data for what they report as 1,400 women in these trials, some of which are publicly available. [Trials they numbered 1, 4 (also here), and 6 (also here, which seems to have 100 more women than Copenhagen logged).]
  • 1 is a comparison of gender-neutral versus female only immunization strategies and herd effect. It was excluded by Cochrane as a phase IV trial. Copenhagen says it is reported as a III/IV trial in one place. It has 20,515 female participants.[Trial 2, also here.]
  • 1 is a feasibility trial, not originally judged to be eligible by the Copenhagen group. It had 406 female participants. [Trial 9.]
  • 1 is a genital warts prevention trial, not cancer prevention. It has 200 female participants. [Trial 11 – and I’m pretty confident there is no trial with the NCT number Copenhagen provided for this trial.]

These trials all have conclusions consistent with that of the Cochrane review. Only 3 have been considered eligible by Cochrane so far, and it’s possible that judgement will remain. Only 1 trial is large, but it’s one that doesn’t seem to be eligible. I think the variance in determining eligibility could be reduced if the Cochrane review’s inclusion criteria were more specific.

I’d be surprised if there was a material difference to the Cochrane review from this. The Copenhagen group has not acknowledged their error with numbers of events versus participants.

There are still issues the Cochrane authors will need to address in their update about adverse events: additional questions were added in this round, too. And in this response, the Copenhagen group criticize the way the word “placebo” is used in the plain language summary for the review. I agree Cochrane needs to change it.

They also make additional allegations about author conflicts of interest, which need to be addressed: Cochrane has a process for that. Not that I agree with everything the Copenhagen group are saying.

For example, there was a major trial in Costa Rica, funded by millions of dollars of US tax payer money. The vaccine manufacturer provided the vaccines to the National Cancer Institute (NCI), and the trialists got it from NCI. They also got assistance from a company related to information in their regulatory materials. “We consider this industry funding”, the Copenhagen group write. Hmm.

So let’s get to the boilover. The hyped-up critique in a trusted publication turned the Cochrane review and Cochrane itself into shark bait. Bad enough, but how did it escalate?


Cartoon of someone watching TV and fearing a conspiracy



There’s a lot of reporting of what went down – but the early reports mostly came from people closely connected to team Gøtzsche, or ideological partisans for whom this is a propaganda/validation gift. The Cochrane statements are couched in a lot of deliberately vague language, given the legal situation they are in, and people’s right to privacy. Here’s what I think seems to have happened.

Cochrane got legal advice about their legal standing in relation to complaints made about Gøtzsche in March. They reviewed the history of previous incidents, as well as communicating with him. Gøtzsche then made allegations about a staff member:

We were advised that various legal consequences flowed from the events – the complaints and the accusations – and that Cochrane should take them seriously.

We asked the lawyers to take particular note of Cochrane’s commitment to transparency. They noted that, but also stressed the importance of confidentiality.

They advised that an independent review was both a sensible and proportionate response.

The board discussed and accepted the advice to get a very senior lawyer to conduct an independent review of the allegations about the staff member in June. The lawyer was engaged on 2 July, with a deadline for a board meeting on 13 September. That was when the international membership would gather for the annual conference (the Colloquium) in Edinburgh.

A few days later (7 July), Gøtzsche and colleagues’ critique of the Cochrane HPV review was accepted by BMJ EBM. It was published on 27 July, and Cochrane received further complaints.

The investigatory process was definitely not a response to the HPV critique.

At the board meeting on the 13th, the report from the lawyer and the HPV critique issues were discussed. There was a motion to expel Gøtzsche from membership of the organization, for 1 of the 3 grounds for expelling a member:

[5.2.1] is guilty of conduct which has had or is likely to have a serious adverse effect on the Charity or bring the Charity or any or all of the members or Directors into disrepute. [PDF]

It passed, but barely. Of the 12 board members other than Gøtzsche, 1 abstained. The motion was carried by 6 out of 11. Of the 5 opposing, 4 then resigned from the board, all but 1 of them being from a Cochrane Centre. It was acrimonious.

It’s striking that so many board members were from Cochrane Centres. That’s one of several specific sub-cultures in the organization, and it’s a small one. It’s also striking that the remaining board was all female, but for 1 Cochrane Centre director (and Gøtzsche). (3 of the 4 who departed were men.)

The Cochrane Collaboration is a registered charity and it has to have a functioning board at all times – so 2 more stepped down to get the proportions of member types conforming to the rules.

Gøtzsche had 7 days from the 13th to in effect appeal the board decision in writing. After that, the board makes a final decision, for which there is no right of appeal. What this means now for the Nordic Cochrane Centre isn’t clear to me. There are formal agreements between Cochrane Centres and the CEO, but I don’t know what provision that makes for this type of situation.

The road to that board meeting was a long and tortuous one: it’s been coming for years. But the events of the last few days went down fast.

Within 24 or so hours, Gøtzsche posted a statement on the Nordic Cochrane Centre website, which we’ll get to. [PDF]

The Board issued a very minimal statement about a day later it seems, and the 4 who resigned in opposition issued their own after that. By the AGM on the 17th, though, the Board had a prepared statement. They emphasized the right of people to a work environment free of attack and harassment.



The Board members who don’t support the decision and resigned see it differently though: their statement spoke only of people’s right to dissent and speak their minds. They viewed the expulsion as more likely to bring the organization into disrepute than Gøtzsche would.

Gøtzsche’s statement describes the decision as a minority decision (6 out of 13, of which he is 1), and says in 25 years no member has been expelled before. But as one of the original founding group (as am I), he knows formal individual membership of the Cochrane Collaboration from which the board could expel you didn’t exist for all those years. In fact, it’s quite freshly minted. And people have been forced out before, one way or another. It would be a rare group if that were not so, wouldn’t it?

Gøtzsche raises wide-ranging concerns:

  • Criticism of the Cochrane CEO’s management style in particular, and “senior central staff” in general.
  • Cochrane’s evolution, strategic direction, and corporate culture, saying it has become top-down, and overly oriented to making money.
  • Allegation that there have been threats to the Nordic Cochrane Centre’s existence and funding because he has been a vocal critic.
  • That there are reviews that are too favorable because of conflicts of interest.
  • Cochrane’s PR of reviews is slanted to reviews with favorable results and is too biased.
  • That Cochrane’s leadership is becoming less committed to open science, progressive civic/political science advocacy, and resistant to criticism of the drug industry – including resistance to tightening its 2014 conflict of interest policy.
  • Objection to the language and ethos of “the brand”, and walking away from the name “Collaboration” and what that represents.

The phrase “moral crisis” is in the title of the statement, and he’s arguing that he, his strategies, his work, his priorities, reflect the original public-minded spirit of Cochrane and that’s why “they” need to get rid of him:

I have been expelled because of my “behaviour”, while the hidden agenda of my expulsion is a clear strategy for a Cochrane that moves it further and further away from its original objectives and principles. This is not a personal question. It is a highly political, scientific and moral issue about the future of Cochrane.

I agree with him on several counts – I finally parted ways with the Cochrane Collaboration in 2012, because of the commercialization, open access issues, and the strategic direction it chose. There are real problems, although there are obviously enough people who want it this way.

But while I would love to see Cochrane change course, I think in this moment, the risk of the organization being pushed to the fringes matters too. Extremism, and the perception of it, can repel moderates and attract more extremists in a vicious circle. And fundamentalist extremism and bullying do tend to travel in tandem, don’t they? Back to the board’s statement:

All our staff, and our members, have the right to do their work without harassment and personal attacks. We are living in a world where behaviours that cause pain and misery to people, are being ‘called out’. This Board wants to be clear that while we are Trustees of this organization, we will have a “zero tolerance” policy for repeated, seriously bad behaviour. There is a critical need for ALL organizations to look after their staff and members; once repeated, seriously bad behaviour had been recognized, doing nothing was NOT an option.

The board speaks of the history of conflict and complaints:

In fact, the earliest recorded goes back to 2003. Many have been dealt with over the years. Many disputes have arisen. Formal letters have been exchanged. Promises have been made. And broken. Some disputes have been resolved, some have not.


Those few words have to carry a very heavy bucket of pain, lost energy, and people lost to the Collaboration, too. But this is a man who is also loved and admired by many, who don’t see the problem.

Like Gøtzsche, I was “first generation Cochrane Collaboration” – part of the group Iain Chalmers assembled to found an international organization out of the roots of the Cochrane Centre he had inspired into life recently in Oxford. I served on the board of the international organization from its founding in 1993 to 2001. Here’s a description by an observer in 2013, around the time I parted ways with the organization:

The Cochrane Collaboration offices are small. If you have ever been to Summertown in Oxford, United Kingdom, chances are you passed them without even knowing. Standing outside their modestly sized single-floor office makes you wonder just how they wield such clout.

After that, the staff grew quickly, and moved into premises in London. There was a period of very rapid change. Even apart from strategic issues, you can see why some would be nostalgic for the old days. But it wasn’t all sunshine and roses. Or at least, there were a lot of thorns on the roses. Change isn’t always bad.

Around the time Gøtzsche began advocating for an end to mammography screening, I wrote a piece in the international Cochrane newsletter (May 2002). It was considered incendiary enough to have a disclaimer about it being my personal view. Now that’s definitely something you never saw!  I was ruminating on whether the valorizing of fighting like it’s a holy war and iconoclasm was antithetical to collaboration:

How much collaboration can there be among people who symbolically denigrate each other’s motivations and work? How do we use critical analysis creatively, without descending into using intellectualism and science as weapons? Isn’t defensiveness, rather than cooperation, the usual human response to attack? So what role does a warrior culture, or a warrior element, have in a collaboration?

I’m even more sure now that all this aggression is antithetical to collaboration, and to science.

Now, the NCC cites the prevention of introduction of mammography in much of Denmark for a time as an achievement [PDF]. I’ve written about the controversy over mammography evidence before, too. Anti-mammography activism made Gøtzsche a champion to many, but increasingly polarizing.

There weren’t as many who thought he was a champion when he called in the BMJ for a stop to longterm use of almost all psychotropic drugs, based in large part on his then soon-to-be-published book, Deadly Medicines and Organised Crime: How Big Pharma has Corrupted Healthcare.

I haven’t read it: here’s a book review on PLOS Blogs with Tom Yates talking about some strengths, but also concern about hyperbole, offensive rhetoric, selective use of evidence, and unsupported claims (like “bipolar disorder … is mainly iatrogenic, caused by SSRIs and ADHD drugs”). Here’s another, from Richard Smith, that shows the other side – the affection and admiration.

It’s seemed forever locked between these poles, but the board is totally right: bullying of individuals is the tie-breaker. It has to be. And the law in many countries these days will back that up if someone wants to fight back.

Now back to the BMJ and concern over the psychotropic drugs essay. Cochrane’s editor-in-chief and mental health review group lead editors responded, to make clear even though Gøtzsche wrote this using his NCC affiliation, the views expressed were not those of the Cochrane Collaboration.

These episodic clashes are more than just inconvenient: they are deeply costly, in time and energy that could be used constructively. And yes, they do damage Cochrane’s reputation, and repel contributors. This is from a piece in Nature last year, when the NCC alleged various kinds of foul play at the European Medicines Agency:

This letter, sent on Nordic Cochrane Centre-headed stationery despite not being an official Cochrane Collaboration communication, focused heavily on the conduct of the EMA review and issues such as conflict of interest, maladministration and confidentiality. However, as the EMA highlight in their detailed rebuttal,12 the authors ignored the limitations of the cited case reports and introduced basic errors such as accusing ‘the wrong Julie Williams’ of undeclared conflicts of interest.

Why does this matter? The Cochrane Collaboration has a long-held reputation of excellence, producing trusted high-quality reviews on thousands of topics related to health and has groups at national and sub-national level. The authors of the complaint, in using Cochrane-branded paper with the header ‘Trusted evidence. Informed decisions. Better health’, give the impression to readers that their views are representative of, or in some way approved by, the Cochrane Collaboration, and this is the view now being promoted in online anti-vaccine communities.

Which brings me back to my iconoclasm piece from 2002. When people self-identify as scrappy, independent, and anti-establishment, it’s a tough transition when you get big and powerful enough that you have to be responsible. But warrior culture is destructive, and inimical to informed decisions and a trusted provider of information.

The last few days, some were saying on Twitter that Cochrane isn’t an organization in crisis, despite the publicity. Trish Greenhalgh pitched in with a thoughtful analysis, including:

21st-century science is an intersectoral endeavour that necessarily occurs in dialogue with society. Maintaining—and funding—the “view from nowhere” requires delicate navigation of tricky political spaces and sometimes accepting hard-won compromises. Board members are presumably expected not to spit in the soup (especially when using Cochrane letterhead).

At this stage in a fast-unfolding story, I am not convinced that the Cochrane Collaboration is experiencing a crisis of either morality or democracy. Its brand, now as ever, stands for rigour, independence, and a commitment to using science to achieve high-quality patient care and social justice. We should cut it some slack while it gets its house in order.

And no, that doesn’t mean kowtowing to the drug industry. I think it’s true that it’s not an organization in crisis. But it is an organization in a crisis. In a systematic review by Mike Clarke and colleagues on communication in disasters, a disaster is defined as “a serious disruption to the functioning of a community that exceeds its capacity to cope within its own resources”.

The super-charging of anti-social social movements and conspiracy theories about vaccine safety and Gøtzsche’s alleged persecution affects far more people than the members of the Cochrane Collaboration. Something big has been unleashed. And a lot of people are outraged.

I’ve written before that I find “the outrage factor” theory in risk communication useful to think through situations like this:

The theory was developed by Peter Sandman [PDF]. You can’t manage communication about environmental risks effectively, he argued, if you don’t consider the level of potential to invoke an extreme emotional response.

A risk, in this view, is never just a hazard: Risk = Hazard + Outrage. The outrage factor may be anywhere from negligible to catastrophically high. Any situation with a high potential outrage factor is high risk, even if only a small direct hazard to people is involved.

He lists potential factors that can cause outrage, and Gøtzsche’s expulsion checks off several. I think the sections on trust and responsiveness (11 and 12) and the chapter on psychological barriers are helpful guides to charting your way through this.

You have to be scrupulously honest, accountable, responsive, and compassionate to be trusted. Faced with outraged people, all that’s extremely hard though, and it’s easy to lose trust:

Apart from the obvious distorting effects of self-interest, conviction probably is an even bigger source of bias. If you think you know the Truth with a Capital T, then cheating a little on some inconvenient lower-case facts does not seem especially dishonest…

Given that trust in industry and government is a slender reed that snaps when you lean on it, you need to stop leaning on it. That is, stop asking to be trusted. The paradox of trust is that the more you ask people to trust you, the less they trust you….

Instead of trust, it seems to me, the bottom line is accountability. The goal is to be able to say, truthfully, to a public that does not trust you, that it does not have to….“Track us, don’t trust us.”

For the people who identify less with Gøtzsche and the Cochrane Centre’s situation, and have a well of goodwill for Cochrane, trust won’t be a problem. But especially as this has splashed across the medical media and further, we’re going to have to work hard to develop the opposite of a vicious circle: spirals of trust.

Sandman points to the importance of responsiveness in building trust:

There are at least five different components of a responsive process: (1) openness vs. secrecy; (2) apology vs. stone-walling; (3) courtesy vs. discourtesy; (4) sharing vs. confronting community values; and (5) compassion vs. dispassion.

It’s hard for the people at the center of an organization being attacked to do this, because they have to suppress their own outrage and hurt about what people are accusing them of.

Here’s a concrete example of what Sandman means. Below, the director of the Canadian Cochrane Centre pointing to an error that the Cochrane board had made – writing something that would escalate conflict, and made a suggestion that offered Cochrane to demonstrate responsiveness. [Update 20 and 22 September]  This post originally included encouraging the Cochrane board to act on this suggestion. I’m pleased to say they did. on the 19th, which is moving quickly. (Thanks, Holger Schünemann and Cochrane Board!) And the Austrian Cochrane Centre removed the statement of the resigning members, for further reconciliation. (Thank you, Gerald Gartlehner & co!)





There is a profound irony, it seems to me, in the timing of this. There’s a fight going on over evidence about the HPV vaccine, with two competing systematic review groups. And both have a cut-off date for data just shy of when the first signs of the predicted drop in HPV-related cancer in trials appeared. Perhaps the most critical information in an upcoming Cochrane review update won’t have anything to do with current criticisms.

We need young people to be the winners here, whatever shape that’s going to take. They mustn’t be collateral damage in someone’s ideological war or ego battle. I’m very proud of the Cochrane Collaboration for pouring so much effort into this particular systematic review.

Cochrane, you have powerful methods for finding out where the evidence takes us and updating reviews and methods in response to new evidence and criticism. Don’t let attacks and wounded feelings take you off course.


Postscript: On 26 September, the Cochrane board announced that it had “voted unanimously on 25th September to terminate Professor Peter Gøtzsche’s membership of the organization, and his present position as a Member of the Governing Board and Director of the Nordic Cochrane Centre”.


Sign pointing to Bias Extinguisher


Much shorter (!) follow-up post: Scientific Advocacy and Biases of the Ideological and Industry Kinds.

[Update] After the smoke cleared, I dug carefully into this conflict to review my position. I posted a timeline and my conclusions, as well as a collection of links, at my personal website on 8 February 2020, as discussion of it began again after Gøtzsche published a book on vaccines: Gøtzsche and the Cochrane Collaboration: A Timeline.



Disclosures: I led the development of a fact sheet and evaluation of evidence on HPV vaccine for consumers in 2009 for Germany’s national evidence agency, the Institute for Quality and Efficiency in Healthcare (IQWiG), where I was the head of the health information department. We based our advice on this 2007 systematic review including 6 trials with 40,323 women, and an assessment of those trials. The findings were similar to those of the 2018 Cochrane review. I have no financial or other professional conflicts of interest in relation to the HPV vaccine. My personal interest in understanding the evidence about the HPV vaccine is as a grandmother (of a boy and a girl).

I am one of the members of the founding group of the Cochrane Collaboration and was the coordinating editor of a Cochrane review group for 7 years, and coordinator of its Consumer Network for many years. I am no longer a member, although I occasionally contribute peer review on methods. I often butt heads with the Cochrane Collaboration (most recently as a co-signatory to this letter in the BMJ). I have butted heads on the subject of bias with authors of the Copenhagen critique.

26 September: In light of developments, some further disclosures which I did not realize may relate to this controversy and the theories around it, when I first started writing about the Cochrane HPV vaccine review. I was invited to speak at Evidence Live, and my participation was supported by the organizers, a partnership between the BMJ and the Centre for Evidence-Based Medicine (CEBM) at the University of Oxford’s Nuffield Department of Primary Care Health Sciences – the director of the CEBM is the editor of BMJ EBM. Between 2011 and 2018, I worked on PubMed projects at the National Center of Biotechnology Information (NCBI), which is part of the US National Institutes of Health. I am currently working towards a PhD on some factors affecting the validity of systematic reviews.

[Updates 20 September 2018] After some questions on Twitter, I added some extra detail to a sentence about membership of the Cochrane Collaboration – thanks, Rosewind! The sentence originally read:

But as one of the original founding group (as am I), he knows individual membership didn’t exist for all those years. In fact, it’s quite freshly minted. And people have been forced out before, one way or another.

I edited and updated a section to include Cochrane’s response to the section which originally read:

Here’s a concrete example of what Sandman means. It includes an error that would escalate conflict, and then a chance to be responsive. Please do this, as soon as possible!

And I changed “5 out of 13” board members being from Cochrane centers to “so many”, because I realized I am not sure exactly how many were – there were at least 4, which means they are disproportionately represented.

[Update 22 September 2018] I added the note that the Austrian Cochrane Centre had removed the statement posted on behalf of the resigning members, one of whom was the director of the center, Gerald Gartlehner, after confirming with him that it was done as an effort towards de-escalation of conflict. (I have kept a copy of this and the original statement from the Cochrane board.)

[Update 23 September 2018] In response to a tweet from @TranspariMED, I edited a sentence that previously read:

Ideological and commercial “merchants of doubt” are causing serious damage, and we’re not good at dealing with it yet.

Thank you, Till Bruckner/@Transparimed!

[Update 24 September 2018] In response to a tweet from Duncan Babbage, I edited sentences that previously read:

It’s also striking that the remaining board was all female, but for 1 Cochrane Centre director. The departing board members were all male, but for 1.

That was only a partial response to the suggestion, though: a virtually all-female board for Cochrane was an amazing sight. However, I also realized when examining the sentences that Gøtzsche remains a board member too, at this part of an ongoing process. Thank you, Duncan Babbage!

[Update 26 September 2018] Added postscript on the Cochrane board’s termination of Gøtzsche’s membership and positions in the organization.

[Update 29 October 2018] When describing the additional 11 trials cited by the Copenhagen group, I corrected the description from “primary outcome (moderate or severe cervical lesions)” to “primary cervical-cancer-related outcomes (moderate or severe cervical lesions or invasive cancer”, and the subsequent shortened version “primary outcome” to “primary cervical cancer-related outcomes”. I realized the original shortened description was misleading when replying to this comment. After further considering this reply, I removed those sentences completely (on 28 February 2020). (Thank you, Aleksi Raudasoja.)


The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)


The sequence of photos of a cooking oil boil-over are by Joergens.mi/Wikipedia, via Wikimedia Commons and the Wikipedia boilover page, with a CC-BY-SA-3.0 license.


  1. Question: was Gotzsche allowed to vote? I could not find any language in the articles of association that would deny him the right to vote on his expulsion. It’s odd that no one has mentioned this.

  2. His conflict of interest would be too high to vote. When a conflict of interest is that high on a board on a matter, you have to leave the room, so people have a chance to discuss it with less pressure. You can’t have every possible point of procedure in a constitution. There is nothing that says explicitly, for example, that you can’t vote on a decision about whether you will receive a large grant you have applied to the organization for, or vote on whether a family member got appointed CEO. But you shouldn’t be able to, because it would compromise your ability to put the organization’s interests ahead of your own.

    How this kind of thing works in practice in boards in the English tradition (and Cochrane is a UK charity), is that you recuse yourself as your conflict gets high, and leave the room when it’s very high, or your presence makes it too hard for everybody else. If there’s a conflict even about this, then the board is really having trouble acting in the interests of the organization and its purposes. There’s a provision [19.2] that “Questions arising at a meeting shall be decided by a majority of votes. In the case of an equality of
    votes, the Chair shall have a second or casting vote.” A disagreement about something like this could itself be put to a vote.

  3. Thank you for the response. I understand the point is a bit moot because in the event of a tie the chair would have cast a 2nd vote. But this logic seems very odd to me. Gotzsche had been highly critical of some members of the board and the general direction of Cochrane set by that board and the CEO. If Gotzsche were too conflicted to vote on the matter, it would seem to me the other members of the board were also too conflicted.

    It’s one thing to recuse on a vote for a grant that would personally benefit you and no one else on the board. It’s another thing when the vote is about whether you will be expelled from membership – and with that removed from the board itself. This removed a democratically elected member of the board and the people who voted for Gotzsche essentially had their voting power nullified. There are usually detailed procedures for things like this precisely because it is legally thorny.

    Another thing that strikes me as odd: does expelling Gotzsche actually accomplish anything? It appears he will continue as head of the Nordic Cochrane Centre. I don’t know that there is a rule against it. If he does continue, he is still entitled to represent himself with Nordic Cochrane letterhead. Will Cochrane be forced to expel the entire Nordic Cochrane Centre?

  4. One more thing to think about: I think the legal status of this vote is very much in question. The articles provide a list of reasons for removal of a board member. None of them specifically contemplate the removal of a member by the other members of the board. The remaining board did this by expelling him from regular membership, which in turn made him ineligible to be a board member. But this is legally dubious. It’s not clear it is allowed. If it were, then a majority on the board could remove and replace any board member that espoused a minority viewpoint. Surely this was not intended by the drafters of the articles. Removing democratically elected members is usually made difficult to protect the interests of minority voters. I think this situation is only going to get worse as more details come out and the legal issues become clear.

  5. They have lawyers and they are using them – hopefully the advice is sound and clear. I disagree with this interpretation of the constitution: members of the board aren’t explicitly excluded, therefore they are ordinary members. It is moot though, because the numbers were there, no matter which way you cut it. Even if you’re elected, you’re not above all rules – and the behavior they’re talking about could have serious legal gravity. This was a protracted dispute, so he had many opportunities to keep coloring within the lines.

    As I said in my post, it’s not clear what happens next with the Nordic Cochrane Centre. Follow the link in the post on this and you’ll see that Centres are supposed to each have a formal agreement. I would be very surprised if there is not some provision for this. I spent a fair amount of time reading on the Nordic Cochrane Centre, but there is very little transparency – I could find no real detail about funding, for example. His statement seems to me be saying the Centre might have been in a somewhat shaky position, even without this. I have no inside knowledge on this, and the organization’s structure and processes have changed profoundly since I was there. But from my experience in general, I would expect this to be a one step at a time process. This is dramatic, but it’s not something unprecedented in the course of human affairs. Most organizations have to confront something like this, at some point or other. It’s not unprecedented at Cochrane either: it’s just that generally, when things become this untenable, the person leaves when they see the writing’s coming up on the wall. People usually care about the organization they have devoted themselves to enough to not want to risk tearing it apart, even if they think they can muster the forces for a coup. So they never get to the Molotov cocktail throwing stage. But there are some people who go scorched earth, preferring civil war to letting go.

  6. It’s a 2016 constitution: some of the framers of the articles of association are essentially in the room, and they know their intentions. These have been issues for Cochrane since day 1. There were times more internally combustible and organization-threatening than this by a long way.

    The organization has been debating this issue, in great detail, for years. Yes, there is a provision for the board to disqualify one of its members [16.1]. That’s a basic need in governance: the entity has to be governed, and all sorts of things can happen to, and among, humans. Everything has to be considered to ensure that the entity survives, no matter how rogue humans go.

    This isn’t a situation of someone being removed because of a minority opinion. But it depends what the opinion would be about, doesn’t it? There are some opinions so socially damaging, a single one would be enough. But Cochrane is enormously tolerant of diversity in opinion and challenge, and it even has an annual prize for it. They haven’t tolerated the opinions for so many years, and suddenly now it’s a bridge too far. The harassment issue takes it to a separate place. Boards have to ensure safe workplaces for their employees: no choice there. That’s law, not just morality (although it is that too).

  7. What is the relevance of the gender of those that left or chose to stay on the Board? if we are going to get into cultural biases what colour were the various Board members? How many were gay? how many believed in the great spaghetti monster?

    There are enough serious issues here without dragging in fashionable concerns about differences of that kind (real or imagined).

  8. It’s not at all because it’s “fashionable”, although the tendency for the genders to have different attitudes to victims versus those who face allegations is admittedly a feature of the current #MeToo/#TimesUp phenomenon. Seeing the photo of the board on stage at the AGM was striking. Gender issues were a major issue for many years in Cochrane, and one of the struggles the minority of us who were women in leadership faced, was very different attitudes to issues of behavior.

    I raised 2 points that I thought were relevant: 1 was gender, because of potentially different attitudes to harassment. The other was the large Cochrane center contingent, who could on social/lobbying grounds (center staff/directors caucus) and/or out of concern for the independence of Cochrane centers see action against a Cochrane center director differently.

    Also….there’s a great spaghetti monster??!!!

  9. Hilda Bastian, you say you “led the development of a fact sheet and evaluation of evidence on HPV vaccine for consumers in 2009 for Germany’s national evidence agency, the Institute for Quality and Efficiency in Healthcare (IQWiG)”, where you were the head of the health information department. You also say you have a personal interest in understanding the evidence about the HPV vaccine as a grandmother (of a boy and a girl).

    Can you please advise, what was the evidence supporting the original three dose regimen for the HPV vaccines?

    Have your grandchildren had three doses of HPV vaccines?

  10. Given the affiliation you’ve linked to, the way this is phrased, and the out-of-line question about my grandchildren – I’m not going to put effort into that: I’m sure you already hold a strong opinion and aren’t really interested in my assessment of that question.

    If you read that statement, then you already know on what evidence the fact sheet was based, as it was part of the statement. As to my grandchildren: their parents and they will be making that decision when they reach that age, not me. My oldest is 7, so it’s a bit of a wait.

  11. We might disagree with some of extreme standpoints expressed by Peter Gotzsche on his blog or published in some of his books that are not peer-reviewed. However, his criticism of the practice of prescribing Specific Serotonin Reuptake Inhibitors (SSRIs) to children, adolescents and young adults is evidence-based and beyond reasonable doubt cast light on potential serious harms associated with their use, including increase in aggression and suicidal behavior in adolescents ( Such findings are not unkown to clinicians and researchers and we can not withhold these safety issues from the public and young patients. We will be better doctors not worse if we admit our treatment limitations. However, in spite of previous FDA warnings and acknowledgment of such adverse effects, some authors openly tried to silence Goetzsche. They questioned and addressed Goetzsche’s team findings, in very biased and extreme way, not presenting any arguments contrary to his team research findings (doi: 10.1136/bmj.i906 and Moreover, some of them, even tried to interfere with British Medical Journal publishing policy, calling for censorship. Of course Peter responded in very scrupulous manner (10.1136/bmj.i915).

  12. Not to mention his scientific criticism ( as well as other Cochrane groups) of flawed review on methylphenidate treatment for adults with attention deficit hyperactivity disorder (ADHD) which was published by Cochrane ( His and his collegues scientific rigorous activism eventually resulted in Cochrane editorial decision to withdraw the Review. No mentioning of this event in your rambling diatribe. To conclude I am sure that with Cochrane or without it, with Peter in Cochrane or out of it, we will have debates and find place for publishing dissident standpoints about effectiveness and safety of vaccines and psychotropics and other medicines, like it or not.

  13. His argument in the BMJ, in case people haven’t read it, was that 98% of all non-acute use of psychotropics should stop. It went far beyond the issue of SSRIs for young people.

  14. Thank you for doing such a thorough job in taking us through. I wonder whether the actual allegations in question will ever be made public.

  15. Thanks! I imagine it would be tough if, for example, it’s about a seriously defamatory accusation and the defamed person doesn’t agree to a libelous statement being broadcast.

  16. In regards to the recently published Cochrane HPV vaccine review[1], this review is severely compromised and cannot be trusted due to the conflicts of interest of authors on the original protocol and the final review document.

    Due to serious conflicts of interests, Cochrane should withdraw this review.

    In February 2016, I challenged David Tovey, Editor in Chief of Cochrane, directly about protocol author Lauri Markowitz’s conflicts of interest.

    Catherine Riva et al raised the problem of conflicts of interest in December 2014 in a comment on the original protocol[2], specifically pointing out the failure to properly disclose conflicts of interest by Lauri Markowitz and Marc Arbyn.

    Lauri Markowitz is an employee of the US Centers for Disease Control and Prevention (CDC), and is involved in HPV vaccination promotion.

    The US Government benefits from the sale of HPV vaccine products, i.e. a letter to Dr Eric Suba from the US National Archives and Records Administration (November 2010) discusses royalties the US National Institutes of Health (NIH) receives from the sales of HPV vaccines. (See a copy of the letter via this link:

    Indicating a stunning lack of transparency, it appears the value of these royalties is kept secret, i.e. it is protected from disclosure under the US Freedom of Information Act.

    The NIH Office of Technology Transfer (OTT) oversaw the patenting of the HPV vaccine technology and licensed the technology to Merck, the maker of Gardasil, which sought approval for Gardasil around the world, working with the PATH group, with support from the Bill and Melinda Gates Foundation, in distributing the HPV vaccine in developing countries.[3] The HPV vaccine technology was also licensed to GlaxoSmithKline[4].

    The Bill and Melinda Gates Foundation has been very influential in promoting HPV vaccination[5]. In regards to the Cochrane HPV vaccine review, Cochrane has a conflict of interest in that it is a beneficiary of Bill and Melinda Gates Foundation funding, i.e. to “support the development of Cochrane’s next generation evidence system, with a specific focus on maternal and child health”.[6] The World Mercury Project has provided critical analysis of Cochrane’s conflicts of interest via the Bill and Melinda Gates Foundation and other organisations[7].

    As a matter of urgency, Cochrane needs to consider conflicts of interest in its undertakings, as these are compromising Cochrane’s mission to provide credible and unbiased information to support informed health decision-making.

    In regards to the Cochrane HPV vaccine review, it’s alarming that an employee of a US government agency promoting HPV vaccination was involved in the Cochrane protocol to evaluate the immunogenicity, clinical efficacy, and safety of HPV vaccines when there is a clear conflict of interest, i.e. it is in the US Government’s interest to justify and defend the use of HPV vaccine products.

    How and why was Lauri Markowitz’s participation in this Cochrane review approved by Cochrane?

    In my previous correspondence to Dr Tovey in February 2016, I noted Markowitz is an author on many papers about HPV vaccination, for example Prevalence of HPV After Introduction of the Vaccination Program in the United States[8], a paper which received acclaim in the mainstream media, see for example this article published in Forbes magazine: HPV Infection Rates Plummet in Young Women Due to Vaccine[9].

    I also noted Markowitz was on the US Advisory Committee on Immunization Practices’ Human Papillomavirus Vaccine Working group in 2006, and that she is the ‘corresponding preparer’ on the ACIP’s document recommending implementation of HPV vaccination[10].

    I queried how Markowitiz could possibly be an objective and independent reviewer of the literature regarding HPV vaccination, and also queried on what basis Lauri Markowitz was engaged to conduct the Cochrane review of HPV vaccines.

    Dr Tovey’s response to me on 1 March 2016 included: “We can’t govern the opinions that review authors hold although we are stricter than other journals about conflicts of interests – in that declaration is not always sufficient. We have safeguards in place to avoid bias due to non financial conflicts although I acknowledge these cannot currently be fully controlled – but these include insisting on teams of authors, peer review at both the protocol and review stage, detailed editing by the appropriate Cochrane Review Group plus oversight by my Editorial Unit.”

    It appears that Cochrane does not have an effective system to evaluate conflicts of interest either of Cochrane itself or its authors.

    Dr Tovey did not clarify on what basis Lauri Markowitz was engaged to conduct the Cochrane review of HPV vaccines.

    It is not clear who initiated the HPV vaccine review protocol.

    Subsequently Lauri Markowitz was not listed as an author on the title page of the Cochrane HPV vaccine review, so it appears there was recognition that it was not appropriate that she be an author of a review on HPV vaccines.

    But the fact remains she was influential in the development of the protocol, which was acknowledged in the final review, along with her “invaluable advice and contributions by reviewing the results and discussion sections”.

    As a citizen interested in HPV vaccination I had hoped to rely on an objective and unbiased review by Cochrane, but I do not trust this review and do not consider it to be a document of value.

    The Cochrane HPV vaccine review is severely compromised. It is demonstrably not independent and cannot be trusted.

    The Cochrane HPV vaccine review should be withdrawn.

    Cochrane also needs to urgently consider its own position in regards to conflicts of interest, and the impact on Cochrane’s credibility, independence and trustworthiness.


    1. Marc Arbyn, Lan Xu, Cindy Simoens and Pierre PL Martin-Hirsch. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Systematic Review. Published 9 May 2018.

    2. Marc Arbyn, Andrew Bryant, Pierre PL Martin-Hirsch, Lan Xu, Cindy Simoens and Lauri Markowitz. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Protocol. Published 30 December 2013.

    3. NIH Technology Licensed to Merck for HPV Vaccine.

    4. HHS-Licensed Products Approved by the FDA.

    5. See for example Summary of Bill & Melinda Gates Foundation-supported HPV Vaccine Partner Activities.

    6. Cochrane announces support of new donor.

    7. Are Cochrane Reviews Truly “Independent and Transparent”? World Mercury Project. 5 June 2018.

    8. Markowitz LE et al. Prevalence of HPV After Introduction of the Vaccination Program in the United States. Pediatrics. 2016 Mar;137(3):e20151968. doi: 10.1542/peds.2015-1968. Epub 2016 Feb 22.

    9. Tara Haelle. HPV Infection Rates Plummet In Young Women Due To Vaccine. Forbes. 23 February 2016.

    10. Quadrivalent Human Papillomavirus Vaccine. Recommendations of the Advisory Committee on Immunization Practices (ACIP).

  17. Further to my previous comment re the Cochrane HPV vaccine review and the undisclosed conflicts of interest of the CDC’s Lauri Markowitz.

    On 17 August 2018, Cochrane Editor in Chief Dr David Tovey responded to me saying “Dr Markowitz withdrew as an author on the review between the protocol and review stage”.

    However, under the Contributions of Authors section of the Cochrane HPV vaccine review, Lauri Markowitz is listed for her participation in the conception of the systematic review, writing of the protocol, and critical review of the manuscript. In the Acknowledgements, Lauri Markowitz is specifically acknowledged for “her invaluable advice and contributions by reviewing the results and discussion sections”.

    Saying that Lauri Markowitz is not an author of this review is a sleight of hand.

    I suggest Lauri Markowitz has in fact been a very influential author of this Cochrane HPV vaccine review.

    Questions remain unanswered by Cochrane, i.e.

    – How and why was Lauri Markowitz’s original participation in this Cochrane review approved by Cochrane?

    – Who initiated the Cochrane HPV vaccine review protocol?

    Lauri Markowitz has significant conflicts of interest via her employment with the Centers for Disease Control & Prevention (CDC), her involvement with the promotion of HPV vaccination, and her publication of papers on the subject of HPV vaccination.

    In the protocol for this Cochrane review, published in December 2013, Lauri Markowitz formally declared “no conflict of interest” in the Declarations of Interest section. I suggest this was misleading as Lauri Markowitz demonstrably does have conflicts of interest in regards to HPV vaccination.

    In their comment on the 2013 protocol, (which now appears to be inaccessible online), Catherine Riva et al pointed out Lauri Markowitz’s conflicts of interest, i.e. her employment with the CDC and her support of HPV vaccination, including via her participation as an author in continuing education programs for medical practitioners, i.e. HPV Vaccine: A Shot of Cancer Prevention, supported by Merck.[1]

    It’s notable that Lauri Markowitz’s conflicts of interest have still not been disclosed under Declarations of Interest in the recently published Cochrane HPV vaccine review.

    As I argued in my previous comment, this Cochrane HPV vaccine review is severely compromised. It is demonstrably not independent and cannot be trusted. The ‘scientific expert reaction’ lined up to support this Cochrane review is also tainted by conflicts of interest.[2]

    It’s remarkable that Cochrane have got this so spectacularly wrong. They’ve really undermined their whole ethos, i.e. to provide unbiased information.

    As a citizen interested in HPV vaccination, I do not trust this Cochrane review and I do not trust Cochrane.

    It’s unconscionable that this obviously biased review might influence vaccination policy.

    The Cochrane HPV vaccine review should be withdrawn.

    1. Catherine Riva et al include reference to CME/CE HPV Vaccine: A Shot of Cancer Prevention, as hosted on Medscape, 2012-2013.
    2. Scientific expert reaction to new Cochrane Review on HPV vaccine for cervical cancer prevention in girls and women.

  18. Dirty laundry shouldn’t be washed in public!
    They should have settled this matter privately first.
    There are many ‘no vax’ people even among doctors, and this dispute can cause potentially serious damage (you admit that it’s very easy to lose trust; and the emotional response could be dangerous).
    With vaccines we only have two possibilities: either vaccinate or not.
    But while vaccines are not perfect and may have various side effects, the diseases prevented by them are definitely much more serious and expensive.
    So let’s work to build on top of this.

  19. Carlo Liverani, do you have any idea how many vaccines and revaccinations are on the schedule now? See for example the Australian schedule:

    Have you had all these vaccines and revaccinations? Would you be willing to roll up your sleeve and have all these vaccinations, a la Paul Offit and his flippant statement that a baby’s immune system could handle as many as 10,000 vaccines, and upping the ante by saying it was probably “closer to 100,000”.[1] Did Offit consider the impact of adjuvants and other vaccine ingredients in his careless remark?

    More and more lucrative vaccine products and revaccinations are being added to international schedules, including for very rare diseases such as invasive meningococcal B. These vaccine products are added without any consultation with the community, and are even being mandated by governments in countries such as Australia, the United States and Italy. We have no idea of the long-term cumulative consequences of this increasing vaccine load.

    Vaccination policy has been captured by vaccine manufacturers, and by influential academics often conflicted by their associations with industry via their participation in vaccine clinical trials, industry-funded conferences etc.

    In my experience of questioning vaccination policy, protagonists seem to be ideologically driven, treating ‘vaccination’ as a religion that must not be questioned, and refusing to be accountable for individual vaccine products.

    In my country, Australia, coercive vaccination lobbyists patrol public forums and sabotage citizens’ discussion of vaccination policy, this has been my personal experience. Citizens questioning vaccination policy are often censored, for example I’ve been banned from participating in public comments threads on the government and university-funded The Conversation website, which publishes articles promoting vaccine products.

    Compliant politicians facilitate burgeoning taxpayer-funded schedules, bolstered by a biased media which often fear-mongers about disease, e.g. HPV and meningococcal B.

    Yes, the desire to prevent disease with a ‘magic bullet’ vaccination is understandable. But are these products all they’re cracked up to be? Problems are emerging, e.g. with the problematic pertussis vaccine.[2]

    We desperately need independent and objective evaluation of vaccine products.

    I suggest the over-use of vaccine products could be leading to a disaster even greater than those already unfolding with the over-use of antibiotics, opioids, anti-depressants and other medical products.

    We must stop the polarised ‘pro’ / ‘anti’ vaccination war that has hindered scrutiny of the booming vaccine market, and start having some considered discussion on the proliferation of these products.

    1. Stomping Through a Medical Minefield. Newsweek Magazine, 24 October 2008.
    2. Resurgence of Whooping Cough May Owe to Vaccine’s Inability to Prevent Infections. BU School of Public Health. 21 September 2017:

  20. This is a lot of effort to go to, to try to discredit the intellectual independence of someone who isn’t an author of the Cochrane review. I’m grateful for it though, because I had found the idea, mentioned by several people, that this was all about the Gates Foundation, quite perplexing. And you’ve spelled out this theory: NIH patents and licenses some tech (a practice which I’ve criticized here, by the way), the NIH is a US government agency, the CDC is also a US government agency, Markowitz works for the CDC, and this somehow affects Markowitz in such a way that renders her incapable of being academically rigorous.

    There are so many long bows being drawn in that part, and so much conjecture, that I believe this meets the definition pf a conspiracy theory (per Oswald 2016). Here’s a brief bio of the person we’re talking about: a remarkably public-spirited career. Here are the CDC guidelines that govern her work at the CDC.

    The argument, though, that there’s an intellectual conflict of interest, because she’s already convinced by the evidence is different. That does not, however, automatically mean that you are incapable of changing your mind if the evidence suggests something different. And sooner or later in the work, you’re going to come to an opinion. The question is, how well can you manage your own biases? Because everyone has them. As Bero and Grundy argue (see my post), there aren’t going to be enough people to do the work who have zero relevant interests.

    That can be done, though. For example, in the UK, a panel was assembled to look at mammography for breast cancer screening. Impeccable, highly trusted and qualified, people were sought, who had never worked on the mammography issue before. Yet even then, when people didn’t like their conclusion, they dismissed their findings. Here’s something relevant the chair, Michael Marmot said:

    …one only has to look at the author of an article to make a reasonable guess whether it will be pro or con”.

    Which I think is the point you are making. The trouble for me here is, it’s being made to discredit a study with conclusions you don’t agree with. Preconceptions are held by people whose work supports your view, too. It has to, for me, in the end come down to the science. And in my opinion, the science of the critique of the Cochrane HPV vaccine is worrisome on multiple grounds, and shows signs of serious bias as well as error.

    It will be difficult, as I point here, for whichever side of this polarized debate turns out be wrong. This one will be more clearcut than many other questions like this, because the magnitude of harm experienced by the large number of women diagnosed with cervical lesions, and the seriousness of cervical cancer for those who develop it, is such that the balance of possible benefits versus possible harms looks likely to tip dramatically towards benefit. And we’ll start getting a better handle on whether that turns out to be right in the next few years.

  21. It’s not of itself sleight of hand to say that someone who helped with the protocol and peer reviewed the review that resulted form it is not an author. Author teams for Cochrane reviews can change from update to update, too. Doing the actual work of the review is a lot more than peer reviewing. Her level of participation is declared.

    Full disclosure of interests of all peer reviewers is not something you see often in a journal – although I’m an advocate of open peer review, partly for just this reason (see e.g. here).

    New in this comment is the concern about teaching in a Medscape course in 2012 that was funded by “an independent educational grant from Merck”. That doesn’t mean she was paid for it: if she did this as an approved work-related activity, she wouldn’t receive payment, for example (see my previous comment for CDC guidelines). But whatever the case, it was years before being an informal peer reviewer.

  22. For those wondering about whether or not fears about HPV are exaggerated, I have gathered some data in this post, and I’ll draw from that here.

    HPV causes most cervical cancers, and some others (including oral cancers). HPV infection is very common in people who have had sex, and can cause genital warts. In terms of cervical cancer, most HPV infections clear up by 2 years, but about 30% lead to abnormal cell changes. The rate of progression to invasive cancer from moderate or severe cervical lesions is about 15%. Invasive cervical cancer leads to death in about a third of the women who get it in developed countries.

  23. Thank you for your evaluation about these things. I want to comment on some of your arguments. At first about the study inclusion.

    Trials numbered 1 and 2 you can find that data on clinical study reports if you look carefully

    Trials 4, 6. in there is no information provided about previous outcome measurements. So I think there is a possibility that there is data that is eligible and you can get that from ema.

    trial 9. There is comment on Original secondary outcomes that they were also assessing cervical lesion prevalence, so I think the data should be available from ema.

    trial 11 they argue that there is only data about safety outcomes that is eligible for inclusion.

    So I think this explains why copenhagen group think these studies are eligible. I think its also misleading to say those trials did not assess the primary outcome for cochrane review. The data was available at least for the first two and might as well have been for the other 3.

    I also think that this is good example to remind that its many times good to look a little bit deeper.

    About SAEs they explain in their last blog post why did they put it that way in the first place. So I think your argument is no more valid about that. Also I understood it the same way as you did at the first place and it looks like to me that they are just trying to hide that they were wrong at first place. I might still be wrong as well about that, it just looks like that.

    In my opinion copenhagen group does not seem like that ”biased” as you are giving us a picture when you take these thing to consideration.


  24. According to Professor Ian Frazer, a co-inventor of the technology enabling the HPV vaccines, the risk of cancer associated with the HPV virus is very low.

    In an article published on the university and university-funded The Conversation website in July 2012, titled “Catch cancer? No thanks, I’d rather have a shot!”, Professor Frazer stated:

    “Through sexual activity, most of us will get infected with the genital papillomaviruses that can cause cancer. Fortunately, most of us get rid of them between 12 months to five years later without even knowing we’ve had the infection. Even if the infection persists, only a few individuals accumulate enough genetic mistakes in the virus-infected cell for these to acquire the properties of cancer cells”. [1]

    “Catch cancer? No thanks, I’d rather have a shot!”. The Conversation, 10 July 2012:

  25. The harrassment didn’t take place though did it? I have been very naughty and read the confidential lawyer’s report.

  26. This wasn’t the subject of the lawyer’s report, though, so implying this was investigated and there was a judgment that it did not occur isn’t a justifiable conclusion. Ongoing harassment of staff was not one of the complaints the lawyer was asked to investigate: indeed, rather the reverse is the case. The second of the 3 complaints related to Gøtzsche’s contention that the CEO acted inappropriately: the lawyer found that claim unsupported by the evidence. In the context of being interviewed about Gøtzsche’s allegation, the CEO reported that he believed Gøtzsche acted with personal animus to him. The lawyer came to no conclusion on this, as he could not investigate this fully, but concluded there was prima facie evidence to support the CEO here (#153 on page 37). But the CEO isn’t the only member of staff on the front line of this, recently, or over the years.

    The lawyer was asked to investigate very specific circumstances, advise on the legal status of those and grievance mechanisms/paths to conflict resolution, and make recommendations on ways Cochrane could improve its governance processes. He made it clear he was not an arbitrator, and that the Board needed to act as it saw fit, subject to its wider obligations (#86 on page 21). The obligation to provide fair working conditions is one of those.

    Several former chairs and other former board members, like me, have publicly stated that we have witnessed this behavior. But even if I had no knowledge of any of the parties or context, the specific set of incidents would show some patterns of behavior: on the one hand, of a senior person in an organization going rogue, and on the other, of a succession of his colleagues across years, trying to find ways to mitigate the consequences of that person’s actions, find accommodations, and develop pathways to prevent further incidents, to little effect. It suggests some contours for the tip of an iceberg, not the totality of what occurred over a couple of decades. No one seems to be suggesting these were isolated or uncharacteristic incidents. For conscientious people who are unable to avoid this kind of behavior from the same person, repeatedly over years, because of the circumstances of their employment, the situation can become intolerable. People are entitled to workplaces (and, indeed, lives) free of bullying.

    Leaving aside staff members, several of the incidents the lawyer describes include behavior towards other Collaboration members that, at the very least, are un-collaborative, and that most members of the Collaboration would likely find distressing if they were ever at the receiving end. (E.g. accusing colleagues of being motivated against patients’ interests, in a letter printed in the BMJ. That part has been deleted online, but he reiterated the accusation while apologizing (sort of) for it here.) This kind of behavior, especially when it is repeated to an expanding group of people over years, is deeply destructive, and antithetical to scientific and community collaboration. But it’s difficult, I think, for those who haven’t seen it, or been on the receiving end, or who aren’t disturbed by it, to appreciate what I described in my post as “a very heavy bucket of pain, lost energy, and people lost to the Collaboration, too”, going back a couple of decades at least. It goes to the heart of current community debates about bullying behavior from powerful academics, doesn’t it? Doesn’t, in the end, the harm an individual is doing to others outweigh whatever their own positive contributions are?

    Finally, assessing the potential impact on patients and the public of activities undertaken in Cochrane’s name was not something the lawyer was asked to assess either. That’s not trivial, and it’s something I believe the board has a duty to put front and center, alongside providing a good workplace and effective governance.

  27. Hilda

    You talk about bias like you’re immune. Do you think hedging your whole career and reputation on vaccine safety just might cloud your judgement a little?

    This is by no means the first time Cochrane’s independence has been questioned. Indeed this has been going on for over a decade. They simply have no credibility left.

    You can lie, twist the evidence and trash dissenters until we’re all blue in the face but you know that in the end the scientific method will win through. History will not judge you kindly, whereas your esteemed colleague Gøtzsche will be remembered as one of the few with the courage to speak out. No doubt your grandchildren will have a particularly hard time understanding.

    Perhaps before you compose another long and vacuous post on HPV, you might remove your head from your behind for a few minutes first and explain the situation to Mia Blesky.

  28. I don’t think anyone is immune to bias. I definitely haven’t staked my career on vaccine safety. And I don’t think I’ve staked my reputation on it, either. The HPV vaccine is the first vaccine I’ve publicly strongly supported, and I have been critical of others. If my reading of the science and politics of the HPV vaccine were to define my work, though, I would be ok with that.

    I obviously don’t know how history will judge me on this, and neither, of course, do you. As I wrote in this post, the evidence is pointing to a reduction in HPV-related cancer for women in countries with high HPV-vaccination rates, but if the evidence shifts against the vaccine, my position will change with it.

    My grandchildren’s parents made their decisions about vaccines without any reference to me, or influence from me. I fear, though, as I wrote in that post, that there could be a mountain of decision regret on the horizon about not vaccinating, if the anticipated drop in HPV-related cancer eventuates.

    I looked at the video about Mia Blesky you linked to. Mia’s experience is heartbreaking, and I very much hope she has recovered, or does so soon.

    I also searched for more information about her. I could only find reports like this and this: the facebook page about her condition set up by her mother has apparently been deleted.

    (For those interested in finding out more about her case, from what her mother said, it sounds like her doctors were saying this was conversion disorder, triggered by bullying or other distress, which her mother denies as a possibility. Here’s a TV report about that condition, including reports of recovery from it.)

  29. As my posts make clear, the risk of cancer from HPV infection is indeed low. However, the infectioncauses most cervical cancer and some other types. The number of people affected by cancer lesions that will be detected and treated, which is distressing and has its own adverse effects is high. Per my posts (here and here) around a million abnormal pap smears in the US each year, and probably considerably more than a quarter of those women will have quite a high chance of being one of the 12,000 who are diagnosed with cervical cancer each year (of whom about a third die of the disease). Not at all trivial.

  30. Thanks for your comment, and apologies that it fell through the net for so long, as it took me a while to sort through lots of comments and “pingbacks” for this post.

    I’ve cked again, and disagree that any of the trials I specified as not assessing cervical lesions did in fact do so. In combing through this, however, I realized that my shortened description of which primary outcomes I meant was misleading, so I have corrected that to be clear I meant only the cervical cancer-related primary outcomes. If you can point to a page number in a document showing a cervical cancer-related outcome was measured in any of these trials, please let me know, and I will check that, and make any needed corrections.

    Yes, it seems to me too that the authors were acknowledging the review and trial had not been wrong about the number of women experiencing adverse events in that particular trial, while obscuring that it had been an error on their part.

    Thanks for your comment – it was very helpful. But I continue to believe the critique was error-laden and seriously biased, and that the material conclusions of the Cochrane review would not change with the incorporation of the additional data.

  31. Thank you for your comment!
    For the first study you can find the data on page 56.
    Actually when I checked again about the second study. There is no data on the clinical study report, but still you can find the data in assessment report released afterwards…
    And for the other trials my argument was that there might still be data. does not prove there is not. To show that jefferson et all were wrong you should have the data ema has.

  32. Thanks, but page 56 is HPV ELISA titers – looking for signs of response to the vaccine. That’s not a cervical cancer-related income. Immune response studies are important. But if that’s all they are, then they don’t last long enough to assess clinical endpoints like cervical lesions and cancer, or longterm adverse effects – and they don’t need to, because that’s not their purpose.

  33. I tried to refer to page one before that so its 55 if you look numbers in the document. Its 56 if taking the numbers on the file… Sorry for unclearness. Did you look at the assessment report?

  34. I am both surprised and grateful you have published my comment and replied, for which you have my respect.

    By your own admission your recent work is strongly supportive of HPV vaccination and will certainly increase vaccination rates. Some parents will have been reassured by your evidence reviews and have chosen to vaccinate their children. Others will have been scared by your assertion their children may develop cervical cancer as a result of non-vaccination. Which is, of course, your unstated aim.

    So let me ask you this. If the evidence shifts against the vaccine, for example if it does not causally reduce cervical cancer rates, and/or if evidence of serious harm continues to come to light, you would be ok with that defining your work? And you would have no problem changing your position? Because I consider myself both rational, skeptical and level-headed, but were I in a position where my reading of the science and politics of a medical intervention directly led to the serious harm of (possibly) tens of thousands of women, I seriously doubt my ability to keep that review impartial. Indeed even a cursory knowledge of behavioural psychology suggests I would (unwittingly) fight tooth and nail to deny any possibility of culpability, and that’s without even having any direct financial interest..

    Your reading of Mia’s experience provides a useful case study. Here we have a mother clearly explaining the timeline of her daughter’s descent from healthy, balanced teenager to complete paralysis following her HPV vaccine. You must be aware from reports such as [a href=””] this one [/a] there is a plausible etiological pathway from vaccine to paralysis. Yet, after providing links showing mother was intimidated and threatened by online trolls for speaking out, you go on to insinuate she was lying and direct us to “Conversion Disorder”, an unexplained condition with NIL scientific basis. Seriously?

    My position is this. Whilst the majority of independent non-industry-sponsored studies raise grave concerns over the safety and efficacy of the HPV vaccine, and while serious and possibly criminal allegations of research fraud and scientific misconduct against international review and regulatory institutions such as GACVS and Cochrane remain outstanding, there must be global blanket ban. Meanwhile parents should choose alternative methods of treatment such as screening, antibiotics and contraception for the 9 – 12 year old girls and boys upon whom the vaccination is targeted, presuming they’re sexually active. If after a thorough, balanced, and completely independent scientific reviews reciprocating results of key industry sponsored studies is complete, and if safety and efficacy is proven, I would have no problem changing my position, nor any guilt over erring on the side of caution and reason.

Leave a Reply

Your email address will not be published. Required fields are marked *

Add your ORCID here. (e.g. 0000-0002-7299-680X)

Back to top