Boilover: The Cochrane HPV Vaccine Fire Isn’t Really About the Evidence – but it’s Critical to Science

 

It was always a volatile situation. If it blew, it was going to be even harder to keep perspective through all the fire, noise, and smoke. And it just did boil over – very, very, loudly. Now, getting clarity is urgent, and critical far beyond Cochrane.

 

First of all, what on earth just happened, and where do we go from here? Why is there so much talk about a Cochrane Collaboration crisis?

“It is about data”, one of the central antagonists said before the public explosion, according to a report in The Lancet. But while, yes, people are fighting about data and methodology, that doesn’t even begin to explain this explosion, its intensity, or its importance to science and society. If it were just about data, we would know what to do, and just knuckle down to it.

We have to get better at handling these incendiary situations that rapidly mobilize increasingly wide circles of people, though. The early days are vitally important. This might be our biggest chance to limit the damage, speed the recovery, and get something positive out of all this.

How did we go from a vaccine evidence dispute into a sudden crisis over expelling the Nordic Cochrane Centre’s director, Peter Gøtzsche, from the Cochrane Collaboration? Let’s try to untangle some of this.

I wrote about the dispute over the recent Cochrane HPV vaccine review here recently. I’ll discuss how that’s been unfolding first.

I’ll pick up where the previous post left off: with the Cochrane chief editors’ response, posted on their website on 3 September. Systematic reviewers from the central team who weren’t authors of the Cochrane review pitched in, to do an analysis of whether the review’s conclusions were dangerously flawed. These systematic reviewers and editors are separate from the ones who authored and edited the review.

They asked Jørgensen and Gøtzsche, who are both from the Nordic Cochrane Centre and authors of the critique, to identify the 20 trials this Copenhagen group had said were eligible and missed. It wasn’t provided.

So over at Cochrane HQ, they slogged through the list for the full index the Copenhagen group had previously published and sent to the Cochrane group who were finalizing the review.

The Cochrane chief editors’ response has a flowchart breaking down how many were unique studies, how many were completed, how many they considered eligible for the review and so on. In all, 137 potentially relevant trials were independently assessed by 2 systematic reviewers, who found 5 eligible trials that the Cochrane reviewers had not found in their own searches. There were also some that would be eligible, but the necessary data weren’t yet publicly available. Trying to get unpublished data had been planned by the authors as their next task: Cochrane reviews can be updated at any time after they are published.

The Cochrane response reads “we do not underestimate the importance of the missing data”, but it wouldn’t, they believe, change the review’s conclusions once it’s added. Only 1 trial had data on the primary cervical cancer-related outcomes – moderate or worse cervical lesions or invasive cancer. (See my original post for an explanation about this.)

There’s new information in the response on the conflict of interest issue:

Jørgensen et al also stated that the lead author of the review leads the European Medicine Agency’s post-marketing surveillance and linked this to funding from a manufacturer. In fact, Professor Arbyn took the initiative to introduce a surveillance study in his country after having been informed that the European Medicine Agency had approved the Gardasil vaccine, remarking that the post-marketing surveillance conducted in Northern Europe was relevant but should include also non-Nordic countries. Professor Arbyn is not funded by the European Medicine Agency nor by any vaccine manufacturer.

While the Cochrane editors concluded that the criticisms were “over-stated” and won’t change conclusions,

Some of the criticisms will inform the next version of this Cochrane Review and the planned review of comparative studies of HPV vaccines.

They will, they said, review any list of trials provided, and try to account for discrepancies.

 

Meanwhile, the editors of the journal that published the critique, BMJ EBM, responded to the Cochrane editors’ response on the 12th. The conflict of interest statement notes that the editor-in-chief (Carl Heneghan) is a close colleague of Tom Jefferson, one of the Nordic Cochrane Center authors of the Copenhagen critique, and an advocate of the same approach to systematic review methods. He’s a Cochrane editor/reviewer, too.

While the Cochrane editors involved people independent of the original work to weigh in, reporting the shortcomings they found there, the BMJ EBM editors see no problem in what they published, or their process. Cochrane criticized the adequacy of their peer review: their response is essentially, How so? We see nothing wrong.

I’ve criticized the adequacy of the editorial process too, so here’s my answer to that question.

Publishing a claim that half the eligible trials and participants are missing required references: how else could peer reviewers, editors, and later readers, reasonably assess the validity of the claim?

When I started fact-checking the critique, it took only minutes of fact-checking to see that some of the criticisms were errors. It took only the normal effort level of peer review to see the serious error of saying Cochrane and the trialists had under-reported the number of serious adverse events in a trial, when what they reported was in fact the number of women experiencing adverse events – which is of course smaller.

I had pointed this out to them too, and asked if they were going to issue a correction. No response, and no correction.

The BMJ EBM editors published the peer reviewer comments and authors’ responses. It doesn’t look as though the peer reviewers considered validity assessment or fact-checking part of their remit, either.

The editors conclude their response with this:

…articles in our journal will seek to hold organisations to account and will and should not shrink from offering criticisms that may be considered inconvenient. Academic freedom means communicating ideas, facts and criticism without being censored, targeted or reprimanded. We believe that the article by Jørgensen et al provokes healthy debate and poses important questions about the need to ensure that all available evidence is included in systematic reviews to properly inform healthcare decisions.

This is useful to parse, I think, because it’s so clearly articulating several issues that are keys to why responses the last few days are so polarized. I’ll come back to the question of accountability for Cochrane later, because that’s also key.

This statement puts academics and their freedom front and center. Academic freedom is, to many academics and others, a protected value. Here’s how I’ve explained this before:

Most of us have some kind of “protected values”. They are ones that have a moral force for us. We don’t like to trade them off against anything. A protected value in the mix is a common source of rigidity and anger that ramps up to outrage.

Invoke the academic freedom value, and enough’s said, to people who hold it that way too. It’s the core of an identity and the ability to pursue the truth, wherever it might lead. To work, as a scholar, in accordance with your own principles. Its opposite is exemplified by academics in prisons. “Academic freedom” is, therefore, an extremely emotive term.

But I’m not an academic, and I don’t see this is as an academic matter. I don’t think academics’ values and interests should be the main point in public debate. There are limits to all freedoms – right where they butt up into responsibilities.

You’re only communicating a “fact” if it’s actually true. That requires serious diligence. Some ideas are extraordinarily damaging, to individuals and society, and have no redeeming value. Lots of ideas academics had/have about race, for example, demonstrate that. And there are prerequisites for healthy debates.

Throwing flammable fuels around open flames isn’t “healthy” – not even for academics. When I read that closing paragraph, it crashed into my protected values, and I was instantly disturbed. It felt familiar and that nagged at me for a while. Then I realized what it reminded me of. And I think the analogy helps explain my protected values here. Bear with me on this.

It reminded of Richard Horton, editor of The Lancet, in 2004, justifying first publishing and then not retracting, the Wakefield paper that launched the “MMR causes autism” societal crisis. Except that instead of not shrinking from being “inconvenient”, the language was about not being “conservative” about “sometimes unpopular thinking”. But the argument was the same one.

It wasn’t until measles deaths returned after a long absence, and the General Medical Council stepped in, that The Lancet retracted the paper… another 6 years. Now Europe is tracking measles epidemics and deaths again, and in the US, a 2015 poll found only 41% of people were pretty sure vaccines don’t cause autism.

Here we now stand on what seems to be the brink of a drop in cervical cancer and other HPV-related cancers for cohorts of women from high HPV vaccination countries. And safety panics are being stoked: so much, that in Denmark, vaccination rates plummeted from around 90% of 12-year-old girls in 2014 to less than 40%. A similar phenomenon happened in Japan.

Ideologically or commercially motivated doubt-sowing are causing serious damage, and we’re not good at dealing with it yet. Journalist Ray Moynihan published an opinion about the Cochrane crisis focused on this, which includes:

What’s at stake in the current bloody fight unfolding within Cochrane’s Governing Board, is not just the credibility of individuals or organisations, it’s the future of reliable trustworthy evidence in a world of increasing falsity and fake news. To see this future threatened foreshadows a disaster for all of us.

The concept of a healthy debate has to take the reality of these consequences into account, and we have to find a balance that has room for the different principles. The stakes are high inside and outside of the ivory tower. That means accountability and responsibility are critical for Cochrane – but they’re just as critical for other scientific journals as well.

When people find critical data they should put it out there. I don’t agree with the Cochrane implication that it shouldn’t be done publicly. But if you’re going to publish evidence about vaccine effectiveness and safety in 2018, or launch major allegations about it, you have a responsibility to exercise a lot of diligence, with openness to ensure verifiability of claims.

Which brings us to the Copenhagen group’s response to the Cochrane response to their critique. BMJ EBM posted it on the 17th. That was right in the middle of the boilover, but let’s finish this strand.

The Copenhagen group report that they have now submitted their own systematic review for publication somewhere, and are now willing to share unpublished data with the Cochrane authors. That means to me, it’s not worth going too deeply into this – I’ll wait for the publication to try to come to grips with much of the detail. Because now, the effort needs to shift to the Cochrane review updating – that’s one of the advantages of a Cochrane review. Whatever its flaws now, they can be fixed.

The first big issue is the discrepancy in judgments about eligible trials. Here’s my analysis, based on the table where they provide details about the trials. [PDF]

Before:

• Cochrane included 26 trials, with 73,428 girls/women.
• The Copenhagen group said half were missing, and there were 46 eligible trials included in their database, for 42 of which there was available data on 121,704 participants altogether. So that’s 16 to 20 trials they said were missing from the Cochrane review.

After:

• Cochrane judged 5 trials with available data to be eligible, along with 3 without data available: for a total of 34 eligible trials.
• The Copenhagen group now reports that there were an additional 17 trials with data, of which 11 are eligible and 6 that were only potentially eligible: for a total of 37 eligible trials.

It turns out, the Copenhagen group only looked at the list of included studies – not the list of excluded studies (studies considered and rejected) or the list of the rest of the references. (You can see how that’s organized here – it’s  standard for all Cochrane reviews.) They wrote:

When we checked again, we found some of the studies in the review’s reference list (3). The Cochrane HPV reviewers chose to use idiosyncratic referencing with study IDs such as “Phase 2 trial (ph2,2v)”, “Immunobridging (ph3,2v)” and “CVT (ph3,2v), which made the study assessment complicated. For numbers of participants, we did not subtract the male participants that were included in three of the studies, as we should have done.

What about the extra 6 trials they deem eligible, in comparison to the Cochrane audit?

• 3 were in the list of excluded studies in the original Cochrane review. because data wasn’t available, either for the whole trial or just female participants. The Copenhagen group has available unpublished data for what they report as 1,400 women in these trials, some of which are publicly available. [Trials they numbered 1, 4 (also here), and 6 (also here, which seems to have 100 more women than Copenhagen logged).]
• 1 is a comparison of gender-neutral versus female only immunization strategies and herd effect. It was excluded by Cochrane as a phase IV trial. Copenhagen says it is reported as a III/IV trial in one place. It has 20,515 female participants.[Trial 2, also here.]
• 1 is a feasibility trial, not originally judged to be eligible by the Copenhagen group. It had 406 female participants. [Trial 9.]
• 1 is a genital warts prevention trial, not cancer prevention. It has 200 female participants. [Trial 11 – and I’m pretty confident there is no trial with the NCT number Copenhagen provided for this trial.]

These trials all have conclusions consistent with that of the Cochrane review. Only 3 have been considered eligible by Cochrane so far, and it’s possible that judgement will remain. Only 1 trial is large, but it’s one that doesn’t seem to be eligible. I think the variance in determining eligibility could be reduced if the Cochrane review’s inclusion criteria were more specific.

I’d be surprised if there was a material difference to the Cochrane review from this. The Copenhagen group has not acknowledged their error with numbers of events versus participants.

There are still issues the Cochrane authors will need to address in their update about adverse events: additional questions were added in this round, too. And in this response, the Copenhagen group criticize the way the word “placebo” is used in the plain language summary for the review. I agree Cochrane needs to change it.

They also make additional allegations about author conflicts of interest, which need to be addressed: Cochrane has a process for that. Not that I agree with everything the Copenhagen group are saying.

For example, there was a major trial in Costa Rica, funded by millions of dollars of US tax payer money. The vaccine manufacturer provided the vaccines to the National Cancer Institute (NCI), and the trialists got it from NCI. They also got assistance from a company related to information in their regulatory materials. “We consider this industry funding”, the Copenhagen group write. Hmm.

So let’s get to the boilover. The hyped-up critique in a trusted publication turned the Cochrane review and Cochrane itself into shark bait. Bad enough, but how did it escalate?

 

 

 

There’s a lot of reporting of what went down – but the early reports mostly came from people closely connected to team Gøtzsche, or ideological partisans for whom this is a propaganda/validation gift. The Cochrane statements are couched in a lot of deliberately vague language, given the legal situation they are in, and people’s right to privacy. Here’s what I think seems to have happened.

Cochrane got legal advice about their legal standing in relation to complaints made about Gøtzsche in March. They reviewed the history of previous incidents, as well as communicating with him. Gøtzsche then made allegations about a staff member:

We were advised that various legal consequences flowed from the events – the complaints and the accusations – and that Cochrane should take them seriously.

We asked the lawyers to take particular note of Cochrane’s commitment to transparency. They noted that, but also stressed the importance of confidentiality.

They advised that an independent review was both a sensible and proportionate response.

The board discussed and accepted the advice to get a very senior lawyer to conduct an independent review of the allegations about the staff member in June. The lawyer was engaged on 2 July, with a deadline for a board meeting on 13 September. That was when the international membership would gather for the annual conference (the Colloquium) in Edinburgh.

A few days later (7 July), Gøtzsche and colleagues’ critique of the Cochrane HPV review was accepted by BMJ EBM. It was published on 27 July, and Cochrane received further complaints.

The investigatory process was definitely not a response to the HPV critique.

At the board meeting on the 13th, the report from the lawyer and the HPV critique issues were discussed. There was a motion to expel Gøtzsche from membership of the organization, for 1 of the 3 grounds for expelling a member:

[5.2.1] is guilty of conduct which has had or is likely to have a serious adverse effect on the Charity or bring the Charity or any or all of the members or Directors into disrepute. [PDF]

It passed, but barely. Of the 12 board members other than Gøtzsche, 1 abstained. The motion was carried by 6 out of 11. Of the 5 opposing, 4 then resigned from the board, all but 1 of them being from a Cochrane Centre. It was acrimonious.

It’s striking that so many board members were from Cochrane Centres. That’s one of several specific sub-cultures in the organization, and it’s a small one. It’s also striking that the remaining board was all female, but for 1 Cochrane Centre director (and Gøtzsche). (3 of the 4 who departed were men.)

The Cochrane Collaboration is a registered charity and it has to have a functioning board at all times – so 2 more stepped down to get the proportions of member types conforming to the rules.

Gøtzsche had 7 days from the 13th to in effect appeal the board decision in writing. After that, the board makes a final decision, for which there is no right of appeal. What this means now for the Nordic Cochrane Centre isn’t clear to me. There are formal agreements between Cochrane Centres and the CEO, but I don’t know what provision that makes for this type of situation.

The road to that board meeting was a long and tortuous one: it’s been coming for years. But the events of the last few days went down fast.

Within 24 or so hours, Gøtzsche posted a statement on the Nordic Cochrane Centre website, which we’ll get to. [PDF]

The Board issued a very minimal statement about a day later it seems, and the 4 who resigned in opposition issued their own after that. By the AGM on the 17th, though, the Board had a prepared statement. They emphasized the right of people to a work environment free of attack and harassment.

 

 

The Board members who don’t support the decision and resigned see it differently though: their statement spoke only of people’s right to dissent and speak their minds. They viewed the expulsion as more likely to bring the organization into disrepute than Gøtzsche would.

Gøtzsche’s statement describes the decision as a minority decision (6 out of 13, of which he is 1), and says in 25 years no member has been expelled before. But as one of the original founding group (as am I), he knows formal individual membership of the Cochrane Collaboration from which the board could expel you didn’t exist for all those years. In fact, it’s quite freshly minted. And people have been forced out before, one way or another. It would be a rare group if that were not so, wouldn’t it?

Gøtzsche raises wide-ranging concerns:

• Criticism of the Cochrane CEO’s management style in particular, and “senior central staff” in general.
• Cochrane’s evolution, strategic direction, and corporate culture, saying it has become top-down, and overly oriented to making money.
• Allegation that there have been threats to the Nordic Cochrane Centre’s existence and funding because he has been a vocal critic.
• That there are reviews that are too favorable because of conflicts of interest.
• Cochrane’s PR of reviews is slanted to reviews with favorable results and is too biased.
• That Cochrane’s leadership is becoming less committed to open science, progressive civic/political science advocacy, and resistant to criticism of the drug industry – including resistance to tightening its 2014 conflict of interest policy.
• Objection to the language and ethos of “the brand”, and walking away from the name “Collaboration” and what that represents.

The phrase “moral crisis” is in the title of the statement, and he’s arguing that he, his strategies, his work, his priorities, reflect the original public-minded spirit of Cochrane and that’s why “they” need to get rid of him:

I have been expelled because of my “behaviour”, while the hidden agenda of my expulsion is a clear strategy for a Cochrane that moves it further and further away from its original objectives and principles. This is not a personal question. It is a highly political, scientific and moral issue about the future of Cochrane.

I agree with him on several counts – I finally parted ways with the Cochrane Collaboration in 2012, because of the commercialization, open access issues, and the strategic direction it chose. There are real problems, although there are obviously enough people who want it this way.

But while I would love to see Cochrane change course, I think in this moment, the risk of the organization being pushed to the fringes matters too. Extremism, and the perception of it, can repel moderates and attract more extremists in a vicious circle. And fundamentalist extremism and bullying do tend to travel in tandem, don’t they? Back to the board’s statement:

All our staff, and our members, have the right to do their work without harassment and personal attacks. We are living in a world where behaviours that cause pain and misery to people, are being ‘called out’. This Board wants to be clear that while we are Trustees of this organization, we will have a “zero tolerance” policy for repeated, seriously bad behaviour. There is a critical need for ALL organizations to look after their staff and members; once repeated, seriously bad behaviour had been recognized, doing nothing was NOT an option.

The board speaks of the history of conflict and complaints:

In fact, the earliest recorded goes back to 2003. Many have been dealt with over the years. Many disputes have arisen. Formal letters have been exchanged. Promises have been made. And broken. Some disputes have been resolved, some have not.

 

Those few words have to carry a very heavy bucket of pain, lost energy, and people lost to the Collaboration, too. But this is a man who is also loved and admired by many, who don’t see the problem.

Like Gøtzsche, I was “first generation Cochrane Collaboration” – part of the group Iain Chalmers assembled to found an international organization out of the roots of the Cochrane Centre he had inspired into life recently in Oxford. I served on the board of the international organization from its founding in 1993 to 2001. Here’s a description by an observer in 2013, around the time I parted ways with the organization:

The Cochrane Collaboration offices are small. If you have ever been to Summertown in Oxford, United Kingdom, chances are you passed them without even knowing. Standing outside their modestly sized single-floor office makes you wonder just how they wield such clout.

After that, the staff grew quickly, and moved into premises in London. There was a period of very rapid change. Even apart from strategic issues, you can see why some would be nostalgic for the old days. But it wasn’t all sunshine and roses. Or at least, there were a lot of thorns on the roses. Change isn’t always bad.

Around the time Gøtzsche began advocating for an end to mammography screening, I wrote a piece in the international Cochrane newsletter (May 2002). It was considered incendiary enough to have a disclaimer about it being my personal view. Now that’s definitely something you never saw!  I was ruminating on whether the valorizing of fighting like it’s a holy war and iconoclasm was antithetical to collaboration:

How much collaboration can there be among people who symbolically denigrate each other’s motivations and work? How do we use critical analysis creatively, without descending into using intellectualism and science as weapons? Isn’t defensiveness, rather than cooperation, the usual human response to attack? So what role does a warrior culture, or a warrior element, have in a collaboration?

I’m even more sure now that all this aggression is antithetical to collaboration, and to science.

Now, the NCC cites the prevention of introduction of mammography in much of Denmark for a time as an achievement [PDF]. I’ve written about the controversy over mammography evidence before, too. Anti-mammography activism made Gøtzsche a champion to many, but increasingly polarizing.

There weren’t as many who thought he was a champion when he called in the BMJ for a stop to longterm use of almost all psychotropic drugs, based in large part on his then soon-to-be-published book, Deadly Medicines and Organised Crime: How Big Pharma has Corrupted Healthcare.

I haven’t read it: here’s a book review on PLOS Blogs with Tom Yates talking about some strengths, but also concern about hyperbole, offensive rhetoric, selective use of evidence, and unsupported claims (like “bipolar disorder … is mainly iatrogenic, caused by SSRIs and ADHD drugs”). Here’s another, from Richard Smith, that shows the other side – the affection and admiration.

It’s seemed forever locked between these poles, but the board is totally right: bullying of individuals is the tie-breaker. It has to be. And the law in many countries these days will back that up if someone wants to fight back.

Now back to the BMJ and concern over the psychotropic drugs essay. Cochrane’s editor-in-chief and mental health review group lead editors responded, to make clear even though Gøtzsche wrote this using his NCC affiliation, the views expressed were not those of the Cochrane Collaboration.

These episodic clashes are more than just inconvenient: they are deeply costly, in time and energy that could be used constructively. And yes, they do damage Cochrane’s reputation, and repel contributors. This is from a piece in Nature last year, when the NCC alleged various kinds of foul play at the European Medicines Agency:

This letter, sent on Nordic Cochrane Centre-headed stationery despite not being an official Cochrane Collaboration communication, focused heavily on the conduct of the EMA review and issues such as conflict of interest, maladministration and confidentiality. However, as the EMA highlight in their detailed rebuttal,¹² the authors ignored the limitations of the cited case reports and introduced basic errors such as accusing ‘the wrong Julie Williams’ of undeclared conflicts of interest.

Why does this matter? The Cochrane Collaboration has a long-held reputation of excellence, producing trusted high-quality reviews on thousands of topics related to health and has groups at national and sub-national level. The authors of the complaint, in using Cochrane-branded paper with the header ‘Trusted evidence. Informed decisions. Better health’, give the impression to readers that their views are representative of, or in some way approved by, the Cochrane Collaboration, and this is the view now being promoted in online anti-vaccine communities.

Which brings me back to my iconoclasm piece from 2002. When people self-identify as scrappy, independent, and anti-establishment, it’s a tough transition when you get big and powerful enough that you have to be responsible. But warrior culture is destructive, and inimical to informed decisions and a trusted provider of information.

The last few days, some were saying on Twitter that Cochrane isn’t an organization in crisis, despite the publicity. Trish Greenhalgh pitched in with a thoughtful analysis, including:

21st-century science is an intersectoral endeavour that necessarily occurs in dialogue with society. Maintaining—and funding—the “view from nowhere” requires delicate navigation of tricky political spaces and sometimes accepting hard-won compromises. Board members are presumably expected not to spit in the soup (especially when using Cochrane letterhead).

At this stage in a fast-unfolding story, I am not convinced that the Cochrane Collaboration is experiencing a crisis of either morality or democracy. Its brand, now as ever, stands for rigour, independence, and a commitment to using science to achieve high-quality patient care and social justice. We should cut it some slack while it gets its house in order.

And no, that doesn’t mean kowtowing to the drug industry. I think it’s true that it’s not an organization in crisis. But it is an organization in a crisis. In a systematic review by Mike Clarke and colleagues on communication in disasters, a disaster is defined as “a serious disruption to the functioning of a community that exceeds its capacity to cope within its own resources”.

The super-charging of anti-social social movements and conspiracy theories about vaccine safety and Gøtzsche’s alleged persecution affects far more people than the members of the Cochrane Collaboration. Something big has been unleashed. And a lot of people are outraged.

I’ve written before that I find “the outrage factor” theory in risk communication useful to think through situations like this:

The theory was developed by Peter Sandman [PDF]. You can’t manage communication about environmental risks effectively, he argued, if you don’t consider the level of potential to invoke an extreme emotional response.

A risk, in this view, is never just a hazard: Risk = Hazard + Outrage. The outrage factor may be anywhere from negligible to catastrophically high. Any situation with a high potential outrage factor is high risk, even if only a small direct hazard to people is involved.

He lists potential factors that can cause outrage, and Gøtzsche’s expulsion checks off several. I think the sections on trust and responsiveness (11 and 12) and the chapter on psychological barriers are helpful guides to charting your way through this.

You have to be scrupulously honest, accountable, responsive, and compassionate to be trusted. Faced with outraged people, all that’s extremely hard though, and it’s easy to lose trust:

Apart from the obvious distorting effects of self-interest, conviction probably is an even bigger source of bias. If you think you know the Truth with a Capital T, then cheating a little on some inconvenient lower-case facts does not seem especially dishonest…

Given that trust in industry and government is a slender reed that snaps when you lean on it, you need to stop leaning on it. That is, stop asking to be trusted. The paradox of trust is that the more you ask people to trust you, the less they trust you….

Instead of trust, it seems to me, the bottom line is accountability. The goal is to be able to say, truthfully, to a public that does not trust you, that it does not have to….“Track us, don’t trust us.”

For the people who identify less with Gøtzsche and the Cochrane Centre’s situation, and have a well of goodwill for Cochrane, trust won’t be a problem. But especially as this has splashed across the medical media and further, we’re going to have to work hard to develop the opposite of a vicious circle: spirals of trust.

Sandman points to the importance of responsiveness in building trust:

There are at least five different components of a responsive process: (1) openness vs. secrecy; (2) apology vs. stone-walling; (3) courtesy vs. discourtesy; (4) sharing vs. confronting community values; and (5) compassion vs. dispassion.

It’s hard for the people at the center of an organization being attacked to do this, because they have to suppress their own outrage and hurt about what people are accusing them of.

Here’s a concrete example of what Sandman means. Below, the director of the Canadian Cochrane Centre pointing to an error that the Cochrane board had made – writing something that would escalate conflict, and made a suggestion that offered Cochrane to demonstrate responsiveness. [Update 20 and 22 September]  This post originally included encouraging the Cochrane board to act on this suggestion. I’m pleased to say they did. on the 19th, which is moving quickly. (Thanks, Holger Schünemann and Cochrane Board!) And the Austrian Cochrane Centre removed the statement of the resigning members, for further reconciliation. (Thank you, Gerald Gartlehner & co!)

 

 

 

 

There is a profound irony, it seems to me, in the timing of this. There’s a fight going on over evidence about the HPV vaccine, with two competing systematic review groups. And both have a cut-off date for data just shy of when the first signs of the predicted drop in HPV-related cancer in trials appeared. Perhaps the most critical information in an upcoming Cochrane review update won’t have anything to do with current criticisms.

We need young people to be the winners here, whatever shape that’s going to take. They mustn’t be collateral damage in someone’s ideological war or ego battle. I’m very proud of the Cochrane Collaboration for pouring so much effort into this particular systematic review.

Cochrane, you have powerful methods for finding out where the evidence takes us and updating reviews and methods in response to new evidence and criticism. Don’t let attacks and wounded feelings take you off course.

 

Postscript: On 26 September, the Cochrane board announced that it had “voted unanimously on 25th September to terminate Professor Peter Gøtzsche’s membership of the organization, and his present position as a Member of the Governing Board and Director of the Nordic Cochrane Centre”.

 

 

Much shorter (!) follow-up post: Scientific Advocacy and Biases of the Ideological and Industry Kinds.

[Update] After the smoke cleared, I dug carefully into this conflict to review my position. I posted a timeline and my conclusions, as well as a collection of links, at my personal website on 8 February 2020, as discussion of it began again after Gøtzsche published a book on vaccines: Gøtzsche and the Cochrane Collaboration: A Timeline.

~~~~

 

Disclosures: I led the development of a fact sheet and evaluation of evidence on HPV vaccine for consumers in 2009 for Germany’s national evidence agency, the Institute for Quality and Efficiency in Healthcare (IQWiG), where I was the head of the health information department. We based our advice on this 2007 systematic review including 6 trials with 40,323 women, and an assessment of those trials. The findings were similar to those of the 2018 Cochrane review. I have no financial or other professional conflicts of interest in relation to the HPV vaccine. My personal interest in understanding the evidence about the HPV vaccine is as a grandmother (of a boy and a girl).

I am one of the members of the founding group of the Cochrane Collaboration and was the coordinating editor of a Cochrane review group for 7 years, and coordinator of its Consumer Network for many years. I am no longer a member, although I occasionally contribute peer review on methods. I often butt heads with the Cochrane Collaboration (most recently as a co-signatory to this letter in the BMJ). I have butted heads on the subject of bias with authors of the Copenhagen critique.

26 September: In light of developments, some further disclosures which I did not realize may relate to this controversy and the theories around it, when I first started writing about the Cochrane HPV vaccine review. I was invited to speak at Evidence Live, and my participation was supported by the organizers, a partnership between the BMJ and the Centre for Evidence-Based Medicine (CEBM) at the University of Oxford’s Nuffield Department of Primary Care Health Sciences – the director of the CEBM is the editor of BMJ EBM. Between 2011 and 2018, I worked on PubMed projects at the National Center of Biotechnology Information (NCBI), which is part of the US National Institutes of Health. I am currently working towards a PhD on some factors affecting the validity of systematic reviews.

[Updates 20 September 2018] After some questions on Twitter, I added some extra detail to a sentence about membership of the Cochrane Collaboration – thanks, Rosewind! The sentence originally read:

But as one of the original founding group (as am I), he knows individual membership didn’t exist for all those years. In fact, it’s quite freshly minted. And people have been forced out before, one way or another.

I edited and updated a section to include Cochrane’s response to the section which originally read:

Here’s a concrete example of what Sandman means. It includes an error that would escalate conflict, and then a chance to be responsive. Please do this, as soon as possible!

And I changed “5 out of 13” board members being from Cochrane centers to “so many”, because I realized I am not sure exactly how many were – there were at least 4, which means they are disproportionately represented.

[Update 22 September 2018] I added the note that the Austrian Cochrane Centre had removed the statement posted on behalf of the resigning members, one of whom was the director of the center, Gerald Gartlehner, after confirming with him that it was done as an effort towards de-escalation of conflict. (I have kept a copy of this and the original statement from the Cochrane board.)

[Update 23 September 2018] In response to a tweet from @TranspariMED, I edited a sentence that previously read:

Ideological and commercial “merchants of doubt” are causing serious damage, and we’re not good at dealing with it yet.

Thank you, Till Bruckner/@Transparimed!

[Update 24 September 2018] In response to a tweet from Duncan Babbage, I edited sentences that previously read:

It’s also striking that the remaining board was all female, but for 1 Cochrane Centre director. The departing board members were all male, but for 1.

That was only a partial response to the suggestion, though: a virtually all-female board for Cochrane was an amazing sight. However, I also realized when examining the sentences that Gøtzsche remains a board member too, at this part of an ongoing process. Thank you, Duncan Babbage!

[Update 26 September 2018] Added postscript on the Cochrane board’s termination of Gøtzsche’s membership and positions in the organization.

[Update 29 October 2018] When describing the additional 11 trials cited by the Copenhagen group, I corrected the description from “primary outcome (moderate or severe cervical lesions)” to “primary cervical-cancer-related outcomes (moderate or severe cervical lesions or invasive cancer”, and the subsequent shortened version “primary outcome” to “primary cervical cancer-related outcomes”. I realized the original shortened description was misleading when replying to this comment. After further considering this reply, I removed those sentences completely (on 28 February 2020). (Thank you, Aleksi Raudasoja.)

 

The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)

 

The sequence of photos of a cooking oil boil-over are by Joergens.mi/Wikipedia, via Wikimedia Commons and the Wikipedia boilover page, with a CC-BY-SA-3.0 license.