Rebranding Retractions and the Honest Error Hypothesis

 

 

We have a serious problem with errors and irredeemably flawed studies: there’s a lot of them, and they keep leading people astray. Few errors get corrected. And it’s very rare for a paper to be retracted (less than half a percent).

Fixing this isn’t going to be easy. It’s always possible to make things worse, too, in predictable and surprising ways. So we should stick to first principles when thinking about whether major interventions are desirable:

• Is it feasible?
• How sure are we that it will have the intended effect?
• What could go wrong?

One of the ideas that’s been circulating quite a bit in the last year is to de-stigmatize retractions by rebranding. It came up again at a recent National Academy of Sciences (NAS) colloquium, [PDF] reportedly advocated by the President of NAS and former editor-in-chief of the Science journals, Marcia McNutt:

 

 

She’s far from alone on this. Daniele Fanelli proposed that there should be a new category, “self-retraction”, for “honest error”. In a poll at Retraction Watch, 90% agreed with that idea (out of 472 who had voted as of yesterday). And this year, Virginia Barbour and colleagues posted a preprint calling for rebranding and new categories for retractions and corrections. (They published this month.)

Let’s work through these ideas with the questions I raised at the start.

 

Is it feasible?

 

This breaks down into two major parts for these proposals: is the classification meaningful and reliably consistent to apply, and could it be implemented at tens of thousands of journals?

Fanelli proposes the name “self-retraction” be applied when:

• The authors solicited the retraction spontaneously for documentable flaw(s);
• The authors “provide evidence of the honesty of the mistake”;
• The retraction statement is signed by every author.

“Spontaneously” is easy to claim, but not easy to prove. Author request for retraction can come after an ultimatum or investigation behind closed doors. “Honest error” is easy to claim, but not all that easy to define or prove. And just because one author doesn’t sign, doesn’t prove dishonesty is afoot: they just might disagree retraction is justified.

Once a journal had a category that was officially “honest”, what would that imply about every single retraction the editors did not stamp that way? And what would the legal consequences be?

 

(FUBAR? It’s military acronym slang.)

 

The Barbour & co proposal is to call everything an “amendment” in one of 3 categories:

• Minor: small changes that don’t affect the substance of the article;
• Major: corrections, clarifications and addenda that aren’t minor;
• Complete: the whole article is considered “unreliable”.

“Complete amendment” is pure euphemism, and doesn’t make sense. Among the comments on the bioarxiv version of this proposal, Matt Hodgkinson pointed out that articles can be retracted with nothing that could be amended. Peter Doshi points out that a dictionary definition of amendment is a minor change. When you consider how dramatic errata can be, “amendment” swings the pendulum to trivialization.

What about the second part here: could a major change to terminology be implemented? Not easy either. There could be sea changes if a group of major publishing players all move in the same direction. But there has never been consistent application across all journals of any recommendation around corrections and retractions. (Retrospectively re-classifying is out of the question.)

 

How sure are we that it will have the intended effect?

 

There’s not much direct evidence to go on here. In Retraction Watch, Fanelli points to the results of a survey of 14 scientists he did with colleagues that supports the existence of stigma around retractions. However, Retraction Watch also points to a study of 1,423 retractions in Web of Science that found no citation penalty for authors who retract for errors under the system we already have.

Would changing what it’s called make people less reluctant to public declare that their paper is grievously wounded? I don’t know of any evidence that supports this hypothesis. In fact, resistance to admitting error at all is high. It’s hard to see how changing what you call it could affect that. Peter Gøtzsche and colleagues found a high level of complete non-response to criticism, including on severe issues. (We’ve found that, too, at PubMed Commons.)

David Allison and colleagues put it this way last year in reporting on their experience of trying to get errors corrected:

Many journal editors and staff members seemed unprepared or ill-equipped to investigate, take action or even respond. Too often, the process spiralled through layers of ineffective e-mails among authors, editors and unidentified journal representatives, often without any public statement added to the original article. Some journals that acknowledged mistakes required a substantial fee to publish our letters: we were asked to spend our research dollars on correcting other people’s errors.

Or, as Retraction Watch put it:

Want to correct the scientific literature? Good luck.

That’s my experience, too.

There’s also no good evidence I’m aware of that changing terminology is an effective de-stigmatization intervention in other contexts. I used to assume it did, and worked hard on changing terminology to reduce stigma and change attitudes and behavior. (Indeed, one of my first-ever journal articles was about trying to change language.)

But it’s not that simple. If the stigma is strong, then it quickly attaches to the new term. Consider terms for people with intellectual disabilities. It only took a matter of years from when medical terms were invented, even sometimes in order to avoid stigma, to them being used as vernacular insults. See, for example, moron and cretin. Unless stigma comes only from using a word that already has a negative connotation, you have to do a lot more to reduce stigma than just rebrand. And you can reduce stigma without rebranding.

Anyway, people will still call them retractions. They have always gone by different names at various journals – withdrawals, removals, etc. “Retraction” is a generic term. It’s not likely to go away anytime soon. And certainly not while Retraction Watch exists!

 

What could go wrong?

 

The “self-retraction” proposal seems to me likely to add time, difficulty, and probably lawyers, to the editorial process. It might increase the stigma, unfairly, to those retractions that are “no fault”, but don’t meet the definition – somebody called for a retraction before authors were aware of the errors at all, for example, or the error or misconduct was the editor’s, not theirs. Or the authors published in a journal that hasn’t implemented the option. On the other hand, some people will manage to come by the designation dishonestly.

For the amendment proposal – and any other name change that’s a euphemism – my main concern is the potential for trivialization. Will people know what it means and therefore take it as seriously as they should when they see it?

The most likely thing to go wrong, though, is that it will turn out to be a waste of effort. As Matt Hodgkinson wrote:

What’s in a name? A retraction by any other name will smell as sour.

 

~~~~

The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)

 

* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.