When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

PLOS BLOGS Absolutely Maybe

Rebranding Retractions and the Honest Error Hypothesis

 

Cartoon about science not being self-correcting

 

We have a serious problem with errors and irredeemably flawed studies: there’s a lot of them, and they keep leading people astray. Few errors get corrected. And it’s very rare for a paper to be retracted (less than half a percent).

Fixing this isn’t going to be easy. It’s always possible to make things worse, too, in predictable and surprising ways. So we should stick to first principles when thinking about whether major interventions are desirable:

  • Is it feasible?
  • How sure are we that it will have the intended effect?
  • What could go wrong?

One of the ideas that’s been circulating quite a bit in the last year is to de-stigmatize retractions by rebranding. It came up again at a recent National Academy of Sciences (NAS) colloquium, [PDF] reportedly advocated by the President of NAS and former editor-in-chief of the Science journals, Marcia McNutt:

 

 

She’s far from alone on this. Daniele Fanelli proposed that there should be a new category, “self-retraction”, for “honest error”. In a poll at Retraction Watch, 90% agreed with that idea (out of 472 who had voted as of yesterday). And this year, Virginia Barbour and colleagues posted a preprint calling for rebranding and new categories for retractions and corrections. (They published this month.)

Let’s work through these ideas with the questions I raised at the start.

 

Is it feasible?

 

This breaks down into two major parts for these proposals: is the classification meaningful and reliably consistent to apply, and could it be implemented at tens of thousands of journals?

Fanelli proposes the name “self-retraction” be applied when:

  • The authors solicited the retraction spontaneously for documentable flaw(s);
  • The authors “provide evidence of the honesty of the mistake”;
  • The retraction statement is signed by every author.

“Spontaneously” is easy to claim, but not easy to prove. Author request for retraction can come after an ultimatum or investigation behind closed doors. “Honest error” is easy to claim, but not all that easy to define or prove. And just because one author doesn’t sign, doesn’t prove dishonesty is afoot: they just might disagree retraction is justified.

Once a journal had a category that was officially “honest”, what would that imply about every single retraction the editors did not stamp that way? And what would the legal consequences be?

 

(FUBAR? It’s military acronym slang.)

 

The Barbour & co proposal is to call everything an “amendment” in one of 3 categories:

  • Minor: small changes that don’t affect the substance of the article;
  • Major: corrections, clarifications and addenda that aren’t minor;
  • Complete: the whole article is considered “unreliable”.

“Complete amendment” is pure euphemism, and doesn’t make sense. Among the comments on the bioarxiv version of this proposal, Matt Hodgkinson pointed out that articles can be retracted with nothing that could be amended. Peter Doshi points out that a dictionary definition of amendment is a minor change. When you consider how dramatic errata can be, “amendment” swings the pendulum to trivialization.

What about the second part here: could a major change to terminology be implemented? Not easy either. There could be sea changes if a group of major publishing players all move in the same direction. But there has never been consistent application across all journals of any recommendation around corrections and retractions. (Retrospectively re-classifying is out of the question.)

 

How sure are we that it will have the intended effect?

 

There’s not much direct evidence to go on here. In Retraction Watch, Fanelli points to the results of a survey of 14 scientists he did with colleagues that supports the existence of stigma around retractions. However, Retraction Watch also points to a study of 1,423 retractions in Web of Science that found no citation penalty for authors who retract for errors under the system we already have.

Would changing what it’s called make people less reluctant to public declare that their paper is grievously wounded? I don’t know of any evidence that supports this hypothesis. In fact, resistance to admitting error at all is high. It’s hard to see how changing what you call it could affect that. Peter Gøtzsche and colleagues found a high level of complete non-response to criticism, including on severe issues. (We’ve found that, too, at PubMed Commons.)

David Allison and colleagues put it this way last year in reporting on their experience of trying to get errors corrected:

Many journal editors and staff members seemed unprepared or ill-equipped to investigate, take action or even respond. Too often, the process spiralled through layers of ineffective e-mails among authors, editors and unidentified journal representatives, often without any public statement added to the original article. Some journals that acknowledged mistakes required a substantial fee to publish our letters: we were asked to spend our research dollars on correcting other people’s errors.

Or, as Retraction Watch put it:

Want to correct the scientific literature? Good luck.

That’s my experience, too.

There’s also no good evidence I’m aware of that changing terminology is an effective de-stigmatization intervention in other contexts. I used to assume it did, and worked hard on changing terminology to reduce stigma and change attitudes and behavior. (Indeed, one of my first-ever journal articles was about trying to change language.)

But it’s not that simple. If the stigma is strong, then it quickly attaches to the new term. Consider terms for people with intellectual disabilities. It only took a matter of years from when medical terms were invented, even sometimes in order to avoid stigma, to them being used as vernacular insults. See, for example, moron and cretin. Unless stigma comes only from using a word that already has a negative connotation, you have to do a lot more to reduce stigma than just rebrand. And you can reduce stigma without rebranding.

Anyway, people will still call them retractions. They have always gone by different names at various journals – withdrawals, removals, etc. “Retraction” is a generic term. It’s not likely to go away anytime soon. And certainly not while Retraction Watch exists!

 

What could go wrong?

 

The “self-retraction” proposal seems to me likely to add time, difficulty, and probably lawyers, to the editorial process. It might increase the stigma, unfairly, to those retractions that are “no fault”, but don’t meet the definition – somebody called for a retraction before authors were aware of the errors at all, for example, or the error or misconduct was the editor’s, not theirs. Or the authors published in a journal that hasn’t implemented the option. On the other hand, some people will manage to come by the designation dishonestly.

For the amendment proposal – and any other name change that’s a euphemism – my main concern is the potential for trivialization. Will people know what it means and therefore take it as seriously as they should when they see it?

The most likely thing to go wrong, though, is that it will turn out to be a waste of effort. As Matt Hodgkinson wrote:

What’s in a name? A retraction by any other name will smell as sour.

 

~~~~

The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)

 

* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.

 

Discussion
  1. JAMA recently introduced, for all its journals (I think), the idea of “retract and replace” (http://retractionwatch.com/2016/06/20/retract-and-replace-jama-may-expand-use-of-this-tool/) “to address honest pervasive error … without the current stigma that is associated with retraction”, in contrast to straightforward retraction, which is “[F]or articles with confirmed research misconduct (eg, fabrication, falsification)”.

    It is not clear how the replacement article will be peer-reviewed. On the one hand, the authors will be very keen to have their retracted article replaced, to avoid the stigma associated with a non-replacement retraction, which seems to clearly imply misconduct (cf. your point about “Once a journal had a category that was officially “honest”, what would that imply about every single retraction the editors did not stamp that way?”, Hilda). On the other hand, the authors previously submitted something with a lot of errors, so shouldn’t the reviewers maybe look extra-hard at it?

    This new scheme recently got tested to breaking point in a well-publicised case (http://retractionwatch.com/2017/10/20/retract-replace-retract-beleaguered-food-researcher-pulls-article-jama-journal/), which I guess may create some kind of scientific history as the only article to be retracted twice from the same journal. I haven’t asked the journal’s editor yet whether the definitive retraction of this article constitutes a suggestion that there has been some form of misconduct.

  2. Thanks, Nick. That’s raised in one of the papers I referred to, but I don’t think it has a bearing on whether retractions are called retractions, though. It’s not new either. “Corrected and republished” has been a publication type for PubMed for a long time. The editorial processes for all forms of correction might vary even more than they do for retractions, given that there seems to be less discussion about it.

  3. Our aim with the “amendments proposal” was three-fold:

    to simplify how changes to published articles are made;
    to be able to make changes quickly;
    and for the change itself to be reported in a way that is ‘blame free’ i.e. can be decoupled from underlying causes, which will be investigated and reported separately.

    The present terms used for post-publication changes are confusing and inconsistently applied – “comment”, “addendum”, “erratum”, “corrigendum”, “correction”, “editor’s note”, “expression of concern”, “partial-retraction”, “retraction” – and we are concerned by the negative connotations, especially those associated with retractions. The name does matter.

    Using the term ‘amendment’ provides a way to indicate a key change to an article in a neutral format without stigma, in the same way amendments are made to the United States Constitution.

    The intention is that amendments can be issued quickly without having to wait for the outcome of lengthy institutional investigations. An analogy here is with product recall, you flag the issue, but the explanation for what happened can follow later. Of course those investigations are crucial, must happen and be transparently reported, but they can be decoupled in time from flagging an issue on a published article.

    We distinguished scale of amendments (minor, major, complete) to aid the reader, but there is no reason these couldn’t be all one class and readers would have to read the amendment notice to learn more. We see these finer details as secondary to the main issue we’re attempting to address, but would love to hear what communities think.

    With regard to the situation of a complete amendment, this would cover cases where an article is found to be unreliable, for example, in the case of a plagiarised article the wording could be something like:
    Complete amendment “This article has extensive overlap with previously published work”.

    For significant change to happen there has to be a will to make the change and the technology in place to help it. Our suggestion is not just theoretical: we plan to test our idea further by initiating a feasibility study and hope to be able to report back in due course.

    Virginia Barbour, Theodora Bloom, Jennifer Lin, Elizabeth Moylan

Leave a Reply

Your email address will not be published. Required fields are marked *


Add your ORCID here. (e.g. 0000-0002-7299-680X)

Back to top