Signing Critical Peer Reviews & the Fear of Retaliation: What Should We Do?
There’s a sort of Godwin’s Law for discussions on open peer review. Sooner or later, someone’s going to say, “We can’t expect early career researchers to sign peer reviews, because of fear of retaliation”. And that’s often a discussion-ender. What kind of monster could argue for carelessly letting early career researchers hurtle towards career suicide?
We shouldn’t let this be the end of a discussion. It should be the start of several more.
Firstly, just keeping peer review closed doesn’t do anything about the problem of abuse of power, does it? It’s not going to automagically disappear. It’s no good for science when people are able to intimidate others into silence or tiptoeing around the weaknesses in their work. This isn’t going to be behavior that appears only during peer review. Scientific communities are harmed by harboring bullies, especially in senior positions.
Secondly, since we know there are authors who would behave this way, and fear of it seems widespread, we should be discussing preventive and responsive mechanisms. There could be close to a hundred science journals with fully open identity peer review systems, including some funders’ journals, and some that have been doing this for nearly 20 years. (More on these journals below this post.)
Then there are journals that offer it as an option, like PeerJ and Nature Communications, and coming soon at Nature, too). Some encourage it, like the Royal Society’s Open Science. And people are having openly critical discussions about preprints, too. On top of that, some of us sign our reviews whenever we review, regardless of the journal’s policies. We don’t know how common that is.
Put all this together, and it’s enough for us to need concrete and productive responses to retaliation and the fear of it. After all, it’s possible that younger researchers are more enthusiastic about signing peer reviews, not less. So let’s talk about the vulnerability of being open – but let’s talk about the strength, too.
Critical thinking and analysis of others’ research are core scientific skills. The skills involved in critiquing, and then communicating your analysis well, are ones we all need to develop and keep honing. It’s not just during peer review either. You could be criticizing or “slighting” someone who could be vindictive at lots of points along the way – what you cite, what you say in the background or discussion section of a paper on your own research, and so on.
Being good at critique and articulation of it can build reputations. When we were discussing this at a reproducibility seminar in 2015, physicist Tony Weidberg said the culture in his field had changed on this over the years. There was a time when it was risky to openly criticize senior people. But these days, he says, it’s more likely to improve your career prospects.
Still, it’s going to be a risk sometimes. In the DOSPERT instrument for rating people’s relationship with taking risks, this makes the list:
Disagreeing with an authority figure on a major issue.
Confronting power is classic risk-taking behavior. It’s not a slam-dunk argument against open peer review at journals, though. You are not obliged to do a peer review if you have concerns. I’m openly criticizing people’s work – and even behavior – all the time, and have been for decades. Still, every now and then I pass on peer reviewing someone’s paper because I don’t want to face the burden of criticizing a person like that at that particular point.
You always have to pick your fights, don’t you? Faced with a prima donna or sheer bully author, I wouldn’t consider blinding a good enough safety mechanism, either. (I’ve discussed the studies on this issue before: from 10% to most authors could accurately guess reviewers’ identity.)
Just because you are asked to review a paper, doesn’t give you a responsibility for the paper unless you choose it. We can’t solve every problem ourselves. You can get practice and visibility for critical analyses in other ways, and for authors you feel confident about.
Saying you can’t take it on right now is a personal solution. But it’s not a solution for the problem. Very public peer review is potentially a solution, as I’ve argued here before. The more open all this is, the more chance we have to develop better norms in framing and dealing with criticism, and to make authors, reviewers, and editors more accountable. The evidence of your critique is there for others to see, if you are contending that your peer review is being held against you by someone. And the more open peer review is, the more it can be studied.
I don’t think any of this is enough, though. What else should we do? Consider this:
Could we agree this is professional misconduct? Calling it that isn’t going to sweep the behavior away, of course. Acting on it would still be hard, too. It would usually be hard to prove, and if someone bore ill-will for a negative peer review, a complaint about it would likely make it worse. However, there are things you can do that can get your paper rejected, or get you banned from a journal. Should aggression to a peer reviewer be one of those things? I think so.
The only relevant thing I could find at COPE (Committee on Publication Ethics) was this: “Authors should respond to reviewers’ comments in a professional and timely manner”. [PDF] Well, I doubt that professionals acting unprofessionally generally recognize it.
Simply codifying retaliation against peer reviewers as misconduct in journal policy and elsewhere might not deter anyone – and the consequences of a person’s ill-will will usually be far more diffuse than Tim Mosca’s experience. Still, if journals and others named retribution against peer reviewers as misconduct with consequences, it could be an important signal. Culture is shaped by “rules”, as well as by repeated actions and habits, small as well as big.
Often, when you get invited to peer review, you get warned of the things you shouldn’t do: on pain of death just about sometimes, you’re told not to show anyone a copy of the manuscript, for example. But what about peer reviewers’ rights? Should there be a note at the journal about what to do if an author acts aggressively?
If authors giving peer reviewers grief is a thing that happens to plenty of people, should we discuss if contact at all is appropriate? It seems to me to be un-collegial and ill-advised at least. You get to respond to peer review within the process, and that should be the end of it. Even if you don’t intend to intimidate or distress someone, that could be the effect.
Discrimination and harassment are codified as professional misconduct for members of the American Bar Association. And this approach is starting to find its way into science’s professional organizations, too. Margherita Perillo recently pointed to this in her post about bullying in science, at the American Association of Cell Biology (ASCB) blog. She points to ASCB’s anti-harassment policy for its meetings, and to the American Geophysical Union (AGU) designating harassment and bullying as research misconduct.
We could get more explicit about our concerns for the effects on scientists and science of authors who intimidate peer reviewers. Journals and others should step up here. It could help shape culture. Clear signals like this make it easier for bystanders to act, and for people to seek support if it happens to them.
I think as a community we can and should do something more about ethics and standards for open peer review. But even if we do, fear of retaliation isn’t going to evaporate any more than impostor syndrome is. The things we fear can hurt us, that’s for sure. Facing fears, taking even little steps at not letting them inhibit us, and surviving and growing when things go wrong can all make us stronger though, too. Courage and character are built by it.
Life offers up an endless supply of opportunities for exercising the muscles we need as individuals to speak up to authority figures and to criticize, inside science and out! Confidence can be built in all sorts of ways. Somehow, too, we have to find the ways that we can take a stand against retaliation together.
More posts on open peer review:
The Fractured Logic of Blinded Peer Review in Journals
Weighing Up Anonymity and Openness in Publication Peer Review
The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)
Journals that have open (named) peer review:
- 70 of the BMC journals, some since 1999
- Atmospheric Chemistry and Physics, since 2001, followed by 11 more journals from the European Geosciences Union and the publisher, Copernicus (more on them here)
- BMJ Open since its launch in 2011, the BMJ since 2014, and BMJ Open Science since its launch in 2017
- f1000 Research (since 2013 I think) and the funders’ journals that have started with this model since 2016 (including Wellcome Trust and the Gates Foundation)
- European Journal of Neuroscience since the end of 2016
I would be pleased to know additions for this list in the comments section below, or via Twitter. [Many thanks to Trish Groves and Irene Hames for additions so far – see comments below.]
* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.
Thanks; another great post.
Could you please add BMJ Open to your list of journals that use open peer review? Since the journal launched in early 2011 we’ve published >5000 articles, all* with open peer review. Indeed, when Publons launched, it used the reviewers’ reports from BMJ Open to populate its site quickly and now carries nearly 9000 of our reviews.
*bar the odd one where the peer review history was withheld on legal advice: under the UK Defamation Act 2013 publications in peer reviewed scholarly journals are ‘privileged’ (ie protected by a statutory defence against libel claims), but there’s a grey area around the libel risk of open reviewers’ reports because the Act assumes peer review to be a closed system.
Thanks, Trish – done. What a silly oversight on my part: my apologies! Added BMJ Open Science too. And that bumps you guys higher up in the timeline too.
One to add to the list of journals. The European Journal of Neuroscience @EJNeuroscience introduced open (signed) peer review in November 2016. It seems to have been a successful transition – the editors have written about the experience and whether their fears and hopes were realised https://doi.org/10.1111/ejn.13762.
Thanks, Irene! I’ve added them.
How does this apply to post-publication review, including by patients, journalists, researchers in other fields etc. I have seen a few authors take aggressive actions against these other people too.
I agree with much of what you said, and I think that Tim should have reported this behaviour to the editor. But I think the answer to whether reviewers should sign reviews will depend on whether the reviews are public. If reviews are published, then , as you note, the reviewer may benefit from being recognised as a person who does a thorough and fair job. Also being transparent just gets the word out more broadly which may make it harder for retaliation to occur. Let’s consider reviewer X, junior person, who has publicly reviewed a paper by Y, big important person, and has raised criticisms of it. If a grant proposal by X comes to a board on which Y serves, then it is more obvious if Y badmouths X that there may be reason for that. However, this is only going to work if the paper by Y is ultimately published. If that is not the case, or if the journal does not publish reviews, then I agree that X should be allowed to remain anonymous.
It is telling that editors are mostly absent from this post. Anybody getting threatened by an author should complain to the journal editor. As an AE, I have prevented a paper from continuing through the review process because the response from the author was highly unprofessional and dismissive of a reviewer’s comments and credentials. There should be no room for bullying by either authors or reviewers and one responsibility of the editor is to assure a professional exchange. The few times I as a reviewer have been in touch with an author were to help them to better understand my criticism, and those conversations went well.
Important points. But I think that even though the circle of participants – authors and reviewers – is small, that, too, can be reputation and network building. New constellations of collaborations could sometimes form.
I’d be more likely to agree with you on reviewers staying anonymous if it wasn’t that X and Y are often flipped, with more senior Y badmouthing and rejecting X’s paper, and X not being able to inform the editor that Y has a conflict of interest the editor doesn’t know about. I don’t know, but I would be surprised if it was more often “senior” authors being peer reviewed by “junior” authors than vice versa. I put “senior” and “junior” here in quotes: replace it with “Anglo-American prominent university” and “author from less prestigious country/institution”.
I think people are being over-trepidatious. The sky hasn’t fallen in at the many journals where this has become normal.
I see what you mean, Craig: I kept using the term “journal”, rather than “editors”, because of course that’s who’s responsible for all this at journals. The direct contact thing is something I’m not so sure about. I think the peer review process could benefit from being more interactive. While contacting a reviewer to take issue with what they wrote seems to me inherently fraught and un-collegial. I can’t recall any study/discussion around this issue, unfortunately.
It concerns me that if potential reviewers decline to review because the author of the manuscript is known to be retaliatory–but without mentioning this to the editors–the retaliatory author may successfully avoid negative reviews, which was presumably the goal of their bullying behavior in the first place. I would rather given an anonymous, negative review than decline to make a non-anonymous one out of fear.