When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

PLOS BLOGS Absolutely Maybe

A Cartoon Guide to Criticism: Scientist Edition

Cartoon about speakers collecting flak jackets from conference registration desk

Science is a profession where critique is integral to the work itself. So you might think we’d all have both criticizing, and handling being on the receiving end of it, down to a fine art. But you’d be wrong – as a rather spectacular recent blowout of norms by a prominent scientist reminded us. Sociologist Robert Merton pointed out in the 1960s that scientists also rely on the acceptance and recognition of their peers, so it’s no wonder things get heated.

I spent several years moderating a high-profile forum for scientific criticism, and my position about this is still broadly similar to the one I described in PLOS Medicine in 2014: I think we need a much stronger post-publication culture in science. I’ve got some notes about that and my position in relation to this recent episode in the disclosures below this post.

Let’s get straight to my (rather wordy) cartoon guide to science criticism – and you can always just skip through the headings and cartoons.

It’s about the work, not you, so don’t take it personally

Cartoon of woman too afraid to look

I know it’s easier to say than do, but this is about the work, not you.

Remember: just because you took it personally, it doesn’t mean it was personal. It gets harder if someone does make it personal (and that’s out of line), but try not to let their bad behavior get to you.

Questions related to potential conflicts of interest shouldn’t be regarded as personal. It’s not ad hominem if the ethos of the person writing/speaking is actually relevant. That said, claims about conflicts of interest can just be excuses to unreasonably undermine an argument or person. (See my reader’s guide.)

Don’t over-react to the vernacular

Cartoon of English lawyer calling fellow lawyer a wackaloon - with all due respect

Whether it’s someone from a different discipline, a different generation, in a different platform, or even all of those at once, communication is culture- and context-dependent.

Civility is important everywhere – the lack of it deters participation, and inequitably so – but what’s regarded as civil or rude varies. What’s appropriate and quite polite on Twitter is very different from the norms of communication in journals, for example.

Generation is a big deal here, too. As academics ratchet up years, publications, and status, they get used to deference escalating into veneration – and many expect it on their behalf, too. Anything less than that might seem like an attack that justifies retaliation: but it’s exaggerated deference that’s the problem, not the mode of criticism.

(If you’re wondering, wackaloon is old internet-speak for a crazy person.)

Don’t take overblown martyr complexes seriously

Unfortunately, some academics have always been in the habit of using the power and status of their positions to retaliate against people who criticize their work.

But in case you’re reading this from a future where this cartoonish behavior is a thing of the past, it really did get to the ludicrous stage of tenured professors from prestigious universities broadcasting on the airwaves and from major newspapers that they were being silenced! Another variant: criticism is a destructive force, and public critique is not a public good – it’s just tearing down more creative/constructive people.

On a more common level, many even claim being blocked on Twitter is some kind of silencing or infringement of their rights. People don’t have a general right to force themselves into others’ conversations when they’re not wanted – not in person, and not in social media either. Sometimes, criticisms of criticisms are important: sometimes, they’re just over-reactions.

Admit you’re wrong every chance you get – it’s good practice

Admitting you’re wrong has a lot in common with apologizing: both are prone to versions of being fudged so far that they are, in fact, “non” admissions.

In this category go things like saying everyone makes mistakes, without actually specifying yours – and claiming your views have changed because there’s new evidence instead of admitting you were out on a limb in the first place.

I think this is much harder if you rarely – or never – do it. You realize it’s not so bad when you’ve done it a few times, even if it is much harder in some circumstances than others.

Be analytical about criticisms – whoever makes them (or how)

Being quick to dismiss a criticism because of who made it, or how it was framed is a mistake. If you’ve erred – and we all do – it’s important to find out, especially if it’s science. Your antagonists might simply be more likely to point out your stuff-ups.

It’s harder to parse comments that rile you in some way, so that makes it even more important to be sure you’re not overlooking something. (Of course, at some point, people get too aggressive/prolific to warrant the cost of analyzing their output, which is a shame.)

Sometimes, though, there is nothing (new) there

Well-founded criticism, no matter how small the point it addresses, or how nastily packaged, is a gift. It comes with the power to sharpen our thinking, science, and communication skills.

But sometimes, it’s not well-founded, or it’s rehashing a point that has been made over and over (sometimes even hundreds of times!). And sometimes, it’s ill-informed and/or straight up trolling. That’s tough territory when you’re the target, or you see others being targeted and fear for yourself. Which is why the next point matters so much.

Great bystander (and editorial) behavior is critical to safe communication culture

Cartoon of editor shocked by reviewer comment

We need social norms to favor criticism and disfavor bad reactions to it – and that means as a community we need to protect those attacked by others for their critiques, or subject to unreasonable criticism. Robust science needs robust criticism – but it also needs scientists not to be driven away.

When it’s running hot, though, everyone can’t stick their necks out – Simine Vazire has written a great post about this. It’s asking a lot more of some people than others, too.

I think there’s personal value from getting involved in defense of someone else: it’s easier to acquire the skills of debating critique and techniques for managing the emotional burden of pushback when it’s not your work on the line. And it’s not only for science that we need skills for being a good bystander when there’s bullying.

But I think there are also ways to be a great bystander for science pile-ons that don’t involve jumping into the fray. When I’ve been the target (which happens from time to time!), I’ve seen an uptick in people doing me apparently unrelated kindnesses – “liking” unrelated tweets, for example, complimenting my other work, or going out of their way to be nice in person. Highly recommended!

Not enough robust criticism is worse than too much

Author with IPSD - Ignored Publication Stress Disorder

One of the prices of your work being ignored by critics is that you’re less likely to find out how you can improve it, and your communication about it.

Peer review at journals just isn’t adequate to the task, and everyone doesn’t have the luxury of being embedded in a large, diverse, critical community. (See my tagged posts on research on journal peer review.) We need a vibrant, high-participation, post-publication criticism culture. Not enough criticism is a bigger problem for science and the public interest than over-energetic critiques.

~~~~

The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny.)

Disclosures: I studied the prevalence of some types of post-publication events in clinical trials and systematic reviews as part of my PhD. I was the lead scientist/editor for PubMed Commons, a commenting system in PubMed that ran from 2013 to early 2018 (archived here).

Related to the recent episode by an author of conflicting reviews of Covid-19 infection fatality rates that triggered this post, including his own: I criticized both reviews at the center of this when they were preprints. I have never met Gideon Meyerowitz-Katz (and haven’t cited or criticized his work prior to that). He responded graciously, both publicly and privately. I have known John Ioannidis for decades (and often cited and praised his publications). I have also (less often) criticized work he’s done, before and during the pandemic (see for example, here, here, and here; and I wrote a post about EBM heroes and disillusion around one of those episodes.) I criticized Ioannidis on Twitter when I saw what he had written (eg here), and communicated with him privately. [Update April 11] Ioannidis requested the removal of the egregious material, responding to concerns in particular from students who criticized him – a welcome outcome.

Discussion
  1. Thank you from Norway. This is so important, especially wrt. medical research where patient safety is at stake. I have criticised studies myself, and that was not well received.

  2. I found this to be well written and I think of great utility to people like myself in emergency medical services. Critiques are usually saved for after patient contact has ended in EMS but sometimes they need to be stated right away to prevent mistakes that could negatively impact patient care. These situations are difficult because there is limited time to reflect on your criticism or on the flip side, respond to criticism.

  3. An attitude I’ve found helpful is to try to act like a policeman, and look for supporting evidence for the criticism; not like a defense attorney, searching for a single flaw in the criticism so that I can dismiss it all.

Leave a Reply

Your email address will not be published. Required fields are marked *


Add your ORCID here. (e.g. 0000-0002-7299-680X)

Related Posts
Back to top