Here’s the thing. If you’re a doctor who is a fervent advocate for a professional school of thought, and you feel the urge to tell patient advocates they are only “true” ones if they totally join your team … maybe just stop right there.
A premise for patient advocacy by patients themselves is that the interests and priorities of the profession can, sometimes at least, differ and even clash with those of the people they are meant to serve. And given the profession’s reputation for paternalism and arrogance towards patients, taking great care to avoid being patronizing about an area of expertise that isn’t their own makes sense, too, doesn’t it?
Vinay Prasad just swept past all that in a post on Medscape arguing that patient advocates “must” become “students of evidence-based medicine” or else they risk doing patients harm:
Without the skills of evidence-based medicine, they may not even recognize this. In my mind, there is nothing more painful than watching someone with good intentions place a self-inflicted wound.
How can we make advocates students of evidence-based medicine? Advocates already have the energy and passion to educate themselves, and need only a direction. There is no better starting point than [name of a book on critical appraisal for doctors that’s not open access]. Books like [name of a book on over-diagnosis that’s not open access] and [name of a book on doctors doing harm that’s not open access] are useful starting points. Once the flame of evidence-based medicine is lit, it is hard to extinguish and will propagate itself.
Even if you see no other problem with this post, that so many absolutes are thrown up without any evidence cited whatsoever should give you pause.
Say for argument’s sake patient advocates do need critical evidence appraisal skills. Is there solid evidence that:
- it’s feasible for patient advocate self-directed learning to provide enough skills for them to be able to rely on their own evidence expertise;
- reading one or more of the prescribed books is a good starting point, it will trigger the conversion experience Prasad describes, and that will “propagate itself”;
- this alone will result in evidence skills being consistently applied and over-riding personal biases – cognitive, self-interest, ideological, financial; and
- patients will always be better off if they do?
No. No, there’s not. All this doesn’t even work easily for doctors, for whom this relates to livelihood and professional responsibility. The most recent systematic reviews I could find don’t establish proof of practice improvement from studying evidence skills either for medical students or practitioners. That doesn’t mean there isn’t some, or that education – self-directed or otherwise – is useless. But it does mean you’re on shaky ground when you start making claims about it being powerful and essential.
You can teach people critical skills – of course you can. You can fire up some people with tales of medicine’s excesses and harms, too. But how does that affect behavior? I think what Balazs Aczel and colleagues point out about cognitive biases – like confirmation bias – applies here too. Skills aren’t enough. We need to recognize the moments when we need to apply them, overcome the competing biases and constraints that can derail us, and control ourselves well enough to use the skills effectively.
What Prasad is prescribing doesn’t teach that. When I looked up one of the books he recommends that I haven’t read, this review of it made me think it might do more to increase a person’s biases than teach them to how to cut through that tangled undergrowth. [Update on 1 January: Prasad tweeted that recommendation was a mistake. It was meant to be another book – a critique of pharma, which is also not open access.]
I was a patient advocate for 20 years. And in some ways I could be a poster child for what Prasad is prescribing. Gee, did I acquire skills, and I sure did propagate them! I wrote, I talked, I cartooned about epidemiology. I developed, ran, and evaluated evidence appraisal skills workshops for consumer advocates – at conferences, in cities around the world, and once even in the outback with indigenous people.
I could only find studies evaluating workshops like that – or training courses – for increasing critical appraisal skills in patient advocates. None were trials or tests, and only this one looked at longer term impact, concluding: “The results of the survey showed a strong impact of the training course on single participants, while a weak outcome on the activities of the organizations was reported”. Even that study, though, didn’t look beyond participants’ views, to their behavior. (I’ve listed the 7 studies I found below this post.)
My own experience? Some patient advocates really do behave as Prasad would like but it’s not a high proportion. Most don’t have the time and resources to develop serious critical appraisal skills, and want to know how to find the experts who have done it. I’d like to see more critical skills of all kinds, but I don’t think this is at all as simple as Prasad suggests.
For my own advocacy, I pretty quickly moved past the idea that critical appraisal skills were a magic bullet. They can strengthen the arms of advocates for anti-social causes too, of course, which underscores the importance of broader issues.
I believe it’s vital that patient advocates be good critics of evidence-based health care and its ideologues, and be on guard for its failures – like an over-emphasis on drugs and doctors. Trisha Greenhalgh and colleagues’ “Six ‘biases’ against patients and carers in evidence-based medicine” is required reading here. Qualitative and social science research have lower status than clinical trials, and that has been exacerbated by the evidence-based medicine movement. Yet it’s vital for the interests of patients and the public, and patient advocates could provide important guardrails for issues like that.
Back in 2004, I wrote an editorial called “Learning from evidence based mistakes” – that’s the other side of the “EBM” coin. I wrote:
My first evidence based mistake came early in my involvement with EBM. When two of the member organisations of the consumer coalition I was involved with were set against each other because of a controversy, we did the evidence based thing. We turned to the systematic review to decide what stand we should take…The systematic review concluded that the drug was effective, with no concerning adverse effects. We didn’t support the call for a ban, and we were wrong. The drug, it turned out, was causing serious harm, including deaths…
Change in practice led by EBM enthusiasts often does a lot of good. But sometimes it causes harm, especially when people react every time that individual trial results become available…It is perhaps an article of faith, more than a matter of evidence, that the people being cared for by EBM enthusiasts are always better served.
The skills patient advocates primarily need, I believe, are around recognizing and trying to address your own biases, being rigorous about what you claim about patients’ views and concerns, and having a strong focus on rights and equity. Media and information literacy is up there, too.
If trials and clinical research are relevant to the area a patient advocate is concerned with – and that’s not everything by a very long way – evidence concepts are a good extra. But knowing the limits of your own expertise is vital. Sure, you can learn some ropes, but over-estimating your own skills is a giant risk. Let’s face it, if this was so easy, why would there even be such enormous differences in risk of bias assessments of studies in systematic reviews by full-time, über-experts?
If you’re interested in learning more critical appraisal skills, there’s an international database of assessed teaching resources that is a rich source of accessible material here.
I recommend Know Your Chances as a starting point for understanding risk – it was tested in trials with “lay” people – and it’s available free of charge, like all the resources I’m suggesting here. Smart Health Choices is a good book introducing critical appraisal. The Testing Treatments book comes in several languages, and is supported by a website of resources too. (I have a cartoon-based blog for this, too, at Statistically Funny.)
And the pioneering “Project LEAD” – of the 2001 study below – has evolved into a free set of online learning materials for advocates at Consumers United for Evidence-Based Healthcare.
Disclosures: I was a health consumer advocate (aka patient advocate) from 1983 to 2003, including chairing the Consumers’ Health Forum of Australia (CHF) from 1997 to 2001, and its Taskforce on Consumer Rights from 1991 to 2001. I was the founder of the Cochrane Collaboration’s Consumer Network, coordinating it for 10 years. I trained consumer advocates at the CHF, and developed and delivered training workshops for consumers on critical appraisal of evidence for the Cochrane Collaboration.
Studies studying critical appraisal skills training for patient advocates:
- Australia (2008)
- Austria (2013)
- Germany (2010)
- Italy (2012) [the only one studying longer term potential impact]
- Norway (2018)
- UK (2015)
- USA (2001 PDF) [this is a training course including advocacy training]