Rifts and Bright Spots in Evidence-Based Medicine
It all starts and ends with the patient. That was a strong message from the first day of Evidence Live.
Trisha Greenhalgh walked that walk on day 2. She showed the limits of evidence-based medicine (EBM) with the story of one patient’s journey through an episode of health care: hers.
Greenhalgh is a scientist and clinician who had a spectacular accident recently, in which her bicycle flew and somersaulted, smashing her to the ground. It left her with two broken arms, numbness in the fingers, neck injury – and a cracked bike helmet. Arms were set, rehab, then neck surgery, and more rehab ensued.
How was this high-impact arm and neck injury in a 54-year-old ex-elite athlete summed up in the hospital record? “55 yr old female fell off bike”. Greenhalgh was then propelled down the highly irrelevant “preventing falls in the elderly” evidence guideline pathway.
The twists and turns, of evidence and decisions, came together in her illuminating analysis: definitely one worth viewing when the filmed plenary sessions go online. Greenhalgh made several compelling cases: the direct evidence we need usually isn’t there, the evidence we have is often just reproducing “our assumptions and prejudices,” and we too often conclude that more evidence is needed when more of the same really wouldn’t help.
Doctors should be trained in the messy complexity of real-life cases, not simulated patient stories where evidence swoops in, neatly packaged, to save the day. Greenhalgh: “The right thing to do is an ethical decision. It requires deliberation”.
“We need more breadth, more balance, and more doubt,” said Iona Heath. “A rift runs through the center of every consultation,” she said on Monday. There is a rift in EBM, between the population and the individual, between significance and meaning.
Evidence is critical, but it cannot be the basis of medicine: “We have been lured into trying to describe a human being in data…Data gives us an alphabet, but as clinicians we remain unsure of the language”.
Heath argued passionately that the impact of guidelines has, if anything, reduced wisdom in medicine. Clinicians, she said, need more support for thinking critically about evidence – and they shouldn’t let drug reps through the door, either!
My own talk on Monday focused less on the evidence, and more on the ways “EBM” as a social movement increasingly clashes with the interests of patients and the public.
Scientific evidence is critical, but the biases of the EBM movement’s own intellectual and commercial interests end up distorting both the evidence and clinical practice. To stake the widest possible claim as “the” way to practice medicine, the importance of less bias has been lost: the phrase “best available evidence” is devoid of meaning.
Social forces, complexity, and bias drive uncritical acceptance of highly biased systematic reviews – and those who criticize such reviews face considerable attack. We’ve lost sight of the fact that systematic reviews and meta-analyses are descriptive studies, so they are very much at risk of bias themselves. I used two examples I’ve posted about before, where bias of systematic review authors plays a role in major controversies: mammography screening and debriefing.
The complexity at the center of these debates masks the extent to which evidence is used in the interests of other ideologies common in the movement, such as being anti-intervention or anti-industry. Many in EBM are not working to support patients’ values: they are seeking to change them. I argued that’s one of the issues at the heart of promoting the NNT (“number needed to treat”) to communicate results (a subject I’ve discussed here.)
The movement’s growing commercial interests put it squarely at odds with the public interest. For example, promotion of brands of evidence for sale runs counter to encouraging and enabling people to seek out the least biased, most up-to-date evidence.
It would help if the EBM movement applied its own principles to itself. Jeremy Howick spoke to this point. EBM is based on a good idea, he reminded us, but good ideas can have bad consequences, too: “If EBM is a good idea, there should be evidence of this. But there isn’t”.
At the conference, too, there were plenty of demonstrations of the EBM movement’s tendency to hyperbole about itself and its own estimation of its role and impact on medicine and in the world – even occasional megalomania. But as happens in the life cycle of social movements that gain force, effective counter-movements emerge.
Those effective counterweights were strongly incorporated into Evidence Live, including the positions about medicine and health research argued by Greenhalgh and Heath, and the open access/open science movements. “Open Trials” is the name of a new project enthusiastically promoted by lead investigator, Ben Goldacre – a highlight for me of the conference.
But for me, the brightest spot for evidence in health was Hywel Williams’ talk.
Williams is a scientist and dermatologist – and there’s probably no field facing a more uphill battle with both superficial and massive commercial interests.
This, along with the plenary talks by Greenhalgh and Heath, is a “must see” talk when the films go online. Williams drew on the framework of the waste in research paper by Iain Chalmers and Paul Glasziou to show a better way to develop and use scientific research to improve people’s lives.
He spoke – and demonstrated – powerfully that a way forward is to use the “creative interplay” of systematic reviews, working with patients and trialists to generate the results that can make a difference to suffering, and then “spin them out to a community of users”. Check out the aptly named GREAT database for eczema as an example.
Williams, like Greenhalgh and Heath, spoke to the problem of too much evidence of the wrong kind. Steered by the biases of industry and others, or impoverished in both methodological quality and meaning, more of this research is definitely not needed. That seems to me the most critical note to end on. Williams: “Stop! Just stop! And start doing decent studies with outcomes that matter to patients”.
See also Evidence Live and Kicking (Part 1).
Major keynote addresses from Evidence Live are here on YouTube.
The photos in this post were taken by me at Evidence Live in Oxford on 13 and 14 April 2015. The cartoons are my own (CC-NC license). (More at Statistically Funny.)
Disclosure statement: I was invited to speak at Evidence Live, and my participation was supported by the organizers, a partnership between The BMJ and the Centre for Evidence-Based Medicine at the University of Oxford’s Nuffield Department of Primary Care Health Sciences.
* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.
I agree entirely with your views about what Iona Heath, Trish Greenhalgh and Hywel Williams said at Evidence Live. Hywel, Paul Glasziou and Ian Chambers have also said that in order to reduce waste of research resources, more attention should be paid to the needs of patients and the doctors who advise them. For example, a RCT result showing that a treatment is better than placebo is not enough for doctors like me who advise patients. In order to use a treatment intelligently we also need to use RCTs to develop evidence based diagnostic criteria and also to estimate the probabilities of benefit for different severities of disease (the predicted benefits to be suggested by patients). In order solve differential diagnoses we also need ratios of sensitivities and not likelihood ratios based on false positive rates, which can be very misleading and of no help to diagnosticians. This is what I said at Evidence Live 2015 and in a recent blog: ‘The way forward from ‘rubbish’ to ‘real’ EBM in the wake of Evidence Live 2015’: http://blogs.bmj.com/ce/2015/04/24/the-way-forward-from-rubbish-to-real-ebm-in-the-wake-of-evidence-live-2015/