A View From the Sicily Evidence-Based Healthcare Conference

It’s a stunning place. And this conference makes the best of it, held in a hotel overlooking its own Mediterranean beach. The weather was pretty much perfect as well.

This was the 10th international Evidence-Based Healthcare Conference in Sicily, intended for teachers and developers of evidence-based healthcare. The first was in 2001. It manages to be both intense and pampering at the same time. And with just 136 delegates, there’s a lot of interaction.

The conference is the child of Nino Cartabellotta. This year he passed the baton to his colleague, Tiziano Innocenti, who pointed out he’s the same age as the term “evidence-based medicine” (EBM), coined 32 years ago. It was the first Sicily EBHC Conference I’ve been to. (I was an invited speaker.) And there was so much going on, I expect this post will be followed by at least one more.

The theme of the conference was the Ecosystem of Evidence, with a subtitle around Covid experiences and future challenges. In Innocenti’s introductory remarks, he drew on Cartabellotta and Tilson’s discussion of an ecosystem as “a community of living organisms in conjunction with the non-living components of their environment (air, water, mineral, soil) acting as a system.” The stakeholders in the evidence ecosystem with their competition, collaboration, and conflicts of interest, he said, interact in an environment that includes non-living components, like evidence and the rules and standards of evidence generation and synthesis.

It was a bit of a plot twist, then, when the main recurring theme in the conference was living systematic reviews and guidelines! I wrote a post about living systematic reviews a year ago, including a look at the history of when we tried it in the 1990s with the establishment of the Cochrane Collaboration, how rarely it was achieved, and an example of an organization that’s been pulling it off for decades in cancer in the US, the National Cancer Institute’s PDQ process. My cartoon joked about calling living reviews that don’t update zombies. Turns out, it’s now a technical term for real! Or at least, it is for Melanie Golob, who presented a talk called, Dead on Arrival? An overview of living systematic reviews and their methodological rigor.

“Living systematic reviews,” Golob said, “can be the Wild West right now.” Her analysis included all 23 Cochrane LSRs, compared with a random sample of 23 non-Cochrane ones from the 113 they had found. Less than a quarter met all the criteria they studied, and those were all from Cochrane. There were already 7 designated zombies – all of them non-Cochrane reviews. It had been more than 2 years since the original publication, and they had not yet been updated. That’s not encouraging.

In his talk on living guidelines, Holger Schünemann stressed the importance of what his group calls “consequential evidence”. You don’t go through the whole updating process for every addition of evidence that doesn’t change anything. They survey their guidelines panel, and if more than 50% agree, then an update in response to new evidence goes ahead. (This is similar to the way PDQ has been doing it for many years.)

Golob said that “living” was a buzzword in the pandemic. Outside of the conference, Mt Etna loomed. The pandemic did the same for the conference – even when it wasn’t the center of speakers’ attention, it tended to make its presence felt in some way.

BMJ editor Kamran Abbasi addressed the enormous level of conflict around evidence during the pandemic in his talk. He said the EBM community “needs to learn to debate as one tribe.” It’s an important point, and I’m still wondering about it. The definition of EBM is about using evidence, in combination with patients’ values and preferences. The pandemic revealed some profound differences in values between many of us. If it is a tribe, it’s definitely not a monoculture.

John Ioannidis discussed the conflicts, too, and his talk was a vibrant demonstration of how alive the divisions remain. For example, there seemed to be enthusiastic accord when he said that the big platform trials, RECOVERY and SOLIDARITY, “were the great [EBM] success story” of the pandemic. But division was instant when, in talking about excess mortality, he said that the very old would have died soon anyway. I wasn’t the only one who was appalled at what sounded like dismissiveness about mass traumatic death. Meanwhile, he’s still apparently appalled by the heavy criticism he faced for his role in the pandemic. (I was one of his strident public critics.)

Ioannidis and I are on the same page when it comes to most evidence issues. On some critical values, though, we might as well come from different planets. Values can affect how scientists weigh evidence, too, just as we acknowledge it does in personal health decisions. It’s not surprising, then, that there are contradictory analyses of the evidence on excess mortality, given the potential for confirmation bias to distort thinking. Being an expert in bias and evidence doesn’t necessarily mean a person is good at managing their own cognitive biases – or even that they always want to.

Abbasi argued for truth and reconciliation after all the emotion of the pandemic. There’s a compelling argument for that, though I suppose it’ll be down to many small interactions, not a major public process in evidence communities. We can already debate divisive issues like pandemic mortality and public criticism with less heat than we could in 2020. I’m not convinced there is a problem beyond that. Perhaps, having learned about how deeply we differ is a form of maturity, and not something that needs repair.

Values were at the heart of one of the most important and impressive presentations, on equity gaps in evidence-based health care. It was by Eleanor Ochodo, from the Kenya Medical Research Institute and Stellenbosch University in Cape Town. Health inequity, she said, is about unfair, systematic, and avoidable differences: “They are unfair because they can be resolved by reasonable action.” Inequities arise in part from history, and in part because they are perpetuated by racism and gender bias, for example, and bias in healthcare delivery and research. Ochodo highlighted the work of Wendy Rogers on this issue (here and here). She also listed equity frameworks in EBHC:

• PROGRESS PLUS
• GRADE equity guidelines
• Equity-Focused Knowledge Translation (EqKT) Framework
• NIHR-INCLUDE Ethnicity Framework
• SAGER (Sex and Gender Equity in Research)
• ETRs Health Equity Framework

Ochodo presented data from participation in Cochrane and use of The Cochrane Library for perspective. Of the 7 to 9 million visitors to The Cochrane Library each year between 2019 and 2021, only around 10% were from lower and middle income countries, and 40 to 47% were using non-English internet browsers. The proportion of visitors who were from countries other than Australia, Canada, UK, and USA was over 60% – whereas the proportion among authors of the reviews was over 40%.

Good news here was that Cochrane had made major progress on gender participation from its early days. (As one of the tiny minority of women at the start, that was heartening – but a reminder of just how much effort it took to change.) The progress was encouraging, but, Ochodo said, there is still much to be done about EBHC’s impact on equity. Her recommendations get the last word for this post:

• Prioritize research and review questions that address health equity.
• Better inclusion, design, collection, and analysis of equity relevant data.
• Commitment to justice in healthcare with accompanying funding.
• More patient, public, and community advisory board involvement in guideline development.
• More advocacy and interdisciplinary partnerships for equity in EBHC.

The next Sicily conference is planned for October 2025.

After the second of these conferences in 2003, attendees published a “Sicily Statement on Evidence-Based Practice.” Now, 20 years later, the conference ended with David Nunan from Oxford kicking off an update.

~~~~

Disclosure: I was invited to give one of the keynote presentations at this conference, with travel support from the organizing group (the non-profit GIMBE Foundation). The categories of sponsors accepted for the conference are listed here, and do not include manufacturers of drugs etc. In terms of the evidence “tribes” with a strong presence at this conference, I have a long relationship with Cochrane and the BMJ, and I was a member of the GRADE Working Group for a time long ago.

The photo of Taormina, Sicily in October 2023 is my own (CC-BY-SA 4.0).