“High Value, Low Wastage Research” Is More Than Just a Catchphrase Now
I don’t recall hearing it before. But I’ve heard the slogan “high value, low wastage research” a lot the last few days. And I think this one is more than just a catchphrase. It’s looking like a non-confrontational enough rallying cry to nudge some important reforms along.
It’s a child of the “research waste” agenda kicked off by a 2009 paper in which Iain Chalmers and Paul Glasziou estimated 85% of health research was a waste of time. In some ways that was a later generation of John Ioannidis’ 2005 “most published research findings are false” provocation. (I wrote about that here.) The “research waste” banner swings over the same issues and more, from research on the wrong questions through to unpublished and unreproducible research, and open access publication.
The symposium I was at was called “Ensuring Value in Research”. It was hosted in Sydney by the National Health and Medical Research Council (NHMRC) and the Reward Alliance, (Disclosure: I was an invited speaker, supported by the NHMRC.) And the reason I came away feeling optimistic that all this waste talk isn’t itself a waste of time, was hearing the commitment and thought coming from the people involved in the Ensuring Value in Research Funders’ Collaboration and Development Forum (EViR). It’s co-led by the UK’s National Institute for Health Research (NIHR), the US Patient-Centered Outcomes Research Institute (PCORI), and the Netherland Organisation for Health Research and Development (ZonMW).
EViR started when Matt Westmore from the NIHR invited a group of other funders to talk informally about responding to the research waste agenda. There were “few opportunities”, he said, for funders to get together, form a community, and share experiences and thinking. They modified the language to talking about increasing value as a positive take, to sidestep negativity many react to the “85% of research funding is wasted” concept. Here’s how Westmore summarized what the funders in this growing and active group are committed to:
There are gaps of course, in both the research waste model and the EViR one – and I spoke about some of them in my keynote. (My slides are here.) But the commitment to practical steps among speakers representing funders at the symposium was clear, and that’s encouraging. The group is developing work packages on stakeholder engagement, study registration, dissemination/implementation, and preclinical studies.
Public funds need public benefits.
That’s the commitment that means the most, isn’t it? That came from Caroline Homer, who spoke about the changes the NHMRC made to the way it assesses track records in their grant application peer review process. (More on that here.) Chipping away at the incentives to churn out loads of bad science was a key theme across the symposium – especially the harm caused by making journal status the proxy for quality and marker for success.
Speaking of journals, Tom Walley gave a great example of the kind of changes funders can make. He’s at a research institute in Australia now, but used to be at the NIHR in the UK. He talked about how the NIHR wrestled the problem of non-publication of research results to the ground. While most funders get a publication rate of about 60%, they got theirs to 98%. How? By creating a peer reviewed, monograph publishing service that everyone they fund has to contribute to within a year of finalizing their study (even if they also publish journal articles). And by having contracts with researchers, not just giving them grants. (More on that here.)
In one of the discussions, Rachael Morton nailed an important point, too. Some of what we call “the reproducibility crisis” is actually “a capacity crisis”, especially in biostatistical expertise. And John Simes pointed out: ““We focused in the past more on innovation than return on investment”. That’s got to change. Nor can funders drive all the change we need. Academic institutions in particular have to accept a lot of responsibility, too.
There were presentations, as well, that slammed home the limits of concentrating on effectiveness research and its processes. Let’s finish with 2 of those.
Erica Spry is a Barda Jawi woman [PDF], and researcher at the Kimberley Aboriginal Medical Services. She and Emma Jamieson talked about their service’s work into the gap between what we know could help, and the reality of indigenous women’s lives. In their study, 20% of pregnant Aboriginal women in the Kimberley experienced adverse birth outcomes that were potentially related to gestational diabetes (GDM). [PDF] Heartbreaking. And 79% happened to women who hadn’t been diagnosed with GDM. One of the contributors was a delay in analyzing screening test results that reduced the effectiveness of screening. We have to support research into implementation – and action on the findings – all the way down the line, where people need it most.
Jess Baker is a psychologist. She spoke about work on refugee children’s health and wellbeing, interviewing young people, parents, and people working in services. [PDF] It was illuminating and moving:
They’ve gone for a specialist appointment they’ve waited three months for and had some hiccup with the translator… They say, thank you very much I’ll wait again, but the truth is they’re absolutely devastated, they’re in pain and they couldn’t express that they were in pain.
It was a powerful reminder of the absolutely critical role qualitative and social science research plays in deepening our understanding of what’s happening and why. And that’s not something that’s captured explicitly in the research waste model, is it? If we don’t truly “get” what suffering, relief, and service use, misuse, and underuse are about, and how change occurs, then an awful lot of research will be low value, high waste.
The Twitter hashtag for the symposium was #valueinresearch18. Thanks to everyone who tweeted at the symposium – it’s not just that it enhanced the conference for me (especially for sessions and photos of slides that I missed): the record was invaluable. And a giant thank you to everyone working so hard to make research better, and all the people who participate in it.
Disclosures: I was opening keynote speaker for the NHMRC symposium I discuss in this post, and the NIHMRC supported my travel and registration. Paul Glasziou is my PhD supervisor.
The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny and on Tumblr.)