A Reader’s Guide to Conflicts of Interest in Biomedicine

The academic clinician in this cartoon needs 15 slides to list all the drug and device manufacturers he’s received money from recently. By the time he gets to the end of all that, will you think he’s not in any one’s pocket or dismiss anything positive he says about a drug? Something in between?

A systematic review by Adam Licurse and colleagues (2010) suggests we tend to want more disclosure of financial conflicts in interest in clinical research and care – and we’re moving towards more ways of achieving that, like online databases. ProPublica’s Dollars for Docs is one example of making that information accessible.

But is it making a difference? Are things getting better or worse? I suspect, all things considered, we might be going backwards – and the problem is spreading sidewards around us too. The push for patenting even publicly-funded scientific developments and tightly aligning science and business interest – “creeping propertization” – has been steadily making this a problem well beyond clinical research and medicine.

With so much financial conflict of interest around us that it’s getting almost ubiquitous, it’s hard to keep it in mind – and keep it in perspective.

Today PLOS Medicine has published a study by Paul Campsall and colleagues on financial relations between industry and the organizations that produce clinical practice guidelines. They did a good job on the study. (Disclosure: I was also the academic editor for the study.) But I found it disheartening to read.

My first instinct reading it was, what? Still??! I wrote the editorial that accompanies the article. And when I looked into the literature to decide what aspects to discuss, you couldn’t really say there has been solid improvement since these studies began in the 1990s. So I focused on the question, is this an intractable problem?

Lisa Rosenbaum, Jeff Drazen, and the New England Journal of Medicine (NEJM) have been taking a lot of heat this month for a series of editorials that go in the opposite direction: arguing we’ve gone too far in being critical of potential financial conflicts, and should ease up. Three former NEJM editors (Robert Steinbrook, Jerome Kassirer, and Marcia Angell) pushed back strongly in a joint essay at the BMJ. And so did a trio of editors from the BMJ itself, Elizabeth Loder, Catherine Brizzell, and Fiona Godlee:

We are deeply troubled by a possible retreat from policies that prevent experts with relevant commercial ties from authoring commentary or review articles. The pharmaceutical and biotechnology industries may well be our medical saviours, but that is not a good reason to return to past practices. Such policies were not motivated solely by a few events, as Drazen asserts, but by recognition of extensive, systemic problems. These problems are far from solved, including internationally, as shown by recent events in India and China.

I’m not going to go over the ground in all this recent writing – and if you read my editorial, you’ll see I’m very squarely in the “managing and disclosing interests matters” camp. I’m going to focus on where this leaves us as readers and audience members.

Adam Dunn and colleagues also published an important review on disclosure of conflicts of interest (COI) this month. (By the way: what was it about May 2016 and the COI issue bursting out all over? Was there a memo?!)

Dunn argues for the importance of public registries for transparency. And he’s written a great blog post about his article (and a piece in Nature as well). What does Dunn suggest you do when you encounter disclosures of financial interests? He says you can either ignore the interests, ignore the research, do the best you can to measure the risk of bias…

Or more simply:  ¯\_(ツ)_/¯

Years ago, I wrote a reader’s guide to author and sponsor bias, including a look at evidence, and the policies and practices at leading medical journals. I’ll copy in the key questions to think about shortly. But it wasn’t restricted to financial interests: money is far from the only interest that should trigger healthy skepticism.

I discuss self-promotion bias in particular – the interest people have in peddling themselves and/or their own ideological position can heavily bias what they will reveal. “Eureka!” bias is another problem – leading to the urge to exaggerate what’s been found. I didn’t discuss the opposite much – nihilism and people wanting to discredit what they don’t believe in themselves. That was an important gap in the article.

Here were the questions to help guard readers against authors’ biases:

Key questions

From journal to journal, the answers to questions about financial entanglement may be sprinkled through different parts of a research article, including: authorship affiliations and contributorship, the methods section, declaration of interests section, acknowledgements and funders’ section. Check all the fine print if you want to assess potential for author and sponsor bias. You need to see the full article—the abstract will not include such information.

• How much independence from the funders did the researchers have, and did they control the data, its interpretation and the publication?
• Is there a systematic review on the topic the research is addressing, and if so, how do the results of this study fit into the other evidence? If there is no systematic review, are there trials?
• What do accompanying editorials, letters and analyses in secondary evaluation publications (such as ACP Journal Club and Evidence Based Medicine) say?
• Is the range of opinion on the issue shown? Do they tell you what the other schools of thought on this issue are?
• Who says so? Is there a high proportion of self-citation? Are there many unsupported claims?
• If you disagree with the author, what evidence do you have to support your position?

That last one is a doozy: it’s our own bias that is probably most likely to trip us up. We tend to be plenty critical of the things that we don’t agree with – and give the things we want to be true an easy pass. (I’ve written some tips about protecting ourselves against our own biases here.)

I mention the issue of digging the information out of the article, too – if it’s even in there to be found. In March, the Center for Science in the Public Interest began a serious campaign for making conflicts visible within PubMed. The Acting Director of the National Library of Medicine (NLM), Betsy Humphries, talked about this at the annual Medical Library Association in Toronto – that was in May, too. NLM can’t require publishers to provide the data, she said, but there will be some developments. So stay tuned on that one.

All of this assumes, though, that potential interests are disclosed. And they often aren’t. Dunn’s argument for transparent public registries can help, but that’s not enough to make a difference if people don’t dig out and apply the information. With so much anonymity about – especially in peer review, both pre- and post-publication – too few people are in a position to address the problem of undisclosed conflicts. (A concern I’ve written about here and here.)

If journals start not just dropping the ball, but even not worrying about the game, then we are going to be even more reliant on others – especially journalists. Health News Review, a media watch dog, emphasizes the importance of journalists identifying conflicts. Whether they have the time to do it – and how well they manage their own conflicts – add further reasons for glum-ness to the picture. Financial pressures are squeezing both scientists and journalists.

I’m not entirely glum, though. That’s in part because I believe a larger historical arc is moving in the direction of more transparency and open-ness in society, and in science. And because I’m optimistic that the skills of identifying bias – and efforts to minimize it – will spread more widely.

Hm. Perhaps this cartoon should say, “Well, she would say that, wouldn’t she?”

~~~~

Disclosures:

• My day job includes work on PubMed Commons, the post-publication commenting system in PubMed.
• This post draws heavily on a May 2016 editorial I wrote in PLOS Medicine, as well as a 2006 article of mine in the Journal of the Royal Society of Medicine.
• I’m an academic editor for PLOS Medicine, including for the study on clinical practice guidelines that my editorial published today accompanies.
• I was a member of the GRADE Working Group from 2008 to 2011 (participating in these 5 standard-setting publications).
• I was a member of the working group the developed the 1999 Australian National Health and Medical Research Council (NHMRC) guidelines on the development, evaluation, and implementation of clinical practice guidelines.
• Another PubMed project I work on, PubMed Health, is used as a resource in ProPublica’s Dollars for Docs database.

The cartoons are my own (CC-NC-ND-SA license). (More cartoons at Statistically Funny and on Tumblr.) 

* The thoughts Hilda Bastian expresses here at Absolutely Maybe are personal, and do not necessarily reflect the views of the National Institutes of Health or the U.S. Department of Health and Human Services.