What if We Can’t Rely on PubMed?

PubMed is incredibly reliable. And a lot depends on it. It’s an ecosystem built around MEDLINE, the steady feed of new publications in biomedicine: It determines which journals count, takes their output in, adds valuable information and linkages, and feeds it back out – free to users globally. And there’s a lot more, too, that we rely on from the NCBI (the National Center for Biotechnology Information at the NIH’s National Library of Medicine).

In turn, many services and scientific tasks are built on PubMed and other NCBI outputs. PubMed itself is not just critical to people who search for biomed information. It’s central to a lot of people’s workflows in the science world. A helluva lot of us know that awful feeling of panic on those very rare days when PubMed goes down for a few hours, and we have a deadline. If it’s under threat, we should fight tooth and nail for it.

That said, between the risks of an exodus of key personnel, understaffing, or goodness-knows-what vandalism when a goon squad arrives at NIH, it’s not paranoid any more to think ahead to the once-unthinkable. What would PubMed enshittification look like? Could PubMed go down more often, and for longer? Might services no longer be free? How else could the quality and reliability of its services be degraded?

Some of that may be longer term. In the short term, just how much junk science will flood in, if (when?) the NLM is no longer a reliable gatekeeper? Here’s how it works now:

“The National Library of Medicine (NLM) decides whether the scientific and editorial character and quality of a journal merit its inclusion in MEDLINE. In making this decision, NLM considers scientific policy set by the NLM Board of Regents, the suitability of the journal for the NLM Collection (according to the criteria in the Collection Development Guidelines), as well as the recommendations of an NIH Federal Advisory Committee, the Literature Selection Technical Review Committee (LSTRC).”

So, a couple of US government committees currently hold this line. Gulp!

Exhibit 1: Consider what Adam Marcus and Ivan Oransky from Retraction Watch just wrote. At his Senate confirmation hearings, the new US health boss brandished a study he claimed showed a link between vaccines and autism. This paper, they pointed out, “appeared in a journal that is not indexed by the National Library of Medicine or by any other organization that might provide it with some scientific credibility.” They go on to show other red flags at that journal. Pointing out that a journal isn’t PubMed/NLM-indexed has been shorthand for un-credibility for a long time, but that table could turn quickly.

Exhibit 2: Consider this headline from an article by Emily Mullin and Matt Reynolds at WIRED: “Donald Trump’s NIH Pick Just Launched a Controversial Scientific Journal.” This one, they write, is linked to a right-wing news site, and only members can publish in it. The political appointees there have been “on leave” since tapped by the new administration, but you get the picture.

Carl Bergstrom is quoted in the WIRED article as saying this journal is part of a strategy: “If you can create the illusion that there is not a predominance of opinion that says, vaccines and masks are effective ways of controlling the pandemic, then you can undermine that notion of scientific consensus, you can create uncertainty, and you can push a particular agenda forward.”

There’s already a quality problem in PubMed, given that all publications from eligible government-funded research goes in, even if the authors published in a non-indexed journal. What’s more, like any institutional gatekeeper with a long history, the NLM has often made problematic errors, in both inclusion and exclusion. The resources to meet their electronic technical submission requirements create another barrier, as does language. Adding an overtly anti-vaccine, anti-climate-science, and anti-critical race theory, anti-diversity, anti-equity, and anti-inclusion agenda is a terrible thought, though. It doesn’t seem paranoid now to think that the floodgates to BS masquerading as science could be opened, deliberately, at PubMed. The global science community has to urgently consider what to do about this.

All of this is obviously scary, but there are options as well as fighting to hold the line inside the NLM. And if we can’t rely on PubMed, it wouldn’t mean a complete return to the bad old days of limited access to reliable searching. There are alternatives for PubMed downtimes. And on a longer term scale, there are places that could step up to help us.

MEDLINE journals submit their records to NLM electronically now, and they can manage their PubMed citations themselves. Journals submit data for the DOI system, too. (More on that later.) Plus, much work that used to be done by human indexers at NLM is now done by machines: Losing it would be catastrophic, but not as much as it would have been when journals were all in print only. All this means that there are some backups, and there are several places we can look to for contingency planning to reduce dependency on these literature-related services. (Although I worked at NCBI for nearly 7 years, my expertise is only in PubMed and ClinicalTrials.gov, so I can only offer information on those services.)

Here’s what I’ll do on days I can’t use PubMed, and some other organizations that could fill gaps.

First stop: EuropePMC

Don’t let the name confuse you. This is not just a version of PMC (the full text repository, PubMed Central). Think of it as “PubMed Plus.” You can read more about it here, and get search tips here if you need them – given how similar it is to PubMed, you might not.

It is Europe-based, run by EMBL’s European Bioinformatics Institute (EMBL-EBI), located in the UK on the Wellcome Genome Campus. A bunch of major European funders support it as a repository for the full texts of articles arising from research they fund. EMBL, the European Molecular Biology Laboratory, is intergovernmental, funded by its member states – about 30 countries. Its full member states are all European, plus Israel. In addition, Australia is an associate member. (Argentina was too, but it withdrew in 2020.)

As long as PubMed content is reliable and the partnership with EMBL holds, there is free access there. But what if it doesn’t? Well, while the PubMed/PMC content depends on the NLM, EuropePMC would still have access to the metadata from the DOI sytem.

Alternative data, first stop: DOIs from Crossref

Journal articles and more have unique Digital Object Identifiers. That’s the number that looks like this:

10.1371/journal.pmed.1004532

The DOIs for the scholarly literature trace back to the standards-setting organization of the UN. Its members are essentially publishers and registration agencies. DOIs are used in PubMed and many other places to link literature together. The metadata from the DOIs is openly available through the nonprofit Crossref. They describe it this way:

“Crossref is the sustainable source of community-owned scholarly metadata and is relied upon by thousands of systems across the research ecosystem and the globe… The information covers elements like titles, contributors, descriptions, dates, references, connecting identifiers such as Crossref DOIs, ROR IDs and ORCID iDs, together with all sorts of metadata that helps to determine provenance, trust, and reusability—such as funding, clinical trial, and license information.”

Crossref is part of the non-profit International DOI Foundation, which has an international governing board.

Another stop, for biomedicine and beyond: OpenAlex

This is called Open Alex, in honor of the Library of Alexandria. It’s also nonprofit, based in Vancouver, Canada, and the French government began contributing resources in 2024. It is an open access alternative to Web of Science and Scopus. It draws heavily on CrossRef and other sources.

This is very different to searching PubMed or EuropePMC, but it gives you an idea of the breadth of resources being developed based on publicly available data that is not reliant on the US government.

You can follow OpenAlex on Mastodon.

What about ClinicalTrials.gov?

This is another NCBI service. If ClinicalTrials.gov goes down, the first stop now is the WHO international trials portal. (Clinical trial registration is a big topic, for another day, perhaps!)

Finally, it’s critical to remember that discoverability of publications doesn’t begin with all these services. It begins with authors. If you write papers or peer review them, there are several things you can do at your end of this chain:

• Make your work more discoverable: Become a more machine-friendly researcher. You are not just writing for humans! Titles and abstracts matter so much – and science as open as you can make it. More on this here.

• Be much more careful about citing correctly, and include ID numbers like correctly formatted trial registry numbers in your abstracts.

• If you don’t already have your ORCID, get onto that ASAP.

• If you’re lucky enough to have the support of librarians and information specialists, ask what you can do to support them – we need them more than ever.

So scream if you need to. But fight for PubMed if you can – and support de-centralization and open access in all the things!

You can keep up with my work at my newsletter, Living With Evidence. I’m active on Mastodon: @hildabast@mastodon.online and on Bluesky.

~~~~

Update on March 4, 2025: The information on the International DOI Foundation was moved to the end of the section to not give the impression they were involved in the day-to-day running of Crossref. Thanks to Ginny Hendricks for pointing this problem out.

Disclosures: I am not a medical librarian or information specialist. I worked at NCBI at the National Library of Medicine (NIH) from 2011 to 2018 on PubMed projects, predominantly PubMed Health and PubMed Commons (both of which no longer exist).

Update: Shortly after publishing, I expanded the section on current quality problems with a section on gatekeeping. (“What’s more, like any institutional gatekeeper with a long history, the NLM has often made problematic errors, in both inclusion and exclusion. The resources to meet their electronic technical submission requirements create another barrier, as does language. Adding an overtly anti-vaccine, anti-climate-science, and anti-critical race theory, anti-diversity, anti-equity, and anti-inclusion agenda is a terrible thought, though.”)

The cartoons are my own (CC BY-NC-ND license). (More cartoons at Statistically Funny.)