What the pandemic revealed about scientific publishing
Here are a few selected excerpts from this excellent article by elementary
Mark Humphries June 3rd · 7 minutes to read
I read my umpteenth news story on Covid-19 science, a story about the latest research on how to protect indoor spaces from infection, whether cleaning surfaces or air exchange was more important. And it bothered me. Not because it was boring (which it was, of course: there are few ways to make air filters and air pumps commonplace). But because of the way science treated it.
You see, much of the research published was in the form of pre-prints, papers shared by researchers on the internet before being submitted to a scientific journal. And every mention of one of these preprints was immediately followed by the information that it had not yet been examined by experts. As if to convey to the reader that the research in it that covered the whole story was somehow less worth, less valuable, less meaningful than the research in a published article, a peer-reviewed article.
Imagine reading about the discovery of the structure of DNA with the same reluctance that we use today: “In a recent letter to the journal Nature, scientists James Watson and Francis Crick of the University of Cambridge proposed a new structure for the DNA before (not yet reviewed by experts). They claim that their “double helix” model, a spiral made of two base strands, explains decades of experimental work and provides a clear mechanism for copying genes. Their proposal relied heavily on data contained in Letters in the same edition of Nature by the teams of Rosalind Franklin (not yet peer-reviewed) and Maurice Wilkins (not yet peer-reviewed). “
Or consider this modern interpretation of a particular scientist’s annus mirabilis:
“The past year, 1905, was a remarkable year for a Mr. Einstein who, in a series of papers, put forward no less than four new theories for modern physics. His first dealt with an eagerly awaited explanation of the photoelectric effect (not yet peer-reviewed), the second on how Brownian motion arises from the collision of invisible particles (not yet peer-reviewed), the third on equivalence between masses and energy (not yet peer-reviewed) and the latest paper updates Newton’s mechanics to be more accurate (not yet peer-reviewed) for objects moving near the speed of light.
These imaginary reports are both ridiculous.
Continuing…
Pandemics don’t comfortably wait for this slow, grueling, peer-reviewed process to judge the science. Science in the time of Covid-19 had to be nimble and quick, had to show its results to the world without the layers of review and revision. We needed rapid exploration of transmission models, immunity and reinfection, public health news and effective interventions, drugs to treat symptoms, machines to assist the critically ill, and the development of radically new vaccines. Hence, pre-prints, those manuscripts that were posted on the Internet prior to peer review, became the weapon of choice. Long common in physics, pre-prints in biology and especially in medicine exploded during the pandemic.
And the media has dealt with this explosion by repeatedly pointing out that the research has not yet been peer-reviewed. Presumably, they are doing this to warn the reader that research lacks the safeguards that peer review brings. The problem with this warning is that peer review does not protect against anything.
Does it catch fraud or data manipulation? No, obviously not: peer reviewers are not omniscient, so they cannot spot fabricated data or review all the results of a laboratory to see if they simply copied and pasted data between papers. If it were, we wouldn’t have the PubPeer website crammed to the gills with people reporting potentially serious offenses in published papers, nor Retraction Watch’s endless coverage of papers so questionable that they were struck from the literature become.
And finally over. This is an excellent piece and should be read at the source in full.
For this reason, the reporting on Covid-19 research bothered me so much: Peer review is not a watchdog, is not a gold standard, and it hardly plays a role beyond gatekeeping. It’s loud, biased, moody. So it is meaningless to point out that some research has not been peer reviewed: peer reviews played no part in deciding what research was relevant in the deep history of science; and played little role in deciding which research was relevant in the ongoing history of Covid-19. The mere fact that messages were being written about the research decided it made sense: because it needed to be done. Virus genomes had to be sequenced; Vaccines need to be developed; epidemiological models have to be simulated. The coverage of Covid-19 research has shown us how badly peer reviews need peer reviews. But hey, you have to take my word for it, because this essay is unfortunately (not yet peer-reviewed).
Mark Humphries is a Computational Neuroscience researcher at the University of Nottingham, UK, and is the author of The Spike: An Epic Journey Through the Brain in 2.1 Seconds (Princeton University Press).
Like this:
Loading…
Comments are closed.