Archive: Unclogging the Peer Review Pipeline
Noah Fierer
June 3, 2021
Peer review is a key part of science. Thorough reviews provide a critical check on the quality of scientific papers. Personally, nearly every paper I've been involved with has been greatly improved by going through the peer review process. Yes - crappy science still passes through peer review (exhibit A and exhibit B) - but, as a reader and contributor to the scientific literature, I have clearly benefited from pre-publication peer review. I would not want to see science move to a model where peer review is scrapped altogether (as has been suggested here and elsewhere). It is already hard enough to separate the wheat from the chaff, and peer review makes that process easier.
The problem is that the peer review process is broken. Maybe not broken beyond repair, but definitely strained. Why? Well, for one reason, it is often very hard to find scientists willing to review papers. Review requests are not evenly distributed across the scientific community. I typically get 10-15 review requests per week (more in December and June when authors are submitting manuscripts before heading out on vacation). At the same time, there are many qualified reviewers that receive few (if any) review requests - often because editors (or the databases they use) do not identify those potential reviewers. This disparity in review requests slows down (and weakens) the pre-publication review process. This is not a simple problem to solve and much has been written on this topic (e.g. here, here , and here). Nevertheless - I want to add my 'two cents' to the discussion, offer some potential solutions, and give some unsolicited advice to those struggling under the deluge of review requests. I recognize that these ideas are neither new or novel.
First - a disclaimer. I am writing this as someone who submits manuscripts to a wide range of different journals and as a frequent reviewer (last year I reviewed 40 manuscripts for 15 different journals). These numbers do not include my duties as an associate editor at Science Advances. I am writing this post from the perspective of an ad hoc reviewer - not as an associate editor and I definitely do not speak for Science Advances.
Decide how many reviews you can handle per year. Clearly individuals submitting more manuscripts should be doing more reviews - but there is no simple metric to calculate the # of reviews you should do. Plus, life gets in the way. While a general rule of thumb is that you should review 2 manuscripts for every manuscript you submit - rules of thumb are not always that useful. Handle what you can handle without sacrificing your mental health, time with friends/family, other professional duties, etc. However, remember that when you submit a manuscript you are asking 2-3 other people (plus the editors) to spend time on your manuscript and, in most cases, that time does not come with any direct compensation. Pay it forward.
Review the manuscripts you actually want to read. Good reviews take time. Thus, when deciding which manuscripts to review – select those you actually want to read. It is far easier (and far more rewarding) to review a paper you are actually interested in reading. Besides, one benefit of reviewing manuscripts is that it is an opportunity to catch up on the literature and learn about new topics/methods/concepts - so reviewing a manuscript is not only a professional service that benefits other scientists, you also reap benefits. Two birds, one stone.
Select potential reviewers carefully. Many journals allow you to recommend reviewers when submitting a manuscript. Don't just pick the obvious choices, like Professor X who has written 50 papers on the topic, as there is a high likelihood that Professor X is overwhelmed with review requests. Instead, pick scientists that might fly under the radar but are likely well-qualified and could provide valuable input.
Volunteer to review. If you are an early career researcher (a graduate student or postdoc) - tell your PI or an editor that you are interested in reviewing. It is valuable to learn how the sausage is made. Depending on the particular circumstances, you can either handle the review yourself, consult with your PI on the review, or co-review with your PI. You will benefit and the scientific community will benefit. You don't need to have published dozens of papers on a topic to be able to give a fair, balanced, and constructive review. Some of the most thorough and useful reviews I've seen typically come from grad students and postdocs. Have a website and a Google Scholar profile so editors can find out who you are, your research interests/experience, and your email address. We need to expand the pool of qualified reviewers if we want to reduce the strain on the system.
Scientists are nomadic, make yourself easy to find. Scientists often switch institutions and old email addresses become inactivated (or the accounts left unchecked). Stick with one email address (preferably one not associated with your institution - like a gmail address). It is surprisingly common for editors to have trouble contacting potential reviewers because email addresses are unused or out of date.
Manuscripts should be shorter. OK - this one is a bit more controversial. I would guess-timate that 50% of manuscripts are far longer than they need to be. I am not proposing reducing the length of the Methods (in most cases) - usually the bloat occurs in the Introduction and in the Discussion sections. Of course - do not leave out relevant details - but write as concisely as possible. This will make your manuscript more readable and, ideally, make your manuscript more straightforward to review. More Hemingway, less Ayn Rand.
Make sure the journal is a good fit for your manuscript. It is OK to 'aim high' - after all, you never know if your manuscript will get accepted by a given journal, even a high-profile journal, as there is a high degree of stochasticity in the process. However, if you are routinely getting manuscripts rejected from the most prestigious journal in your field - maybe you shouldn't submit all your papers there (and, yes, I know scientists that send nearly all their manuscripts to Science/Nature). There are a lot of great venues out there and no reason to think that every paper you write needs to be published in tabloid journals. Maybe I’m naïve, but I do think good manuscripts will be read/cited no matter where they are published, and submitting all your papers to a select few journals that are believed to be more prestigious than others can consume a lot of time from editors/reviewers and cause unnecessary mental anguish for the authors.
Be patient with editors. Reviewers and editors are usually handling manuscripts without compensation and handling manuscripts is usually just one small part of their workload. Yes - it is OK to contact a journal if your manuscript has been languishing for months and you fear that is lost in the system - but do so politely. Definitely do not send nasty emails if you haven't heard back about your manuscript after a few weeks (and, yes, I've seen this happen). Even if the stars align, the review process will take at least 7 weeks. Patience is a virtue (says the guy who often lacks the required patience).
Respond to review requests! If you are contacted to review and you just can't review the manuscript for whatever reason - that is fine. However, please respond to the request and recommend an alternate reviewer as soon as possible. Simply not responding to review requests slows down the entire process and ghosting is incredibly frustrating to editors (he says before immediately checking his inbox to make sure there are no review requests languishing in there).
I’m sure I’m missing other important points, so feel free to chime in by adding a comment. In the meantime, get back to those reviews!
