Who are the people reviewing far more papers than they're submitting for review?

Question

I've submitted more papers for review than I've reviewed myself. I try to review as much as I can, but it's limited by incoming requests. Publons says I'm somewhat above the median (of 0.3 reviews : 1 publication), although this data is incomplete.

It seems there must be people out there who are reviewing far more papers than they're submitting for review. But who?

Question: Who are the people reviewing far more papers than they're submitting for review?

I'm just after some mental picture of who is receiving, accepting, and completing large numbers of review requests. I can't imagine it's high-level professors, who seem constantly busy. Nor do I expect it's early-career researchers, because they're unestablished and unlikely to be thought of when editors email reviewers.

Answer 1

I can't imagine it's high-level professors, who seem constantly busy.

If you want something done, ask someone who's constantly busy.

Nor do I expect it's early-career researchers, because they're unestablished and unlikely to be thought of when editors email reviewers.

I can't point to specific posts, but I've seen enough references on this site to the scenario that I believe it's not uncommon for the editor to ask the high-level professor and the professor to delegate to an early-career researcher in their department.

Answer 2

First, that ratio is based on published papers, not submitted. But the principle is the same. I was given advice several years ago that you should be reviewing 3 or 4 times as many papers as you submit.

Think about it this way; if papers need 3 reviewers and half of the papers don't get published, then the break even point is 6 reviewers per published paper. If you are in a field where 3 authors is typical, then all authors would need to review 2 papers per paper. But not all authors are able to review equivalently, so 3-4 seems to work for my discipline.

An editor would be able to give you the best information about who they ask, but senior postdocs and low to mid level academics seem to be the ones reviewing in my discipline. On the other hand, I checked publons and the highest count is actually a semi-retired professor.

Answer 3

People who submit good reviews are asked to do more of them, and then more, and more, and....

Let me tell you about my own refereeing history. My research profile is not stellar, but it is adequate for a tenured theoretical physicist at a middling R1 institution; I typically publish something like two to five papers per year. On the other hand, I referee something like ten times that number of papers, and the refereeing work ends up being a major component of my professional service. In the last couple of years, I have won refereeing awards from several major journals.

Over four years as a post-doc, I got one paper per year to referee. Around the time I was established enough to get a tenure-track job, the number jumped to about one paper per month, and it has continued to increase (roughly linearly) in the decade-plus since. Unlike some of my colleagues, I try to referee practically everything I am sent that I am qualified to evaluate, and as I have built up a relationship with some journal editors, I have both been sent more papers and have been assigned to adjudicate more complicated situations, such as cases where previous referees have disagreed or appeals submitted by the authors of rejected papers.

Frankly, this does cut into some time that I could probably be usefully for research. On the other hand, I feel that, in a certain sense, I owe the professional community my expertise; I want to do at least my fair share. I work in a somewhat niche area, and there are not a lot of people who are able to give complete evaluations of research in this area. That means both making sure that papers with fundamental mistakes are not erroneously published, and making sure that the good papers are properly vetted and corrected, to maximize their usefulness. Recognizing the unacceptably bad papers quickly is a skill, but once the serious problems with a manuscript have been identified, writing a rejection recommendation is not too time consuming. Checking through good papers for minor elisions and errors, on the other hand, can take quite a bit of time, and that is where there is probably a real tradeoff between refereeing work and research output. However, as I said, I am happy with where I am currently situated in this regard.

Answer 4

Authors in developed countries tend to review much more papers than they submit, while the three biggest countries for which the reverse is true are China, India, and Iran.

Source

Answer 5

Before retiring I worked for a company that (inter alia) did paid research for industrial companies. We had little incentive to publish in the peer refereed literature. There was some incentive to publish in the trade literature, but not much - we were very well known among potential clients. We did participate in industry and regulatory groups as contributions to the "greater good" for the industry. I and many of my senior colleagues reviewed many more papers than we submitted; I can't speak for my colleagues but I regarded it again as contributing to the "greater good".
Edited to add:
Similarly, I had no interest at all in recording my contributions in Publons, which Wikipedia describes as (my emphasis) "...service for academics to track, verify, and showcase their peer review and editorial contributions...".

Answer 6

I am one of your mystery reviewers. I work in research in industry, and I have gotten a positive reputation in my field for my work. As such I am frequently requested to review journal articles and since my combination of fields is relatively rare, I generally feel a duty to help ensure the quality of the contributions in that field through reviewing articles (which I frequently do on my own time).

On the other hand, submitting articles is more of a secondary (or even tertiary) function of my job. Any publications have to be cleared through our IP team to determine if we need to get a patent filed first or even try to protect the knowledge as a trade secret. When I do publish, it is usually to establish the value of a product or it is part of an academic collaboration I have been engaged in. As such, I only publish about once a year to every other year.

Answer 7

I suspect a major factor is that ratios will vary significantly by field. In my field (pure math), a typical paper has 1-3 authors and will get reviews from two people (more if it gets rejected post-review and sent to another journal, although overlap of reviewers in that case is not uncommon). So I'd expect the median ratio there to be somewhere around 1. Mine is 1.3 FWIW (I am not on publons but keep records for my own interest).

In other fields the typical number of authors per paper could be much higher without the number of reviewers per paper significantly increasing, so I would expect a much smaller ratio. 0.3 seems reasonable for fields where 10+ author papers are the norm.

Answer 8

Actually the most reviewing is, in my field and related ones, done exactly by the two categories that you wanted dismiss.

A reputed professor receive a large number of invitations, so that the ratio is high in spite of refusals.

A successful and young researcher receive relatively few but s/he tends to accept all as it is a new and certainly formative experience.

I suspect, as another answer points out, it's very much field and subfield related. I do personally reviewed much more papers than those I've published.

While a researcher in synthetic chemistry publish and might be invited by a few good journals, one in material science can do that in a broader range, spanning from physics and chemistry to dedicated publications. When preparation of new materials is involved, basically all papers are getting interesting and reviewing becomes a way to stay tune.

Similar scenario is plausible in many applied sciences.

In my field the number of reviewings largely exceeds that of the submitted/published papers for literally every graduate researcher that I ever met.

Also a ratio as those given in your Q, if not individually calculated, would block research publications. Simple maths suggests that ratio lower than one would be a big problem in the current procedure, except perhaps in fields where having about ten authors or more is common (still some of those fields have also more abundant literature, e.g. medicine). Also having the name in a author list doesn't make you a referree, at least not one of first and even second choice for a good journal.

Edit 1 after the comment.

Edit 2: Stimulated by this I've finally registered on Publon. While importing most of my papers was a simple click, the thousand review I have done in the past will never show up. So at the current stage I have contributed X papers to literature without having been referee at all. I've overlooked the fact that OP inspiration was Publon. Basically she can be confident of how much she is reviewing (not enough in my opinion, as for what I and others have written above) but in general one should be highly skeptical of those data, independent of their actual significance.

Answer 9

This is just a theory: The most active reviewers are editors of low-ranked journals, who assign the reviews to themselves.