Is there anything that has been tried to stop publication shopping

Question

I find it very annoying as a reviewer when I get asked to review a paper several times as it gets submitted to different journals/conferences or, worse still, I help send it to the reject pile only to have it published elsewhere with my comments ignored. I definitely get the impression that a small minority of authors "shop" their papers around too much, which wastes a lot of the community's time. Often these papers are completely unpublishable, but sometimes it seems the authors are just going down some list until they reach the appropriate "level", rather than submitting appropriately in the first place.

I often devise hypothetical schemes to prevent this. For example:

1. Require authors to pay to submit.
2. Require authors to disclose the submission history their article.
3. Have journals/conferences publish a list of rejected articles.
4. Calculate an acceptance ratio for authors, along the same lines as bibliometrics.
5. Somehow publish rejected articles too, so that they cannot be published elsewhere. E.g., have a journal that accepts nearly 100% of submissions, but includes a review or rating with each article, so readers know which ones are the good articles.

Of course, these are just wild ideas, many of which are impractical and would greatly annoy authors. I'm curious to know if any such schemes have actually been tried and, if so, whether they seem to have worked. Surely, in some far-flung corner of academia, one of these ideas has been tried.

Question: Are there systems that have been tried by journals/conferences to disincentivize authors from submitting their articles to too many journals/conferences? If so, did they work?

I am only aware of one experiment along these lines: ICLR - the International Conference on Learning Representations. All submissions and all reviews are made public and they stay online even if they were rejected. I don't know what effect this system has had. I would like to hear other examples and also know if there is any evidence regarding the effect of such policies.

Answer 1

A version of your suggestion 1. is implemented in my academic field (mathematics).

In mathematics authors usually pay dearly to submit to any particular journal, not in money but in time. Namely, an unacknowledged but hard to dispute stage of the refereeing process in mathematics is the part where the referee lets the submission sit on their shelf for N months. It would be interesting to track the dependence of N on various factors, but based on my own experiences as an author (and okay, yes, as a referee!) I believe the median value of N to be about three months, with a higher mean. Especially, when I submit a paper to a large range of reputable journals in my field, I feel that I am incurring a risk -- small, but not negligible; let's say 3-5% -- that the referee will wait a year or so before rejecting the paper with a wholly superficial accompanying report (say, one paragraph that does not make specific reference to anything in the paper past the introduction).

This makes it rather impractical to shop a mathematics paper around to a large number of journals at about the right level. (If you send a less than excellent paper to a truly excellent mathematics journal, the chances are good that it gets bounced back to you quite quickly. I have heard stories of the absolute top journals in mathematics keeping a paper for a year or more and then responding negatively...but then I have also heard of them doing this and then accepting papers that I would have guessed they would reject.) This certainly works for me: I have submitted over 50 distinct math papers for publication. Most often I step down in quality after the first rejection and always after the second rejection. For me, having three or four journals consider the same paper feels like quite an ordeal....because it is likely spread out over several years!

Maybe this sounds sarcastic. It is not intended to be. If anything, it is a relief to think that there might be some good effect of the current endemic slowness of mathematical refereeing. Because we have a thriving preprint culture, it is hard to argue that slow publication has a significant effect on the intellectual progress of the field. If only the slowness were equidistributed, I think there would be little to complain about. (But it isn't, and I suspect that my suffering is as nothing compared to that of some of my junior colleagues...In fact, just now I realize that while for many years now my papers have been treated better by top journals whether they take them or not, at the beginning of my career I had several papers languish in top journals for a really long time. The rich should have to pay more, not less.)

Answer 2

When a paper is rejected, there are two possible reasons:

1. The paper is bad
2. The review is bad

The majority is 1. (hopefully), but 2. is also very common. Some random examples:

• 8 Scientific Papers That Were Rejected Before Going on to Win a Nobel Prize
• The Curious Case of Srikanth Krishnamurthy: acidentally submitted two versions of the same paper to MobiCom. One version of the paper scored 1,2 and 3, and was rejected, the other version scored a 3,4 and 5, and was accepted!
• My postdoc advisor got a best paper award after 2 rejections

As there is no way to distinguish 1. and 2., if their authors resbumit, should we (i) punish both or (ii) ignore both?

Require authors to disclose the submission history their article.

An example: see the Reviews from Prior Submissions section of the CFP of Oakland, one of the big 4 in security.

Answer 3

I'm not aware of any disincentives. However, I can describe a process we're currently undertaking to preserve the valuable information that is often lost when a manuscript moves from one journal to another. I consider the present situation wasteful of time, effort and resources. I participate in the Editorial Board of a general medical journal considered the premier medical rag in my country. At the present time, we're running an experiment of sorts in collaboration with other medical subspecialty journals and craft groups to share submission histories, reviewer comments, manuscript revision and decision histories. This will enable us to track a submission, at least within the consortium.

We were surprised that editorial processes haven't been subject to the rigorous experimental design that we ourselves expect from our manuscripts. Thus, we decided to design one ourselves. In this experiment, the author is free to choose to which journal he or she submits. If an accept decision is eventually made (regardless of the number of revision rounds), then there is no further action on the consortium's part. However, if a reject decision is made (again regardless of the number of revision rounds), then the manuscript's meta-info are logged onto the consortium's database. If the author chooses to submit to a member of the consortium, then the manuscript's history is made available to the editorial team of the second journal. And so on.

There are flaws in this idea, of course. Journal coverage is incomplete, for instance. We will not have access to histories from journals outside the consortium. Some of the previous decisions may colour subsequent editorial decisions is another potential concern.

That's the general idea, at least. We're in the early stages of its development.

I do hope that this helps.

Answer 4

I can’t help noticing some contradiction in your post. You observe a small minority of the authors shopping around and are concerned about the uneven communities’ waste of time.

Your main problem seems to be your personal annoyance, especially seeing a paper being published without your valuable comments (given while rejecting it) being addressed.

By now you probably have your list of this small minority of authors in your field which ‘shops around’. You are certainly not obliged ‘to help’ these authors’ papers to the pile reject. In addition I follow BruceET here. Just don’t review a paper you previously rejected. It will save you a lot of time.

Your suggestions (the legal ones) are interesting as a mind experiment. In addition: as bad reviewers are just as annoying as bad authors I (solely as a mind experience) would suggest the following: let each review be accompanied by statistics on the reviewer.

• the reviewer’s field of specialism, years of experience, number of publications published as first author and the journals published in.

• the reviever’s review history (number of review reports, their length, the journals reviewed for, duration of the reviews, reviewer’s decisions made in comparison with final editorial decisions).

• of course, as you propose for authors, these reviewers’ statistics are shared with the journal’s consortium.

In addition we could think of similar statistics for associate editors.

To continue:

• after an editorial decision, authors are requested to provide a review on the quality of the review reports (validity, coherence and structure) and editorial guidance and responsiveness. These scores will be added to the reviewers’ statistics mentionned above.

In this way we can more easily identify bad reviewers and bad editors. This may certainly help authors, reviewers and editors to maintain the quality of the review process.

Answer 5

There are two important perspectives here.

Author's point of view. Journals have standards of various levels, and 'topics of interest' can change in important detail over time from one chief editor to the next.

I think it is entirely reasonable for an author to submit to the highest-level journal that seems realistic. In my field, rejection will usually be very quick if the paper is far below standard for the journal.

If an associate editor feels the paper has some merit, then it will go to review. If reviews are positive it may be accepted, provisionally accepted pending suggested/required revisions, or rejected (due to the editor's taste or too high a backlog of 'worthy' papers). If the paper is rejected, one can hope that useful suggestions for improvement will be included. Also if reviews are unenthusiastic, the paper will be rejected. Then it is up to the author to get another idea or to re-submit elsewhere, possibly with some modifications.

I have served on publications committees of a couple of societies publishing journals in my field. Serious efforts have been made to speed up the review process. Some major journals now make final decisions on 75% of papers within six months. I detect no deliberate attempt to use delay as a penalty for successive submissions.

I see no reason an author should take one or two rejections as a sign to abandon the essence of a paper. However, feedback should be used to prompt revisions, and after a couple of strategic downgrades in the prestige of journals submitted to, perhaps it is best for an author to move on to other work. These days there are dozens of predatory 'journals' that will accept almost anything, and self-respecting authors will protect their reputations by avoiding them like the plague they are.

Reviewer's point of view. Reviewers should set reasonable, but generous, limits on the number of papers they agree to review and try to respond quickly (the task will not get easier if delayed).

My view is that the only proper response to a request to review a paper for which one has previously recommended rejection is to refuse immediately to review it again, saying why. That will take almost none of the reviewer's time. Thus the reviewer can spend precious time giving a helpful review of the the next relevant paper to come along.

The author deserves an unbiased, fresh review process. However tempting it may be to believe in one's preeminence, few reviewers are uniquely qualified to pass judgment on any one paper.

Answer 6

I find it very annoying as a reviewer when I get asked to review a paper several times as it gets submitted to different journals/conferences or, worse still, I help send it to the reject pile only to have it published elsewhere with my comments ignored.

I believe you are taking excess ownership in the outcome. It's not your job to set yourself up as the ultimate gatekeeper of the entire scientific literature. As a reviewer, you are an extra pair of eyes (perhaps with some especial subject matter familiarity) to help the editor make a decision. But he still has full ability to ignore your advice and accept a paper.

And authors have the right to not make changes they don't agree with, knowing full well that this may/not lead to rejection and movement to another venue. And to pull papers from journals pushing for changes they disagree with.

It's the author's paper, hence the byline. And the editor's magazine. Your position is much more limited and advisory than theirs.

Of course it's annoying to see the paper again, but in that case, I recommend you to just tell the new editor that you previously rejected the paper and won't review it again. Don't send them a copy of the old review or justify your reject recommendation. Just pull yourself from the process. No more cause for annoyance then. (This also gives the author a fair chance from different eyes.) Of course, the simple note that you saw it before and refuse to review it may influence the editor. But you should not worry about that pro/con (if he wants to give the paper a fresh chance or just gets rid of it).

Answer 7

Unfortunately, the process user St. Inkbug suggest, might be illegal. According to EU, sharing info without the approval of users.

Somehow, I need to agree with number 5.

Somehow publish rejected articles too, so that they cannot be published elsewhere. E.g., have a journal that accepts nearly 100% of submissions, but includes a review or rating with each article, so readers know which ones are the good articles. and it is now done by Sci, Sci — Open Access Journal http://www.mdpi.com/2413-4155/1/1/1/htm