17

Suppose you find out that someone you know well or work with is reviewing the same journal paper you are reviewing (i.e., finding out this information on your own, and not via the journal or editor, assuming a blind process).

Can you discuss the paper? Can you discuss your reviews?

11
  • 4
    I would lean towards no.
    – bobthejoe
    Commented May 19, 2012 at 3:34
  • 9
    You've already broken the rules by discovering the identity of the other reviewer. Don't make it worse.
    – JeffE
    Commented May 19, 2012 at 7:02
  • 2
    @RanG. Nope, not at all. Referees are supposed to be anonymous and independent. You are supposed to act in public as though you don't even know the paper exists.
    – JeffE
    Commented May 19, 2012 at 18:07
  • 2
    @RanG. Of course things happen. (Rules can be broken unintentionally.) But when they do, you're supposed to pretend they didn't.
    – JeffE
    Commented May 19, 2012 at 19:23
  • 7
    @JeffE in most cases, I would probably be more inclined to explain to the editor the situation and suggest that another reviewer be found rather than to pretend it didn't happen. Commented May 20, 2012 at 3:41

3 Answers 3

23

If it's a blind process, you're not meant to find out. As various people have pointed out in the comments already, discussing your review with them is therefore a big no.

One of the main principles of reviewing by several people is that you get independent opinions, so that if a particular (maybe well-known and well-respected) person doesn't like the research, they can't just make it disappear by convincing everybody else that it's bad.

For some conferences (and maybe journals?), you are able to discuss your review with the other reviewers after the initial submission. But in this case the facilities for doing this are provided by the submission system. There are two important differences to the situtation you've described though. First, your initial review will not be influenced by the discussion and second, there is a record of the conversations and changes made.

2
  • The reasoning of having independent opinion makes a lot of sense.
    – Ran G.
    Commented May 19, 2012 at 18:07
  • The system where the reviewers can discuss their reviews (and the authors responses) is an excellent scheme. We all make mistakes in reviewing from time to time, and this is a good way of dealing with them. Commented Oct 13, 2012 at 18:10
9

I largely agree with @Lars Kotthoff's answer, but I think there's another important aspect to the reviewers not discussing the paper with each other.

It encourages you not to rely on the other reviewers.

If I knew that X, who is really good at the mathematical side of things, was reviewing the paper, and talked it over with them, I might very well be tempted not to go over the math aspects of a paper with a fine-toothed comb, figuring "they've got it". Similarly, they might rely on me to pick over the data analysis or parameter choices, etc.

That's bad. Reviewers should be reviewing papers, not chunks of papers. It's one area where division of labor isn't desirable.

3

I think it depends upon the context and field.

For instance, on program committees in my field, once you've submitted your own review, it is normal to know who the other reviewers are and be able to see their reviews. (As a program committee member, I've always been able to see other reviewers' reviews and see their identity. If I was barred from doing so, I would probably refuse to serve on the program committee.) I always read the other reviewers' reviews, and sometimes discover that I'd missed something important. It is not uncommon for me to adjust my review in light of insightful comments found in other reviews. Therefore, I think it is helpful to be able to read other reviews.

Generally, I think the reviewing process assumes that reviewers will be independent. This is important, to give authors a fair shake and combat groupthink. For instance, in my field, we generally try to ensure that every submitted paper receives at least 2 reviews (usually at least 3 reviews). Why do we do this? Because we know that reviewers are human and can make mistakes, and it is possible for one reviewer to dislike a paper that is nonetheless great and worthy of publication. Therefore, our system relies upon reviewers to independently evaluate the paper in their initial review, without talking to other reviewers. If all reviewers got together and shared notes before forming their own evaluation or submitting their own review, it would create significant dangers of groupthink and undermine the purpose of having multiple reviewers read every paper.

On the other hand, once you and the other reviewer have submitted your initial review, it's probably fair game to exchange notes -- but this may be dependent upon the culture in your field or conference/journal. If you are not familiar with the culture, I recommend that you contact the program committee chair or journal editor to find out what process they want you to follow.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .