Was I wrong to rebutt unjustified rewiewer comments in the review?

Question

Recently submitted a manuscript which got a decision of Reject & Resubmit, i.e. the editor invited a resubmission provided we can address the reviewer's comments. In my previous experience, all my manuscripts were always accepted after a resubmission was invited, so I thought the editors basically use this type of decision to give themselves a little more leverage.

One reviewer's comments were very negative, but unjustifiably so. For example, he/she stated that the "experimental treatments" weren't independent, while our study was purely observational and didn't include any experimental manipulation. The comments looked like they were made by someone who didn't bother to put any thought into the review and had a poor understanding of statistics (our study is loosely in the field of ecology/conservation).

We rebutted these comments firmly, but politely. The other comments, including those made by the second reviewers, were minor and were addressed by revising or clarifying the text in certain places.

We also explicitly drew attention to the issue with one reviewer's comments in the cover letter accompanying the resubmission. We stated that we carefully considered these comments, but they were unjustified, and again explained why.

This resubmission was rejected outright very quickly, within a week. The editor acknowledged the revision did improve the manuscript, but still rejected citing the standard "our acceptance threshold is very high" etc.

It is the first time my invited resubmission was rejected, and I can't help wondering whether the firm response to the reviewer's comments was the cause.

Answer 1

It's fine to push back against reviewer comments, but it can be a bit of a delicate dance to do so.

First, I would suggest you take reviewer comments that are unjustified to mean that you didn't do a good enough job explaining what you did. Although it's possible the reviewer was simply sloppy in reading your paper, a lot of people are going to sloppily read your paper: give them as few ways to get it wrong as possible by being extremely clear. Communication is between the reader and the writer: it isn't only the reader's fault if the reader gets it wrong. You can apologize for your role in the miscommunication and do better in the revision.

Second, I would suggest whenever you plan to push back against large parts of an individual review, rather than just a modest misconception or two, have a less-involved person that you trust to give honest feedback review your remarks. Sometimes a coauthor with a smaller contribution is sufficient (i.e., not the primary or senior author, who may feel most personally involved in the work), other times it may be helpful to reach out to someone else outside the authorship group. Ask them explicitly whether they think your response is sufficiently polite. Ask them to play the role of that reviewer or the editor. Additionally, give yourself a bit of time to sleep on both the reviewer comments and your response, at least a day or two and maybe a week or two. Personally, I sometimes write my angry/snarky response in a private document for myself to "get it all out", spend a couple hours complaining to people close to me about how worthless peer review is, and then come back to the original comments later and start working on real responses to what turns out to be helpful criticism. You can skip the first two parts of that if you want, but the last part is crucial.

Third, make sure you aren't really missing something in the comments you think are unjustified. Your question here is vague, so I am only guessing, but you say:

he/she stated that the "experimental treatments" weren't independent, while our study was purely observational and didn't include any experimental manipulation

If you responded to the first part of that comment and say "this comment doesn't apply because our study was purely observational" I would, as the editor, probably reject your paper. The real, important comment there is the lack of independence, which is often a critical weakness in an observational study. It may be that you interpreted your data in a way that implies an experiment has taken place, when it hasn't, and that makes the reviewer's comment very very relevant, even if they misused the word "experiment." Be careful that you haven't used causal language, that you have done field-appropriate comparisons between any groupings you made in your observations, etc.

Finally, it's quite possible for you to do everything "right" and still be rejected. For a more selective journal, especially, an editor may reject if there is insufficient support from the reviewers, even if you are able to refute some of the criticisms. They are looking for papers that the reviewers agree are excellent, and it's possible that observational work is simply not a good fit for this journal.

Answer 2

Whenever you get a negative reviewer comment that misunderstands an issue in your paper, or is otherwise unjustified, you should take that as a signal that you have not explained the relevant issue sufficiently clearly. The response should therefore usually entail a substantive change to the paper (to add a clearer explanation) accompanied by a response to the referee comment that identifies the additional explanation, perhaps with some additional discussion justifying your change.

Aside from improving your paper, one advantage of this approach is that it gives the referee a response that shows that you do not think he/she is stupid or misguided. By putting the blame on yourself, you avoid a negative reaction from the referee (or editor), and by making a substantive change to the paper as a result, you show that you have improved the explanation so that others will not misinterpret the paper in the same manner as occurred in the referee comment. Also, this kind of change usually does entail an actual improvement to the paper, since it prevents other readers from making the same mistake as the referee. Here is an example of what this would look like.

Referee: The experimental treatments in the study are not independent.

Response - Revised: On review of the paper, we can see that we failed to adequately explain the experimental setup. Our study is actually an observational study, which did not entail any controlled treatment variables (i.e., the explanatory variables were observed without intervention). There is indeed multicollinearity between some of the explanatory variables, and we had not addressed this sufficiently in our previous draft. We have now made a number of changes to the paper to clarify the experimental setup, and to discuss the effect of multicollinearity between the explanatory variables.

We have now added an additional sentence in the methods section (p. 3, line 8) to note explicitly that this is an observational study, and to stipulate that we have not "controlled" the explanatory variables of interest. We have repeated this in the results section, in our remarks on the difficulties of making causal inferences from the data (p. 12, lines 14-16). We have also added an additional paragraph to the results section (p. 11, para 2) discussing the multicollinearity in the explanatory variables, and its effect on our inferences. For the reasons set out there, we are satisfied that multicollinearity does not invalidate our statistical inferences or conclusions in the paper.

Answer 3

Unless your literature uses a different terminology, a reject and resubmit is not a review and resubmit. The editor will have told you what you have to do, before submitting again (from fresh). Typically this means a larger revision of what you have done. Therefore, it's a rejection: to let you know that the hurdle of acceptance is higher, and there is much more work than usual required. With that information, you may simply want to submit elsewhere instead.

However, a new submission also means that the editor is free to change the reviewer if they want to. A Reject+Resubmit is often a nice way to get rid of a reviewer that the editor regrets having assigned to a paper.

For this very reason, if indeed you have a reject+resubmit and not a review+resubmit, do not dwell on the reviewers' comments, but focus on what the editor wanted. Did you really manage to address everything you were asked?

Answer 4

Yes, there is nothing wrong with writing a rebuttal whenever necessary.

However, carefully ensure the response to the reviewer's comment is not directed personally at the reviewer (this should be observable from your tone of response and choice of words), rather it is directed at the comment.

Also, try to remind yourself (or pretend if that be the case) that the comments are not directed at you but the content of the paper. This implies that while responding, ensure that you emphasize that the content of the paper (or the revised paper) has answers to the reviewer's comment.

Answer 5

If they were indeed unjustified, it is certainly good to correct them. But you must think, "Am I right?" or is the other person right? (in this case the reviewer). Being right is not a shield, however, for the necessity to be polite at all times, and treat the comments with respect. Often adding clarification (such as which of some instance of something you are using) can make it more understandable. So, respond would be a better usage than 'rebut'. The latter implies a possibly negative response. (NOTE: in my post, I originally had 'rebuke' here incorrectly, and that would be very negative). However, according to dictionary.com 'rebut' does have negative as a related word:

https://www.dictionary.com/browse/rebut?s=t