Is there any good way to determine when it appropriate to reject a manuscript versus merely recommending major revisions? I.e., at what point do you declare a manuscript a lost cause? I understand there are some situations where the decision is very clear. For example, if a paper has methodology problems but is otherwise solid "major revisions" is an easy decision, as is the case if the manuscript presents clear examples of plagiarism or results that do not advance the field.
However, there are cases where the proper decision to make for a manuscript is not clear. As an example, I was recently asked to review a manuscript on this topic. Originally, I thought the paper was a straightforward "major revisions", but the more I look at the more I am not sure. Specifically...
- The paper is poorly written and has a large number of grammatical and syntax errors. The journal is an English journal but I know authors are ESL speakers (they are from Argentina and France). In general, I have a strict policy as a reviewer not to reject papers for spelling/grammar/wording issues if English is likely not their first language since the issues could be out of unfamiliarity, but merely make suggestions as to how the syntax could be improved. So this is a minor issue from my standpoint but combined with the other issues it starts to add up.
- The data are incorrectly formatted and input into their analyzing program of choice, which is likely causing bias in the data (and at the very least raise questions as to its results). Specifically, the data's formatting contradicts what was described in the methods. The methods state the values were based on the published values for the data points but the actual data corresponds to broader (and much less precise) categorical bins used in my field. They authors claim the data was compiled from the previously published literature but it is clear they merely got it from an online database, as it contains errors that are only in that particular database's version of the data.
- Their definition of terms contradicts that of all previous studies, and they do not justify this change. They find a non-significant result but this is almost certainly because their data are nonsense because of noise introduced by their different definition of the term
- Inconsistencies in reporting of the data. I.e., in some places the authors say two variables are negatively correlated and in others they are positively correlated.
- The authors make claims in the discussion that they say are "evident" but provide no support for this either in their own results or via citation of the previously published literature
- The authors do not sufficiently cite the previously published literature. Specifically, the authors appear to be avoiding citing papers from my research group. In their manuscript when reviewing the previous literature and in the discussion they bring up some of the exact same results obtained by previous papers by myself and my lab group, but do not cite them once. Checking through the references they cite pretty much every major research group working on this topic except ours. Discussions with other colleagues in the field suggest this may not be an honest oversight, but may be personally influenced. My concern is that leaving a highly critical review would result in academic retaliation or being blackballed by the authors, and our field is small enough it would be hard to leave an anonymous review.
- The lead author is a graduate student (approximately my age), but the others are my senior by at least 10 years (I am a PhD student) I suspect the paper was primarily written by the graduate student based on literature errors that suggest a lack of familiarity with the subject. People have told me that if reviewing a manuscript from someone that is my senior I should be very lenient because they have power over me. There's is also the whole aspect of not being too hard on a very young researcher.
- The paper is also scooping research that at least two other groups, one by myself and other colleagues and the other a third party, presented at the main conference for our field last year. The abstracts of these presentations were publicly accessible. I have no problems with this fact, I want to see this group's paper accepted. However, examining the data in detail suggests that they may have just thrown the dataset together using previously published databases without critical review. EDIT: Based on the comment by @Buffy I thought I should add a bit more clarification. Our research group was actually planning to scrap our project because we weren't making much progress on the manuscript and it wasn't high priority, and this will likely be the nail in the coffin. The issue is more the fact that our lab group presented on this same topic is publicly accessible knowledge.
The paper might be salvageable and contains results that would be worthwhile if they stand up to scrutiny after the data has been fixed, but at the same time it shows extreme sloppiness that would not be acceptable for any scientific publication. I suppose, more broadly the question could be formulated as if there is any point where comparatively minor issues are so numerous that rejection becomes a better option than major revisions, or is it always better to choose major revisions over rejection? As you can see, this is a pretty complicated situation that makes it difficult to determine whether major revisions or outright rejection is the better course. Personally, I'd like to recommend major revisions for this manuscript, but I'm not sure if it's the right thing to do. I usually don't like to reject a paper unless there is something fundamentally wrong with it that cannot be fixed with revisions.
I've talked about this with some colleagues, and I've had some who say I should not only not reject the paper, but I should also not bring up the issues as to improper data formulation, definition, and citation issues and give a recommendation of minor revisions at worst so as to avoid getting blackballed by the other researchers. But I'm not sure if this is the right choice.
