15

I was chosen as a reviewer for a conference in the field of computer science in which the authors can also submit some extra materials like proofs and additional implementations separately (until a week after the deadline), but their papers should be understandable and self-contained without considering these additional materials.

The paper I’m reviewing has a good scientific contribution and I like the way they tried to solve the problem, but some parts are not so understandable unless you read the additional submitted PDF file. For example, their pseudo-code for their algorithm has some mistakes, but they provide the correct algorithm in the additional material. Also, they provided more implementation results in the extra PDF file, which I think are important to support their claim. The proof and in-detail explanation of their method is also provided there.

The organizers have emphasized that the authors shouldn’t use the additional documents as way to submit a more complete version of the original paper.

So to be fair, if they add some of the parts from their supplementary document to the main paper, then the resulting paper would be perfect and I’d definitely accept it. However, I’m not sure whether it is better to accept it with feedback on the lacking parts, telling them to move these parts from the additional PDF to the main one to make it complete, or to only consider the main paper to be self-explanatory, in that case I’d more likely reject it!

Improve this question
3
  • 1
    How close is the paper to the page limit? Maybe the authors know full well some of the material you listed would actually improve the paper if included, but did not have any space left for that.
    – O. R. Mapper
    Commented Nov 22, 2016 at 13:29
  • They have enough space to add them to the paper, but i think they didn't have enough time to finish the paper before the deadline! Also there are some written mistakes in the algorithm which they have corrected it in the additional documents, however they were advised by the organizers not to do so!
    – Bob
    Commented Nov 22, 2016 at 13:46
  • 11
    Mistakes/weaknesses of the paper that can be fixed in less than 15 minutes should never form the basis for rejection. Your example of mistakes in the main part that are fixed in the additional material are a prime example for this.
    – lighthouse keeper
    Commented Nov 22, 2016 at 14:02

2 Answers 2

Reset to default
18

Your job is to provide critique and suggestions by which the authors can improve the paper. The published paper should be self-contained; that does not mean that as a reviewer you should ignore the supplementary material.

It sounds to me like you already know how to advise the authors to improve their paper. I would definitely ask them to make improvements and submit a revision, rather than reject — especially if the paper would be "perfect" with your suggested changes.

Improve this answer
3
  • Good point, but if it was a review for journal paper, i'd be more confident that they'd make the revisions that way. But for a conference paper there is no guarantee that they would move these additional parts (around 1 page!) to the main paper for the camera ready, however they have enough space to do that.
    – Bob
    Commented Nov 22, 2016 at 13:42
  • @Bob I don't get this. If you believe that the article should be self-contained (and your question implies that is a rule of the conference too), but you are concerned about the fact you can't guarantee that the authors will edit the paper to make it self-contained, then surely you reject the paper?
    – Ian_Fin
    Commented Nov 22, 2016 at 13:49
  • 1
    @Bob as a reviewer, you have zero power to force anybody to do anything. If you think a paper is complete **** (and explain clearly and objectively why you think so) the editors can still publish it if they disagree with you. A reviewer's job is simply to say whatever needs to be said about the paper, and then get back to living their own life.
    – alephzero
    Commented Nov 23, 2016 at 1:17
7

You say that a paper should be self-contained and understandable in its own right, and as this paper isn't there could be grounds for rejecting it. However, it's not the case that you've been sent a flawed paper and have no evidence that the authors can do better. The supplementary material demonstrates that the paper could relatively readily be improved to make it self-contained and understandable.

As Dylan Richard Muir has highlighted, one of your roles as a reviewer is to provide guidance on how a paper could be improved. Take this opportunity. Accept the paper (if the only options are accept or reject, as your question suggests) but strongly suggest that the authors improve the manuscript by bringing the relevant content from the supplementary materials into the paper.

If the authors choose to ignore your advice, and it's far from obvious that they will if the material exists in some form and there's enough space in the paper for it to fill, then it's really only their paper that suffers.

Improve this answer
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .