If a question asks about how to improve a puzzle, is an answer which only solves the puzzle acceptable?

Question

The answer in question

The question asks, in short, "How can I improve the wording of this puzzle?". Most of the answers address this question by providing wording suggestions. This is on-topic for the site; puzzle-creation and preventing confusion in a puzzle's presentation fall under our scope.

However, there is one answer which solves the puzzle without addressing wording at all. I do not think this is an answer to the question as posed. From the three Looks OK votes in the review and comments on the answer, clearly others disagree. Since I can't address the thoughts of the reviewers, as sadly I lack the required telepathy for such a feat, I'll deal with the arguments brought up in the answer/comments.

I don't intend to strawman the counter-arguments I present here, but I am going to put all of them and (naturally) some are of higher quality, while some are lower.

• From the answer:

  I couldn't resist

  Wanting to do something which breaks the rules is not, in and of itself, an excuse for breaking the rules. If someone wants to post "Where did you learn how to make this puzzle?" as an answer to a question, that "answer" will rightly be deleted. Moving on to stronger arguments.

• From a comment:

  I couldn't post a picture that way

  As I said in another comment, you can post pictures in comments by linking them. Indeed, people have linked pictures in comments elsewhere on the site.

• From the same comment as before:

  solving puzzles more closely aligns with what this SE is usually used for

  What standard questions ask has no bearing on what this specific question is asking. This question does not ask at all to solve the puzzle. To make an analogy to Stack Overflow, a normal kind of question on that site is a "debugging" question. People will post some code, their goal, and ask how to fix their code so that the goal is achieved. However, if someone posts a question asking about the history of a buggy example in a famous coding textbook (no idea if this question would be on topic, I don't know Stack Overflow's scope well), an answer which fixed the bug would fundamentally fail to answer the question.

• From a comment by the questioner

  I have no objection to you posting solution(s), in fact, I thought that someone might do it.

  If the Stack Overflow questioner from my previous analogy didn't mind the bug-fixing answer, that wouldn't mean the answer was acceptable by the moderation rules of the community it was in. Plus, that's addressing an entirely separate question. (I concede that it could be a "bonus" question, but answering just a bonus is not quite kosher, and is not mentioned in the Puzzling question anyways.) Questions should be focused to have one question so that answers can address just that question; the desire or expectation of the questioner to get multiple answers addressing separate questions is moot.

• From a final comment

  the puzzle is so charming it’s worth having here

  A puzzle being fun to see or solve does not mean that posting a solution to it is automatically acceptable. Again, that has to be what the question was asking. Which, again, was not what this question was asking. A separate question could be created, linking back to the original, which asked specifically for solutions to the puzzle. Then it would be perfectly fine and charming and answering it with a solution would be perfectly fine and charming.

So, is this an acceptable answer? If the answer is “yes” then the correct action is obviously “leave it alone”. If the answer is “no” then should some action be taken (deletion, downvoting) or is this a special enough case that, despite breaking site rules on answers-must-answer-the-question, it should be left alone anyways?

Answer 1

Short answer:
Posts (like the example given) that do not answer the actual question should be deleted

Longer answer:
In this sort of situation (a question about the details of another puzzle), the initial meta-question about wording/construction/layout/etc should be restricted to that purpose. If the puzzle itself is in a state where it can be answered as-is, and is worthwhile answering, it should be posted as a separate question. Where answers to the latter appear on the former, I would vote for deletion.
But clearly (as this question shows), opinions differ, and others think it is fine to let those answers stand. It really depends how strict you think PSE should be on this and related one-question-per-post conventions.

And an observation:
Part of the problem may be down to a combination of a poor question title, and a deficiency in the current review UI.
The review page only gives you the question title, not the full question text. So a reviewer-in-a-hurry could see an answer that, while it could be better, does appear to be an valid (not low-quality) answer to the question asked. It is only when you click through to read the full post that you find the answer doesn't address the actual question. (Of course, I can't tell if that has actually happened in this case. It may simply be that the reviewers have a different opinion on the acceptability of these sorts of answers.)

Answer 2

The general principle is that an answer that doesn't in any way answer the question that's been asked is Not An Answer, and therefore shouldn't be there.

The other general principle is that we should take actions that make PSE a better place, by (for instance)

• providing interesting and on-topic content
• in the case of questions like this one that are actually seeking help (typical for xSE sites but actually quite unusual on PSE), making it easy for readers to find good answers
• rewarding people for posting interesting puzzles, clever solutions, intriguing questions, and informative answers
• being a friendly place for newcomers.

So, in the present case, the "answer" is clearly, unambiguously, Not An Answer to the question that was actually asked. On the other hand, it doesn't particularly get in the way (the actually-accepted answer answers the question actually asked pretty well, and so do the other answers below it; the non-answer here is down at the bottom of the page and no one will even see it unless they go scrolling around), and anyone reading the question will likely be curious about the answer.

I think the Right Thing for the person who posted that answer to do was: make another PSE question, consisting of the puzzle itself, with a link to the original question for context; post a self-answer answering that question; add a comment on the original question saying "in case anyone's wondering about the answer to the puzzle, here's where you can find it". (Or: same, but without the self-answer, inviting other people to solve the puzzle, and therefore with a slightly different comment on the original question.)

That's all well and good, but it's not what actually happened; how to proceed? I think flagging as Not An Answer is entirely reasonable. I think disagreeing with that is also reasonable; it really is Not An Answer, but it doesn't do much harm and maybe does a bit of good because some readers will be curious about the solution to the puzzle. And there are "second-order" considerations pointing both ways: on the one hand, we don't want to encourage people to ignore the rules; on the other hand, it seems the person who posted the answer was upset at being told it wasn't an answer and perceived it as an attack (they posted an answer here in meta, which they have since deleted), and it's a shame when people get upset. To be clear, I don't at all think it was an attack, but I can understand how someone might feel that way.

This is the kind of situation where my preferred moderation strategy is to get out of the way and let The Will Of The People prevail: any outcome seems defensible. I don't think PSE needs a definite official policy on this specific point; it's a weird edge-y case that arises very infrequently. The meta-strategy I recommend for weird edge-y cases is: take whatever actions you think appropriate (flag, don't flag, vote to approve or decline flags, upvote, downvote, etc.), and if things end up not going the way you hoped just sigh and move on. Sometimes the Will Of The People isn't what you'd like, and there's limited value in fighting it.

Of course, then there are the meta-meta-debates where one person says "this is a weird edge-y case" and another person says "no it isn't, this is straightforwardly against the rules" or "no it isn't, it's obvious that the answer does more harm than good" or whatever. I don't have any magical techniques for recognizing cases where the good folks of PSE are going to disagree with one another, other than waiting to see whether in fact they do disagree with one another :-).

Incidentally, I agree with fljx that suboptimal UI design in the review process is probably part of what happened here, and furthermore I suspect that it leads to quite a lot of review decisions that the reviewers would change their minds about if presented with all the relevant information in a manageable way. But that's not a thing we can do much about.