88

I've come across many posts that can only be described as okay (see examples below). They're not good enough to deserve an Upvote and not bad enough to deserve a Downvote, there's no point in editing them except maybe to correct one letter, there's no need for a comment and they don't have spam or bad language to flag.

Because the numbers are decreasing the more you "review" it gives you a some-how rewarding feeling, that you're helping decreasing the problems. But at the same time they're putting a pressure on the user to do something in order to unlock that awesome I'm Done button. So the reviewer might do unnecessary edits or leave unneeded comments.

I suggest having a Looks Good button, and when enough people (3, 4?) think it looks good it can be treated normally.

I don't think Not Sure is the only option we need here, because it isn't the same as Looks Good ("Low Quality Posts" have both). I think that we need something to mark the post as okay. Because I'm not Not Sure, I'm pretty sure it Looks Okay.

I also strongly agree with sixlettervariables, this has discouraged me many times from using those two sections in the Review system.

Examples: This, this, this, this, this, this, and this.

Improve this question
11
  • 1
    Do you have any actual evidence that supports your claim that people are performing unnecessary actions just to unlock I'm Done?
    – Servy
    Commented Sep 21, 2012 at 15:38
  • @Servy, I believe I haven't made such claims. "the reviewer might do unnecessary edits or leave unneeded comments."
    – Adi
    Commented Sep 21, 2012 at 15:39
  • 18
    It's encouraged me to not use the system. I am sure that it needs nothing.
    – user7116
    Commented Sep 21, 2012 at 15:44
  • 3
    @Adnan Exactly. You're just speculating and I'm drawing attention to that fact. The entire review system is in beta; it's there for the SO team to see how people are using it and how effective it is. Once the beta is over (or more than a few days old) they will take the time to see if this concern (which has been addressed several times already) is actually a problem or not.
    – Servy
    Commented Sep 21, 2012 at 15:44
  • @Servy, while I agree with your first point I think I'm making very reasonable speculations. I'd love to hear what you think about the idea itself rather than whether the problem exist or not, that's pretty much why I added the discussion tag.
    – Adi
    Commented Sep 21, 2012 at 15:48
  • 2
    @Adnan I don't see the point in having such a discussion. The beta exists specifically to allow the SO team to observe the results in practice so that nobody needs to speculate at all. Once the results are in and it comes time to interpret those results, rather than speculate without information, a more productive discussion can take place.
    – Servy
    Commented Sep 21, 2012 at 15:50
  • 1
    As to the links that you posted, none of the answers were in my area of expertise, so I could only not sure them. All of the questions needed editing. If you're looking for answers to "what should I do here" feel free to look at the revision history for those questions.
    – Servy
    Commented Sep 21, 2012 at 15:54
  • 1
    See also meta.stackexchange.com/a/146991/56338
    – sth
    Commented Sep 21, 2012 at 16:53
  • I don't see any moderator answers that were deleted, and the one diamond answer I see that isn't deleted is from a community team member.
    – BoltClock's a Unicorn
    Commented Oct 1, 2012 at 12:19
  • @BoltClock'saUnicorn, ops.. sorry, I saw it was greyed out so I assumed it's deleted. I got used to seeing deleted answers after they're deleted on Stackoverflow.
    – Adi
    Commented Oct 1, 2012 at 12:22
  • @Adnan - BTW: Nice about me section. Curious to know how did you set condition for smiles?
    – Himanshu
    Commented Apr 22, 2013 at 10:58

5 Answers 5

Reset to default
31

We've now implemented this. Obviously, my initial optimism in assuming folks who didn't feel strongly about voting would do something else useful was completely misguided. I was wrong, and I apologize sincerely for not realizing this sooner.

I'm quickly souring (again) on the idea of allowing voting in these queues at all; not because of the people who vote without reviewing at all, but because it's made it impractical to implement something that's been a staple in other queues: multiple reviewers per review item. Indeed, we badly need a locking mechanism to prevent multiple reviewers (3-4 votes on new posts can really break things), however implementing this will be tricky.

So for now, we're adding a "No Action Needed" button. We'll be watching to see if it is effective in siphoning off folks with no real opinion on the posts they're reviewing.

No Action Needed

Improve this answer
5
  • 7
    I think No Action Needed button is fine. But add those posts again in queue (one time) if it is reviewed with No Act.. so we can ensure that the post is reviewed fine. (And don't show those posts to old reviewer.) :-)
    – Himanshu
    Commented Dec 15, 2012 at 4:26
  • 5
    @hims056: yeah, I think that's a really good idea. But let's wait to see if this calms some of the more ridiculous voting first, and give the devs a chance to try cracking the locking problem.
    – Shog9
    Commented Dec 15, 2012 at 4:27
  • 1
    It might be a good idea to implement an underlying "review weight" for each reviewer (for these two queues). Their weight increases only when a post that they marked "looks good" or upvoted also gets upvoted outside the queue i.e., from users using the site normally. This way, you can be somewhat sure that the upvote from normal use was genuine and not from someone gaming the queue, hence serving as a form of validation. The weight goes down if they upvote from the queue, but gets downvoted heavily or closed outside the queue.
    – Lorem Ipsum
    Commented Dec 15, 2012 at 16:16
  • Taking this a step further, you can perhaps "safely" decide that a post really looks good (or not) if 3 reviewers with "sufficient review weight" agree on the outcome or keep a post in the queue longer if 3 new reviewers approve, but 1 high weight reviewer disapproves, etc.
    – Lorem Ipsum
    Commented Dec 15, 2012 at 16:18
  • per my brief testing, new feature works fine. By the way did you consider increasing delay for non-skip review buttons along with implementing it? Thing is, new button is faster than upvote+done, thus giving robo-reviewers even better chances than before to out-click those who care
    – gnat
    Commented Dec 18, 2012 at 9:06
44
+50

I agree with the OP on the matter. In my humble opinion there is a difference between a perfectly fine question (that does not need a comment or edit) and a post that I want to upvote (which should be a great post).

Looking through the first post and late answers queue I find a lot of questions that are good posts in terms of being well written, well formatted, and seemingly in valid in its content, but I don't feel like upvoting them because of one of many reasons. In these cases pressing Not sure feels like I am doing nothing, but in fact I am approving the post in its current form and I would like to feel that I have done something.

As an alternative name for the button maybe Valid post could be considered.

Improve this answer
3
  • 2
    You seem to be very stingy in terms of giving upvotes. If I can't find anything wrong with a post then why shouldn't you upvote it?
    – Servy
    Commented Sep 24, 2012 at 20:15
  • 22
    Hehe, nah, I don't consider myself to be stingy with the upvotes, but I think that upvotes should be given to good posts, not to valid posts. A good post, imo, is a post that is useful to the community, a valid post is just a post that fulfill the requirements in the FAQ. Besides, the number of votes per day is limited, whilst the number of possible reviews is not(?) and I can easily find more than 30 review posts that I consider valid but not great.
    – Avada Kedavra
    Commented Sep 24, 2012 at 20:21
  • 1
    Agree with @AvadaKedavra here.
    – Ren
    Commented Nov 30, 2012 at 13:40
14

Ok, so having just been in the review panel for a few hours, I've run across several instances of this which have really ruffled my feathers. The main problem I'm seeing here is that users are reviewing posts which they may or may not even have any experience with, seeing that there's nothing to do, and blindly upvoting things. And I mean blindly as in, there have been a couple answers which got 2 upvotes and the answers themselves were flat-out wrong. One I downvoted, but it's still sitting at a +1 because of the reviewers. This is outrageous, there really needs to be a middle option so that users can stop damaging our site in pursuit of a silly badge!

This brings me to my second point: the whole point of reviewing is community moderation. Saying that, the policy has always been that moderators are not there to judge the accuracy of the content, but only to clean up the crap off the site. Is that not exactly what we're supposed to be doing by using the review panel? And yet, you are practically forcing us to try and do so on posts that oftentimes aren't anywhere within our realm of expertise when the queues are near empty.

Jaydles says in another answer that if we don't know enough about the subject, we should be clicking the Not Sure option, and I couldn't disagree more. The review panel has always been a tool for keeping the site clean, and that statement right there is completely changing its meaning, and I don't like it. You're effectively taking away our tool to keep the site clean and transforming it into an "aww, these users are new, let's be nice to them and upvote everything of theirs." The First Posts queue should be "these are new users, we need to review their posts and make sure they understand the proper use of our site." The review system needs to go back to reviewing, the process in which we get rid of crap and let potentially good content stay. Let the people interested in the question determine if the content is good or bad. People who naturally come across the question are much more likely to take the post seriously and judge the content accordingly than some random user in the review queue.

Improve this answer
6
  • as of now Not Sure option label is replaced with Skip
    – gnat
    Commented Nov 13, 2012 at 7:52
  • +1 If you would not have posted this, I would have done it right now. I review the "Not an answer" flag and found 3 posts that were so clearly not answers, but where upvoted. e.g. this or this
    – stema
    Commented Nov 13, 2012 at 9:06
  • 3
    @gnat The problem is the "Skip" button does not bring the user closer to a badge, so it seems some users are just blindly upvoting and clicking next, no matter what the post is, even when they are not answers written in all CAPS.
    – stema
    Commented Nov 13, 2012 at 9:14
  • @stema fake reviews are known problem with current system, even in queues that don't have "Skip" button
    – gnat
    Commented Nov 13, 2012 at 9:19
  • Huh? I don't follow your disagreement. What should you be doing in the review queue if you don't know enough to upvote, comment, flag, etc.
    – Jaydles
    Commented Dec 17, 2012 at 15:32
  • It would be good to get clearer guidance on this. Specifically, if you come across a late answer that's well formatted and looks to be well researched and may provide valuable information but isn't within your expertise to evaluate. Upvoting isn't an option as you don't know whether it is a real solution, no action needed won't help to uncover 'hidden gems' and skip will potentially just pass on the decision to a reviewer with similar uncertainty. The decision depends on the purpose of the review - no action needed if we're reviewing quality vs skip if we're reviewing validity of answer.
    – Tone
    Commented May 24, 2016 at 23:46
13

I think there's a really valid point here:

Even though the goal is to give more feedback, sometimes you can't, for one of two reasons:

  1. You don't know enough (like you're not sure if it's a good answer or not - this is covered (well) by "Not Sure"
  2. A post isn't bad enough for a down vote, but isn't good enough for an up vote. In this case, the ideal action would usually be a comment, to help the poster improve it enough to be upvote-worthy (or, at aminimum, to understand why it isn't).

The problem in case 2 is that if there is already a comment, there's not much you can do that doesn't feel capricious (voting) or redundant (adding another, unneeded comment).

And yes, clicking "not sure" feels wrong when you are sure that the post is "covered"; plus it fails to clear it from the queue.

So, what if one of the acceptable actions was upvoting a comment? That would seem to cover almost all the cases where there's really no opportunity to add feedback.

Improve this answer
8
  • 2
    Exactly. The case of measures already haven been taken and time not having shown improvement are exactly when "Not sure" is the wrong answer. You are sure, there just isn't anything more than can be done.
    – Caleb
    Commented Oct 1, 2012 at 17:31
  • 3
    FWIW: this was implemented
    – Shog9
    Commented Oct 5, 2012 at 15:00
  • I agree with you. First, I am using the "Skip" button a lot of times when I feel I don't have the proper knowledge to evaluate the post. Second, I also upvote comment (if good enough) when I don't think the question deserve an upvote or downvote. But the problem still arise when there is no comment.
    – ForceMagic
    Commented Nov 10, 2012 at 7:41
  • @forceMagic, I'm not sure I follow. If you're saying that sometimes there's no comment, but you lack the knowledge on the specific topic to leave one (or to vote, edit, etc.) then that's exactly when you should be hitting skip, which is essentially saying, "I'm not the best person to weigh in here; keep this in the queue till someone more familiar with this topic opines."
    – Jaydles
    Commented Nov 10, 2012 at 12:40
  • You understood correctly, that's what I am doing when I do not have enough knowledge. Sometimes though, I know that the post is Ok, I don't see how I could improve it with an edit or a comment. In those cases, a button "Looks fine" should be available IMO. That`s what I was trying to say, sorry for the misunderstanding.
    – ForceMagic
    Commented Nov 10, 2012 at 12:43
  • 1
    @forcemagic, no apology needed! The theory is that if you know it's got nothing wrong with it, and it can't be improved, an up vote would probably make sense.
    – Jaydles
    Commented Nov 10, 2012 at 22:39
  • 1
    I disagree that the proper action is adding a comment. You asked me to review the post. I reviewed it and determined that no action on my part was necessary. Just because I don't feel it's worthy of an upvote from me doesn't mean I should comment so they can improve it, I just don't want to upvote it. I only get so many votes per day, why should I waste one on some random post I saw in review that I don't really care about. Further, commenting on it involves me in the post. I don't want to be involved with the post. I was only temporarily pulled in to review it, which I did.
    – animuson StaffMod
    Commented Nov 13, 2012 at 1:28
  • @animuson, okay then. You don't have to. But one of the goals of the "first posts" review queue is to provide new users feedback. Positive when appropriate, redirecting when helpful, and negative (but hopefully constructive) when necessary. So, where it's possible to do one of those things, but not necessary, or where you feel you don't have the time or inclination, which is totally cool, hit "skip". That way, it stays in the queue for someone who does.
    – Jaydles
    Commented Nov 15, 2012 at 15:33
4

I guess I don't understand something fundamental here in this argument, and those about "Doing XYZ should enable the I'm Done button..."

That is: what is wrong with pressing "Not Sure"? If you can't think of anything worth your time to do, why not just click Not Sure? Do people really have some reason to "do something" to a post that doesn't seem to really need any action, when clicking "Not Sure" brings you to another post, where you could potentially find some action you could easily want to do?

I understand there's a drive to do something in general - but is it not faster and easier to just click "Not Sure" and move on to a post that potentially has an easier/more obvious action to do?

I know this is posed as a question, but I mean it as a statement: If people don't know/can't think of something obvious to do with a post, it's easy to click Not Sure and move on.

Improve this answer
7
  • 3
    At least on other discussions on this topic, people seemed to want the system to record distinct actions for "not sure" and "nothing to do", so posts with enough "nothing to do" votes could be removed from the queue at some point.
    – bfavaretto
    Commented Sep 21, 2012 at 19:40
  • 14
    I'm not the OP, but I understand his point. Not sure means "not sure". It does not mean "Ive reviewed the post and nothing needs to be done.". What I don't understand is the resistance.to such a thing. It only makes sense
    – Barak
    Commented Sep 21, 2012 at 19:49
  • 1
    @Barak "Looks Good" could have entirely different meanings, too; there've been posts about confusion regarding that (since it implies approval of a post, when you really just mean to say "Meh; it's not great, but I don't think anything needs to be done about it". It's just all too much parsing of wording of buttons, IMO. Besides, to literally express "Looks Good", would one not consider up voting the content?
    – Andrew Barber
    Commented Sep 21, 2012 at 19:51
  • 2
    I think the opposite problem is a much bigger issue. That is, people clicking "Looks good" when a post has clear, obvious things to be fixed.
    – ale
    Commented Sep 21, 2012 at 19:53
  • 5
    I didn't ask for looks good... :) "Looks OK" would do fine. :P. My main point being I see a huge distinction between "not sure" and "acceptable but not voteworthy"... there ARE questions and answers that fall into that category. Besides, if every answer that was acceptable got an upvote from every reader just for being acceptable, there goes the intent of letting the best rise to the top.
    – Barak
    Commented Sep 21, 2012 at 19:58
  • 1
    @AlEverett: How is that any different from people editing something (thus removing it from the queue) but still leaving "clear, obvious things to be fixed?"
    – Nicol Bolas
    Commented Sep 22, 2012 at 21:44
  • 5
    @Andrew: "That is: what is wrong with pressing "Not Sure"?" Because that's not helping. The post will still be in the review queue for others to look at. The point is to say, "a human has reviewed this and it is passable as-is." The "Not Sure" button is to say, "I don't know what to do with this, so someone else should take a look."
    – Nicol Bolas
    Commented Sep 22, 2012 at 21:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .