78

Update (2021-09-14): This is now live

See below for details on how it works. Carryover credit for First posts review work is now being taken into account when calculating eligibility for the Reviewer and Steward badges for First questions and First answers.

If you go to the Stats page for either queue, and you have earned any carry over credit, you will be able to details on the amount of tasks carried over.

Image showing stats for First questions, giving stats on how many items carried over from the First post queue

In the recent post announcing the final review queues workflows release, we wrote that we did not plan on transferring progress from First posts reviews to the First questions or First answers queues. The initial decision was made based on an estimate of the amount and complexity of work that would be needed to make this happen. The response to this was overwhelmingly in protest of this decision.

We realize now that this is an area that was deserving of another look and examination to see if/how we can find a solution that will be satisfactory to our reviewers. Reviewers spend many hours helping our communities and users with their reviews, and it would be wrong for us to leave any of these efforts unrecognized when it comes to badges. I apologize for the way that we initially planned and communicated this.

Since then, I have come up with a new plan for how this can be accomplished that seems to meet our main objective of not losing credit for First posts work (and that also passes muster with our DB team). If knowing this is enough for you, you can stop here and check back soon. If you want to know all of the details, then read on…

As I begin to work on this, I would like to present the proposal here for feedback.

  1. No changes will be made at all to badges awarded based on activities on the First posts queue. If you earned a badge there, it is yours for keeps.
  2. Similarly, there will be no changes to review history entries in any of the queues to support this (so we will not be doing anything like transferring records from First posts to First questions and First answers).
  3. For the Reviewer badge (a silver badge that is awarded after 250 reviews, and which can only be earned one time per queue):
    1. If you have not already earned this for First posts, then the number of reviews you made on First posts for questions will be applied to earning Reviewer for First questions, and the number of reviews you made on First posts for answers will be applied to earning Reviewer for First answers.
    2. If you have already earned this for First posts, then no credit for reviews over 250 will be transferred to FQ/FA. As this is a one-time badge per queue, just like reviews over 250 no longer helped you for Reviewer on First posts, so the credit will also not be transferred to other queues.
  4. For the Steward badge (a gold badge awarded once for every 1000 reviews, which can be earned multiple times per queue):
    1. Two “carryover” numbers will be calculated based on First posts reviews, one for reviews done on questions, and one for reviews done on answers. These numbers will be equal to the total number of [FP reviews for type] that have been awarded since (five minutes before) the last Steward badge was awarded to the user for First posts (or if none were earned, then the total numbers for each badge).
    2. These carryover numbers will be added to the total number reviewed in First questions or First answers when awarding the Steward badge.
    3. So for example: A user with 2,548 reviews in First posts will have been awarded two Steward badges for First posts. Since the second badge was awarded, they reviewed another 250 for questions and 298 for answers — these are the carryover numbers. So they will then earn their first Steward badge for First questions after they review 750 items, and for First answers when they review 702 items.
    4. This does mean that if a user has reviewed more than 250 items in First posts, they will not get any credit to the Reviewer badge, but they will get credit towards the Steward badge in FQ/FA. (And as Catija pointed out to me, a user who has reviewed 800 questions in FP will get Steward after 200 reviews in FQ, and Reviewer after 250 reviews in FQ. This is interesting, and is by design.)
  5. No credit will be given on First questions or First answers for the Custodian badge. This bronze badge is earned after the first review made on a queue.
  6. The changes will be made to the badge award queries that are run by the site. The carryover numbers from First posts will also be taken into account in the display of the badge progress queries that are shown for each user on the respective review queue stats page. I will try (but cannot yet commit) to find a way to show (on-screen, or in a tooltip), the exact carryover number being applied based on First posts reviews.

This work will probably take several weeks to complete and make live (I will update here when that has happened). It is complicated logic to write and to test (and I have some upcoming holidays as well). However, please rest assured that when it is made live, it will take effect retroactively.

I think that I have found a way to ensure that credit towards the appropriate type of review badge is received for every single review done on First posts, while not double-counting any reviews either. If you have thoughts on the plan, please leave them below.

Improve this question
8
  • 5
    It would be nice if you edited this post to add in additional solutions that were brought up in your internal discussions along with explanations as to why those were found to be non-viable. As an example, my preferred solution is to combine the counts for both modern queues for badge purposes, even for newer users who've never reviewed in the old queue. This way, users who have counter-solutions won't have to spend the time writing their counter-answers if they see that it's already been discussed and decided to not be viable.
    – Sonic the Anonymous Hedgehog
    Commented Sep 2, 2021 at 17:44
  • 8
    Thank you for reconsidering this. I'm not much of a badge hunter myself, but I can see how it would be discouraging for someone to lose progress, especially on a long term goal like the Steward badge.
    – ColleenV
    Commented Sep 2, 2021 at 18:02
  • 2
    @Dudecoinheringaahing please see point 4.4 that I just added (expanding along the lines of what Glorfindel wrote)
    – Yaakov Ellis
    Commented Sep 2, 2021 at 18:44
  • 12
    @SonictheAnonymousHedgehog I am not going to answer each alternate scenario in the post. If there are some that you want to know about, feel free to suggest it in an answer. But to address your suggestion here, one of the goals is to maintain the badges as they exist right now. We don't want to unaward or change the old ones. FQ/FA are successors to FP, but not identical (more posts can go into the new ones than would go into the old). So we want to - as much as we can - recognize reviews made where they were made, and not merge things as you suggested.
    – Yaakov Ellis
    Commented Sep 2, 2021 at 18:45
  • 3
    As a former staff member said about badges: "We are not in the buisness of taking things back." So excellent call.
    – bad_coder
    Commented Sep 2, 2021 at 22:43
  • Out of curiosity, could you also share a screenshot of how the badge progress would look like for the user mentioned in point 4.4 (800 FP reviews carried over to FQ)?
    – Bart van Ingen Schenau
    Commented Sep 15, 2021 at 7:37
  • 2
    Thank you for including this in the way you did. We now have a meaningful new review queue with descriptive and accessible information, keeping progress intact and making distinction so easy. Awesome.
    – Akixkisu
    Commented Sep 17, 2021 at 12:29
  • YaakovEllis, please read this comment: meta.stackexchange.com/a/379560/282094, thanks.
    – Rob
    Commented Jun 15, 2022 at 15:56

2 Answers 2

Reset to default
17

I think that I have found a way to ensure that credit towards the appropriate type of review badge is received for every single review done on First posts, while not double-counting any reviews either. If you have thoughts on the plan, please leave them below.

(emphasis mine)

While I don't think this is a problem that needs fixing, there is a potential for double counting in the proposed solution, where reviews that were already counted for a First Posts Steward badge could get recounted for the First Questions and First Answers Steward badges.

For example, suppose a user has reviewed 999 questions and 999 answers, so a total of 1998 posts. They would then have one existing First Posts Steward badge. Completing one review each in First Questions and First Answers would then give them a Steward badge in each. So they would end up with three Steward badges for completing 2000 reviews.

In case this is unintentional, this could be fixed by first finding the remainder of all reviews from First Posts when dividing by 1000, and then splitting into First Questions and First Answers. In other words, if the total number of FP reviews is 1000x + y (where 0 <= y < 1000), split and carry over only the last y reviews into First Questions and First Answers.

Improve this answer
1
  • 8
    You are absolutely right, this is something that I hadn't considered. I will be modifying the logic so that in the case where the user has earned a Steward badge on First Posts, we only count carryover starting five minutes before the last Steward badge was awarded (we award badges every five minutes, so I want to make sure that we dont miss any potential carry-overs). I'll update above as well. Thanks!
    – Yaakov Ellis
    Commented Sep 12, 2021 at 13:10
15

I recommend you make it visually clear to see that progress the reviewers have established in the new queues. I say this because, on the smaller site that I moderate, we've now glaringly lost the FP queue, and that represented years and years and years of work by many many contributors. There is now no one who has a Steward badge on an extant queue. It looks like it has all just been wiped away. For some, that's a crushing blow. I read the original announcement. There was nothing in it that I saw about losing progress towards that historical effort. If the new queus can atleast clearly represent to those discouraged users that they may not be far off the new relative badges, and their efforts haven't been forgotten, maybe we can keep them contributing to the curation of the site. The sooner the better, because that missing queue really stands out - it took a lot of reviews with it.

Update (2021-09-15): So this is looking good.

enter image description here

One curious thing about this though is that for some users, the carry over could make the remaining tasks to the Steward badge less than that remaing for the Reviewer badge. So some users will be awarded a Steward badge in First Questions/Answers before they get a Reviewer badge, which is hilarious, but I think a natural result. For me this is perfectly acceptable.

Improve this answer
1
  • 3
    Regarding your curious thing, yes. Yaakov actually did mention this odd case in the question. ;) bullet point 4.4
    – Catija
    Commented Sep 15, 2021 at 16:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .