Today I was reviewing answers through Low quality posts on Stack Overflow and found myself reviewing suspiciously similar answers. It turned out that they were the same answers that I previously reviewed but clicked Skip. Here is a screenshot of my review history annotated to show the issue:
Although I first noticed the behavior with some user scripts active, I have reproduced the issue without any user scripts.
As mentioned on MSO, this is fixed.
I shipped a change yesterday that allowed users to return to their skipped LQP tasks, and boy did I let them go back to them.
Basically, I fixed the "can a user review this post" sub-query to support returning to skipped posts, but the "give me the next post for a user" query was depending on that to exclude previously skipped tasks rather than including the default logic for that.
This problem was compounded for users with the trusted privilege. If any of the tasks that they can review have a recommend deletion vote and can be deleted from review, they are going to be served a task from a pool of no more than 10 tasks. (Something similar happens with watched and ignored tags.) This meant instead of getting the skipped post 1 out of 125 times, you maybe got it at least one out of 10 times.
All queues now consistently filter out tasks you have reviewed in that queue instead of depending on the subquery to manage that exclusion.
