I have been surfing meta for a few weeks now and I am seeing a large number of reports about failed audits and review bans. Obviously if someone is getting banned they are going get on meta and vent hoping they can get back to reviewing. I, too, have been in this boat. There are obvious flaws in the honeypot system as you can see by all the posts on meta. But I wanted to break down some of the issues I have seen and offer a couple suggestions:
Issues with the current audit system
- It is an automated honeypot system. This means that if you have different view points for verifying if a post is valid than the rest of the users you are assumed to be wrong. This is great for black and white post where it is clear that a user is spamming or wrote a two page answer with references, but it is less helpful for gray-area posts. If people can fall on either side of an issue, the honeypot can trick you.
- Audit bans are exponential. Meaning if you fail audits and get banned you are banned for longer and longer periods of time. This is obviously a good system for spammers, but what if you started reviewing before you understood the system and you accrued a few failed audits. You could then pass the next 100 audits and fail one, and you are banned for a month. I know this, it has happened to me.
- Commenting and editing are considered to be inherently bad. So if you go to comment on a post you think is good, you will get a failed audit.
These are just a couple of the issues I have spotted, but there are surely others.
Here are my suggestions:
- Review audit atonement - If you get a bad audit, don't just incrementing a failed audit count on your account, make the reviews a ratio. If a person has a
pass:failof20:10then their failed audit ratio would be.5. The more passes they have, the lower their fail ratio will be. This would allow people to atone for bad audits. You could then determine at what ratio a person is considered to be doing more harm than good. You could even build a tracking system to see if people have been at a high fail ratio for a long period of time. You could then weight failed audits higher for that person. - Ban review queue - If a person is banned for failed audits, allow them to have their ban reviewed by their peers. List out the failed audits and let others decide whether or not a ban was warranted. Additionally, you could allow the user to comment on their ban - this could avoid the legions of posts on Meta about audit bans. You could also allow peers to determine if the length of the ban is justified (I envision a "shorter ban length", "longer ban length" and "this ban is justified" buttons). If someone is banned for a month because they keep commenting on good posts, they obviously shouldn't be. Which brings me to my next point...
- Adjust commenting and editing - The current functionality is flawed. There is no reason that a user should be penalized for commenting or editing. I see two possible solutions. Either allow a user to comment/edit without incrementing their review count like a standard audit OR treat comments/edits as a "skip this review" and simply redirect them to the next audit like you would a failed review. Perhaps even mention that "other users found this post to be satisfactory without the need of edits or comments".
I realize that there are already tons of posts about audits on Meta. However, that is exactly one of the issues my question is trying to resolve. I ask that you read my question in its entirety before plastering duplicates all over the place.
Please let me know what you think! Thanks folks!
