15

I occasionally see recommendations here to scrap specific review audit questions. It isn't clear whether there is actually a mechanism to do that.

I assume the system uses the audit examples over and over to audit different users. It would seem logical to have a mechanism where examples that are recognized as bad could be deleted from the library.

On the other hand, the system doesn't need to be perfect to accomplish its purpose. Some percentage of bad examples could be an acceptable price for an automated process. Also, the service life of audit examples might be too short to warrant a mechanism to delete bad ones.

  • Is there an existing mechanism to cull audit examples the system gets wrong?
  • If so, is this the place to identify them?

Example

One of the answers to the recent question Do rackmount servers need to be in a rack? was used for an audit. The answer was a suggestion to use Ikea Lack tables for a rack. The original question has a number of other identical suggestions, including a comment by Journeyman Geek, and tombull89's answer, complete with picture.

The answer was picked for the audit because it received 12 upvotes (which likely occurred because it was an early answer and the first answer to mention the tables). In the audit, however, it is presented as a new answer to an older question.

As a "new" answer, it probably should be deleted because it does nothing more than regurgitate an idea already mentioned several times, and covered well in another answer; it adds no additional value. It also contains an admission by the author that someone else had already suggested the idea, so it's really just a "me too" post in this context. Responding to the audit appropriately produces the wrong result.

Example 2

A four year old link-only software recommendation that received 14 upvotes and was the accepted answer. It was presented as a new answer, which is unacceptable today. I got dinged immediately upon posting a comment suggesting that the author read the canonical Meta post on software recommendation answers. Even if there is no mechanism to identify and cull bad audit questions, at least the selection bot should be smart enough to skip answers from years ago that contain just two sentences and a link.

2
  • 2
    This is a great point. Pre-canned audit elements should only function for a certain number of elements, and as such, the criteria for these should very clearly be defined. Just because a question or answer gets plus votes doesn't mean it is "good". A number of crap answers (low quality to boot, because they're link-only) get plus votes. That DOES NOT mean the answers are something that the community should permit. In fact flags for "low quality" would be appropriate. But, the community (especially crusaders) will disagree. Automation that includes crusader decisions, sucks.
    – killermist
    Commented Apr 15, 2015 at 19:40
  • 1
    I'm all in favour of some mechanism to remove the duff audit questions - as I'm currently suffering a month's ban for hitting the 'wrong' answer for the 3rd time this year, on an answer that pretty much said 'do some updates + link'. It wasn't link-only, even though that's what I was baulked for. Not being a good answer did not make it 'not an answer'. It makes me not want to even bother restarting the review process after the timeout elapses.
    – Tetsujin
    Commented Apr 22, 2015 at 19:12

1 Answer 1

7

There are couple of feature requests on Meta Stack Exchange

There haven't been any official response to these.

There was also a call to review for audits -

This has been locked and I doubt it will be tried again

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .