What rules/guidance do we want to give for homebrew review questions, if any?

Question

There seemed to be a strong consensus in this question that we're not happy with how homebrew review is working on the site right now, and that the way forward is to get some additional rules around it to try to cut down on the most common types of problems these questions have.

What sorts of rules/recommendations do we want for homebrew review questions, if any?

This post is for discussing potential rules/guidance; once we hammer things out here we can make a new post that clearly lays them out for ease of understanding.

Please post each suggested rule/guideline as a separate answer; well-received suggestions will be included in the final "official" post.

Answer 1

Vote to Close rapid iteration as duplicates

I suggested this in the other thread and there was some support for it. Really this is just an enforcement method for our current guidelines. So here is a formal suggestion.

Leave at least 72 hours between iterations.

Homebrew is an iterative process and the correct method for this is to post new questions for each iteration. Posting new iterations too quickly has a number of issues including:

• Gathering insufficient feedback from the previous version.
• Flooding the question feed
• Very similar versions with insufficient testing

To prevent these and other issue we will enforce this by closing any iterations, posted within 72 hours of the first, as duplicates.

Answer 2

Very few to no special rules

I am a fan of adding one particular rule (I'll get back to that) but adding on a lot of new rules unique to this one type of question is not going to make homebrew reviews succeed. In fact it's probably the best way to ensure they fail, and the more rules we add the faster the failure process will go.

When game recs were on shaky ground right near the start we tried to add new rules to help them work well. The problem was nobody really knew those rules until they broke them, so really all the new rules did was mean people were breaking even more rules (that didn't exist before) all the time and made game recs look worse. Those who did know the rules didn't want to follow them, giving us a this-is-why-we-can't-have-nice-things kind of situation. More severe application of the rules didn't make things better, either. The rest is history: game recs are banned.

We should accept that people are going to mostly act the way they're already acting. Additional rules will see newbies crash into them and more experienced users refuse what they'll see as weird arbitrary constraints. This will make people see a higher rate of failure with the tag (because things aren't meeting the new rules) and unhappier with homebrew review questions on the site. It happened with game recs, it's not going to happen differently for this case: in a year or two people will be voting to ban the topic specifically citing how often Q&A doesn't follow the simple rules laid out for homebrew review (that are special and unique only to it).

We should focus on guidance that funnels them in a good direction but accepts what we're currently getting as still OK. I'm a fan of prompting people to ask more than just "is this balanced?" but we have to accept that kind of question as well. We can focus on diversifying from there.

---

I want to op-ed a bit in terms of why I support one of the rules that's been proposed, even though I don't want us to go overboard, because I'm hoping it's going to help clarify part of my thought process here.

I'm a fan of adding the 72-hour duplicate closure because that actually isn't a hard roadblock (despite involving a question getting closed): usually this happens because someone's incorporated feedback from only one answer. We'd be telling them hey, slow down, take the time to receive more feedback. We don't waste much of their time -- they've improved their homebrew, they're just showing us the result of that. Meanwhile the original question may still get get more feedback to help them polish it further. We improve the overall health of homebrew reviews by removing that situation where you've just come back after a day ready to write a review and they've already posted a second version and you have to start from square one, and where every version has barely any commentary (let alone contrasting reviews available). 72 hours isn't even that long, we're just catching the worst cases, and when that 72 hours is over it's likely they'll have an even further developed version for us to address.

So for this thing ease of enforcement here is simple, we're not wasting anyone's time (and removing some time-wasting factors), we're mostly accepting how people already do things -- we're just telling them to take it a bit slower and wait another day or two, however long it is, before iterating again. I don't see this causing failure or reducing overall health of this question category (where health is also "people are less happy with having it on the site") and we should keep an eye on things to see what effect it does have on homebrew review health.

Answer 3

Enforce linking to previous and followup iterations

To understand the development it is mandatory to link to a previous iteration - and to make sure that one is on the most recent version, the next iterations also need to be linked. Offending questions shall be edited to comply.

Answer 4

Encourage context and purpose

An explicit explanation of what the homebrew is trying to achieve, and why the already existing resources are inadequate in achieving said goals.

A lot of the questions I see are simply here's X, is it balanced? As a community, all we can do is say yes or no to best of our ability. Putting context on homebrew enables the community to leverage more of their expertise. Many "issues" that users are trying to solve via homebrew can be resolved via alternative suggestions that the community can provide, but only if the context is given. Plus, requesting that users justify why official resources are inadequate may spur the critical thinking in some users to find their answer before they post.

• "Here's my Frogperson homebrew, is it balanced versus officially released races?" "Yes/no/maybe."
• "I want a character that comes from a swamp and has a swim speed. Here's my Frogperson homebrew, is it balanced versus officially released races?" "Yes/no/maybe. Have you also considered the Lizardfolk race?"
• "I want a character that comes from a swamp and has a swim speed. The closest thing available is the Lizardfolk, but I want my character to be an amphibian. Here's my Frogperson homebrew, is it balanced versus officially released races?" "Yes/no/maybe. Have you considered reskinning the Lizardfolk as a Frogperson? Change scales to protective slime covering, and bite to a tongue attack"

Answer 5

Please use consistent rules

We should encourage askers to use rules language in their homebrew material that is consistent with the language used by the core system. Questions with homebrew which is "improperly" written tend to (rightly IMO.) get closed as unclear. To help against unclear questions we should have a point to the tune of (thanks @Powerdork):

The intent of your material is often better conveyed when you follow the style guidelines and use the correct system terms of the game in question