19
$\begingroup$

Statistics and conclusion below.

We have been struggling for a long time with the homework close reason. There are shortcomings to all approaches we have tried so far to change it. I believe one of the problems with this is that actually enacting a different strategy comes with the worry that it'll drastically change the dynamics of the site.

However, continuing like before seems hardly an option anymore. On several occasions the use of this reason has been questioned. We even ran an experiment being more lenient about closure in general. I personally think homework has become a boiler template for closing questions that we do not want to deal with on the site. In a way it has gotten too easy to close questions, but the problem with that strategy is that it does not utilise closing as a tool. This is not the most efficient way of creating a repository with great content.
Closing a question (putting it on hold) should have the focus of improving it, reopening it, and answering it. It should (in the majority of the cases) not be used as a standard way to get rid of content; that's what voting is for.

To encourage more communication towards the question authors and the community, we decided to run another experiment. In June we will be disabling the homework close reason. The current homework policy stays in place, however, to use it, you need fill out the custom close reason.
We hope that this will encourage users to actually name the problems of the post, ask follow up questions for clarification, offer guidance on improving them. It will hopefully give us new data on what kind of questions get closed and why, as well as the most common shortcomings of such questions.

During that time we hopefully can have another discussion about the policy, or are even able to create a new one from scratch. There are quite a few approaches already in play, but we have to find a general compromise that works for everyone.
If we cannot figure out anything during that time, we will re-enable the close reason, and nothing ever happened.

If you have any concerns that need to be addressed before the start of the experiment, please share it as an answer below.

Disclaimer

Because it is likely to come up again: Homework questions are still allowed as long as they follow our current homework policy. This experiment is not changing this. If you would like to talk about the scope (including homework) in general, please open another meta discussion.

$\endgroup$
25
  • 1
    $\begingroup$ So... high level summary -- VTC means post has potential to be a good question, but needs improvement, but downvote means post likely does not have potential to be a good question, regardless of improvements? $\endgroup$
    – hBy2Py
    Commented May 22, 2017 at 19:45
  • 4
    $\begingroup$ @hBy2Py not always imo - some closed posts are just hopeless, e.g. if it's primarily opinion-based "how can i studi chemistryzzzz halp" $\endgroup$
    – orthocresol
    Commented May 22, 2017 at 20:44
  • 6
    $\begingroup$ the way I see it now, it is something like this: downvote = "I personally don't like this question"; closure = "this question is entirely unsuitable for SE" $\endgroup$
    – orthocresol
    Commented May 22, 2017 at 23:27
  • 1
    $\begingroup$ @orthocresol That seems somewhat orthogonal to Mart's perspective on things. How can we possibly construct a coherent close reason/policy if we haven't agreed on what closure is for? $\endgroup$
    – hBy2Py
    Commented May 23, 2017 at 1:18
  • $\begingroup$ @hBy2Py Sorry, I think I probably didn't express myself very well but I'll try. Not every question is salvageable - that is a given. Primarily opinion-based questions, like this one, can't be salvaged no matter how hard you or I try. That's why it's written "in the majority of cases" above (these are the minority). But for something like too broad, it's often possible to narrow it down. For something that's closed as HW, it's very possible to edit it into a shape that is acceptable for the site. And what Martin means by the bolded sentence $\endgroup$
    – orthocresol
    Commented May 23, 2017 at 1:31
  • $\begingroup$ is that for such questions, closure doesn't mean "get lost and don't ever come back", which (I feel) has become a bit too common. Instead it should mean "right now in its current form, your question is unsuitable for the site. Here's xyz information on how to edit it so that it becomes suitable and we can reopen it". We would like to, at the very least, push the community towards writing something substantial rather than just dropping a simple close vote, which tends to come off as "get lost". If this doesn't make sense ping me in chat, I'll be happy to discuss it.. $\endgroup$
    – orthocresol
    Commented May 23, 2017 at 1:37
  • $\begingroup$ @orthocresol No, I follow, I think -- some close reasons are for 'unrecoverable' characteristics of a post, while others are '(potentially) correctable'? $\endgroup$
    – hBy2Py
    Commented May 23, 2017 at 1:58
  • 1
    $\begingroup$ @hBy2Py (sorry for pinging you again) Yes, that's what I think right now, at least. Closure means "right now it's not suitable for SE", and it doesn't inherently come with any judgment as to whether it might potentially be useful. [side note, it would also be a lie to pretend that we've got everything sorted out. We have an idea of the direction we would like to move the policy towards, but the details are not fleshed out, there's lots of discussion to come, hopefully.] $\endgroup$
    – orthocresol
    Commented May 23, 2017 at 2:08
  • $\begingroup$ In general, closed AND/OR downvoted questions generally receive less attention that the ones that are upvoted (less views, answers, etc). If our objective is to "encourage users to actually name the problems of the post, ask follow up questions for clarification, offer guidance on improving them", shouldn't we find ways to reward people who look/fix these types of questions? $\endgroup$
    – CoffeeIsLife
    Commented May 23, 2017 at 7:24
  • $\begingroup$ @Quantum I am not sure how that would work. If you don't want to deal with those questions, simply move on, someone else will, or they will tumbleweed out of existence. The reward of looking at those questions is that you are helping people out, make the site function better. I am just asking that questions, which are not up to our standards will not be dismissed by default, and if you have already read the question, I thinks it is easy to drop a comment; especially when you are the first one to do that. You don't have to, you can simply go to the next question, nothing is a must here. $\endgroup$
    – Martin - マーチン Mod
    Commented May 23, 2017 at 8:27
  • $\begingroup$ @hBy2Py Closing means it is off-topic, and that is what the standard reasons do; but first the posts are on hold so that they can be improved. I would very much like to use this time to give, especially new users, the opportunity to do just that. The current custom close reason doesn't do this job well, because it reads like this: Don't do that! Don't do this! You must do that! I would rather have it read like this: You might not get an answer for your problem, but you can try the following to make other users more interested: ... . I don't want questions immediately rejected. [...] $\endgroup$
    – Martin - マーチン Mod
    Commented May 23, 2017 at 8:46
  • $\begingroup$ [...] In general, putting on hold (and later closing and eventually deletion) should follow easily understandable and objective rules and it should be done with respect to our own set scope. That's all problematic and explains our whole conflict and why this is taking so long. Voting is something very personal. It says I think this question does not help anyone. And you should utilise it at your convenience. It doesn't (or shouldn't) be dependent on closing or not. But if voting was more, a lot of the closing that we are doing now would be unnecessary. $\endgroup$
    – Martin - マーチン Mod
    Commented May 23, 2017 at 9:00
  • 1
    $\begingroup$ @airhuff Bear in mind that downvoted questions with no or only non-upvoted answers are subject to automatic deletion after a certain amount of time. So, less closing != more bad content sticking around necessarily. The cleanup would just proceed by different mechanisms, if I understand the mechanics correctly. $\endgroup$
    – hBy2Py
    Commented May 26, 2017 at 15:27
  • 1
    $\begingroup$ As a <3k user who can't use custom reason to flag questions as off-topic, which reason should I use then? I have another option 'blatantly off-topic' instead, but that doesn't seem applicable. $\endgroup$
    – Glorfindel
    Commented Jun 1, 2017 at 21:28
  • 1
    $\begingroup$ Also, regarding Closing a question (putting it on hold) should have the focus of improving it, reopening it, and answering it: closing a question can also have the purpose of feeding Roomba if it's a really awkward question. $\endgroup$
    – Glorfindel
    Commented Jun 2, 2017 at 9:29

4 Answers 4

Reset to default
15
$\begingroup$

And now I think, it is quite apparent that we need a close reason for zero-effort questions to simply save out time, because the absolute majority of questions that were previoulsy closed as homework ones are now closed with this very custom reason (zero-effort).

Yes, I know that judging an effort is subjective, but:

  • It is not the tough to spot zero-effort questions. Not specifically, I'm not talking about low effort, or not enough effort, which is always subjective (what is easy for me might not be easy for someone else), I'm talking about this highly typical cases of the "here is a problem, please solve it for me" type.

  • As a community we have to do that, for otherwise we well end up with what they have on SO: tons of (extremely highly upvoted!) question à la "How to print a number in language XXX?" which simply sucks. Science is basically based on making a lot of efforts in solving problems when facing them and we have to encourage that.

  • We are already doing that anyway through the custom close reason.

$\endgroup$
11
$\begingroup$

PSA: Please do not flag homework questions for moderator attention...

...because we will not closehammer them.

We would really like to move away from the current homework close reason, this experiment is not here simply to make everybody's life more difficult. If we thought status quo was alright then we wouldn't be doing this.

So far we have gotten quite a few homework mod-flags, and so far I have been marking them as helpful, as I know that flaggers have good intentions in mind. However, now that I have made this clear, soon I will start to decline them if they still pop up.

Thank you for understanding.

$\endgroup$
0
10
$\begingroup$

In conclusion ...

Although Wildcat basically already said it, I would like to add a few thoughts and numbers. I have to admit that this illdesigned, untargeted "experiment" did not turn out the way I would liked it to have. Some of my initial thoughts I have already shared with you: Let's talk about effort, shall we?

This, however, did not generate a discussion about what might be a good direction for change. It also did not provide an argument for keeping the effort criterion. And my suggestion was thoroughly rejected as the numbers will prove. I have the feeling, that we are all tired of talking about change and rather accept the status quo of the current homework policy. I think Brian put it best:

It's been hashed over so many times, I think I've lost perspective on it.

Well, let's get to the numbers. If someone wants to know the values from the 30 days before the experiment, find them here. Remember, you can find more statistics (10k+ reputation) in the tools section.

Closed and Asked questions

  • Questions Closed: 325
  • Questions Asked: 855
  • Close percentage: 38.01 %

Closure statistics

  • Close reason, no. closed (% of total), no. edited (% of closed), no. reopened (% of closed), no. edited and reopened (% of edited)
  • too broad, 53 (16.92%), 1 (1.82%), 0, 0
  • off-topic other (custom), 107 (32.92%), 5 (4.67%), 1 (0.93%), 1 (20.00%)
  • off-topic medical, 8 (2.46%), 1 (12.50%), 0, 0
  • duplicate, 55 (16.92%), 1 (1.82%), 0, 0
  • unclear, 82 (25.23%), 18 (21.95%), 4 (4.88%), 4 (22.22%)
  • opinion-based, 10 (3.08%), 1 (10.00%), 0, 0
  • off-topic (other SE), 2 (0.62%), 0, 0, 0

Custom closed as off-topic questions

  • (Questions Closed) Comment
  • (7) I'm voting to close this question as off-topic because we're not a homework service.
  • (4) I'm voting to close this question as off-topic because there is no effort in the question.
  • (3) I'm voting to close this question as off-topic because the OP demonstrated no effort to solve the problem.
  • (3) I'm voting to close this question as off-topic because it's homework without sufficient input from OP
  • (2) I'm voting to close this question as off-topic because you have not demonstrated any effort to solve the problem.
  • (2) I'm voting to close this question as off-topic because the OP has not demonstrated any effort towards solving this problem.
  • (2) I'm voting to close this question as off-topic because the OP has not demonstrated any effort towards an answer.
  • (2) I'm voting to close this question as off-topic because the OP demonstrated no effort to solve his question.
  • (2) I'm voting to close this question as off-topic because no effort has been shown by the OP.
  • (2) I'm voting to close this question as off-topic because no effort has been made by the OP.
  • (2) I'm voting to close this question as off-topic because it can be considered homework with no OP's effort shown.
  • (2) I'm voting to close this question as off-topic because a specific point of trouble in solving the problem isn't given. Please edit the question to show where you're having difficulty.
  • (1) Welcome to StackExchange. Your question shows zero effort and might be closed as such. Edit it to include more info.
  • (1) Pinpoint exactly where your stuck, otherwise this question will get closed as no effort.
  • (1) I'm voting to close this question as off-topic. Even though the OP has requested examples of ions that are isoelectronic with Ca (2+), he has made no apparent effort to solve this himself.
  • (1) I'm voting to close this question as off-topic because, at very least,"liquation" isn't a word.
  • (40+) I'm voting to close this question [...] no effort [...]
  • (10+) I'm voting to close this question [...] homework [...]
  • (1) I'm voting to close this question as off-topic because the OP should make some attempt at a solution so that we can diagnose their issue with this problem, and attempt to make it a more general question so as to appeal to a wider audience.
  • (A couple) I'm voting to close this question as off-topic because this is something that can be answered via Google.
  • (A couple) I'm voting to close this question [...] better suited at [...] SE.

With that being said, I will re-enable the homework close reason as promised.

$\endgroup$
4
$\begingroup$

SE is not about today, it is about the interstellar mission to another star system that needs to have available the collected wisdom of humanity soaked into an AI that will be able to offer advice on the problems that crop up.

  • A question should be allowed to stay if it can be answered.
  • A person can answer if they want to.

Those that don't like homework question should let the new blood cut their teeth on the easy questions and focus on the unanswered questions, perhaps even automate it that with 2000+ reputation you see new questions only later if they have no answers, no more frustration.

The amount of abuse will be zero as those that do not believe homework questions deserve any of their time will not even see them. Those that want to get reputation fast will soon loose the ability to see homework questions and so they become slow to get answered. The OP will get an answer from an enthusiast sooner than from a specialist but hopefully too slow to submit an answer into the test paper. A Win-win-win situation.

Homework is not a crime, lack of personal effort should retard the speed of getting results, not a way to prevent the collection of knowledge in the stack space.

How do we copy this post to all the other META stacks that keep having the homework question problem come up repeatedly.

$\endgroup$
5
  • 1
    $\begingroup$ 1. I agree in principle with your two bullet points, but I'm afraid that's about all I can agree with, because: 2. Most of the stuff you have written about are things that we simply do not have the ability to implement here. I'm not commenting on how good or how bad an idea it is, I'm just saying there's no way we can do it. 3. I don't really care who answers homework questions and whether the OP benefits from it. We aren't here to make sure everybody does their homework in the most proper fashion. $\endgroup$
    – orthocresol
    Commented Jun 19, 2017 at 11:36
  • 1
    $\begingroup$ @orthocresol It does rather sound as if you are certain that a final solution to the perennial and vexing problem of homework close determination would be ignored simply because there is coding required. SE is moderated by humans and operated by humans and I have a strong gut feel that if there was a need for some coding assistance the members would chip in. My proposal only requires those who have high reputation (or frown upon homework questions) to have newest unanswered questions sorted BY DEFAULT after older unanswered questions, selecting an explicit sort order would work as before. $\endgroup$
    – KalleMP
    Commented Jun 19, 2017 at 22:34
  • $\begingroup$ Yes, of course SE is run by humans, and anything is possible. It is just not within the realm of meta.chem. We have power here, but not enough power to fundamentally change the way the site works. I might be a moderator but I really do not have access to any of the underlying code of the site. For that sort of thing you would have to go to meta.SE. I hope you get what I mean. $\endgroup$
    – orthocresol
    Commented Jun 19, 2017 at 22:50
  • $\begingroup$ @orthocresol, I get what you mean, my point exactly is that if there was consensus that a simple fix could resolve the problem then the meta.SE folks could get the job done. I keep seeing this same problem in meta on many exchanges and it seems that the problem is merely perception and convenience rather than any desire to collect information or record answers. Who cares if someone finds all of last years test or homework answers on SE, it means that knowledge is now safe which is the prime directive here. $\endgroup$
    – KalleMP
    Commented Jun 19, 2017 at 23:00
  • 1
    $\begingroup$ I can see where you're coming from, and I do agree with the points that you're making, particularly about the aims of SE. I think sometimes we lose sight of what we are trying to do. Sigh, I guess I am just afraid that all of this might not be possible. Perhaps it is a somewhat defeatist attitude, but based on my experience there I doubt this would be well-received on MSE. It may be more productive to focus on a solution that we can implement ourselves - and that has been the focus of all our discussion, really. I guess you could try... $\endgroup$
    – orthocresol
    Commented Jun 19, 2017 at 23:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .