Are the wrong questions and answers getting too many upvotes?

Question

On the Tour page for Security.SE it states:

Information Security Stack Exchange is a question and answer site for Information security professionals.

However, when I see questions like this Does clicking lead to DoS attack? with 10 upvotes, it doesn't strike me as the sort of question an information security professional would be asking. For a start it is a very basic question, and the OP doesn't appear to possess knowledge of how a web page actually works (i.e. the click resulting in a GET or POST made by the browser).

I wouldn't mind questions like this as I like helping people, but the fact that it has got 10 upvotes indicates that it is high quality information security question where it is clearly not. Obviously the OP may be an IT professional and wants to ask about security, but that doesn't make them an infosec professional - should the Tour page be updated to reflect this?

Another example is an answer to the above question which states

DoS is not about sending some specially prepared data that will kill your site (like exploits, ping of deaths etc.).

which is clearly wrong, but appears to have 6 votes.

Another is this answer to a different question - 25 votes for an incomplete answer.

Maybe these examples are simply anomalies - but it concerns me that these posts are the ones currently receiving the most positive attention on a site for "information security professionals".

Now is the scope for Security.SE for the average user with security issues, or is it for infosec professionals? Can we have a site that does both without the professional questions being watered down by basic ones that appear to attract the most attention.

The solution that this most reminds me of on the SE network is MathOverflow vs Mathematics.SE where they have a different site depending on the level of the question. However, this may be confusing and may result in two watered down sites if Security.SE did the same.

Answer 1

This is the joy of Hot Network Questions. All of the questions you mentioned were recently featured in the Hot Network Questions sidebar, which is shown on all sites. Many Stack Exchange users (including lots of people who are not members of the Security.SE community) see those questions and click on them.

This exposure has advantages and disadvantages. The advantage is that we get more exposure for these questions and can document answers to some questions of broad interest. Others elsewhere on the Stack Exchange network benefit from this knowledge, or at least have moment of entertainment and intellectual enrichment.

The disadvantage is that we get a massive influx of people viewing and voting on these questions and answers, many of whom might not be members of the Security.SE community and might not be information security professionals. Anyone who has at least 200 reputation on any Stack Exchange site anywhere can associate their account with Security.SE with just one or two clicks, and they are instantly provided an account on Security.SE with 101 reputation. This gives them the privilege to upvote (but not downvote) questions and answers. Also, the exposure sometimes prompts others to try answering the question.

So, when a question gets featured as a Hot Network Question, it often accumulates more answers than usual (sometimes from people who might not have previously been a member of Security.SE), and the question and answers often get many upvotes from folks passing by from another site. As a result, vote counts on Hot Network Questions can be pretty dubious and don't always reflect the Security.SE community's views; votes from new folks can outnumber votes from our own active members. So, the votes on these questions aren't always reliable, and the answers aren't always reliable, either.

I've noticed that a bunch of the questions I've seen featured on Hot Network Questions have been what I would consider to be lower-quality: they did not show much evidence of prior research, many were duplicates or were a variant of something that had been previously asked, and they might be about something of broad interest but that maybe an information security professional would likely already know.

So, I think these might be isolated cases that don't necessarily reflect the rest of the questions on this site. I'm not convinced we need to change the tour page, create two sites, change the scope, or anything like that. Overall, things seem to be working pretty well, apart from these exceptional cases caused by Hot Network Questions.

There's not a lot we can do about the effects of questions being featured in Hot Network Questions. About all we can do is try to moderate these questions rapidly: vote on them; if they need to be closed, place appropriate close votes; when there are many answers encourage moderators to clean things up and delete answers that don't add much (e.g., by flagging the poor/incomplete answers for deletion, even if they have some upvotes), etc.

One thing you can do: if you see a Hot Network Question that has attracted poor answers from drive-by users, please flag it for moderator attention and ask the moderators to protect the question. This will ensure that the question can only be answered by users who have earned >= 10 reputation points on this site. This situation is exactly the sort of thing that protected questions were designed to solve.

The side effects of being featured as a Hot Network Question have been discussed at length on the Stack Exchange Meta; they're not unique to our site. You can read there for more on the arguments in both ways. See, e.g.,

• Dealing with “Find out who's going to buy the croissants”

• The Anatomy of a Hot Question

• Filtering “hot” questions

• Better criteria for the hot questions list

• What is the Goal of “Hot Network Questions”?

• Trolls in our Halls

• The association bonus should not enable users to vote on every site

Answer 2

I agree that the wrong things get upvotes. Look at my profile: +167 for a banal answer on chip & pin, while a pile of more interesting answers get only a vote or two. And it's not just because of hot network questions.

I would love to see a site more aimed at Infosec professionals, but I think this is unlikely to happen. A flow of more basic questions gives a reassuring volume of traffic (but are there too many dupes?). The most likely fix is to rebrand slightly, perhaps "a site for Information Security discussion".

Something that exacerbates the problem is that questions of a professional nature tend to get closed as "too broad" or "opinion based". This has frustrated me a few times and is something the mods could easily change.

Another change we could make now is for high-rep users to upvote more on these kinds of questions. Many high-rep users do not vote all that much. I do wonder if votes from high-rep users should actually count for more - but this is a SE change that I expect is unlikely to happen.

Many high-rep users say "don't worry about rep" but I think that is a bit of a cop-out. SE is designed to be all about rep - it's always shown next to your name. The reality is that rep is not all that accurate - but it's the best anyone can come up with. SE is very effective at being an online community that is better than mailing lists - and that's why I stick around.

I notice some high-rep users are very adept at playing the rep game. They understand what questions will be popular and write popular answers. Fair play to them - but I do wonder if there are perverse incentives here. Sometimes the right thing to say is not the popular thing.

Answer 3

MathOverflow and Math SE are a special case. For those who are unfamiliar: anyone can create an account on MO, but in practice, the community is largely restricted to research mathematicians (think people who have doctorates in mathematics and work at universities); on the other hand, Math SE is more of a "typical" SE site.

MO was originally started as a Stack Exchange 1.0 site, in the days before Area 51 existed. The SE 1.0 system, in a nutshell, allowed anyone with enough money to buy a copy of the SE software and start a Q&A site about any topic. Most of the sites that were started in the SE 1.0 days failed; MO is one of the few exceptions. After some discussion on both sides, we eventually worked out a way for them to join the current network, but by that time, Math SE already existed.

The point of this short history lesson is that MO and Math SE shouldn't be used as a model for other sites. We really don't like the idea of splitting up a topic into "the experts" and "everyone else." Aside from having an uncomfortable elitist vibe, that hurts the chances for most people to get good answers.

Instead, we try to have the best of both worlds by creating sites that are aimed at experts, but accept questions from just about everyone, as you surmised. (On a related note, we've actually stopped using "professionals" in new proposal and site descriptions.) In our experience, experts are generally happy to help non-experts interested in their fields, as long as they're not completely clueless. We've also found that experts don't need 100% of the questions on a site to be super-expert-level for them to stick around; the number just needs to be significant.

If you don't want the tour and other pages to say "professionals," please post a separate feature request. Note that many new users don't actually read that page, so the impact of the change will be limited.

As D.W. pointed out, the Hot Network Questions were involved in this specific case, and I have nothing to add to his good answer about that.

Answer 4

Using many related sites, the framework has all of the right controls and balances IMHO but... in this particular case what is lacking is better moderation. That last control is what can/should weed out bad questions and non-empirical discussion. However, there are perhaps not enough/qualified moderators within the security sub-domain.