I come from lands afar, also known as Hardware Recommendations Stack Exchange. Over there, we have a serious problem with first-time Stack Exchange users coming along and asking off-topic questions - to the extent that 50% of incoming questions are closed. We've had a few attempts at solving this, but none have really worked.
So, this afternoon, I got together a group of people who had never seen Stack Exchange before. I gave them each a question (on various topics, each intended for a different site in the network), and asked them to find the most appropriate site to ask it, noting down the troubles they had along the way.
(They were all terrible questions, though the topic was clear. I did ask them not to actually post, but, by happy chance, the one who actually did demonstrated my issue perfectly.)
This post is a list of their most common problems with the sites as first-time users.
Help! How do I start?
People didn't start reading from the top bar. When they landed on a Stack Exchange site, the most common behaviour was:
- read the site title
- read the banner, be confused by it (more on that later)
- read question titles
After that, actions start diversifying. The point is, they don't see the topbar. Which, in turn, means they don't see the help link. Of 20 participants, only one ever saw a help center - and that wasn't by getting there from the topbar link. The topbar help link fails its purpose.
It's Free! Now sign up.
When you're a first-time user, your view of a site has a dismissible banner across the top giving you a 30-second introduction to the site:
That seems helpful enough, until you read the wording for the first time and notice this:
It's 100% free, no registration required.
Sign up
It's not technically incorrect or in any way wrong, but it reads very strangely for a first-time user. "You don't need to register - click me to register!"
Why can't I answer?
Okay, my test group weren't meant to be posting answers. Still, they identified this as another weird thing. Having just read that banner, and seeing "anyone can post an answer", the next thing many of them did was click on a question to see if its topic matched their question. When they scrolled down to read the answers, there weren't any, and they discovered that they couldn't post an answer either.
"What? But it said anyone could post an answer - how am I meant to do that?"
Of course, the question they're looking at has been closed. This raises several points:
- they didn't notice [on hold] in the title (or didn't place any significance on it)
- they scrolled past the close message
- there was no way to tell why they couldn't answer
When they did notice the close reason, it was generally fairly good at explaining what had happened, so that much is working. Perhaps we just need to emphasise the reason, and add a note to the bottom of the page in place of the answer controls explaining why the controls aren't present.
Hot Network Questions - what?
HNQ was one of the most confusing features mentioned. On a page full of questions about gardening, looking at the sidebar and seeing questions about Star Wars and Windows 10 next to each other confused my test group. Some sample reactions:
Aren't those questions off topic here?
What's the network?
What's Hot?
Why are they here with the gardening questions?
Ah, so I ask here. Or do I?
Many sites' scopes are not immediately obvious. That's not a bad thing, it's just a product of the system that a site can't take absolutely all questions about one topic. However, it seems that we do need to make sure the guidance is available.
On the Ask Question page (/questions/ask), there's an info box on the right-hand side of the page. Some sites have customised this with a custom message; others use the default.
A customised message (Hardware Recommendations):
The standard message (Open Source):
In both cases, my test group didn't notice, or didn't fully read this information until after they'd typed their question into the box.
At that point, if the guidance makes the question they've just typed off-topic, the majority reaction was "but I've already typed it out... they won't mind, right?". Clearly, that's not the reaction we want. If this guidance could be moved to the left hand side of the page, in consistency with most people's reading direction of left-to-right, I think more people would read the guidance before posting an off-topic question.
I've got a small table of results, which shows the site I intended the question for, the site the user decided to post on, the time it took them to reach a decision (hh:mm) and whether the question was on-topic at their decided site.
| Intended | Decision | TTD | On Topic? |
|---|---|---|---|
| SU | HR | 00:12 | n |
| HR | HR | 00:08 | y |
| SO | SO | 00:03 | y |
| Progs | SQA | 00:20 | n? |
| Workplace | Workplace | 00:19 | y? |
| SF | SO | 00:05 | n |
| Gardening | Gardening | 00:15 | y |
| ELL | EL&U | 00:10 | y? |
| CS | TCS | 00:12 | shrug |
| Apple | Apple | 00:07 | y |
| SF&F | Movies | 00:25 | y? |
| Bicycles | Bicycles | 00:09 | y |
| Aviation | Aviation | 00:11 | y |
| Security | Progs | 00:07 | n? |
| Physics | Physics | 00:06 | y |
| Homebrew | Beer | 00:17 | n |
| Writers | WB | 00:13 | n |
| DBA | SO | 00:15 | y? |
| Crypt | Security | 00:08 | y? |
| Biology | Biology | 00:06 | y |
So that's the results of my mini-research study on Stack Exchange. No doubt there are some obvious and some less obvious solutions to each of these points; if we could consider doing some of these things we'd make the sites a lot friendlier to first-time users. As a side benefit, we'd also give them more guidance, potentially reducing off-topic questions on some sites. So - can we make SE more friendly to first-timers?
