41

This is not a question of "Why does The Loop Survey ask about age?" That has been asked elsewhere.

This is not a question of "Why can't users under 18 take The Loop Survey?" That has been asked elsewhere.

The first question in The Loop Survey is:

    1. To continue, please confirm that you are at least 18 years of age.

If you answer "I am under 18", the survey ends.

If you answer "I am at least 18 years old", the survey continues.

Question # 6 is:

  1. What is your age in years? (optional)

The first response listed is:

Under 18

Selecting "Under 18" does not abort the Survey.

Was there no thought at all put into the Survey questions?

Is this a "trick question" to find all the teenagers using Stack Overflow while pretending to not want to talk to them?

Improve this question
1
  • 21
    Yeah, that thing wasn't thought out very well in the first place. Inner contradictions are one of its most harmless characteristics.
    – Frédéric Hamidi
    Commented Nov 28, 2019 at 7:13

2 Answers 2

Reset to default
20

Typically you want complete coverage for a demographics question, but because this is after a screener, yes, you don't need the under 18 option for question 6.

It might be used an attention check, but it's also possible the survey author didn't think through the branching logic. When rushed, I've made similar mistakes.

It's one of the reasons you should always field test a survey first. Considering the other errors, it seems like they skipped this step.

Improve this answer
6
  • 12
    "rushed" is no excuse when this is being pushed as a big deal on the blog of a big company. Plus the survey has been revised at least once already to try (arguably unsuccessfully, but at least some attempt was made) to fix the race question. You would think that once issues were raised that someone would take a few minutes to review the whole thing.
    – manassehkatz-Moving 2 Codidact
    Commented Nov 28, 2019 at 7:21
  • 3
    The "Loop" was scheduled to be released a week ago but there were hitches and its arrival was announced last Monday. I wonder what fixes had to be made then.
    – Mari-Lou A Слава Україні
    Commented Nov 28, 2019 at 7:30
  • 16
    A missed opportunity to recreate a sense of collaboration and instil a relationship based on mutual respect if management had had asked users on SE to go through the questions spotting any incongruences. The more eyes the better etc.
    – Mari-Lou A Слава Україні
    Commented Nov 28, 2019 at 7:36
  • 1
    @MariLouA I couldn't agree more
    – De Novo
    Commented Nov 28, 2019 at 7:40
  • 2
    Obviously, they didn't like "The Loop" much more than we do.
    – Gloweye
    Commented Nov 28, 2019 at 7:50
  • 9
    Considering the fact that they already moved out the announcement a few days, to "get it right" ... all of this is just such an example how to not do it. In some years, business schools will study this time at SE Inc, and write a book "what to not do".
    – GhostCat
    Commented Nov 28, 2019 at 8:55
6

My guess is the 18+ requirement came down from on high (legal) late in the process and thus wasn't considered in the original design of the survey. Having been added at the end, the age category wasn't revisited to remove the redundant option.

This is substantiated by Catija's explaination in her answer to the question of why the loop requires respondents to be 18 or older:

I asked about this when we were rolling it out and the response I got is that we consider this to be user research, which, for legal purposes, we restrict to users who are 18 and up in order to opt in.

It's plausible that the data science team designing these surveys was not aware of this requirement when designing them.

Improve this answer
2
  • 2
    I am not sure this was produced by a data science team. They would have us an external source for the ethnicity questions. The first thing I did is to cross-reference their options with the ones used by the US Census Bureau.
    – Alex
    Commented Nov 28, 2019 at 7:35
  • 4
    I think Yaakov assured us somewhere in a comment that they have people of matching professions preparing all that stuff. Maybe they are hiding their professional attitude to come over more human-like. You know, not like total expert nerds. They mess up on purpose so that we feel better and have something to rant about?!
    – GhostCat
    Commented Nov 28, 2019 at 8:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .