24
$\begingroup$

When taking a MOOC in calculus the exercises contain 5 options to select from. I then solve the question and select the option that matches my answer. Obviously only one of the options is correct. But there are (quite a few) times where my solution is wrong even though it is one of the available options.

My question is, how does a maths teacher know how to create wrong options that are as plausible as the correct answer?

I've found these two good questions but I am wondering how would an educator come up with wrong answers in general.

What are some common ways students get confused about finding an inverse of a function?

How are students messing up in this Khan Academy surface area problem? (right triangular prism: 3-4-5 triangular base, height of 11)

Improve this question
$\endgroup$
8
  • 11
    $\begingroup$ Useful google search: "item writing" + "rationales" + math + "distractors" $\endgroup$
    – Dave L Renfro
    Commented Mar 14, 2020 at 13:53
  • 23
    $\begingroup$ One thing to beware of is the "answer in the middle" strategy. Suppose the available answers to a question are (a) $\log(x)$, (b) $\log(x)+c$, (c) $x^0/0+c$, (d) $-1/x^2+c$. Well, you can see that the answer must have a $+c$ because all but one of them do. So (a) is wrong. So why is (a) there? It must be one hop from the right answer, which must therefore be (b). Basically, the answers form a star-shaped graph, with the right answer in the center and others 1 simple move away. Similarly, faced with (a) "dog" (b) "dig" (c) "hog" (d) "doe", what would you choose? $\endgroup$
    – Adam Chalcraft
    Commented Mar 14, 2020 at 23:17
  • 2
    $\begingroup$ @Adam Chalcraft: Your (a) shouldn't appear in a carefully designed item (no option should have an especially unique appearance), and in the work I've been associated with, options are ordered (or reverse-ordered) by numerical value or by some type of visual or mathematical complexity pattern. Also, checks are made (especially for large CAT item pools) to ensure a balance of correct letter-choices, and often the choice to use a reverse-order pattern in newly written items is to correct for imbalances that might have been found. $\endgroup$
    – Dave L Renfro
    Commented Mar 15, 2020 at 8:12
  • 6
    $\begingroup$ @AdamChalcraft, if I wanted to be mean, answer "a" would be the correct one. $\endgroup$
    – Mark
    Commented Mar 15, 2020 at 21:53
  • 2
    $\begingroup$ An evil strategy I've seen on high-stakes multiple-choice tests is to always have "None of the above" available as an answer $\endgroup$
    – Nayuki
    Commented Mar 17, 2020 at 16:55

5 Answers 5

Reset to default
44
$\begingroup$

I freelance as an item writer, someone who writes questions for standardized tests. When making up alternate choices, I always have to justify my reasons for the "wrong answers" or distractors. Here are some strategies I use.

  1. I focus on common misconceptions for students at that grade level.This is easier after years of classroom experience.
  2. In a multi-step problem I often use the answer to an intermediate step.
  3. When calculator use isn't allowed, I use answers with common calculation errors.
  4. I look for errors students might make if they don't read the problem carefully.
Improve this answer
$\endgroup$
3
  • 17
    $\begingroup$ As I'm sure you know, sometimes it can be challenging to come up with 4 or 5 distractors, and often you just have to let your imagination run wild. For instance, suppose on a non-calculator test the question was "Which of the following is equal to $4^{1/2} + {36}^{1/2}$?" Here are possible choices, roughly ordered by how likely I suspect someone might pick them (last 2 or 3 very unlikely): $\;$ (A) $2 + 6\;$ (B) $2 + 18\;$ (C) $(4 + 36)^{1/2}\;$ (D) $4.5 + 36.5\;$ (E) $(4 + 36)^1\;$ (F) $(4 \cdot 36)^{1/2}\;$ (G) $(4 \cdot 36)^{1/4}\;$ (H) $(1/2)^4 + (1/2)^{36}$ $\endgroup$
    – Dave L Renfro
    Commented Mar 14, 2020 at 18:48
  • 3
    $\begingroup$ Number 3 is mean... and brilliant. $\endgroup$
    – Cort Ammon
    Commented Mar 15, 2020 at 5:51
  • 4
    $\begingroup$ @CortAmmon: In physics class I frequently depended on the multiple choice questions containing no calculation errors (all consumed by the conceptual errors) to get through pop quizzes with a four function calculator as we had to pass one advanced calculator between me and my sister. Which means if we don't know in advance when an exam is, ... $\endgroup$
    – Joshua
    Commented Mar 16, 2020 at 1:59
21
$\begingroup$

If a teacher has taught the course before, and has asked questions that are free-response (not multiple-choice), then the teacher can look at the incorrect answers previously given by the students.

If not, then the teacher can ask other teachers who have had this experience.

Errors that "appear to stem from consistent application of a faulty method, algorithm, or rule" are called "systematic errors" (or "bugs") in the literature. Use these terms to search the mathematics education literature. (See my answer here, for example.)

Improve this answer
$\endgroup$
2
$\begingroup$

One thing common, especially at middle school/high school levels is to list the unknown (for example, x = 3) as one of the options, especially when the question actually asks you to solve for something else that requires knowing x. At the middle/high school levels, several students have a habit of thinking that "x is the answer, or you're done once your find x". In such questions, listing x = 3 (or whatever the unknown actually is) as one of the answer choices tends to throw students off. This is how I come up with wrong answers sometimes.

Improve this answer
$\endgroup$
2
$\begingroup$

If the answer requires a complicated formula, I would present several "permutations" of the formula, only one of which is right. Example:

If $f(x) = \frac{u(x)}{v(x)}$, what is $f'(x)$? (Quotient Rule.

Answers:

a) $u'(x)v(x) + v(x)u'(x)$. Product rule formula, wrong.
b) $u'(x)v(x) - v(x)u'(x)$. Correct numerator, no denominator, wrong.
c) $\frac{u'(x)v(x) + v(x)u'(x)}{(v(x))^2}$. Correct denominator, wrong numerator, wrong.
d) $\frac{u'(x)v(x) - v(x)u'(x)}{(v(x))^2}$. Correct denominator, correct numerator, correct.

Improve this answer
$\endgroup$
-1
$\begingroup$

So. I started working as a teaching assistant for a course, and the professor showed me "test generating" software. It comes down to the fact that the editor of a textbook creates various open, multiple choice, and true/false questions and answers, and you - using the software - can just click "generate X number of MC questions". No need to think about it at all.

So to create a midterm, I just downloaded the software, hit "generate" and got all the questions and answers. I know this does not exactly answer your question, but I did not know this existed and it completely changed the "they must think so hard about all of this" idea I had in my head and I had to share this. Especially as I was shown this only AFTER coming up with questions and answers for the previuos midterms.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.