What are the pros and cons of the practice that includes some questions in a math exam which are directly discussed in the lectures?

Question

What are the pros and cons of the practice that some questions of a (written) math exam are exactly from the course? For example, asking about the proof of a theorem proven in a course.

The argument I have heard against such practice is that some students tend to just memorize the material, but on the other hand, some people say that the results covered in a standard course are usually essential theoretical results and even memorizing them has some pedagogical benefits.

Edit: The majority are average students are able to do routine exercises, are more and less attentive. Few students are outside average range (very exceptional students or below average). Just a normal class.

Answer 1

Have a look at Bloom's Taxonomy. Asking students to understand a proof from class well enough to recreate it is near the bottom of the taxonomy. This is perfectly fine because you definitely need the students to be at that level before they can apply their understanding to new situations. But the questions you need to ask yourself are:

1. how much of my assessments focus on understanding vs on application?
2. what other questions could I ask to test their understanding?

For a proof-based class, the students should be engaging with higher levels of the taxonomy so you don't want the majority of your questions to be "recreate what I did in class." One or two questions should be ok given some thought about the key concepts in the proofs you expect your students to know.

Answer 2

One big advantage of this sort of question is that it tests what it aims to test extremely well. If a student knows the material they will be able to do the question (even on a bad day) and if they don't they won't (even on a good day).

When you ask about something new, there is inevitably some randomness about which students will spot what they need to do differently. A slightly different question might suit different people, and someone having a bad day could miss the key idea even if they would probably get it under better conditions.

Of course, that is also why you need plenty of questions higher up the taxonomy - you need to measure those skills too, and since they are harder to measure you need to put more effort into doing so.

Answer 3

Variation in familiarity/difficulty is desirable in exams

An ideal examination will have questions that range in their level of difficulty and familiarity, including some questions that are familiar to students and offer an opportunity for "easy marks". Including questions that have already been asked and answered in lectures or tutorials (or which were assigned as homework) meets the function of giving familiar questions with no nasty "twists" that would require higher-order skills. You should also include some questions that are minor variations on things the students have seen before, some questions that are major variations on things the students have seen before, and also some questions that are totally unfamiliar, but still use the same mathematics that is being taught in the course. This variation will help students get used to adapting to solving new problems with varying levels of familiarity.

There are some areas of applied mathematics where it is useful to memorise a set of equations or results, particularly when what is being memorised is the correspondence of an equation or result with its name. For example, for students doing work in probability/statistics, it is useful to have memorised the density functions of all or most of the core families of probability distributions (e.g., being able to easily identify the normal distribution, gamma distribution, binomial distribution, etc.). This is useful in the same way that it is useful for medical students to memorise the names of parts of the body to aid in learning the substantive parts of medicine.

Aside from these types of cases, one of the key things you will need to impart to your students is that "memorisation" of results that is not backed by understanding is not a method that can lead to long-terms success in mathematics, even over the period of a single course. It is possible that a student could memorise a proof and regurgitate it, but it is not really feasible to pass an entire mathematics course merely by memorisation and regurgitation (and it is certainly not feasible to have a successful mathematics career this way). Memorisation of an entire proof comes with a significant cost in terms of cognitive load. Memorising the steps of a proof you don't understand requires significantly more cognitive load and likelihood of error than merely remembering the core method and main steps of a proof you understand. Moreover, the lack of understanding means that you can only reproduce the same proof and you can't handle variations on the problem if these arise.

In a learning situation, the student who memorises the proof (in lieu of understanding) learns that there is a significant cost to not understanding something and merely reproducing it by rote and that this is an inefficient method over the long-term. At most, memorisation of a proof can offer a minor short-term benefit. It is useful to impart this reality to students explicitly, but they will also learn it as they go when they attempt to "cram" for exams.

Answer 4

My sense of your question is that this is entirely normal and common. But the question isn't (shouldn't be) one that emphasizes memorization, but, rather, insight into how such a proof actually works and contributes to the overall theory. That requires thought, of course.

I'll note that some proofs in math are especially useful when they provide not just proof, but insight into the underlying concepts of a field.

I'll also guess that this is most useful for the important theorems, rather than the corollaries and such.

For this approach, I don't see any downside.

Answer 5

Often I do exams like this:

• First part, lesson questions
• • Question 1) Recall the definition concept X.
• • Question 2) Recall theorem Y.
• • Question 3) Recall the proof of theorem Y.
• Second part: solve a math problem (divided into several subquestions to guide the steps).
• Third part: another math problem (usually one of the two problems is much shorted than the other)

Not surprisingly, the students who fail questions 1, 2 or 3 also completely fail both problems, and most students who succeed at questions 1, 2 and 3 mostly succeed at the problems as well, although there can of course be a few mistakes along the way.

So, it looks like these three sets:

• the students who memorised the lessons
• the students who understood the proofs
• the students who are able to apply the concepts to solve a problem

are mostly one set, not three different sets.

The exact score earned in the exam due to the two problems vary a lot between students, due to varying number of mistakes or of how well they handled the time constraint of the exam; but the first part very clearly divides the class between those who have a basic understanding of the lessons and those who don't.

If I had to explain why, I would guess that the concepts are actually easy enough that if you have memorised them, you will not have too much trouble understanding them; but they are complicated enough that there is no hope of understanding them if you haven't even memorised the basic building blocks.

Answer 6

One disadvantage – or at least restriction – is that it doesn't work in an open-book exam.

Answer 7

You've received some good answers already, I'll just add an example of what might happen in case you overdo it.

At the uni where I studied there is a lecturer in theoretical physics who was (and maybe still is, I don't know) known among students for basing her exams largely around (minor variations of) exercises that were studied in the exercise groups during the semester, and thus for the exams being easy to pass even when you struggle to understand difficult theoretical material.

And then, of course, more than on average perhaps students that are looking for an easier way to pass the exam attend courses by such a lecturer, for example students that have failed the course before or, interestingly, sometimes even students who take the course one year earlier than scheduled just to get that opportunity (thus not having attained all the knowledge from preceding theoretical physics lectures they might need for the course).

I'm saying nothing about the extent of such an effect and one whether you should put this down as a pro or a con... ;)

Answer 8

Okay, so I would like to share my experience working with some professors and also in the creation of test papers for university students in Russia. Some questions on math exams are taken directly from the course material, not just math but everything.

Here are the pros:

It solves the problem of a lack of question materials. You cannot keep creating new questions all the time. In fact, professors often create a list of questions based on what was elaborated on during lectures. And, of course, the lecture program was created based on a "course" book. Even if let's say you can create new questions, can you keep doing that for years, because after each years, the current year of students will just made a compilation of known questions and sell it to the next year. New questions become known questions.

It makes it easier for students. Now, you may ask, why does it need to be easier? Simply because many professors know that students don't have time to study all of that. Life in university or college is really busy, and many students work part-time while attending classes. So having repeated questions each year based on lecture materials reduces the time required for studying.

It also reduces the chances of disputes. When you create new questions, different students will answer differently because they will have to answer based on their understanding. University students' wording is not always the best, so in cases where the answers to those questions were not regulated, it's hard to deny the answer of someone without disputes coming from them later on, which costs a lot of time and work.

Answer 9

Well, I don't get what the alternative would be. As a student, I expect the exam material to reflect the notes, readings, and lectures. If the exam doesn't do this, then the notes, readings, and lectures are useless, from my vantage point.

I haven't started teaching yet, but it would be unfair of me to ask students details we've not covered in class or in the material, because everything I'll cover is worth covering under the time constraints.

I should note that the questions I would ask on an exam depend on the audience. If I'm teaching a masters level class on causal inference, I will only ask why, for example, the convex hull constraint is believed to be useful. I won't, for example, teach them about changing the constraints to allow for negative weights, allowing for intercepts, or taking away the adding to one constraint.

Why? Well because I honestly would expect very few masters students to even try to do well on a question like that. If I've only covered the classic method, then that's all they'll need to know about.

And even if it were a PHD course on synthetic controls, I would still at least talk about these different penalties and approaches to constraining the weights. I would demand they have knowledge of the basic setup, and then be able to talk richly about why changing these details might change our results. Why? Well, they're PHD students, and this hypothetical course would only be an elective, so if you're here in the class, I presume you want to know deeply about this method in econometrics.

You've already mentioned that your students are average (which is fine!!!). Why even bother to ask them things you know they won't do well on, if the median student in the class is average? Part of being a teacher is teaching to your audience, and putting things in a way they will understand, and testing them in a tough, but FAIR manner.