What does math education research know about difficulty vs. effectiveness?

Question

I've asked basically the same question previously on on math.SE, then cogsci.SE without much response, surely here is the place to ask this.

As anecdotal evidence is plentiful, but unfortunately for whatever conclusion you wish to draw, I'm hoping for answers that are backed by actual education or cognitive psychology research.

There seems to be a prevalent view among mathematicians and math educators that forcing students to struggle is an effective way to improve retention and understanding.

Since I first asked the question, several points have become clearer:

• There's sufficient (for me) evidence that required math courses are sometimes used to "weed out" weaker students as a form of "population control". I'm ignoring those.
• There are researchers who teach (historically, surely) who consider teaching to be a distraction from research and so have an incentive to embrace any teaching philosophy that minimizes their efforts, such as "let them struggle with it and prevail". Putting aside ethical and/or utilitarian arguments on the wisdom of such an approach, here too the nature of the course is not motivated by a belief that difficulty increases the benefit for students. I'm ignoring these too.
• There are various "non-traditional" approaches such as Problem-based learning, peer-learning, constructivist approaches, the moore method (and variants), lecture-skepticism (my term) and others which claim, and can often demonstrate quantitatively, improved performance by students (yet often, interestingly, decreased confidence). They share a de-emphasis on the traditional lecture in favor of students taking a more active role in the learning process (So in a sense, working harder). They often simply require a larger investment of time by the students which might itself account for the difference.
• Research suggests that humans have a natural tendency to "skim" when a suitable opportunity presents itself, so that increasing the difficulty involved in processing the material (even extremely artificially, as in using a less readable font) forces more engagement from the reader/student and improves retention (see papers). Similarly, when students have a strong preexisting intuition on the subject, such as when studying newtonian physics (which we all have much experience with), students appear to resist the learning of new concepts that replace their existing intuitions unless actively confronted with their shortcomings (see papers). This also can be interpreted as evidence that learning is improved when students are forced to break out of a tendency to use "cruise-control".
• The theory of "cognitive load" has been pointed out as relevant on cogsci.SE, but from what I've skimmed (naturally) so far, the idea is that under increased cognitive load learning breaks down. I've not seen yet results demonstrating that increasing cognitive load improves learning other then, as just mentioned, by forcing engagement.
• Vygotsky's theory on the "Zone of proximal development" is intriguing in it's claim that learning is most effective when students tackle problems which they "cannot solve unaided, but can solve with some guidance". The discussion is centered on children's education, it's unclear to me how widely applicable it is to a collegiate math setting.
• The related idea of a teacher as providing "scaffolding" also matches up with research on human memory: we remember new facts better when we connect them to existing memories and concepts. It seems plausible to claim that a student who derives a result on his own has built up a more substantial structure around the discovered fact and is therefore more likely to remember it.

That's an outline of 3 different senses of "difficulty" in learning:

• Challenge ("He sets impossible standards, so you work really hard or crash. or both.")
• Engagement ("I really have to pay attention to make sense of it, it's exhausting!")
• Time spent ("It took me 3 days to do that assignment, it was really difficult!").

Engagement seems specifically to be an issue of growing importance in recent decades, vis the debate on the effects of TV/video games and the increased diagnosis rate for attention/hyperactivity disorders.

I'm looking for answers that shed more light on the issue, point out research results and generally make the issue clearer. Though I am not an educator (math or otherwise), a good guideline are answers indicating research that would help someone with teaching duties, in math, at the college level, form an informed policy on the difficulty level of a course intended (one would hope) to help students achieve their best.

This question is admittedly quite too broad, but I care about the answer and so I'm asking anyway. I've done my best to make it constructive.

The original question on Math.SE was sparked by a bunch of quotes I've accumulated through readings on math. I've included them here:

The first is by Lebesgue:

When I was a rather disrespectful student at the Ecole Normale we used to say that 'If Professor Jordan has four quantities which play exactly the same role in an argument he writes them as $u$, $A''$, $\lambda$ and $e_{3}'$ Our criticism went a little too far but, nonetheless, we felt clearly how little Professor Jordan cared for the commonplace pedagogical precautions which we could not do without, spoiled as we were by our secondary schools. <...> Professor Jordan's only object is to make us understand the facts of mathematics and their interrelations. If he can do this by simplifying the standard proofs, he does so; <...> But he never goes out of his way to reduce the reader's trouble or compensate for the reader's lack of attention.

The "Moore Method" (and variants thereof) is a well known approach to math education. Here's an excerpt from P.R. Halmos' autobiography "I want to be a mathematician":

Can the mathematician of today be of any use to the budding mathematician of tomorrow? Yes. We can point a student in the right direction, put challenging problems before him, and thus make it possible for him to "remember" the solutions. Once the solutions start being produced, we can comment on them, we can connect them with others, and we can encourage their generalizations. Almost the worst we can do is to give polished lectures crammed full of the latest news from fat and expensive scholarly journals and books—that is, I am convinced, a waste of time. You recognize, I am sure, that I am once more advocating something like the Moore method. Challenge is the best teaching tool there is, for arithmetic as well as for functional analysis, for high-school algebra as well as for graduate-school topology.

Lastly, here's a quote from the preface to "Mathematics Made Difficult":

there is no doubt that an absolute ignoramus (not a mere qualified ignoramus, like the author) may become slightly confused on reading this book. Is this bad? On the contrary, it is highly desirable. <..misleading redaction...> it is hoped that this book may help to confuse some uninitiated reader and so put him on the road to enlightenment, limping along to mathematical satori. If confusion is the first principle here, beside it and ancillary to it is a second: pain. For too long, educators have followed blindly the pleasure principle. This over-simplified approach is rejected here. Pleasure, we take it, if for the initiated; for the ignoramus, if not precisely pain, then at least a kind of generalized Schmerz

Previous questions elsewhere

• math.stackexchange.com/questions/702044
• cogsci.stackexchange.com/questions/5921

Related Papers

• Designing Effective Multimedia for Physics Education - www.physics.usyd.edu.au/super/theses/PhD(Muller).pdf by Derek Muller TED talk: www.youtube.com/watch?v=RQaW2bFieo8
• Fortune favors the bold - web.princeton.edu/sites/opplab/papers/Diemand-Yauman_Oppenheimer_2010.pdf by Diemand-Yauman & Oppenheimer

Answer 1

Perhaps the key-word needed here is not just struggle but productive struggle.

Hiebert and Grouws (2007) discuss two key features of mathematical teaching/instruction for "promoting conceptual understanding" (p. 383). Their paper can be found here:

Hiebert, J., & Grouws, D. A. (2007). The effects of classroom mathematics teaching on students’ learning. Second handbook of research on mathematics teaching and learning, 1, 371-404. Link.

To the point, their second feature is called Students Struggle with Important Mathematics.

The authors trace this line of thought back at least to John Dewey (p. 388):

The article continues with references to work by cognitive theorists such as Lev Vygotsky (whose Zone of Proximal Development is mentioned in the OP), the mathematician George Pólya (whose "How to solve it" effectively laid the groundwork for the study of mathematical problem solving), and the developmental psychologist Jean Piaget.

The authors continue with a discussion of some material on work done outside of mathematics before returning to studies specific to mathematics and mathematics education. You remark that your "question is in natural peril of being too broad," but hopefully this source (and the references contained therein) can get you started on tackling it.

Answer 2

Here is an NPR article that discusses how teachers' efforts to engage learners in productive struggle (or not) may be culturally situated. (Of note, Benjamin cites Jim Hiebert above, who has written The Teaching Gap with Jim Stigler, interviewed in this article.) Jim and Jim have conducted research on how instructional approaches differ culturally between nations.

Answer 3

The keyword you're looking for is "desirable difficulty," which refers to a practice condition that makes the task harder, slowing down the learning process yet improving recall and transfer.

Your question is incredibly broad so I'll devote the rest of my answer to addressing specific confusions you mentioned.

Active Learning

In order for students to have learned something, they need to be able to consistently reproduce that information and use it to solve problems. None of these things happen when students watch a lecture, even if they understand it perfectly. The same reasoning applies to watching videos, reading books, re-reading notes, and all other passive learning techniques. If students don't actively practice retrieving information from memory, it doesn't get written to memory. It just falls out of their brain.

Relationship with Cognitive Load

Now, here's the thing. The goal of active learning is not to blow up a student's cognitive load. It's actually the opposite -- to get students actively retrieving information from memory, while minimizing their cognitive load.

When a student has a heavy cognitive load, their working memory is running low on processing power, which means that

• they will have a harder time noticing general patterns and seeing the forest for the trees, and

• they'll be more likely to fail in their attempts to retrieve information from long-term memory.

This is why it's so important to scaffold instructional material and introduce new material only after prerequisites have been learned. New material needs to be

• broken down into bite-sized pieces small enough that no piece overloads any student's working memory, and

• introduced after the prerequisites have been learned so that the prerequisite knowledge can be pulled from long-term memory without taxing working memory.

Issues with Pure Discovery & Radical Constructivist Learning

Unfortunately, that's where some extremist non-traditional approaches get it wrong: they get students performing activities, but they don't minimize cognitive load, and students just spend the whole time in a state of cognitive overload, getting nowhere.

Optimally active learning doesn't mean that students never watch and listen. It just means that students are actively and successfully solving problems as soon as possible following a minimum effective dose of initial explanation, and they spend the vast majority of their time actively and successfully solving problems.

Reality vs Perception of Learning

Finally, there's one catch: even if students are engaged in optimally active learning, they're typically not going to perceive it as being optimal. Active learning produces more learning by increasing cognitive activation, but students mistakenly interpret that extra cognitive effort as an indication that they are not learning as well, when in fact the opposite is true.

And here we arrive full circle back at desirable difficulty. Active learning creates a desirable difficulty that makes class feel more challenging but improves learning. Passive learning, on the other hand, promotes an illusion of comprehension in which students (and their teachers) overestimate their knowledge because they are not made to exercise it.

Further Reading

It sounds like you have lots of questions about all this and want a bunch of scientific references, which is beyond the scope of a single StackExchange answer. Here's a draft that I'm working on that goes into all this stuff (and more) in way more detail with over 300 references and relevant quotes pulled out of those references. It will clear up all the things you're asking about in this question, as well as more things that you'll naturally want to ask about afterwards.