One of the ways computers are being used in math education is to provide students with online practice. There are a bunch of serious problems with most of these programs.

 

Here is one example from the Khan Academy (apparently at least one of the flaws outlined below no longer applies to the Khan Academy. But that same flaw still applies to IXL. And Prodigy Math. And a thousand other practice apps out there.)

 

Feedback is terrible or nonexistent

Many programs will, as David points out above, only allow a student to progress after they have gotten a certain number in a row correct. But if a student is struggling to complete an activity and the feedback to the student is terrible, how exactly are students meant to achieve the streaks necessary in order to advance?

Note: Watching a video of a concept isn’t feedback if the learner has already watched that video before. That’s information the learner already has.

 

Impossible to see patterns

One way that people learn math is by observing patterns in their work or solution strategies as they work on a set of problems in a row.For example, what pattern do you see if you try the following exercise:

5 x 3 = ?
4 x 3 = ?
3 x 3 = ?
2 x 3 = ?
1 x 3 = ?
0 x 3 = ?
-1 x 3 = ?
-2 x 3 = ?
-3 x 3 = ?

I’ve tried this with students and most of them notice that they are subtracting each time to find the next product, and so then they make a leap and decide that -1 x 3 must be -3.

But if an online practice program only ever shows one question at a time and the numbers for these questions are selected randomly, there will be very limited opportunity for students to notice and subsequently use any patterns that emerge.

 

Blocked practice

Except for a handful of studies, there is a lot of research that suggests that for most people, if the goal is to remember some mathematical idea, practicing topics in blocks will take longer than if different topics are interleaved together. Almost all of the programs out there focus on students practicing discrete topics. Caveat: I did read a study recently that suggested that for students entering a course with weak prior performance, while interleaved instruction was beneficial, interleaved practice was less effective for these students than blocked practice. Further caveat: I cannot find the link to this study.

 

Too easy or too hard

For some students the exercises are too easy. Sometimes this is because kids select easier problems for themselves, sometimes this is because students already know a bunch of mathematics and do not need this particular practice activity. Either way, needing to work through a streak of 5 or 10 problems just to be able to move on is ineffective for these students.

For other students the exercises are too hard. A student who really doesn’t know a particular area of mathematics doesn’t benefit from practice in that area – they need teaching or access to information.

 

Inappropriate medium

For many, many math problems, the best choice of a medium to work on the problem is a piece of paper. Or maybe the best choice for working on a particular problem is a programming language.

These online systems offer neither. This means students are often working in a possibly unfamiliar medium without the most useful tools available for them to work.

This also restricts the people who design questions for the system as they end up likely severely restricted as to what kinds of questions they can ask if they need the answer to the problem assessed by a computer.

 

It obscures information from teachers

If you are a teacher and you are using one of these online systems for your students to practice, there is usually a dashboard you can look at to see how well your students are doing with a particular exercise. But these dashboards truncate an enormous amount of information about the progress of learning and actually make it harder for you as a teacher to gather the information you need to be able to act to improve your students’ learning.

They also are likely to lead to teachers looking for students making mistakes instead of looking for student conceptions, which promotes a deficit view of students instead of treating students as sense-makers.

 

It can lead to bad practice

Virginia Tech has an online remedial math program where students go to sit at a terminal and watch videos on math and then take quizzes on what they learned, over and over again. There is a Facebook post where almost all of the students complain about how much they hate this mathematics class. If the online practice programs did not exist, neither would this course.

Teach to One uses a computer online practice program to inform teachers when small group instruction should occur. But in this middle to upper class neighbourhood, parents revolted and the program was scrapped. But what about districts where parents have less power?

Dan Meyer outlines the many problems with Rocketship Learning Labs, another personalized learning model, in this post.

 

It isn’t really mathematics

If you ask a mathematician or anyone who uses mathematics regularly what mathematics is, literally none of them will answer “it is a series of multiple choice questions or short response questions asked and answered on a computer screen.”  While practicing mathematics is a decent way to get better at what you know how to do, it isn’t really the goal of teaching students mathematics.

If answering a series of problems is the only experience of math that students have, they are likely to end up with a very limited definition of what mathematics actually is.

Note, I’m not opposed to students practicing math at all. This is obviously an example of good practice and there is plenty of research to support this claim. I’m opposed to this being the primary experience of math that students have.

 

Conclusion

If you can possibly avoid it, don’t use these programs. Or at least try the program yourself for a couple of hours to see what the experience is really like for students. And if you are a designer of one of these online computer practice systems, for the love of God please do a better job than the industry currently is.