You are here

Ineffective feedback and the Khan Academy

According to Grant Wiggins, there are seven essential elements of effective feedback.

Feedback should be:

  • goal-referenced,
  • tangible and transparent,
  • actionable,
  • user-friendly,
  • timely,
  • ongoing,
  • consistent.

 

I recently looked at the new Common Core Math section of the Khan Academy. I'd like to analyze it within this framework of feedback.

Here is a sample exercise from the Khan Academy, related to the Common Core grade 3 standard of 3.G.A.1. Not familiar with this standard? Just follow the previous link; the language is identical to what is written on the Khan Academy website.

 

Khan Academy - 3rd grade geometry problem

Goal-referenced:

There are two places where goals are embedded within this question. The first is an implicit goal, described by the Common Core standard linked to this problem. The second goal is embedded within the problem; get five correct in a row.

An obvious issue here, given that the Khan Academy is mostly intended to be used independently by students (the definition of personalized learning the Khan Academy is apparently using), the language of the standard is probably not student-friendly, and in particular it does not lend itself well to be a goal for students.

Getting five questions correct in a row is clearly a goal that a 3rd grade student can understand whether or not they have achieved, although the question remains about whether five correct questions in a row means students understand the standard being assessed by these questions.

 

Tangible and transparent:

There are two main ways students get feedback from the Khan Academy exercises; they can check their answer, and they can ask for a hint. If students click on check answer, and their answer is wrong, the button they just clicked on shakes back and forth. Presumably this means the answer is not correct, but both of these feedback opportunities are opaque.  

The box shaking back and forth does not tell the learner WHAT they did incorrectly, nor is a shaking button even explicitly a signal that something is wrong.

Next, feedback that is tangible is something a student understands. Look at the series of four hints students are given when working on this problem.

Khan Academy hints

Now imagine you are a 3rd grade student, and you do not know what a right angle is, what is meant by 4 equal sides, or what a rhombus, rectangle, or square is. How does this feedback help you understand these key concepts or the connection between these concepts and the diagrams given. Finally, telling students that the correct answer is "none of these" is not helpful feedback because it does not respond to whatever thinking a student may have done about this problem, it just states something that should be accepted as fact.

 

Actionable:

Giving students the correct answer to a problem or telling them that they are wrong does not give them actionable feedback because it doesn't give them any actions they can take in order to improve. A student who receives this feedback has no other alternative but to continue guessing until they are able to get the answers right. This does not lead to student understanding of mathematical concepts.

 

User-friendly:

The key to user-friendly feedback is to give a small amount of feedback at once so that students have an opportunity to make use of the feedback and not be overwhelmed, but at the same time give them feedback that they know how to act on. Feedback that is constructed in language students may not know, or which requires them to use knowledge they do not have is not user-friendly. What evidence do the developers of the Khan Academy have of the effectiveness of the feedback they have included in their system?

 

Timely:

What little feedback the Khan Academy exercises give, it is certainly timely.

 

Ongoing:

Again, this feedback is ongoing. If students want another chance to try again, they just need to do another exercise.

 

Consistent:

The feedback offered by the Khan Academy is certainly consistent, in fact for any given exercise, it is likely virtually identical. This actually leads to problems, since some variance in the language used in feedback is often necessary in order for students to make sense of it.

 

This analysis of the feedback offered by the Khan Academy platform suggests that their mechanisms for feedback fail on four out of seven of the criteria shared by Grant Wiggins. One open question here; is it possible for a computerized feedback system to give feedback that passes all seven of these questions? Next question; what is learned from using a system that gives poor feedback?

 

 

About David

David is a Formative Assessment Specialist for Mathematics at New Visions for Public Schools in NYC. He has been teaching since 2002, and has worked in Brooklyn, London, Bangkok, and Vancouver before moving back to the United States. He has his Masters degree in Educational Technology from UBC, and is the co-author of a mathematics textbook. He has been published in ISTE's Leading and Learning, Educational Technology Solutions, The Software Developers Journal, The Bangkok Post and Edutopia. He blogs with the Cooperative Catalyst, and is the Assessment group facilitator for Edutopia. He has also helped organize the first Edcamp in Canada, and TEDxKIDS@BC.

Comments

About the most consistent thing in Khan Academyis just dumping more advanced skills and knowledge into their problem sets and "hints" and feedback. 

He's got a pretty good explanation for figuring out areas of rectangles ... but the problem set includes areas of triangles (*and* his drawing of the problems have the "bigger" area drawn smaller than the "smaller" one, which I only noticed because commenters were confused). 

Add new comment

Subscribe via email

CAPTCHA
This question is for testing whether you are a human visitor and to prevent automated spam submissions.
ICZNMOUJOASTANG
Enter the word shown above.

Links

Subscribe Subscribe to my blog

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer