At the ASU-GSV summit this year, I spoke on a panel about the transformative nature of games in assessment. Assessment of learner performance is closely related to learning, so when we spoke on the panel, we often drifted between the two topics.
Wikipedia defines the critical attributes of games as that which has goals, rules, challenge, and interactions.
Here are the notes that I wrote in preparation for this panel discussion.
It already exists:
Game-based assessment is nothing new. My son, a swim instructor, uses games in his assessment all the time in his lessons. Instead of asking his students to put their heads under water or blow bubbles, he plays “Find the Treasure” and can quickly assess whether his students have the skills he wants to assess. The game creates a context for why performing the skills is important. Using games as part of assessment and to drive learning is older than our species!
Gradual increase in difficulty:
Games like Super Mario Brothers let you make choices and slowly and gradually scaffold the gameplay so that it increases in challenge but in a special way. The last level of any world is the hardest, but the first levels are easier and usually easier than the previous levels in the last world. Also, students can and do go back to levels they have completed over and over again because sometimes doing something better than you know you can do is more fun than struggling over and over again to complete a level you can’t yet do.
Digital games work partly because the game’s challenge level varies between easier and harder concepts, ideally moving up and down in a cadence that captures the learner’s interest and engagement.
Authentic:
We need to avoid game designs where students do some fun stuff, then pause and step out of the game to do some math problems, and then jump back into the game. This doesn’t lead to authentic learning and makes math a roadblock to having fun rather than part of the fun.
I designed a very simple graph game where students move a little stick man across a horizontal bar while trying to mimic a particular graph given on Cartesian coordinates. One axis is time, and the other is the distance the little stick person is away from their starting point. The game provides instant feedback that one has matched the graph by showing the position of the stick man in two places, one on the bar and one on the distance-time graph. It’s an extremely engaging and interesting little game, and it makes learning mathematics an integral part of the game rather than a side dish.
Not everything is a good fit for a game, but an awful lot of stuff we do is. We need to find ways to make the concepts we want students to learn authentically a part of the game and have a purpose for advancing the game. Often, these purposes can be drawn from our original invention of the concepts. When Descartes lay in bed watching the flies on the ceiling, he wanted a way to describe where the fly was on the ceiling, so he invented our modern coordinate system.
Depth of gameplay:
Really good games teach their creators, too. I once played a simple probability game where students placed ten tokens each on the numbers from 2 through 12. They take turns rolling the dice and taking off their tokens with the goal of removing all of their tokens before their opponent. After playing this for a couple of days, I built a simulation that allows students to place their ten tokens (or any arbitrary number) wherever they want and simulate playing this game 10,000 times. From this simulation, I learned that my intuition about the best distribution across the eleven numbers was wrong.
Enable connection:
We must not forget that digital games can and should enable connection between students rather than separating them all the time by screens while they work in their own environments. The best games are played with other people. Both of my sons always prefer to play their games with other people, often spending hours online playing Roblox or Minecraft. Collaborative learning is also known to be a very powerful learning experience. We should support this innate desire to connect with other humans.
Invisible:
Assessment should be invisible to the learner, but we should also give them control of the results. Every child who plays Super Mario Brothers knows which levels and worlds they have completed successfully and which ones they still have left to do, but they never stop playing the game to “take an assessment” before moving on—the game is the assessment.
Agency:
Learners should have agency. Education should be done with the learner, not to them. Games give their players choices, and these choices are part of what makes the game fun.
Student: “So, I think.. well we believe, we noticed that expression number two combines with vision number B because it’s parenthesis x plus two, right?So all the visions and all the squares are repeated three times because it’s being multiplied by three. And so the x I believe, well we believe, would be the rectangle and the squares would be the two.”
What do you think is being discussed in this example? It’s impossible to understand this conversation without a visual representation. So, let’s look at the task the students are talking about.
Now, consider the same questions. What mathematics is being discussed?
Let’s look at the task again but with some annotation meant to draw students’ attention to the salient details of the explanation.
Now imagine the conversation. What is being discussed? What strategy is being shared?
Here is a generalizable principle: if one doesn’t have a visual way for students to track a conversation, one should assume they cannot.
An instructional routine is a consistent way of interacting with learners. The primary benefits of a routine emerge when the teacher and the learners know the routine well enough that the steps fade into the background, and everyone can focus more on the ideas being shared.
All successful teachers use instructional routines, but not all routines are equally effective. Below are some routines that educators have tested, and all have one important trait in common—they treat learners as sense-makers.
Choral Counting
In this routine, students look for structure in numbers that emerge as they count those numbers as a whole group. First, given a starting number and an amount to count by, the class counts together as an entire group, then the teacher pauses the class to give students opportunities to look for structure. Students then share some of the patterns and relationships they noticed.
This routine, developed by Amy Lucenta and Grace Kelemanik, asks students to use mathematical structure to connect two visualizations of the same mathematical idea represented differently.
After this activity is launched, students are presented with two different kinds of representations. They then make matches between one type of representation and the other. Once they have matched made with their partner, the teacher orchestrates a classroom discussion where students describe how they made their matches. Typically, a representation goes unmatched, so students create the missing representation next. Finally, students reflect on what helped them make their matches — what might be helpful to pay attention to next time.
This routine, also developed by Amy Lucenta and Grace Kelemanik, asks students to use mathematical structure to create shortcuts for calculations.
After this activity is launched, students are given a glance at a mathematical image. They are asked to share what they notice while their teacher records these noticings for everyone. Students are then given the mathematical image again, this time the remainder of the class, along with a question to address related to the image.
Angela Chan, Megan L Franke, and Elham Kazemi describe this routine in detail in their book Choral Counting & Counting Collections. In it, students count objects and then represent how they count them.
The magic of this routine is in the time spent by educators watching students count and keeping track of students’ ability to use one-to-one correspondence, mathematical structure, place value, skip counting, etc… as they count.
The goal of a number talk is usually to expose students to multiple strategies for solving the same mathematical problem. Students are given a mathematical problem to solve, usually a problem involving arithmetic or counting, and asked to come up with a solution in their heads. Once sufficient numbers of students have indicated that they have a solution, the teacher leads a discussion where several solutions are shared.
This routine is terrific for celebrating learning that has occurred but doesn’t necessarily press students into using new strategies. For that, one wants to use a problem string instead (see below).
A problem string is a deliberately selected sequence of problems given to students, one at a time, to help students develop a new mathematical strategy. They are similar in how they are run to number talks but are not necessarily restricted to arithmetic or counting problems. Problem strings lend themselves well to all areas of mathematics.
Pam Harris has some resources for problem strings on her website here. She’s also written a terrific book on problem strings in high school. There are resources here on implementing a specific kind of problem string called a number string here: https://tedd.org/number-strings/
Three Reads
Three Reads is a routine used to help students make sense of contextual problems and learn how to deconstruct mathematical problems they have read. First, the routine is launched with an explanation of what students will work on, why they are working on it, and how they will work on it. Next, students read the same mathematical problem three times, each time for different types of information. Finally, they share what they understood from their reading with each other. Over time, students get better at mathematical reading.
Amy and Grace wrote a chapter on this routine in their book Routines for Reasoning, and I highly recommend reading their book to learn more about it.
Which One Doesn’t Belong?
The goal of this routine is to spark mathematical creativity, give students opportunities to construct mathematical arguments and show them that there are many mathematical questions one can ask that do not have a single correct answer. The most important part of this routine is the argumentation students develop as they justify their choice.
Students are given four different mathematical objects in a collection and asked to explain which one doesn’t belong. Usually, students do this first on their own, share their ideas with a partner, and then their teacher leads a mathematical discussion based on their ideas.
In this routine, students are presented with examples and non-examples. They analyze each pair and come to a better understanding of the concept presented by the examples.
Using examples and non-examples is perfect when you have a mathematical concept for which you want students to have a definition. This is especially helpful as students often struggle to understand definitions given to them without sufficient explanation.
A guide for getting started with this routine is available here.
Sharing Skepticism
The Sharing Skepticism routine is intended to support students in constructing arguments, critiquing each other’s arguments, and reflecting on what makes an argument good. Over time, as students debate which arguments are more convincing, they will develop the habits of mind necessary to construct mathematical arguments. The overall structure of the routine is for students to convince themselves, convince a friend, and then convince a skeptic.
First, the routine is launched so that students know why they are sharing skepticism today, what they will learn, and how the routine proceeds. Next, students solve a problem independently and then share their solution with a partner. Two or three solutions are presented to the whole class; then, students work with a partner to select their favourite argument and try to improve it somehow. Some ideas for improvement are shared with the class, and then students reflect on their experience and consider what makes a good argument.
Any task that every student can devise a strategy for solving, which requires some level of mathematical thinking and has multiple strategies for solving, can be used as a task for this routine.
More information on this routine is available here.
Paired Examples
This routine aims to help students make connections between different mathematical concepts, allowing them to explicitly build a procedure or algorithm from something they already know.
First, the routine is launched, and then students are presented with the first step in each pair of mathematical procedures. Students look for and name connections between each step. Each step is unveiled in the same way, with students sharing the connections they notice in a structured way with the whole class. Once the entire procedure is unveiled, students are presented with another example of the new procedure for them to try, first independently and then working with a partner. A couple of these new procedures are selected to be shared with the whole class. Finally, students reflect on what they learned from the activity.
This routine is ideal anytime there is a mathematical procedure one wants students to learn when there is an early similar mathematical procedure students have already learned. The earlier example does not need to be identical to the new procedure. Students noticing the differences between the two can be as powerful as noticing the similarities.
Further resources related to this routine are available here.
Language Routines
There are times when the focus of a mathematics classroom is on building language, both the academic language of the discipline of mathematics and the everyday language needed to converse about mathematical ideas with each other.
For those teachers who teach multiple subjects, the subject-agnostic group learning routines developed by All Ed may be ideal. Each routine is described in detail, and different routines are provided for different use cases.
One of my favourite group learning routines is Idea Carousel, and I frequently use it in professional development sessions. In this routine, learners have a big idea they are unpacking, and the components of this big idea have been separated into separate posters. Learners are given one marker for each group and rotate through the posters around the room, swapping the marker to another recorder on each rotation. As learners circulate, they add detail and annotate the previous ideas added by other groups. At the end of the carousel, the teacher leads a whole group conversation based on the posters.
A sample poster from one of many workshops.
Routines for Teacher Learning
While all of the routines I’ve described before lead to educator learning, there are specific routines designed for people who facilitate teacher learning to use.
I asked ChatGPT to write a TED talk about why we need to change how we teach math. Here’s what it came up with.
Title: The Mathematics Revolution: Unlocking the Power of Learning
[Introduction]
Speaker: Good morning, ladies and gentlemen! Today, I stand before you to ignite a spark of curiosity, to challenge the conventional wisdom, and to advocate for a fundamental change in how we teach one of the most critical subjects in education: mathematics.
[Opening Hook]
Speaker: Imagine a world where mathematics is no longer a daunting, mysterious puzzle, but rather an open gateway to creativity, critical thinking, and problem-solving. Envision classrooms buzzing with excitement, where students are not merely memorizing formulas but actively engaging in the beauty and applicability of mathematics. This is the vision I propose we embrace today.
[Establishing the Problem]
Speaker: We live in an era of unprecedented innovation and technological advancements, yet our traditional approach to teaching mathematics has remained largely unchanged. This outdated method focuses on rote memorization, stifles creativity, and fails to inspire a genuine passion for the subject. It’s time for us to acknowledge that this approach is limiting the potential of our students and hindering progress in a rapidly evolving world.
[Point 1: Shifting Focus from Answers to Understanding]
Speaker: Our current education system fixates on finding the right answers, often neglecting the importance of deep comprehension. Mathematics is not merely a series of procedures to reach a solution; it is a language that empowers us to decode the complexities of the universe. We must shift our focus from answers to understanding, encouraging students to explore the “why” behind mathematical concepts and nurturing their ability to think critically.
[Point 2: Embracing Real-World Applications]
Speaker: Mathematics is not an isolated subject confined within the pages of a textbook. It permeates every aspect of our daily lives, from calculating the optimal route to work to analyzing statistical trends in healthcare. By integrating real-world applications into the curriculum, we can show students the tangible relevance of mathematics and inspire them to apply their skills in meaningful ways. Let’s bridge the gap between theory and practice, allowing students to see the direct impact of mathematics in fields like technology, finance, and even art.
[Point 3: Fostering Collaboration and Creativity]
Speaker: Mathematics is often wrongly associated with solitary and rigid thinking. However, it thrives when we encourage collaboration and embrace creativity. By introducing group projects, open-ended problem-solving, and mathematical investigations, we can cultivate an environment that nurtures teamwork, communication, and innovation. Let’s unleash the power of collective thinking and teach our students that mathematics is a dynamic, collaborative endeavor.
[Point 4: Redefining Assessment]
Speaker: The conventional method of assessing mathematical proficiency through standardized tests falls short in capturing the true essence of mathematical competence. We need to redefine assessment to encompass a broader range of skills, including problem-solving, critical thinking, and logical reasoning. Embracing alternative assessment methods, such as portfolios, projects, and presentations, will provide a more accurate representation of a student’s mathematical abilities and foster a holistic understanding of the subject.
[Conclusion]
Speaker: Ladies and gentlemen, the time for change is now. We have the opportunity to revolutionize mathematics education and unlock the immense potential within our students. Let’s move away from a system that stifles creativity, instills fear, and perpetuates the myth that math is only for a select few. Instead, let us embark on a journey of discovery, exploration, and collaboration, where mathematics becomes a tool for empowerment and innovation.
Speaker: Together, let’s create a future where the beauty and practicality of mathematics are celebrated, where every student is equipped with the skills and confidence to tackle the challenges of an ever-changing world. Thank you!
[End of TED Talk]
Here’s a video of the TED talk that AI also created using the same script above.
Neither the script nor the video are amazing, but one doesn’t need much imagination to see that some of our ways of working are about to change pretty dramatically.
I’ve seen a lot of interest in applying artificial intelligence to education, but the efforts appear to be starting in the wrong places.
What is being built
We have AI being applied to write lessons for educators. These early efforts create lessons that are not likely helpful for their target audience — educators who need lessons targeting specific concepts for which the educators do not already have lessons. The most likely educator in this situation is an early career educator or an educator teaching a new grade level or course. In the long run, neither of these groups is well-served by poorly written lessons by a “novice” AI curriculum designer.
This sample prompt might give an educator an idea of how to generate this task, but it isn’t something that anyone should put directly in front of students.
We have AI being used to tutor students. Dan Meyer wrote a great post outlining one of these efforts. The critical issue is that the AI tutors are fumbling around in the dark, trying to support students without understanding their needs. Where are the supportive visuals? Where are the simplifying examples? Why does every tutoring program rely almost entirely on questions?
We have AI being used to grade students. This is a time-consuming task for educators, to be sure, but auto-grading forgets the key reason educators do the task in the first place—to learn more about their students! A summary report reveals little about student learning. By removing educators from the job of looking through their student work, we make educators blind when they plan future lessons for those students.
This is my first attempt at using a custom AI to create an assessment on the Common Core standards K.CC.A.2, K.CC.C.7, and K.MD.B.3. This isn’t even in the ballpark of useful.
What should be built
AI categorization of student thinking I read a fascinating tweet the other day about an AI being used to translate between two languages. It made me wonder—can AI be used to translate between the language students use when initially approaching a concept and the language educators as people who have already mastered the concept?
Students use their partial understanding to make sense of concepts, which results in them using novice language to describe their ideas. Frequently, the best person to explain a concept is someone who just learned it rather than someone who knows it so well that they forget what it felt like when learning it. This is part of the reason group work works.
This novice language can be hard for educators to understand, especially when they have many students in their classes with different ways of understanding an idea. Decision-making about what to do to support students at the moment is hampered by this challenge in translation. What if AI could be leveraged to translate what students say into how it connects to what educators know about the topic? This would help educators build what Deborah Ball calls Mathematical Pedagogical Knowledge for Teaching, but more immediately, it would help them make better decisions about supporting their students.
Another main reason to start with a project like this is that AI tutoring and other educational uses of AI almost all depend on this capacity.
Insights about student learning from student work
What are these students thinking? What strategies are they using?
This is a task every mathematics educator needs to know how to do. Take a sample of student work, figure out what students are doing, and then make a plan of action around this thinking.
It is also a task on which AI could shine. Most AI programs are well suited to categorization problems, which is exactly what this is. Large samples of similar student work on similar problems are impossible for educators to analyze, except over a lifetime of working with many students, but AI would trivialize this task.
Note that this task is fundamentally different than auto-grading work. Grading is about evaluating work; here, we want to understand the work and how it connects to our goals for teaching.
The Future
It is obvious that AI has great potential in education. Who is working to realize that potential?
One of the most challenging topics to teach in high school mathematics is “Completing the Square.” This is because educators do not always fully understand the topic and because it includes several algebraic steps that are incredibly challenging for students.
An eye-opening experience was when I first generated visuals for each step in completing the square using an area model.
Algebraic Step
Visual Model
x² + 6x + 8
x² + 6x + 8 + 1 – 1
x² + 6x + 9 – 1
(x + 3)² – 1
The name “Completing the Square” is not arbitrary! Visually, one can see that we are literally taking an incomplete square (at least in cases like the one above) and making it complete. This visualization makes the algorithm’s goal obvious and helps students see what they are trying to accomplish.
However, the steps above are not sufficient for students. The area model above is much easier to understand if you already know the mathematics it represents. Since children don’t, we need to introduce the area model with more straightforward examples before using it for more complex ones. Here’s a worksheet that aims to do this.
Once we have used the area model to establish the purpose of the steps when completing the square, we gradually remove the visuals. This is because we don’t want students to need to draw out the visuals each time to help them solve the equation. The goal is for students to understand conceptually what completing the square is and what steps are needed to complete it. The visuals are an aid for this goal.
There are other cases to consider (for example, expressions like x² + 6x + 10 and 2x² + 6x + 10). Once students have a handle on the simpler cases, these examples will be easier for them to manipulate algebraically.
The key idea here is that some ideas are much more obvious when represented visually than when we focus purely on a symbolic approach.
