(Image source: MindShift blog)
Stephen Downes just shared this study suggesting that students see a 20% improvement in their test scores on their state exam after using an iPad loaded with HMH Fuse.
I am a supporter for using technology in mathematics education, but it’s probably worth examining these results closer. Here are some quotes from the study itself, and my unpacking of what this means for the reliability of this study.
- “Earhart has been a school eager to employ new technology in the classroom…” (p5):
This suggests a selection bias. Further, it also suggests that this program has been attempted at one school, or that all that has been shared with us are the results from one school. Were there other schools that had an opportunity to pilot this program which have not been shared in this study?
- “Coleman approached his teachers about this opportunity and two teachers, Jackie Davis and Dan Sbur, were ultimately chosen to take part in the study.” (p5) :
This suggests that the process for choosing the teachers was anything but random. The study makes careful mention that the students were carefully chosen, but underplays how teachers were selected.
- “…this meant more work and time required by the teachers…” (p5) “Like any new technology, there was a slight learning curve with adopting a tablet in the classroom. “In the beginning of the year I tried a little bit of everything, trying to find out what was best for my class and for me,” recalls Jackie Davis. Dan Sbur also found that “Over time, it became easier to use and I could use it more in my class as I became comfortable with the device and app.” (p6) :
The teachers who were chosen (or volunteered?) had to work harder to implement this program. This suggests that at least part of the effect on their test scores could be attributed to the efforts their teachers put in.
- “…This meant that students were allowed to take the devices home and “customize them,” adding their own music, videos, and additional apps. This approach also allowed students to have 24/7 access to the HMH Fuse: Algebra I program.” (p5) :
So now, are we measuring the effectiveness of the program, or the effectiveness of time spent learning math? Students who spend much more time working on math are obviously going to see an increase in their test scores.
- “As one would expect, those students who were randomly selected to be part of the HMH Fuse study were very excited – as were their parents. In fact, Coleman quickly found that one benefit of the HMH Fuse: Algebra I app was enabling parents to provide more support to their children: “Parents could watch the videos or review problems with their children to help them if they did not understand.”” (p6) :
Clearly parental involvement makes a difference in a student’s education, and if this app helps parents be more involved, that’s excellent. If this program wasn’t considered so innovative, and new, would parents be as involved? In other words, if we standardized this program, would parents get excited by it?
- “In addition, Mr. Davis found students took the initiative to use HMH Fuse: Algebra I to check their work during class, freeing him up to do more one-on-one work with struggling students in need of individual attention. In this regard, the HMH Fuse app essentially enabled a “flipped classroom” model in which students learned and worked independently at home, and then came to class ready to do problems and practice what they had learned (see Bergmann & Sams, 2011). This “flipped classroom” dynamic gave both Mr. Davis and Mr. Sbur the ability to provide personalized instruction to many students during the normal school day.” (p6) :
If the HMH Fuse app allows students to work in a more self-directed way, that’s a good thing. If their teachers are changing their pedagogical approaches to suit the affordances of the device, that’s probably a good thing too. So one wonders how much of the learning effect was due to this personalized attention. Did the two teachers in this study also find ways to personalize and give individual attention to their students in their other non-iPad classes?
One thing not at all discussed in this study is what they hope to accomplish by improving mathematics instruction. Test scores are one measure we have for mathematical ability, but they are not the only measure. Did this program give students additional time to work on improving their mathematical reasoning and their problem formulating & solving skills? Hopefully the authors of this paper will submit it for formal review so that any of the issues that I’ve addressed can be peer reviewed.
Bess Ostrovsky says:
Hi David!
This is so right! Why every time that I read your blog I want to clap my palms and smile happily?
So far I have read almost every post in any place I can find about integration of iPads in BC education. I started to do so to understand (it was my first thought that I do not get it, but it is my problem)where is a true benefit of iPads in education? Most of the articles were written by teachers/educators which have admitted at the beginning of the research/article/report that they are passionate fans of iPads. My silent question was: you already know what kind of the fish you wanna catch! And the story-tale of Andersen about naked king would try to sneak into my head…
Now, iPad is good for the future that we will not need to print books and we will have money to provide students with iPads (one more possibility: the Provincial educational administration will pretend that iPad is equal to pen and papers, so kids will purchase iPads instead of notebooks and crayons). iPads can be very efficient to get projects done too. BUT it cannot be considered as an educational tool, like we have never considered the notebook or the bag, or pens as educational tools!
Maybe someone will explain (educate me, please)what is the real reason for this passionate obsession?
January 25, 2012 — 4:42 pm
Kaitlin Boatman says:
Hi! I am Kaitlin Boatman and I am an Elementary Education major and am in an Educational Media class at the University of South Alabama in Mobile, Al. I was assigned your blog to comment on, my blog address is http://boatmankaitlinedm310.blogspot.com/ my twitter name is @kaitlinnmariee.
I agree with you when you say that this tool is very useful in the classroom, but I too wonder what they hope to accomplish by improving the math instructionu using the Ipad. I understand where the study was coming from when they wanted to use the Ipad in the classroom, today students are spending more time on technology so why not try to incorporate education into that technology? When I become a teacher I would love to use my Ipad to better my students learning, and they can have fun while doing it! Great post!
January 29, 2012 — 12:50 pm
Chris A. says:
I use this textbook series at school – if I was handed a set of ipads for my classes, this would be a nice app to have. I’m always trying to encourage my students to utilize the online textbook resources that Holt provides and this would clearly be a more accessible and interesting way to use those, also providing some extended uses on the ipad. So I’m not negating that this might be nice to have, but it’s not revolutionary.
I don’t think it’s fair to judge the usefulness of ipads as simply a way to automate textbooks. Seems like the potential for ipad use to stimulate interest, explore creative problem solving, or support differentiating instruction should involve SO much more than the use of a fancy textbook.
January 29, 2012 — 2:38 pm
David Wees says:
I think that iPads are likely a very useful tool for schools, but they are expensive, and so when we hear about a study that says that their use results in a 20% gain in learning for students, we should evaluate it carefully. What exactly caused that 20% "gain"? Was it the iPad, or the fact that the teachers using the iPad had to change their instruction to accommodate the use of the iPad?
January 29, 2012 — 4:29 pm
Craig says:
…. and could that accommodation have been facilitated without the use of iPads?
January 30, 2012 — 4:33 am
David Wees says:
Exactly. Why spend money on an expensive device if we can achieve the same aim with a different means? Let’s consider the use of iPads in the classroom to do things that we cannot already do, at the very least.
January 30, 2012 — 1:22 pm
Bryson Norrish says:
I’d speculate that the ‘forced’ reform of the teachers educational practice is a major factor in the students grades rising. How many teachers do we know who use the same materials they created in previous years, when a teacher is required to revitalize their program (and subsequently refresh their content with better teaching practice) we should expect a rise in grades.
Additionally though, I’ve found the iPad to be a hook in the classroom. Students want to use them, students find their work more enjoyable, and I believe having a ‘cool’ piece of tech is a factor in higher preformance because my experience shows students are more willing to complete assigned work because they are finding it fun.
November 14, 2012 — 3:15 am