Education ∪ Math ∪ Technology

Assess what students can do

An exam
(Image credit: miss_lolita)

I’ve worked on a few programming projects in the past few days, and as I’ve been doing it, I’ve realized that if someone gave me a test on JavaScript, or PHP, or creating Drupal modules, I’d likely not do very well. In fact, I might fail that test.

The problem is, when I program, I spend at least a quarter of my time looking up functions, finding solutions to problems I’m having, and just generally muddling about. My code virtually never works the first time I run through it, and I spend another half of my time trouble shooting and finding errors in my code, some of which are dead stupid. The final quarter of my time is spent on inspiration and actually writing down the code I’m producing, as well as checking that it works. Some of the projects I work on take hundreds of hours, many of which are spent when I should be sleeping. 

I’d fail that test because I don’t have enough of the programming knowledge in my head to be able to regurgitate it as needed. I might be able to rough out solutions to algorithmic problems, but I’d never produce a masterpiece as part of my exam, and I certainly would have to look up many functions and ideas to be able to finish any program.

As it happens, I do well enough on my programming to make money on the side. I actually have made many thousands of dollars over the past few years doing small programming jobs (although less recently). Why? I solve the problems people need fixing. I’m tenacious. I don’t stop when my code doesn’t work, I forge on and find a solution. I know how to find the pieces of information I need to fit into the code puzzle I’m solving.

When I think about my success in programming, and how I couldn’t easily represent my knowledge in an exam, I wonder how useful exams (particularly timed closed book exams) are for finding out about what people know. How many of us have things we "know" but for which we often have to look up additional information? A chef has a recipe book, a mechanic has a car manual, and a chemist has a periodic table of the elements (as well as countless reference books). Everyone who works professionally has some resource they access to remind them of things they know, and to fill in the gaps in what they don’t know. 

Yes, there is specific information related to programming which I know without having to look it up, but if that was all that I knew, I wouldn’t be able to successfully create anything but the simplest of programs. I’d be completely unsuccessful as a programmer based on the sum of the knowledge currently locked in my head.

We must find ways to assess what students "know" how to do which represent the actual conditions of the life they will enter. While there are a small number of times in our lives where we will have to do something without access to information outside of our head, the vast majority of the time we will be able to fill in the gaps in our knowledge whenever we need.

Should we assess students on  what they can memorize? Or should we assess them on what information they can access?