Education ∪ Math ∪ Technology

Tag: bias (page 1 of 1)

Bias in assessment

Every form of assessment of learning has bias. This bias may be hidden, or it may be quite obvious. As Cathy O’Neil points out, assessment is a proxy for what we want to measure – learning. We cannot measure the building of connections between neurons that is happening in the brain directly (or even potentially understand what that growth even means) so we use a proxy in the form of an assessment of the externally visible signs of learning.

One bias therefore is our tendency to forget that we are not actually measuring learning, we are measuring a proxy for learning.

Another bias is the language we use to do the assessment, whatever form that language takes. Language is necessarily a construct of our minds and no matter the appearance, it has deeply personal meaning based on our own experiences. As such, our use of language to communicate an assessment to a student contains an inherent bias based on our understanding of the language we have used.

The medium of the assessment also introduces bias. An assessment done on paper on pencil is limited to what can be collected in this form. As Dan Meyer and Dave Major are showing, digital assessments have different affordances (and different biases) that can change how students can interact with the assessment. Students who share their understanding verbally may have a very different explanation than if they write down their explanation.

There is further bias in the assessment based on our interpretation of the results of the assessment. If one knows the names (or race) of the people doing the assessment, is one sometimes more or less lenient? Look at this video of people assessing whether or not someone is stealing a bicycle. Do they exhibit any bias? Is it possible that teachers may experience similar bias (hopefully to a lesser degree!) when examining student work? How much of a difference to the student’s marks does it make if the work is messy or neat?

There are no doubt other biases in our assessments that I have not mentioned.

To combat bias, we must be first aware that it exists, and next that we should look at our assessments and ask ourselves, what bias is likely to exist in this form of this assessment? Can I address this bias? If not, can I use a second assessment in a different form and compare results?

“Thin slicing” and its effect on educators.

I’m reading Malcolm Gladwell’s book, Blink.  He uses a variety of arguments to show the power of information we can receive from a very small amount of information, and the subconscious ways we make decisions very quickly.  It’s a fascinating read, I highly recommend checking it out.

One point he brings up a fair bit in the first half of the book is the value in making quick decisions based on limited information and how it can both be extremely beneficial for making decisions but that it can also be problematic occasionally.  He calls this process thin-slicing.

First, the power of thin-slicing is that it allows you to make decisions quickly.  You can also trust your instincts when making decisions about directions of programs, assuming you have the expertise in that area.  His book shares some research that shows that people who are experts in their area can make a judgement about their area of expertise in 10 seconds, just as easily as in 2 hours, or 3 days.  The amount of time necessary to analyze a situation and think about a recommendation is very short, with high levels of accuracy achieved in a short period of time.

However there is a darker side to this story.  Unfortunately in areas in which we are not experts, or where there is a great deal of cultural stereotyping, our unconscious decision making can betray us.  The messages which are broadcasted into society subliminally can cause us to make unconscious decisions and snap judgments that betray what our personal sense of morality would lead us to believe.  Essentially, we not only all judge a book by its cover, but we color all of our analysis of the book by the decisions we made when we saw the cover of the book.

Most educators in North America would probably not consider themselves racist but our society sends out messages of racial stereotypes on a regular basis.  We read statistics about how 5% of African American men are incarcerated and recognize that this is much larger than the number of people from any race, and we make presumptions about African Americans.  Of course there are lots of other examples of racial stereotypes in our society, most of which are present on television.  

Now as teachers, we are likely to believe that people are deserving of equality and we almost certainly are not consciously aware of our bias.  If asked, we will say that all of our students deserve equality, and that we should treat them equally.  We may need to give some our students more attention than others because of their individual needs, but we wouldn’t openly treat them differently.

Unfortunately, we cannot avoid making snap judgments about students based on our prior experiences, and the influence of the stereotypes in our society is strong.  These snap judgments will colour all of our interactions with our students and can prevent us from treating them as fairly as we would consciously like to do.  We may downgrade papers from students whom our cultural stereotypes say are supposed to struggle with literacy, or treat unfairly students who may differ in their external packaging (their dress and mannerisms) than their peers.

There is a solution to this problem, or at least a way to make the process of educational evaluation more fair for all students involved.  One of the stories Malcolm Gladwell talks about is how the orchestras around North America and Europe have been transformed by blind auditioning.  Apparently as recently as the 1980s and 1990s, most orchestras were predominantly filled with men, and women had difficulty advancing in this area.  Orchestras recognized this (you have to read the book to find out how they recognized the problem) and began to institute policies that required the gender and race of the musicians to be hidden during the auditions.  In only a few short years, the problem of diversity in orchestras has begun to be solved.

So what can we do as educators? Any time you evaluate student work, make sure the identity of the students is unknown to you while evaluating it.  Have students turn in their work with a number which is matched to their name (randomized for each assignment) and in electronic form to avoid recognizing hand-writing.  After you have read and graded each piece of work, match the numbers to the students and record the grades or feedback.

My guess is that if you institute this policy, you will be surprised on a regular basis of the quality of work that is produced both by your supposed superstars and your weaker achievers.  You will also begin to lose some of your bias as your professional experiences begin to overcome the initial stereotyping to which you have been exposed in society.