Why Do We Compare Students to Each Other?

Why do we use so many norm-referenced standardized tests for accountability when what we really care about is learning?

And learning is much easier to measure with a criterion-referenced test.

When a teacher gives an assessment in the classroom, that is almost always a criterion-referenced test or assessment that measures specific content.

Are students able to demonstrate mastery of that content?

That is a criterion-referenced test.

And if you go to the driver's license office and take the driver's license exam, you have to get a certain score, but your score is not compared to other people's scores, right?

Like, you just get a certain number of questions right and wrong, and if you score high enough, you pass.

Otherwise, you don't pass.

But the standardized tests that are used for school accountability don't work that way.

Tests like the SAT and the ACT that are used for college admissions don't work that way.

So many tests, including the state accountability tests that schools are measured with, are norm-referenced tests.

And in a norm-referenced test, students are compared to one another.

There's a norm, there's like a national average or an average score for the test, and you get a percentile on that test.

And a lot of the tests Now that we even use for progress monitoring, like MAP, are norm-referenced tests.

And we say those measure growth, but they don't really measure what students have actually been taught and what they've learned in the same way that a criterion-referenced test does.

And I think this matters a great deal when we think about raising test scores and our expectation that schools should be able to raise test scores.

Because if we're talking about a norm-referenced test, what we're saying is we want more of our students to be above average.

And I'll give you a second to think about that when it comes to accountability.

Like if you want more of your students to be above average, how would you do that?

And we've seen some notable examples where schools have taken steps to manipulate their student population and say, okay, we can't really make more of our students above average.

So what we're going to do is we're going to try to change who our students are.

We're going to disincentivize some of our students from coming back.

We're going to try to incentivize maybe more involved parents to come to school here because their kids are probably going to have higher test scores.

Like there ends up being all this distortion and manipulation and gamesmanship when it comes to trying to raise norm-referenced test scores.

Like you just can't raise norm-referenced scores that much, but you can raise criterion-referenced test scores quite a bit by teaching better, by learning more.

And there was actually a pretty big controversy this year in AP US History circles I saw some accusations from people who had scored the AP US History exam that there was pressure in the organization that administers that to not give too many passing and high scores.

Like the number of students who are passing AP US History is going up and there is internal pressure, so I've heard, to keep those scores down.

So let me know what you think about these incentives.

I think we've got to be really careful about norm-referenced tests when maybe we should be using criterion-referenced tests.

Let me know what you think.