Focus on Process Accountability, Not Outcome Accountability

Holding people accountable for outcomes in education turns out to have not been a great idea, and there are a couple of reasons why.

One is the fact that almost all measurable effects in education statistics are selection effects.

In other words, they're not measuring what people did, they're measuring who you measured in the first place.

And in a lot of cases where there seems to have been some sort of dramatic change or dramatic result, coming from some improvement that people made in a lot of cases you will find that the people involved changed or the students who were included in the school changed there was some sort of change other than what people were doing and what we want to learn from any kind of education research and what we want to emphasize in any kind of improvement work is is what we can do differently right the reason selection effects are so disappointing to us like we're not really interested in selection effects is because they're not something you can do And a lot of schools have honestly touted amazing results that supposedly come from what they're doing that are actually coming from how they're manipulating their student population, right?

There are lots of ways that schools can be selective about their students, even if they're not allowed to be selective about students or about families.

There are little tricks.

that schools have figured out that allow them to make it seem like they're having a bigger impact than they are.

And when we don't really have control over outcomes too much beyond selection effects, what happens is according to Campbell's law, people cheat, people manipulate the statistics in various ways.

And we're seeing that subtly as we always have with the selection effect manipulation, but now we're also seeing it blatantly as in, uh, grade inflation, right?

People are making up points to give students on assignments they didn't do.

Uh, I'm waiting for the day when we have attendance inflation, when we're taking students who are absent and marking them as present.

I feel like it's just a matter of time until the news of something like that breaks.

And we also, of course, have discipline statistic inflation, or since discipline statistics are negative, deflation, where we take bad things that have happened and we downgrade them.

We don't suspend kids for them, even though students should be suspended for them because they'll make our statistics look bad.

Ultimately, in education, we want to improve the underlying reality.

We want to improve students.

the experience of students and teachers.

We want students to learn more.

But when we're held accountable for specific measures, it's too tempting.

Campbell's Law tells us it is too tempting for people to manipulate those measures rather than change the underlying reality, which despite our biggest efforts, may not actually change the numbers very much.

So here's what I think we should be doing instead if we want to improve teaching and learning, if we want to improve the student experience, if we want schools to be healthier places, is we should focus on process.

We should focus on what schools are doing to make people safe, to guarantee that students learn, to intervene when students aren't learning, to support students even when they're struggling.

If we just focus on the outcomes, it is very, very likely that people are going to cheat, people are going to manipulate various factors and try to get those outcomes by any means necessary, and that is unlikely to actually improve things for teachers and students.

Let me know what you think.