The Perfect Assessment System

[00:01] SPEAKER_00:

Welcome to Principal Center Radio, bringing you the best in professional practice.

[00:06] Announcer:

Here's your host, director of the Principal Center and champion of high performance instructional leadership, Justin Bader. Welcome everyone to Principal Center Radio.

[00:15] SPEAKER_01:

I'm your host, Justin Bader, and I'm honored to be joined today by Dr. Rick Stiggins. Dr. Stiggins is founder of the Assessment Training Institute and the author of the widely respected book, Assessment for Learning, as well as his new book, The Perfect Assessment System from ASCD, which we're here to talk about today.

[00:36] Announcer:

And now, our feature presentation.

[00:39] SPEAKER_01:

Dr. Stiggins, welcome to Principal Sam Radio. Hi, Justin. Glad to be here. Thanks for inviting me. Well, let's talk first about where this book comes from.

[00:46]

Now, you've written one of the kind of definitive books on classroom assessment. What prompted this new book, The Perfect Assessment System?

[00:53] SPEAKER_02:

You know, it's a culmination of about 30 years of frustration. And, you know, I spent the early part of my career, I finished my graduate work at Michigan State, PhD in 72, all those years ago, and did advanced psychometric work after that for a few years. And then something really important happened to me. I became a dad. And I had a chance to watch through 13 years of public school education as our daughter lived the assessment experience of the classroom. And about that time, my career reoriented.

[01:22]

And I've spent the last 35 years trying to understand the task demands of classroom assessment and help teachers and school leaders manage those task demands effectively. While all that was going on, I watched the accumulation of high pressure, high stakes district-wide, statewide, national, international standardized testing and eating up all the resources with leaving nothing to support teachers as they face the task demands of classroom assessment. And this book, The Perfect Assessment System, is the culmination of that frustration. I'm saying stop it. And we can talk more about what stop it means because that's really important. But it is the culmination of a very long emotional experience of growing in the profession of educational testing.

[02:06] SPEAKER_01:

Well, let's talk about the era that we are just now leaving, the era of No Child Left Behind and high-stakes testing, which coincided with my entire career in public education. I became a teacher, became a principal during the era of adequate yearly progress and you know, annual high-stakes testing. And now we're seeing a shift away from some of those mandates to put all of our efforts into the standardized testing bucket. And yours has always been a voice calling for classroom-based assessment, you know, assessment for learning, assessment designed by teachers to inform teachers' decisions, to help students improve their learning. What did you see happening during the No Child Left Behind era That kind of got us to the the situation that we're in today. What what occurred in our profession?

[02:53] SPEAKER_02:

It clearly is the case that the frustration built in our society about that kind of testing I need to say here at the outset that I'm not opposed to annual accountability testing I think under the right conditions it can be very helpful but what became painfully obvious to us is that is that that system is out of balance and where all of the resources tip toward meeting the information needs of policy level decision makers. And nothing's left in the balance for the other key decision makers, primarily students and teachers and parents to a certain extent also. So I think what we realized as a society is that we need to balance our assessment systems. And that's what the perfect assessment system is about. It's about, it defines assessment as As gathering information to inform instructional decisions, then the key matter is who are the decision makers?

[03:47]

Well, it turns out there's a wide array and their information needs vary profoundly. We build a perfect assessment system when we build it to meet the information needs of all users, not just those who hold the purse strings or have political power. It's not that they're unimportant. It's that their decisions compared to those made by students and teachers pale in terms of their importance in terms of promoting student academic well-being. I'm not the only one that realized that. I think we are a society that's realized it.

[04:14] SPEAKER_01:

Well, let's talk about the frontline users of assessment, because as you said, we've had a lot of attention paid to federal policymakers and state policymakers and district administrators in terms of their need for assessment information about what's happening in schools. But on the front lines, what do you see as some of the biggest opportunities for our profession to make better use of assessment in the classroom with students and teachers? That's a huge one.

[04:38] SPEAKER_02:

There's one huge thing we can take advantage of. We've been operating for decades on the belief that assessment is something adults do to students. And they do, and they need to do it well. But in addition to that, what we failed to understand and are beginning to understand very keenly now is that students are data-based instructional decision makers too. They read their evidence. the evidence of their own performance and make critical decisions about what to do about that evidence on the basis of that evidence.

[05:11]

They're asking questions like, can I learn this or am I just too dense? Or is learning this worth the energy that I'm going to have to expend to get it? Or think about the struggling learner. Is trying to learn this worth the risk that I might fail again in public because that failure just hurts too much? critical decisions. The power we have in our hands now is to use the student's role in the assessment process as an immensely powerful teaching and learning tool that has quite literally yielded results in terms of profound achievement score gains around the world.

[05:46]

In the perfect assessment system, what I'm saying is let's balance our assessment systems to meet the information needs of policymakers and school leaders, to meet the information needs of teachers, and to meet the information needs of students. And when we build a system like that, we'll have something going for us. We've been so far out of balance that it just makes me sick.

[06:03] SPEAKER_01:

And I think, you know, as any kind of district administrator can appreciate, we do need that kind of school-to-school comparison information. We do need that kind of standardized test information to figure out where we have inequities, where we're not serving our students well. But we know that in schools, things happen a lot faster than those systems can really measure. And we've got to give teachers the tools, give them the opportunity to act.

[06:26] SPEAKER_02:

As a matter of fact, that typical teacher makes instructional decisions at the rate of one every two to three minutes. Is it any wonder, therefore, that a once-a-year test isn't of much value? That's fine. It's valuable to somebody else. But what we need to make absolutely certain of, and what I talk about in the perfect assessment system is, how do we make sure teachers have in hands the tools they need to, A, gather good evidence of student learning? You know, 99.9% of the assessments that happen in a student's life happen at the behest of their teacher.

[06:53]

And so they need to know how to do that well and then how to use the assessment process and its results to promote student success, not just to measure it, but to bring students into the assessment process so they can team up with their students to manage the learning and help kids believe that success is within reach for them. When we get that kind of a perfect system in place that balances all of that and gets the student teacher team working powerfully like that. We know how to do that if we could just get people the opportunity to learn how to do it. And that's the big bugaboo. It's very hard to get people the time to learn how to gather quality data and how to use the assessment process in terms that I'm talking about. If I were king and I would decree the allocation of resources, it would be in professional development.

[07:39]

I know you've been involved in professional development to get people the time and opportunity to learn both teachers and school leaders how to use the assessment process in these wonderful, new, almost revolutionary ways.

[07:51] SPEAKER_01:

And we definitely see threads of that in all of, I would say, all of the great movements in education over the last decade or two. We see the importance of assessment in John Hattie's research on effect sizes with feedback being at the very top of his list of effect sizes, you know, bigger than any other by a pretty substantial margin. We see that in the professional learning communities structure that has taken the profession by storm over the past two decades. and we see it you know just in in terms of mindset and and self-efficacy and i was wanting to draw a connection to student self-efficacy uh and that idea of a growth mindset that if if we give kids the opportunity to see that they can learn we have the potential to undo a lot of the messages that they might have received up to that point in life or that they might have been telling themselves uh or you know based on previous school difficulties

[08:45]

I think there's just tremendous potential to build student self-efficacy through the kind of work that you're talking about.

[08:50] SPEAKER_02:

And to use a sports metaphor for just a minute here, what that translates into in the classroom is get kids on winning streaks and keep them there. Because if they are on winning streaks and believe that success is within reach, they can handle the occasional setback, you know. But if they're on what they believe to be an inevitable tumble of failure, then negative feedback is destroyed. and just drive them out. But you're absolutely right, Justin. The emotional dynamics of this experience, the assessment experience, from the student's point of view, is something we have, A, neglected, and B, really need to pay careful attention to.

[09:26]

We know how to do all this stuff. There is, and you mentioned the PLCs is important. We have a professional development program literally built for professional learning communities around developing this kind of understanding. If we could just give people the opportunity to learn. And I just continue to think we're going to get there. That's got to be our priority for the kids' sake.

[09:47] SPEAKER_01:

I want to ask an opinion question here about who designs assessments and who should be designing assessments, because there's been a big kind of land grab, I would say, in the last 10 or 15 years into formative assessments. And it used to be, at least my recollection is that formative assessment used to be something that teachers came up with and that teachers did. And now we're seeing, you know, you can't go to a conference without seeing a vendor booth pitching formative assessments. And we have all these overlapping terms. We've got somewhat standardized assessments that we give multiple times throughout the year. What is your definition of a classroom-based formative assessment?

[10:24]

Because in my experience, that's something the teacher does, but now I see I can go online and buy those. Help us out.

[10:31] SPEAKER_02:

The marketplace responded to the growing priority. If there's money to be made in it, there are people who make a lot of money in the testing industry. They have for decades. And besides that, people who work in that context are kind of trapped by a sort of a 1950s version of excellence and assessment where the belief is, well, teachers can't create them and make them of high quality. We better create them for them. Oh, and let's sell them to them.

[10:57]

I'm not opposed to that if those assessments are of high quality. And now here's the key point. If they fit into the learning progressions that teachers happen to be involved in in their classrooms, But the greatest sensitivity to those professions and what's needed in order to determine students' needs to help them move forward in a student-involved way are assessments that are created by teachers when they need them. Now, to the extent that we can help them by creating quality assessments in the context of text materials, or I'm okay with commercially produced formative assessments, but they've got to be designed to fit into instructional progressions, not just overlaid on top of them. That's been a problem with our annual standardized tests. We have instruction going on, and then we impose a test, and whether it reflects what's been taught or not, the score stands.

[11:47]

Well, that can't help in the formative context. Formative assessments is gathering information to inform instructional decisions by students and teachers while they're learning. That's a continuous flow. Teachers need to be in charge of that.

[12:03] SPEAKER_01:

Right. And I think some of the kind of jumble of information that we're getting throws us off a little bit because, you know, at the district level, I know a lot of districts are getting better about collecting more frequent information. But it's still not necessarily curriculum based information like the NWEA map test would be a good example. I know it's used in districts all over the country. We used it very good. sense of, you know, comparison for a student in terms of their growth over time, as well as comparisons between classrooms and between schools.

[12:33]

So it gives us numbers that mean something that are comparable to each other. But what it doesn't give us is any information about the top curriculum. And I think that's that's part of what we're getting. Right.

[12:45] SPEAKER_02:

And to the extent that tests like that can be encouraging, like we really are growing. I'm fine with that. It can be part of the balance. But what we need to understand is that learning is almost a moment to moment thing. And what we need to understand is that the level of precision that's needed in information. I mean, you're a teacher.

[13:07]

You know that the decision making can unfold almost spontaneously sometimes about, oh, my God. We need to take some action on this. That level of precision requires that teachers have the capacity to create and use assessments. They may be micro assessments. They're going to be formative. Ultimately, teachers need to be accountable for summative assessments, too, for grading or standards-based report cards.

[13:30]

But there's a level of precision, a level of frequency, a level of precision you simply can't buy. It has to be managed by the leader, by the coach. You know, I think of assessment for learning as being like a GPS system for learning. You know, I have a boat, I fish some and I've got a GPS system in my boat and I can go out from the dock and then I can follow the waypoints right back to the dock even if the fog comes in. Well, assessment for learning is about that and my demonstration of what it means precision. I need to be able to take my students from where they are now to the dock and that requires that we go from waypoint to waypoint.

[14:05]

And we can only do that if we have a clear sense of that progression and how to get there on a step-by-step basis. And when we think in those terms, we don't think about periodic assessments, although they can help. We don't think about annual assessments. We think about classroom assessments.

[14:21] SPEAKER_01:

Right, and you talk a lot in the book about learning targets and learning progressions and then kind of scaffolding students through those progressions.

[14:29] SPEAKER_02:

Yeah, the key points of the book are we've got to be clear on purpose and balanced. We've got to be clear about our learning targets and share them with our students from the beginning of the learning. We need quality assessments and the perfect assessment system. We need effective communication systems, formative and summative. Both are important, but they're different. And then we want to weave the student deeply into the assessment and learning process while they're learning in the classroom.

[14:52]

If we put those pieces in place, we'll have, I promise, a perfect assessment system.

[14:57] SPEAKER_01:

So let's play a game that I like to call the magic wand. If you could wave a magic wand and get all school leaders everywhere to do something, to take some action to move in the direction that you would like in terms of the perfect assessment system, where would you start? What would your first wish be when waving that magic wand?

[15:15] SPEAKER_02:

I would want all of them to step up. and take responsibility for the development of their own assessment literacy and the promotion of assessment literacy of all around them. They may be non-educator policy makers on the school board. They may be teachers. I'm saying that in the perfect assessment system, there needs to be in place a universal foundation of at least a basic understanding of the principles of sound assessment. This is not something that has traditionally been woven into teacher prep programs, although yours sounds like it was the exception, and that's great.

[15:49]

Relevant, helpful assessment training remains almost nonexistent in leadership preparation programs across the country. And yet in these times, I don't know how that's even justifiable. The typical teacher spends a quarter to a third of her or his available professional time involved in assessment-related activities. Most do it without having been trained to do it well. Those in leadership positions, I would have them step up and make sure we determine what it is that teachers need and make sure they get the learning time that they need to do it. If we can put this foundation in place, that will underpin what I'm calling a perfect assessment system.

[16:24] SPEAKER_01:

And what I'm hearing you say there is that leaders go first and that teachers have to understand what they're asking students to do and leaders have to understand how those systems are designed. By and large, it's not happening. I think something a little bit terrifying to me is happening where we as leaders are taught how to hold teachers accountable. but we're not taught how to develop the measures that we're using. And in a lot of cases, if I can step onto a soapbox briefly, we're evaluating teachers based on systems of assessment that nobody even understands. I mean, maybe you and Jim Popham and three other people in the world understand them, but I can say definitively as a profession, we do not understand a lot of the systems that are in place right now.

[17:08] SPEAKER_02:

You know, two years ago, I wrote a book on that very topic about defensible teacher evaluation, in which I talked about the prospect of evaluating teachers based on student achievement. And I think it can be done, but it has to be done arising out of the classroom level of assessment. And I map out a specific way to do that. In doing that, I cite at least a dozen and a half reasons why relying on annual standardized test scores to evaluate teacher performance is patently indefensible. It cannot be defended. but there are better alternatives.

[17:40]

And to the extent that there's a foundation of assessment literacy in place, school leaders will understand. what that appropriate process would be like.

[17:48] SPEAKER_01:

Well, and I know this is a time in the United States especially when we're really starting to kind of get into the consequences of a lot of those systems that were designed in response to Race to the Top. We have a lot of value-added systems that are, you know, now the consequences are kicking in even as certain other aspects of federal accountability are being rolled back. And I know there were some very notable letters that were written by assessment experts that raised some concerns about some of the systems that are being used, about value-added measures. If we can geek out here a little bit on the assessment side, since it's not every day that I get to speak to someone with a PhD in educational measurement, what are some of the aspects that you've seen that make certain systems indefensible from that standpoint?

[18:33] SPEAKER_02:

First of all, there's the danger of fundamental misalignment between what's tested and what an individual teacher is assigned as their instructional responsibilities. There may be a fundamental mismatch there. Holding them accountable for scores on a test that fails to reflect their teaching responsibilities is patently indefensible. The shallowness of the coverage of such tests call into question their instructional sensitivity, that is their ability to detect the impact of any individual teacher, I know Jim Oppen and others have called that into question. There's the very content of the test. If a test spans more than one year or more than one grade level, how do you hold teachers accountable for the score if they're kind of at the end of that and the first two teachers didn't do their job?

[19:18]

It's not fair. So there are fairness issues that come into play. There are technical issues that come into play. I think it's very dangerous. I address the value-added dilemma in my book also. And we have people setting policy who simply lack the assessment literacy they need to set sound policies, and that harms kids.

[19:39] SPEAKER_01:

Well, certainly I've benefited from your work and from understanding what those balance systems look like. If we were to go back to something you said a moment ago about using the right kind of assessments in teacher evaluation, what would that look like if we wanted to make some moves in that direction?

[19:55] SPEAKER_02:

First of all, we teach lessons about four keys to quality assessment. One is pick a proper method given the learning target. We have a variety of methods at our disposal. They're not interchangeable. They need to line up well with the target. Then teachers need to know how to frame a sample of student performance, how to gather enough evidence to inform the decision without wasting time gathering too much.

[20:17]

And in addition then, quality assessment requires that we create high quality ingredients for the assessment, that is good exercises and scoring schemes. And then finally, that we communicate results effectively. This is not technical stuff. This is very practical stuff. And if everyone can understand those things, then whether they're selecting an assessment for use from a publisher or creating one for use in their classroom, they'll understand the criteria by which to develop or to select. And that is foundationally essential.

[20:50]

for kids' wellbeing. And then if they know when and how to engage students in the process, which I think is the big breakthrough idea, watch, just watch what happens. I've watched classrooms light up. Kids light up when they get the opportunity to do that stuff. I know it works. Literally around the world has worked.

[21:07]

We need to tap into that.

[21:08] SPEAKER_01:

So the book is The Perfect Assessment System, published by ASCD. And Dr. Stiggins, if people want to find out more about your work online, where can they go to find you? My website is rickstiggins.com.

[21:20] SPEAKER_02:

There are videos to watch there and information and papers and stuff to read, ways to contact me. Or if they wish to contact me directly, rickstiggins, R-I-C-K-S-T-I-G-G-I-N-S at gmail.com. That's the pathway.

[21:35] SPEAKER_01:

Well, thank you so much for your time today. Thanks for joining me on Principal Center Radio. It's been my pleasure. Thanks for listening.

[21:42] SPEAKER_00:

And now, Justin Bader on high performance instructional leadership.

[21:46] SPEAKER_01:

So high performance instructional leaders, what did you take away from the conversation with Dr. Rick Stiggins? One thing that stands out to me is the importance of assessment literacy. As decision makers, we're responsible for setting policies and for applying those policies about how we use student assessments to make decisions about teachers, to make decisions about students. And as Dr. Stiggen said, I think it's essential that we understand what those assessments are telling us.

[22:14]

And we're in a very strange time in our profession where we're now making decisions based on assessments that honestly we don't understand. And I certainly have experienced that as a principal where. I was seeing assessments being given and being given reports about those assessments, but honestly did not understand what those assessments told me and had to really kind of fight to learn what they meant and more importantly, what they didn't mean, what they didn't tell us. Because often when we're not as familiar with how an assessment works and what it really means as we should be, we assume that it tells us more than it does. And often assessments are designed to answer very narrowly tailored questions, and we apply them to all kinds of things. And if we can get clear on the purpose of an assessment, get clear on the design of assessment, we will be better users of that assessment.

[23:03]

And I know we have lots of professional development to pursue, lots of initiatives to implement in our schools, but I think it has never been more critical to push for assessment literacy. So I want to encourage you to check out Dr. Stiggen's book, The Perfect Assessment System from ASCD. And let me know what you think. Shoot me an email, justinatprincipalcenter.com.

[23:24]

I would love to hear from you. Thanks so much.

[23:27] Announcer:

Thanks for listening to Principal Center Radio. For more great episodes, subscribe on our website at principalcenter.com slash radio.