Audio Webcast Program #4
Monitoring what gets taught: insuring an adequate opportunity to learn
Summary: This program continues the assessment discussion and focuses on monitoring what gets taught: insuring an adequate opportunity to learn. At the heart of No Child Left Behind is the demand that every child be given a meaningful chance to graduate high school with more than a piece of paper of sometimes dubious value. Giving students an adequate opportunity to learn what they will need to graduate with a good start in life is within our control and this program discusses how good tests will play an increasingly important role.
Introduction: Historically, the promotion of students from grade to grade and even
their graduation from high school has been based on classroom grades
and passing required courses. Standards for grading systems and what
constituted passing varied greatly from school district to school
district within a state, from school to school within a district, and
even among classrooms within a school. To some extent what a student
left school having learned was a matter left to the professional and
personal decisions of teachers, and a good deal of chance.
The academic standards movement changed that by making public each
state’s requirements for what students should know and be able to do
before they graduated. Now that knowing what is required is public
knowledge, attention needs to be focused on buildings systems that
ensure that every student has access to that content. One of the most
important new roles for a testing system is to monitor the relationship
between published standards and what is actually taught in classrooms
(the enacted curriculum) to make sure that all students are given an
equal “opportunity to learn.”
Listen Online (8:09)
Download Audio Webcast MP3 File: Program.mp3 (~7.6 MB)
Audio Webcast Speakers:
| Marci Dianda |
National Education Association |
| Sandra Feldman |
American Federation of Teachers |
| James Pellegrino |
University of Illinois |
Additional Audio: This audio webcast program includes
four additional audio segments.
Audio Webcast Transcript:
Host
Welcome to the fourth of our audio webcasts on how education can be improved
through more objective and more public evidence of performance. I'm Ed
Janus.
We thought we'd continue our discussions about making assessments
valuable for teaching and learning with a sort of "primer" on the
subject. Since of course there is much slinging about of technical
terms we asked Andrew Porter of Vanderbilt University to help us. Dr.
Porter is one of the preeminent experts on the proper measurement of
performance in education.
Dr. Porter, you obviously believe in the power of testing to provide
useful feedback to students, teachers, administrators and policy
makers. But it isn't simple is it?
Porter
Well, assessment and accountability is a two-edged sword. Anything
worth doing can be done poorly, and assessment and accountability is no
exception. I've come up with a, you know, if you use the word kind of
loosely, theory for how you design a good quality assessment and
accountability system.
I think if you set a good target, you're symmetric in the sense that
you hold students and teachers accountable, and you set up a fair
system which includes opportunity to learn for the students and
adequate resources for the teachers, then you'll do good things for
students.
But if you don't do those things, then you can have some bad things
happen. For example, if you don't give students an adequate opportunity
to learn, and then you test them and you don't give them a high school
diploma, is that a fair thing? No.
So, there are plenty of places where you can go wrong. Let's say that
you test what's easy to test rather than what you should test. Then you
set a bad target. You skew the whole system down on that bad target,
maybe get an unbalanced curriculum, maybe stamp out from your
curriculum some of the things that are most important to be taught and
learned. That would be very bad as well.
Host
A great danger that critics of testing make is that it can too easily
force teachers to narrow what they teach to insure success on mandated
tests. A good testing system shouldn't do this though?
Porter
There are a lot of things one can do to make sure one doesn't narrow
the curriculum. And one of the most basic and fundamental things to do
is to not use the same form of the same test year after year. And
everybody has to think about the test as a sample, so that they stay
focused on the standards, not the specific test. If you change the form
of the test every year, you have a different sample every year, that
forces the focus on the standards and not the test.
Let's talk about that word "standardization" for a minute, because it's
gotten a bad name. And you know, it's gotten a bad name because people
are misusing the word. Standardized test has come to mean a paper and
pencil, multiple choice test. But what really is meant by
standardization is that every kid gets an equally fair opportunity to
do their best.
Standardization is an issue of fairness, fairness to the student,
and of course if there's going to be teacher and school accountability,
fairness to the schools and teachers as well. So you always want to
have a standardized test in that sense.
Explain the difference between "reliability" and "validity."
Well, reliability is just a notion of precision. And everything else
being equal, it takes more items on a test to get a more reliable
assessment of the student's accomplishment. So that when you get a
score it's an average across a lot of items.
Validity is the more important and more complicated idea. Say
you're a state or a school district and you have some content
standards, then you want your test to assess what's in your content
standards. This is the concept of alignment. Is your test aligned to
your standards? Is everything in the standards tested? Is everything in
the test in the standards? There are two sides to this; often people
ignore one side, but there are two. And if everything is, if both sides
work, if everything in the standards is tested and vice versa, then
you're going to have a valid test from a content point of view.
Now many content standards of districts and states include things that
are almost impossible to assess on a paper and pencil test. So, that's
a problem for validity. Now, if you're going to use a test for making
decisions about student promotion from grade to grade, here's where it
gets very interesting and very tricky. And it's hard to get a test
that's an accurate reflection of the standards.
If you're going to hold a student accountable – and let's take
something big like high school graduation – then actually the law
requires that you can demonstrate that you provided the student an
adequate opportunity to learn what's on the test.
Now, some people when they talk about opportunity to learn, they
want to say, "What's the per pupil expenditure? How many books are in
the library? Is the teacher certified?"
That's simply too distant from performance on the test for my tastes.
You can have a student who experiences a curriculum, that provides them
the opportunity to learn, but the delivery of that curriculum could be
so poor that they don't learn it, or the student could put in so little
effort that they don't learn it.
So, it all has to come together in a nice package. You've got to get
some effort out of the student, you've got to get some good delivery
out of the teacher, and then you've got to be delivering the right
stuff. And opportunity to learn, in my mind, is the right stuff. Do you
have a curriculum that aligns well with the test, and do you have a
test that aligns well with the standards? And when you've got that kind
of consistency, I would say you have a good opportunity to learn.
In my research and that of others, it's clear that what is taught is a
better predictor of gains in student achievement than how it's taught.
And furthermore, we can't take content for granted because you can go
into an elementary school and go into a fourth grade classroom, you'll
find two teachers at the fourth grade level, in mathematics. By
Christmas, one will have already taught as much mathematics as the
other teacher will have taught by the end of the school year. Enormous
differences. Content simply cannot be taken for granted. But we have
not developed good systems for monitoring what is taught. So this is a
real frontier for us.
What we need, I think is a program of teacher evaluation and
feedback that's formative in nature more than summative. That is to say
that it's a partnership with the teacher to help the teacher see what
they're doing well and what they're not doing so well, and then
provides the kind of support they need to become more effective in
areas that need shoring up.
For this to work, the teachers, the whole school has to come on board
and say, "We want to get better. We want to look at our practice. We
want to use this tool to help us understand our practice."
Host
In some of our most progressive states, like Maryland for example, end-of-course exams are being instituted. Do you like them?
Well, I think end-of-course exams are just the greatest, and here's the
reason. What it does is it really nicely couples together instruction
and the test. And the more tightly coupled you get instruction and the
test, the better I like it; the more likely it is that you're going to
satisfy the opportunity to learn requirement.
If you have an end-of-course exam, and let's say then you say, "Well, a
sixth of this student's grade in the course is based on the
end-of-course exam." Then you start to have leverage on what is taught
in that course. Well, if you're going to start to have leverage on,
say, what's taught in Algebra I, and if you want to be responsible and
fair to your teachers, then they're going to have to come together and
form a partnership and say, "Here's what's important to be taught in
Algebra I," and we're all on the same page. So, and then that, and
they're going to jointly approve your end-of-course exam. I would hope,
I hope it would be a partnership.
Host
Partnerships among teachers, administrators, policy makers and students
to improve the effectiveness of teaching and learning will be the
subject of many of our subsequent audio webcasts. Please join us. For the
Center for Comprehensive School Reform and Improvement at Learning
Point, this is Ed Janus. Thanks for listening.
|
