Audio Webcast Program #3
Trends in using measurement to improve learning
Summary: This chapter looks at trends in using measurement to improve learning and the emergence of a growing approach to assessment. The standards movement is an important piece of the history of assessment and continues the discussion on the value of basing decisions on evidence rather than personal opinion in education.
Introduction: Testing is a big topic in education today. It is also perhaps the strongest single source of anxiety among educators. Measurement, with its implications of evaluation and judgment, makes us nervous. But we also know that it's hard to improve without objective assessments. Knowing what to measure and knowing how to use the results productively is absolutely essential to making progress.
Listen Online (10:50)
Download Audio Webcast MP3 File: Program.mp3 (~8.6 MB)
Audio Webcast Speakers:
| Marci Dianda |
National Education Association |
| Sandra Feldman |
American Federation of Teachers |
| James Pellegrino |
University of Illinois |
Additional Audio: This audio webcast program includes
four additional audio segments.
Audio Webcast Transcript:
Host
Welcome to the third audio webcast in our series on developing greater
objectivity in educational practice. Again, I'm Ed Janus. On this
webcast we'll begin our discussions about the importance of the
measurement of performance. In other words we begin our ongoing
discussions on assessments of student learning, teacher effectiveness
and school progress. And although there is much to dislike in our
current assessment regime, many people who can see just over the
horizon, people you'll be hearing from on our programs, argue that a
kind of new age of measurement is coming; an age in which we will be
able to measure what's important, learn well from our measurements and
improve our efforts greatly as a result.
And as always, there is much more on each of our topics on our web pages and webcasts.
Let me now introduce our first guest. Marci Dianda is a senior policy
analysis and program consultant for the National Education Association
– the teachers' union. She was also a member of the important
Commission on Instructionally Supportive Assessment. How goes the
standards movement?
Dianda
I was just at a conference last week where someone said that we're at
the point where we need to make mid-course corrections in
standards-based reform, and I think what's happening is that they're
very, trying very, very hard to deal with the implementation of not
only standards, but more particularly of statewide testing programs and
statewide testing requirements. And they're trying very hard to figure
out how the tests can be useful for teachers' decision making, and can
be used to drive school improvement.
Host
Sandra Feldman, who has unhappily recently died, was president of the
American Federation of Teachers, the other national teachers' union.
Ms. Feldman, like her predecessor, AFT's late president Albert Shanker,
was a leading voice in the movement towards high academic standards for
all children – and measurements as a way to get there. When I talked
with her not too long ago I asked: where goes testing?
Feldman
I think that we're on the way to a better system, but we're certainly
not there yet. And as you know, a lot of teachers are very upset about
the nature of the testing. I think the testing has a lot of catching up
to do with the kind of education that teachers want to see in their
classroom.
Host
And finally, James Pellegrino is a professor of psychology and
education who has spent his career on the measurement of cognition. Dr.
Pellegrino, is it possible to really measure important knowledge and
skills?
Pellegrino
I think it's extremely feasible to do that. We know a lot more now than
we did 20 or 30 years ago about how to think about the nature of what
it is that kids understand, and how we can begin to get at some of the
more interesting and complex and important aspects of what it means to
know something.
What about tests now being used?
Those tests are perfectly adequate for certain kinds of conclusions.
And I say that because I don't want to come across as damning the
standardized tests. The dilemma comes when we try to use those tests
and the information they give us for purposes that are well beyond the
scope of what the tests can do.
Dianda
Although most states now have tests that are supposedly standards
based tests. And what's happening is that those tests are not fully
aligned. The tests assess some of the standards. They're not aligned
with curriculum frameworks, and they're not really aligned with what's
taught in the classroom. So that's a huge problem.
Pellegrino
Many of the standardized tests that are in use in many states, the ones
that come from the so-called Big Five publishers, they're not connected
to any curriculum. And it creates a real dilemma because there is this
assumption that, if we develop this one test, that it's going to meet
the purposes of several different audiences relative to evaluating
performance on the state standards. That is asking something from one
instrument that it can't possibly deliver.
Ideally what you would have is an assessment system that could operate
from the classroom level on up, where you would have assessments that
were designed to meet the needs of each context and purpose. If you're
a teacher who's teaching physics, you want an assessment that will
really help you understand, what is it that the kids are understanding
and misunderstanding? That way you can do something about it, rather
than just evaluate the fact that they don't seem to know how to solve
certain kinds of problems. That usually means that you've got to have
very carefully crafted sets of problems that will give you that kind of
conceptual information. That's not necessarily what you would want to
do if at a district level what you're trying to determine is, in
general, how well are these kids doing?
Dianda
Another problem is that the results from those tests aren't provided
back to schools or teachers in a timely manner. So that if there's any
information on the test that might be relevant to a state standard, or
relevant to what you're doing in the classroom, they get the results
back so late in the school year that you can't possibly use it for any
kind of instructional planning for what you're going to do with
students that school year.
Pellegrino
Here's a kind of metaphor that may help think about the problem we're
facing. Remember what used to happen usually once or twice a year in
all sorts of retail businesses? They would close down the store and
they would bring people in and they would do an inventory or an audit
of what was on the shelves, okay? Do you see that happening now?
Through the use of technology, they now have available on an ongoing,
daily basis, information about what products are being purchased. And
so they can not only get the kind of summary or snapshot information
that they used to get, but they can get a continuous stream of
information which tells them what products are being bought with what
frequency, etcetera.
But the point is that we have to think of a way of being smarter
about doing business which doesn't require us to shut down the place
once or twice a year just so somebody can come in and do an external
audit. And I think the most important thing about this is to think
about this in terms of having information available in the system on an
ongoing basis which allows the system to intelligently adapt. Because
right now, most of what districts have is very weak data that comes
from standardized test scores, which typically doesn't tell you what
the – if it tells you, if it says that the kids are having a problem in
math, because they're low achieving in math, that typically doesn't
tell you what the nature of the problem is.
Host
Ms. Feldman, people are pretty up tight these days about test scores aren't they?
Feldman
People have to understand better that test scores are going to be very
variable, that it, you really have to look at test scores over a period
of years. That just statistical variations will cause ups and downs in
test scores. And we've got to get off, you know, basing property values
on test scores. I mean that just doesn't make any sense. Tests are
about seeing and measuring how children are learning, enabling us to
put help where help is needed. And there will be variations up and down.
I think that it also should not be necessary, if you have a good
testing system, if you have a good assessment system, if it's based on
curriculum, based on high standards, it should not be necessary to
narrowly teach to the test. Teaching to the test, if the test is
measuring a rich curriculum, should not be a problem. And I think that
there are lots of growing pains out there. And because, unfortunately,
in a lot of places, states put this accountability program into place
as if it's punitive instead of diagnostic, there is tremendous
trepidation and tension in the local school districts about how their
kids are going to do on the test.
Pellegrino
One of the big weaknesses in our current approach to assessment is that
we haven't invested the time, effort, resources, in developing the
kinds of materials and tools the teachers could use to make these kinds
of more diagnostic assessments. We rely far too much on teachers to be
able to have to invent these things. What I would want to see is having
available to teachers a rich set of assessment tools that was woven
into their instructional practice. So that it wasn't just viewed as
assessment for assessment's sake, but it was actually woven into the
fabric of the teaching and learning process.
We need information that is geared to the level of decision for the,
for where we sit in the educational system. And we have to recognize
that we need to have assessments – and I use the plural – that map
against those purposes and needs, and not try to do everything with one
kind of assessment that represents a weak compromise that doesn't do
anything well.
Feldman
I think that if we had somehow in a lot of states started out better,
if it were handled less politically and more educationally, we would
have been better off. But I'm hoping that we're going to stay the
course, that we'll do the mid-course corrections that are needed. And
that we'll continue to see a rise in achievement as a result.
Host
Assessments are, as we just heard, simply sources of information. If
this information is used, as Dr. Pellegrino said, to "intelligently
adapt," they can become extremely useful tools in the hands of
educators. Making assessments and the data derived from them "useful
tools," will be the subject of many of our future webcasts.
For the Center for Comprehensive School Reform and Improvement at
Learning Point Associates, this is Ed Janus. Thanks for listening.
|
