Audio Webcast Program #3
Additional Audio from Audio Webcast Program #3:
Trends in using measurement to improve learning
Additional Audio 1 of 4: Peter Robertson
Title: The new age of testing: serving teachers, students and the public
Listen Online: (2:45)
Download Audio Webcast: robertson2.mp3 ~1.9 MB
Summary: A "standardized" test is merely one that is given to all students in a fair and consistent manner. Today, "standardized" has come to mean "large scale" testing; testing that is thought to miss the mark as much as hit it. These tests, more and more are being considered really too "large scale" for students and teachers to use to improve day-to-day learning. That is, many of today's assessments are summative, rather than formative, in nature. To make matters worse, they are not well enough "aligned" with the curriculum to be useful.
Peter Robertson was the chief information officer for the Cleveland Public Schools. He is a true believer in the power of feedback to improve things.
Introduction: As our first two audio webcasts have made clear, we are living in a new age of science. More and more, the problem solving strategies of the scientific system are being brought to bear on nearly every aspect of our lives. Why? Because science is a method through which many minds can be brought to bear on a problem. Examining and discussing objective data, using scientific processes, can help us separate fact from opinion. And knowing what is fact and what is opinion is the first step in finding solutions to persistent problems, including those in education.
Over the last 15 years, public education in America has been challenged by critics to re-model itself into a more transparent and self-correcting system. These critics have argued that too often the personal and private judgments and expectations of educators rather than objective and public judgments and expectations have driven policy and practice. These arguments have been strengthened by the widespread standards, assessment and accountability movement that has required that public education put itself on a more scientific footing and develop a more public culture of measurement and self-correction.
The first phase of this movement -- a phase some feel is coming to a close -- was characterized by public shock at the discovery that educators had long been free to decide who got taught what, how well, and to what effect. This realization engendered a strong reaction and spurred change. Content and performance standards were written, and large-scale tests and sanctions were employed to ensure a more standardized approach to classroom teaching.
However, these large-scale assessments and their results, although useful to measure the progress of a group of students at a point in time and to make determinations about the progress of institutions, are not particularly helpful to classroom teachers working day to day in classrooms with individual students.
Transcript:
Ed Janus
A "standardized" test is merely one that is given to all students in a
fair and consistent manner. Today, "standardized" has come to mean
"large scale" testing; testing that is thought to miss the mark as much
as hit it. These tests, more and more are being considered really too
"large scale" for students and teachers to use to improve day-to-day
learning. That is, many of today's assessments are summative, rather
than formative, in nature. To make matters worse, they are not well
enough "aligned" with the curriculum to be useful.
Peter Robertson was the chief information officer for the Cleveland
Public Schools. He is a true believer in the power of feedback to
improve things.
Peter Robertson
I think that the conversation about, you know – and we use standardized
testing to drive alignment. But the mistake we make is, because the
standardized test doesn't serve the, the teacher and the student in a
formative sense, it, it is often externalized and not used by the, in
that classroom. And therefore, it actually doesn't do much to drive
alignment except out of fear. You know, it doesn't drive, it doesn't
drive alignment out of understanding; it drives alignment out of fear.
Working with teachers to get them to use the same authentic tasks in
their classrooms, and to compare each other's scoring of those tasks
and to have conversations about what it means that a student did this
on this task versus that on this task, is a much more laborious thing,
effort; it is much less reliable from a statistical validity
standpoint. But it is ultimately much more effective in aligning the,
the actually implemented curriculum.
We're trying very hard to be true to the standards-based lesson
planning mandate, which is, once you've figured out what the standard
is, figure out how you're going to assess students against that
standard, and then work backward to lesson plans. So as we are starting
to build the materials that will support teachers in classroom
implementation, classroom assessment tasks is an important part of that.
And what we're trying to do, and we're really at the infancy stages of
this, is really two-fold. One is, and the piece that we've already made
some headway on, is we are using some standards-aligned classroom tasks
requiring teachers to give them in their classes. And then those tasks
are being collected and scored at district wide scoring camps by tables
of teachers looking at these student tasks.
And, and it's very expensive. If you view this strictly from an
assessment standpoint, it's very expensive. But it's some of the most
effective professional development we've done. And if you view this as
assessment and professional development, it's a reasonably priced
initiative.
Additional Audio 2 of 4: Eliot Asp
Title: Aligning standards, curriculum, assessments and lessons to improve learning
Listen Online: (2:19)
Download Audio Webcast: asp.mp3 ~2.1 MB
About this audio clip: States, districts and schools are now attacking the problem of alignment between standards, curriculum and lessons. Many take the approach that good -- really good -- assessments can be used to bridge these separate "islands," and create a more seamless road from legislated curriculum to enacted ones.
Eliot Asp is the assistant superintendent for research and assessment for the Douglas County, Colorado schools, a leading district in the effort to create useful systems of assessment.
Context: The "new assessment movement" that we'll be hearing about in this audio webcast reflects an attempt by educators to bring attention to the role of formative assessments. Its proponents emphasize that formative tests can be used to shape instructional strategies in the classroom and increase student achievement before summative assessments are administered. In other words, good formative assessment can help students learn throughout the school year, and large-scale, summative assessments can remain in place as accountability measures at the end of the school year. Both types of assessments have their place, but if we really want to increase student achievement, we must move beyond summative assessments and into developing appropriate benchmarking techniques or formative assessments. Doing this will allow educators to satisfy the public demand for accountability but still create useful solutions to daily educational problems.
Transcript:
Ed Janus
States, districts and schools are now attacking the problem of
alignment between standards, curriculum and lessons. Many take the
approach that good – really good – assessments can be used to bridge
these separate "islands," and create a more seamless road from
legislated curriculum to enacted ones.
Eliot Asp is the assistant superintendent for research and assessment
for the Douglas County, Colorado schools, a leading district in the
effort to create useful systems of assessment.
Eliot Asp
For a long time it was just thought of as a summative measure: did the
kids learn this particular set of knowledge and skills? But now we've
really seen that evolve over the last number of years so that it's much
more integrated with curriculum and instruction. It's a formative
measure that teachers use along the way to adjust their instruction,
but it also sets targets for kids and, and for teachers. In other
words, if you're thoughtful about this, clever about it, the kind of
assessments you choose allow you to really have a good understanding of
what you want kids to know and be able to do, and you can share that
with kids ahead of time, and you can use it to guide your overall
instructional program.
We've almost taken a step backwards in the last few years. The
large-scale assessment has really driven out, in some sense, reliance
on classroom assessments, and really moved us toward thinking of
raising test scores in some ways, rather than thinking about improving
students' achievement in authentic ways.
Can you combine classroom and large-scale assessment?
In fact we're trying to build compatibility between large-scale and
classroom assessment. We're, that's really critical here. Certainly we
need to stay in touch with what the large-scale assessments are
measuring. But it's really the teacher who knows the most about student
achievement. Much more so than any large-scale test will tell them.
The most important instructional decisions are made by teachers every
day in a classroom. If they don't have good information, good
assessment information about where a kid is, then those decisions are
based on faulty, on a faulty premise. And so, the more assessments we
can provide and the more examples of student work tied to those
assessments that would provide fodder for people to engage in
professional dialogue around what it means to meet the standards, and
what they do next really, and as they move to, "How in the world do
they teach a kid to do this?" And they start exchanging instructional
strategies.
Additional Audio 3 of 4: Rick Stiggins
Title: Student-centric assessment - the next frontier
Listen Online: (2:38)
Download Audio Webcast MP3 File: stiggins.mp3 (~3.3 MB)
About this audio clip: Feedback is one of the fundamental steps in learning. How else can we know if we are right or wrong, skilled or lacking in necessary skills? Generally, we think of tests and assessments as providing the necessary feedback. Yet, it doesn't matter if teachers, superintendents, or state legislature knows what the feedback is. If the feedback does not make it's way to the student in a meaningful and constructive way, assessments have not done their job. Learning is what happens when students come to understand the feedback provided by assessments, for themselves, so that they may right the wrongs and develop skills where they were previously lacking.
Rick Stiggins is perhaps the leading advocate for a student-centric system of assessment. Mr. Stiggins heads the Assessment Training Institute which works with teachers and principals to help them develop such a system.
Context: Through the application of honest scientific measurement and statistical methods, as we'll say over and over, facts are established. And having agreed-upon facts -- data -- can center and focus discussions that in turn lead to solutions. Take for example, the issue of academic expectations. Research tells us where expectations are high, achievement tends to be high. The opposite is also true. So how do we get to a solution that provides high expectations for all students? Science has a suggestion.
No Child Left Behind requires that test data be broken out into statistical pools that represent distinct socioeconomic and racial groups. Using these "disaggregated" data and statistical methods to analyze them provides an increasingly accurate picture of the "facts" of student achievement, by illustrating, among other things, achievement gaps. Once one is aware of gaps through the use of data, one is better equipped to deal with them and create solutions to address them. Achievement gaps between high and low performing student groups may indicate, for example, inconsistent expectations for all students. They might reflect an uneven allocation of resources or variations in teacher qualifications. But, without starting with measuring achievement of all students -- and then analyzing and acting on the data -- no credible solution can be achieved. The ongoing work of developing better formative assessment techniques through the use of new and more sophisticated measurement techniques will result in additional data for examination and analysis.
Transcript:
Ed Janus
Feedback is one of the fundamental steps in learning. How else can
we know if we are right or wrong, skilled or lacking in necessary
skills? Generally, we think of tests and assessments as providing the
necessary feedback. Yet, it doesn't matter if teachers,
superintendents, or state legislature knows what the feedback is. If
the feedback does not make it's way to the student in a meaningful and
constructive way, assessments have not done their job. Learning is what
happens when students come to understand the feedback provided by
assessments, for themselves, so that they may right the wrongs and
develop skills where they were previously lacking.
Rick Stiggins is perhaps the leading advocate for a student-centric
system of assessment. Mr. Stiggins heads the Assessment Training
Institute which works with teachers and principals to help them develop
such a system.
Rick Stiggins
The dimension of the understanding of cognitive processes that we tap
into is the need to have the learner actively involved in the learning.
And so, we want them deeply involved in the assessment process, on a
day-to-day basis.
We want them to understand the learning targets from the beginning of
the learning; we want them to have continuous access to descriptive
feedback. So that in effect what we're doing is use the classroom
assessment process as a mirror to show students themselves growing.
This allows them to feel in control of the learning process, to feel
confident enough to take the risk of continuing to try. And that turns
out to be the key to the learning process.
So we want them deeply involved in the assessment process, we want them
deeply involved in the record keeping process, and deeply involved in
telling the story of their own achievement in the communication process
over time.
This kind of active involvement allows them essentially to take
responsibility for constructing their own meaning. In our parlance we
refer to the distinction of assessment of learning and assessment for
learning. In the assessment for learning context, we want students to
have continuous access to descriptive feedback that informs them about
how to do better the next time. It's not that we're opposed to
assessment of learning, or summative assessment. We think that plays a
critical role.
It's just that during the learning, students need evidence of their own
current level of proficiency so they can understand how they're doing
in closing the gap between where we want them to be and where they are
now. All of that requires – yes, that students have continuous access
to evidence of their own current level of proficiency. In light of, and
we want them to have access also to a vision of what success will look
like when they get there.
There are three conditions that need to be in place for students to
improve. The first is that they need to see, need to understand what
good work looks like. Secondly, they need to understand where they are
now in relation to the vision of excellence. And thirdly, they need to
learn how to close the gap between the two.
The only way they can do that if they have that continuous feedback;
that continuous descriptive feedback so they can monitor and take
responsibility for their own development over time.
And you're talking, almost day to day perhaps?
I'm talking moment to moment here. Absolutely. When students are
learning to write for example, they're going to be producing pieces of
writing pretty much continuously. Each one of which is a valuable
source of information about how they're doing. If we carry out proper
principles of assessment for learning.
And what I mean by that is: they would come to understand a
student-friendly version of what good writing looks like. They'd see
models of strong and weak work, pretty much continuously. So they can
know where they are on that achievement continuum. You know that it's
the scaffolding that they're climbing on their journey to being a good
writer. In order to do that, one must have pretty much continuous
access to that evidence.
We're constantly working with a student right at the edge of their
capabilities and helping them take the next step. It's that kind of
interaction that turns out to be critical.
Additional Audio 4 of 4: Peter Robertson
Title: More on student-centric assessment
Listen Online: (2:03)
Download Audio Webcast MP3 File: robertson.mp3 (~2.5 MB)
About this audio clip: Peter Robertson continues the discussion on the difference between student-centric and adult-centric assessments.
Context: On our web pages, through the audio webcasts and extended audio interviews on the topic of using measurement to improve learning, we'll explore in some depth the role that assessments and testing can play in giving educators the information and tools they need to improve the effectiveness of their efforts.
Transcript:
Ed Janus
Peter Robertson continues the discussion on the difference between student-centric and adult-centric assessments.
Peter Robertson
The key for teaching teachers about assessment is to help them
understand that there's no point in giving a kid a mystery test at the
end of the, at the end of the week or the end of the quarter. And you
know, if it's – if the point of it is, you know, "I told you it was
going to be on the test to make you listen," then you're back in that,
"going gets tough, tough get going" psychological motivation theory.
And again, that's going to work for some of your kids, but not for the
ones who are having trouble being successful. Those kids need to see a
clear path to success if they're going to engage in their classrooms.
And that requires that the teacher understand that assessment is
actually an open book conversation between them and the students. You
know, "Here's the rubric by which I'm going to grade this paper. Why
don't you and I talk about what this rubric means. Why don't you write
the paper against that rubric. I'll give you feedback on that rubric.
I'll let you rewrite it." You know, "I'll let you keep working until
you reach the point on the rubric that we have agreed to." That's a
very different mindset for teachers, and until they get their heads
around it, and until they see that actually that makes their lives
easier because kids are now managing their own learning, it's a very
daunting thing.
But the most difficult cognitive task in it is for a teacher to begin
their lesson planning, not by saying, "What am I going to do today?"
But by saying, "What is it that I want my students to be able to do,
how will I assess that?" And to spend a lot of time up front thinking
about, "How will I assess that?" and is that good way to assess that?
And if they're at least trying to do that, over time, you know, you get
better at that. But honestly, you know, I challenge anyone to come up
with an assessment task that no one can find any fault with. So, it's
more about the process and the dialogue than a perfect, a perfect
assessment task.
|
