Audio Webcast Program #7
Additional Audio from Audio Webcast Program #7:
Using shared assessments to unpack standards and compare instruction
Additional Audio 1 of 3: James Pellegrino
Title: Measuring complex cognitive tasks is necessary if students are to learn
Listen Online: (2:58)
Download Audio Webcast MP3 File: pellegrino1.mp3 ~2.9 MB
Summary: Using data to look beyond our own, limited experiences, is important if we are to open ourselves to better ways of doing things. Knowing the outcomes achieved by others with students similar to our own sets us on this path.
Introduction: Unpacking state standards and translating them into objectives that
can be measured is the key to standards-based reform and growth in
student academic achievement. Teachers need support in learning to
answer day-to-day instructional questions like:
- Which standards-based objectives should my lesson contain?
- What activities will help students practice these objectives?
- How and to whom should I re-teach the objective?
- How do I define proficient work?
Giving teachers the information they need to make these kinds of
decisions is fast becoming one of the most important new roles for
assessment. Not only will new assessments give teachers concrete ways
to deploy standards in their instruction, they will also give teachers
tools for objectively measuring their success in teaching
these standards. The new assessments provide a much needed common
language which teachers and instructional leaders can use to fairly
compare the outcomes they have achieved. This new lingua franca of instruction will give educators a way, perhaps for the first time, to truly help each other improve.
Transcript:
Ed Janus
Because
the tasks required in today's curriculum encode deeper cognitive
procedures it is necessary for today's assessments of those tasks to
unpack these deeper structures. This is work that requires deep
understanding of how people learn to think. Creating these tests of
deeper cognitive structure cannot be left to busy teachers alone. They
may not have the deep background necessary.
James Pellegrino
No matter what assessment we're doing, it's connected to a deep
understanding of what it means to know something and develop competence
in an area of the curriculum. So it should be grounded in not just a
sort of loose description of what the topics are, but in a real deep
analysis of what does it mean to understand how to deal with things
like fractions and proportional reasoning. It's not just about whether
you can do fractions, but do you understand what fractions represent in
the form of ratios and proportional reasoning?
So any assessment should be, at least should be grounded in
some, you know, the best knowledge we have about what it means to
really know how to think in an area, whether it's in science, or what
it means to comprehend, to be good at comprehending text and things
like that.
Then, it should be designed in such a way so that the tasks you
give, the observations that, the data that you get is connected to that
conceptualization for the purpose you want to use it. Now what do I
mean by that? If you're a teacher who's teaching physics, you want an
assessment that will really help you understand, what is it that the
kids are understanding and misunderstanding? That way you can do
something about it, rather than just evaluate the fact that they don't
seem to know how to solve certain kinds of problems. That usually means
that you've got to have very carefully crafted sets of problems that
will give you that kind of conceptual information.
And I think that one of the big weaknesses in our current approach
to assessment is that we haven't invested the time, effort, resources,
in developing the kinds of materials and tools the teachers could use
to make these kinds of more diagnostic assessments. We rely far too
much on teachers to be able to have to invent these things. And quite
honestly, they can't utilize assessment effectively unless we give them
better tools and better ways to collect the information and condense it
down into things that they can turn into actions that will help
students. It's beyond any one person's ability.
And so I think that the challenge here is, at district level or a
state level, is to begin to work with your curriculum publishers,
etcetera, to provide those kinds of quality assessment materials that
teachers could use at the classroom level.
Additional Audio 2 of 3: Peter Robertson
Title: Requiring teachers to use the same assessment tasks gives them a common language to talk about work that meets standards
Listen Online: (2:45)
Download Audio Webcast MP3 File: robertson.mp3 ~3 MB
About this audio clip: N/A
Context: The efforts of the Douglas County, Colorado school district
illustrate the new role assessments play in creating a common language
of instruction and improvement. The district strongly believes that the
curriculum can be "enacted" only through the decisions made by its
classroom teachers. However, the judgments of these teachers about what
constitutes high quality work must be accurately "calibrated" to
minimize variations in expectations. Douglas County demands a uniform
standard so that all children have a more than adequate opportunity to
learn.
To implement their initiative, Douglas County has developed a "body
of evidence" guide to selected district standards, and "checkpoints"
that define those standards in some detail. A body of evidence is a
collection of student work that answers the question, "what will
convince my teacher and others that I have actually mastered the
knowledge or skill detailed in the checkpoint?" Each checkpoint
includes one or more required, district-produced assessments, called
"anchor assessments," as well as other, optional district-produced
assessments. The checkpoint also identifies other tests and work the
teacher assigns to help gain greater understanding of the student's
progress and may also include assessments created by other teachers or
teams in the school or district.
Anchor assessments allow the test administrator to establish a
performance baseline that can be used to calibrate newer assessments.
Using the anchor as a guideline, teachers plan their instruction by
determining the complex skills contained in the checkpoint and making
them explicit, both to themselves and to their students. By embedding
the anchor and other model assessments into their instructional
planning, teachers build a standards-based instructional plan for the
year.
More importantly, anchor assessments are shared. They are
administered in every appropriate classroom in every school in the
district and become the focus of shared teacher instructional problem
solving and professional development. As teachers talk with each other
about the success, or lack of success, their students had performing
commonly-assigned tasks, they can begin to examine their instruction.
And since they are measuring performance on the same tasks, using the
same standard of quality, teachers can compare their efforts to those
of the teacher next door or across town. These activities provide
effective professional development because they allow educators to
learn from each other.
Transcript:
Ed Janus
The best professional development for teachers might well be talking
together about what constitutes good work. These discussions can only
be useful if teachers all use the same, sophisticated tests that get at
important learning goals.
Peter Robertson
Once you've figured out what the standard is, figure out how you're going to
assess students against that standard, and then work backward to lesson
plans. So as we are starting to build the materials that will support
teachers in classroom implementation, classroom assessment tasks is an
important part of that. And what we're trying to do, and we're really
at the infancy stages of this, is really two-fold. One is, and the
piece that we've already made some headway on, is we are using some
standards-aligned classroom tasks requiring teachers to give them in
their classes. In fact one of them this semester, we're requiring that
the teachers include in the grade of the student in that class. And
then those tasks are being collected and scored at district wide
scoring camps by tables of teachers looking at these student tasks.
And, and it's very expensive. If you view this strictly from an
assessment standpoint, it's very expensive. But it's some of the most
effective professional development we've done. And if you view this as
assessment and professional development, it's a reasonably priced
initiative.
But I think that what we hope to do, and one of the things we've
already started to do, we did a sample of this fall where fourth grade
students had done a reading task in this manner that we had scored at
scoring tables, and they had also been required to take the state, the
state fourth grade reading proficiency tests, and we also had their
report card grades. And the question that we're asking is, "What's a
reasonable alignment to expect between those three kinds of
information?" And we would provide information back to teachers on that
alignment, and encourage them to bake that into their conversations on
their faculty and with their principals about, "How do I get a better
handle on what the standards actually are, how I measure against those
standards? What," and here's a real, here's the real insight from this
work so far, help teachers understand, "What quality of student work I
ought to be expecting."
Because one of the real "Ah-ha's" from the teachers who have been
coming out of these scoring camps is that their expectation of what
quality of work, you know, I should have, what quality of work I should
expect of my students is going up as these people come out these
sessions, because they're seeing the kind of work that the kids can do,
and they're seeing how that relates to the standard.
So, we would want, we would want this to be used in, in formative
evaluation conversations with te-, you know, within faculties. Of, "How
do I get better at doing the things I need to do?"
We've got fourth grade teachers now able to pull down that fourth
grade reading task and score it on-line. And they can score student
work on-line and then we could presumably send them back a report if
they wanted it of how well their scoring correlated with the scoring
that was actually done of these items in the scoring camps. And we
could set up kind of—and our goal is over time, you know, as teachers
need to get themselves recalibrated to the standards periodically, then
they might log on to a Web site, pull up an assessment item and
actually score a few students' work, and then get back, you know, how
their scores compared with the scores that were given to this work.
Additional Audio 3 of 3: James Pellegrino
Title: Professional development that unpacks complex learning tasks
Listen Online: (2:25)
Download Audio Webcast MP3 File: pellegrino2.mp3 ~2.3 MB
About this audio clip: N/A
Context: N/A
Transcript:
Ed Janus
Helping teachers unpack complex learning tasks is real professional development.
James Pellegrino
I mean, well, it sounds so obvious, but I defy you to go read most
states' standards and figure out, or have anybody figure out, exactly
what constitutes evidence of having met that standard. It's, you know,
one way to think about it is many of these standards documents are
like, you know, the classic Rorschach Ink Blot.
Mr. District Administrator has got a big problem here. Mr. District
Administrator needs help from the curriculum people, the assessment
people, and you know, the people like myself in cognitive psychology to
more clearly lay out for the administrator or the teacher or everyone,
exactly what we're looking for. And what are the differences in terms
of levels of performance or levels of understanding that define
different sort of benchmarks?
We should start out with a clearer sense of what it is we want and,
for example, what we know about how kids, you know, how kids understand
things like fractions, or how they understand a rational number, or how
they understand certain kinds of concepts in science, and then develop
assessments that will get at those different levels of knowledge and
reasoning.
Well, you know, they could, this would be a part of the professional
development activities. And we, you know, districts typically invest a
great deal of resources in professional development to begin with. But
most professional development teachers will tell you, it's not very
meaningful. It has far too much of the one-shot character to it,
etcetera. And another thing that teachers will tell you is oftentimes,
when they are involved in programs where they're either designing
assessments or they're evaluating student performance, student work on
various kinds of assessments, that can be the most valuable
professional development they've ever had.
So that, I would view this as part of the professional development
investment that we already make. And also build it into the ethos of
what it is that teachers are supposed to be doing as a part of their
instructional practice.
But I don't want to, again, laying all the burden on the
teachers is difficult. I would, you know, use teachers along with other
experts to develop a quality assessment system. And I would also put
pressure, and I think states should do more of this, should put more
pressure on, you know, the textbook publishers and the test developers
to actually develop more of these kinds of quality materials and
methods as a part of the procurement process for educational materials.
|
