Podcast Programs 1-8 Podcast Home Podcast #1: An introduction to basing our practice on better evidence Podcast #2: Changing the nature of the education conversation Additional Audio Podcast #3: Trends in using measurement to improve learning Podcast #4: Monitoring what gets taught: insuring an adequate opportunity to learn Podcast #5: Making science usable - engineering evidence-based knowledge into lessons and learning Podcast #6: Comparing ourselves - honestly - with the best Podcast #7: Using shared assessments to unpack standards and compare instruction Podcast #8: Making Mistakes and Moving Beyond Them is in the Scientific Spirit

Podcast Program #2

Changing the nature of the education conversation: What do we mean by "scientific," and how can the spirit of science help educators become better professionals?

Summary: This program looks at changing the nature of the education conversation. The panel discusses what we mean by "scientific," and how the spirit of science can help educators become better professionals. Also discussed, what would the educational equivalent of the FDA be like?

Introduction: Most of what we do in our professional lives -- and certainly in our personal ones -- is not driven by rigorous appraisals of the effectiveness of various alternatives. Instead, we are, what pragmatist William James might say, creatures of habit. That is, we do not think about most of what we do, we just do it. We are guided by a kind of personal sense, call it "intuition," of what seems to have worked, at least moderately well, in the past.

As professionals, we add to our own experience by seeking out the wisdom of others in our profession, frequently from those with whom we agree. Therein lies the rub. As highly social beings, we seek confirmation of our beliefs about the world. Unfortunately, such confirmation may not have the best consequences. Science, on the other hand, is an organized system of intellectual rigor that seeks what might be termed, a "dis-confirmation" of beliefs. Science tests beliefs -- hypotheses -- against reality. Science questions our habits, going against the grain of our normal way of doing things, to see if there is a better way.

Listen Online (9:12)

Host

Welcome to the second of our podcasts on the movement towards greater reliance on evidence in the practice of education. I'm Ed Janus.

Our guests on this edition are Howard Bloom and Robert Slavin. Dr. Bloom is chief research scientist at MDRC, a well-respected social research institution. Dr. Bloom has pioneered the use of experimental design to answer questions about the effectiveness of various approaches to welfare reform -- an area of social policy that many had thought beyond the ability of real science to address. Dr. Bloom is convinced that vexing questions about education too can be answered through the application of rigorous science.

Dr. Slavin, as many will know, is the co-founder of the widely-used school reform model, Success for All. He has been outspoken on the need for education reform ideas to submit themselves to the scrutiny of numbers.

Dr. Bloom, let's start by asking you, on a scale of 1 to 10, just how scientifically based you think classroom practice is.

Bloom

I'd say one-half.

Okay, so a lot of voodoo in the classroom?

Well, I wouldn't say that. A lot of professional opinion in the classroom. Which I'm not saying is wrong or bad. "Look, there's an art and a science to teaching." That, you're not operating in a vacuum; there are, I believe, certain fundamentals and certain near universals–I hate to say universals–but there are some forces at work there that are important to know about, and that hopefully you can, you know, influence things by understanding these forces, and that there are better and worse ways of doing things.

Now, but I truly believe there are better and worse ways to proceed for almost any profession, and a willingness to understand that and to seek out that, I think is important in a professional.

Slavin

I think it's going to happen only when government or researchers themselves or others are able to, first off, provide more useful research on topics that people really care about, but also when there's an opportunity for the results of research to be synthesized in a compelling way that enables people to use the results with more confidence; it's not just one person's opinion or what have you.

You know, "Here's what the evidence says. Here's where the evidence is lacking." But you know, here, this is really a guide to practice that I'm hoping will be, will help change the discussion a great deal, to a situation in which a, you know, somebody who is having to make a very difficult policy decision will have something much firmer to use as a basis for that decision.

Host

Why do you think then that educators have ignored science as a way to improve their practice?

Bloom

I mean, if, if you're not taught to think about how to process the information you get, and you're not taught to think that science has anything to say to you that's worthwhile, you won't care. But if instead you are taught about, you know, the role of science in progress in any field, and how then to use that science to achieve your own personal goals, if you're taught that, then you're more likely to think about it, you're more likely to use it, you're more likely to let it affect the art of teaching that you develop, I think.

Slavin

Now, you must be familiar with this, the concept that education should become more like medical science. Do you think that's a good analogy?

Well, it's not a perfect analogy, but I think it's helpful in many ways. And remember within medicine there's a huge range of kinds of research and a huge range of, of purposes to which that research is, is put. And there are many areas of medicine that really do resemble education more than others. So for example, people who are concerned about the medical analogy will say, "Well, in medicine you just give somebody a pill and the pill is always the same. Well, in education of course, everything is much more context bound and depends so much on the characteristics of the school and of the teacher and of the learners and so on, that it's impossible to make those kinds of generalizations.

Well, in fact there are lots of situations in medicine like that as well. There are many, many variables that go into whether a particular operation is successful or not. But that being said, for a particular class of, of diseases, you can find very reliable differences between using one procedure versus using the other, and those differences are taken very seriously by physicians. And, and I think, I don't see any reason why in education we couldn't get to that same spot where, not for every question we face, but for many of the really core questions of practice, there's no reason that we shouldn't be able to say, "This method works better than that method on average in these particular circumstances." And then in other circumstances, maybe some other method is called for.

Bloom

Now as you are, I'm sure, very well aware, No Child Left Behind demands scientifically based practices. Do you think this is a very useful way to move this revolution forward?

I do. I think, until somebody really pushes the issue and calls the question, nothing much will happen, which is what's been going on for the last decades. So I think some forces, which are clearly at work, have to push this issue.

Now as in any paradigm shift or any new way of thinking, if you're not careful you can overdo it, and the pendulum can shift too far in the other direction, and you have to be mindful of that.

How sanguine are you that in the next 20 years or so, education will begin to be much more scientifically-based practice, like medicine and welfare reform?

I'm pretty sanguine. Now I'm not totally naive and I, you know, things can go wrong. But I think there are so many parallels out there. And there's a lot to be learned by looking across those other fields as we proceed in education, and there's been enough, in my mind, enough success in those other fields that I think, a fair bit of success is, is to be expected, and certainly hoped for, and expected, in education.

Slavin

Okay, I'm going to name you as my sort of science czar in my district. Give you pretty much power to transform my district into science seeking organization.

Okay.

So, what would you do?

I guess I would, I would cause to happen, one way or another, a review of evidence on each of the big practices that, that the school district has to take on. I mean so for example, I would -- I would look at: What should kindergarten be like? You know, let's look at the research on effective practices in kindergarten. What reading program should we use? You know, for beginning reading; what reading program should we use for upper elementary? What reading, how should we deal with reading issues in secondary school? We'd also then look at: What are appropriate ways to organize schools? You know, to deal with issues of grouping and to deal with issues of promotion or retention, or issues of special education, or gifted. You know I'd be, be kind of looking at the research on each of those things.

I would be looking for evidence of effectiveness for each of those, each of those programs, and then basing the district practices on what we find, and then building around those the appropriate kinds of professional development and, and materials and, and incentives and everything else, to see that the effective practices that we have identified are being effectively used.

Do you think it's necessary to sort of raise the level of objectivity of your leadership staff, your administrators, your principals, and the teachers too?

I think so. I think certainly people need to be focused on objective data, both data that, from the outside, as well as data within the school system, so that things can be held up and examined as objectively as possible before you're making decisions that are consequential for children. And that's not an easy process because, you know, education is, has a tradition of being very ideological and being very much driven by belief systems rather than by evidence. And, but I do think that, you know, with consistent leadership, that could change.

I think that, that eventually it's got to become acceptable, it's got to become mandatory for people to begin to look more at the evidence, and we need to have–and I think we will have–you know, more investment in high quality research in the first place, so that we can have more and more confidence in the results of the research.

Host

Nothing is more important, I think then in becoming more confident that what we are doing is the best that can be done. To do that we really must put aside our over reliance on what we have been doing, or what we think is good to do, and ask, where is the evidence and how good is it. Then we really must let the chips fall as they may.

For Center for Comprehensive School Reform and Improvement at Learning Point Associates, this is Ed Janus. Thanks for listening.