If you’d like to see the debate on whether teacher unions are to blame for failing public schools, you can watch a video here. Just click on the tab that says “Audio/Video.” It’s better in living color than in a transcript.
Teacher Union Smackdown Video
March 25, 2010
1 Comment | 
teacher union | 
Permalink
Posted by Jay P. Greene
Vegas, Baby
March 23, 2010
I recently returned from an excellent conference organized by the Nevada Policy Research Institute in Las Vegas. As you can see in the above photo, we had a good time (I’m the one in the middle — the Jewish guy).
At the conference it dawned on me that health care is likely to have a huge, unintended effect on education policy. By placing significant new health care costs on states, the bill will almost certainly strain state finances. Since education is the only other really big expenditure in state budgets, look for states to become much more open to ways to economize on education than they have been. State may become much more interested in virtual education, choice, and other lower cost ways of delivering education. The days of regular annual increases in education spending are over. Once that happens the political landscape will almost certainly change.
Patrick Gibbons, the smart and energetic organizer of the conference, has already blogged on my health care/education trade-off ideas on the NRPI blog.
This was my first time in Vegas, so I should make a quick observation about the place. I think Las Vegas could best be described as a giant vacuum cleaner that sucks money out of people’s wallets. The people are glad to have the money vacuumed; they aren’t being robbed. It’s just that everything about the place has been carefully designed to extract as much money as people will willingly part with. If you don’t like gambling, they have shows. If you don’t have shows, they have shopping. They have food, they have prostitution, they have booze, they have spas, they have luxurious accommodations. Whatever you want to spend money on, they will provide the service and take your money. It is the Platonic form of commerce.
I know a lot of people have negative judgements about Vegas, saying that people are exploited or manipulated to give up their money. That wasn’t my experience. People were thrilled to have their money taken and seemed to enjoy the process knowing full well what was happening.
My sense is that it is no more exploitative than the local shopping mall in every city. It is only far more efficient and on a larger scale.
4 Comments | economics | Tagged: Las Vegas | Permalink
Posted by Jay P. Greene
Teacher Union Smackdown
March 23, 2010
Intelligence Squared sponsored a debate in NYC on March 16 on whether teacher unions were to blame for our failing schools. On the union side was Randi Weingarten and two union bosses whose names are not worth remembering. On the other side was a dream team of Terry Moe, Rod Paige, and Larry Sand.
Let’s just say that the debate wasn’t close. Before the debate the audience was polled and 24% believed teacher unions were not to blame, 43% believed they were to blame, and 33% were undecided. By the end of the evening 25% believed the teacher unions were not to be blamed, 68% believed they were, and 7% remained undecided. Given the quality of the arguments made by Moe, Paige, and Sand and the lame responses from Weingarten, et al, it’s easy to see how the union side gained virtually no supporters while the union-critics won over an additional 25% of the audience.
Here is Terry Moe’s opening salvo:
What we are saying is that the unions are and have long been major obstacles to real reform in the system. And we’re hardly alone in saying this. If you read “Newsweek,” “Time Magazine,” the “Washington Post,” lots of other well respected publications, they’re all saying the same thing: that the teachers unions are standing in the way of progress. So look. Let me start with an obvious example. The teachers unions have fought for all sorts of protections in labor contracts and in state laws that make it virtually impossible to get bad teachers out of the classroom. On average, it takes two years, $200,000, and 15% of the principal’s total time to get one bad teacher out of the classroom. As a result, principals don’t even try. They give 99% of teachers — no joke — satisfactory evaluations. The bad teachers just stay in the classroom. Well, if we figure that maybe 5% of the teachers, that’s a conservative estimate, are bad teachers nationwide, that means that 2.5 million kids are stuck in classrooms with teachers who aren’t teaching them anything. This is devastating. And the unions are largely responsible for that. They’re also responsible for seniority provisions in these labor contracts that among other things often allow senior teachers to stake a claim to desirable jobs, even if they’re not good teachers and even if they’re a bad fit for that school. The seniority rules often require districts to lay off junior people before senior people. It’s happening all around the country now. And some of these junior people are some of the best teachers in the district. And some of the senior people that are being saved are the worst. Okay. So just ask yourself, would anyone in his right mind organize schools in this way, if all they cared about was what’s best for kids? And the answer is no. But this is the way our schools are actually organized. And it’s due largely to the power of the unions. Now, these organizational issues are really important, but they’re just part of a larger set of problems. Our nation has been trying to reform the schools since the early 1980s. And the whole time the teachers’ unions have used their extraordinary power in the political process to try to block reform and make sure that real reform just never happens. Consider charter schools. There are many kids around this country who are stuck in schools that just aren’t teaching them. They need new options. Well, charter schools can provide them with those options. But charter schools are a threat to teachers’ unions. If you give kids choice and they can leave regular public schools, then they take money and they take jobs with them. And that’s what the teachers’ unions want to stop. So what they’ve done is they’ve used their power in the political process to put a ceiling on the numbers of charter schools. As a result in this country today, we have 4,600 charter schools. There are like well over 90,000 public schools. So this is a drop in the bucket. And mean time charter schools have huge waiting lists of people who are desperate to get in. In Harlem, for example, the charter schools there got 11,000 applications for 2,000 slots recently. So just to give you an idea of about how the politics of this works out, in Detroit a few years ago, a benefactor came forth and said he was willing to donate $200 million to set up additional charter schools for the kids in Detroit who obviously need it. What did the union do? The union went ballistic. They shut down the schools, went to Lansing, demonstrated in the state capitol and got the politicians to turn down the $200 million for those kids. This is good for kids? I don’t think so. This is about protecting jobs. The same kind of logic applies with accountability. Accountability is just common sense. We obviously need to hold schools and teachers accountability for teaching kids what they’re supposed to know. But the teachers’ unions find this threatening. They say they support accountability but they don’t want teachers held accountable. Any sensible effort to hold teachers accountable, they brand as scapegoating teachers. They don’t even want teachers performance to be measured. Right here in New York City, Joel Klein indicated a while ago that he was going to use student test scores as one factor in evaluating teachers or tenure. What did the union do? Now, this is something that Obama supports, that Arne Duncan supports. It’s unbelievable. What the union did is they went to Albany and they got their friends in the legislature to pass a law making it illegal to use student test scores in evaluating teachers for tenure anywhere in the state of New York. It’s just outrageous. And makes no sense from the standpoint of what’s best for kids. The “New York Times” called it absurd. This is how the unions approach accountability. Okay, well, I don’t have a whole lot of time left here. So let me just quickly say our opponents are going to say tonight, and Randi has already said, there is really no conflict between standing up for the jobs of teachers and doing what’s best for kids. But the thing is there is a conflict. And that’s why we can’t get bad teachers out of the classroom, because they protect them. That’s why the schools have totally perverse organizations imposed on them, and that’s why totally sensible reforms are seriously resisted in the political process. Now, what you’re going to hear, I’m sure, throughout the evening is that union leaders and unions around the country, they’re actually reformers too. They want to get bad teachers out of the classroom. They say they’re for charter schools; they’re all in favor of accountability. Well, not really. Talk is cheap. What counts is what they actually do. And what they do is to oppose reform. This is the reality.
A union boss from Lowell, MA responded:
I’m from the commonwealth of Massachusetts. Which I’m very proud to say is number one in the country. Our students perform higher than anybody else in this country academically. Yet we have the strongest collective bargaining rights in the country. So do me it just doesn’t add up. And then I started thinking I’m also a doctoral student so I’m trying to learn as much as I can about research and so the next thing I did is I went straight to the literature. Even Professor Moe said publications are all saying the same thing. There is no research to support what he is saying. There is no research out there that correlates student achievement to collective bargaining rights to teach unionism, either for or against.
And check out this exchange:
[Moderator]: And I want to begin with a couple of specifics — specific charges that were laid out there without responded to. Terry Moe specifically saying that teachers unions operate against the whole notion of charter schools, that they try to stop them wherever they find them. I want to hear from the other side, true or not true. Let’s start with Randi Weingarten.
Randi Weingarten: Well, given that the United Federation of Teachers under my watch, started two charter schools in Eastern York, it’s totally and completely untrue. What we want to do is we want charters to be held to the same accountability standards including the ones that we started, as any other school and what the evidence has been in New York, like the evidence around the country, is that charter schools instead of, as Diane Ravitch said, should take more of the most at-risk kids are actually taking fewer special needs kids and fewer kids with limited English proficiency. So we’ve open to, we think charters could be a great incubator for instructional practice and could be a great incubator for labor relations practice. But Terry, I don’t want New York to be as much as an evidentiary zone as Washington D.C. seems to be, which means let’s look at the Credo story which were done with a pro-charter advocate. What they said was, where 17 percent of the charters are better than public schools, 34 percent are worse, and the rest are the same. The idea is to actually find what works, make it sustainable and make it replicable. That’s what we’re trying to do and that’s what I’m trying to do.
[Moderator]: Terry Moe, Randi Weingarten is saying no, it’s not true that they are against all charter schools.
Terry Moe: Well let me first point out that New York State has a cap on the number of charter schools. It has a cap because this union put it there.
[applause]
Terry Moe: And even under the pressure of race to the top, they wouldn’t lift the cap. Right, so this is not an organization that’s in favor of charter schools. They’ve done everything they can to keep charter schools down. What they’re doing now in New York City is they’re running three charter schools to show if they can, that unionized charter schools can work, because what they want to do, is to unionize all the charter schools. That’s the only reason they’re doing it….
[applause]
Randi Weingarten: I mean, what’s interesting Terry is that I didn’t know you were in my head so much. We are not running charter schools to unionize all charter schools.
Terry Moe: Where’d the cap come from?
Eventually the debate is opened to questions from the audience and someone asks how many teachers had been fired for poor performance in New York state. Rather than answering Randi Weingarten begins to question whether the audience was packed with opponents, She says:
Well, but I think that the tone — what I have experienced in terms of New York City is that in a — most teachers right now, as we are speaking, are at home actually grading papers and marking lessons. And frankly, from my perspective when I was the teachers union president here, I never actually asked people to come or pack an audience or do these things, from my perspective.
When someone dodges a question and says she doesn’t have enough allies in the audience because they are busy working at home, you know it’s over. Put a fork in her, she’s done.
There are too many great arguments from Rod Paige and Larry Sand that I’ve left out, so I urge you to read the whole thing.
9 Comments | teacher union | Permalink
Posted by Jay P. Greene
The Wheels Are Coming Off the National Standards Train
March 21, 2010
Less than two weeks ago Andy Rotherham was declaring victory for a national standards consensus:
If the only person WaPo’s Nick Anderson can find to critique the push for common standards on the record is Susan Ohanian, does that mean it’s close to a done deal? That pierced my skepticism more than anything else in this process so far!
But not everyone jumped onto the national standards train. In fact, the wheels seem to be coming off. Strong resistance to adopting these national standards has developed in Minnesota, Virginia, Massachusetts, and California — joining Texas and Alaska who already declared their opposition.
Now the Wall Street Journal has joined the rising chorus of nat stand skeptics. Here’s the meat of their argument:
The biggest challenge may be reaching agreement on what a national curriculum should include. In the 1990s, the Bush and Clinton Administrations advocated national history standards. But the process became dominated by educators with a multicultural agenda preoccupied with political correctness and America’s failings. The Senate censured the history standards by a vote of 99 to 1. The recent brawl over the Texas social sciences curriculum suggests that what works in Nacogdoches isn’t going to fly in Marin County, and vice versa.
Under the 2002 No Child Left Behind Act, states are free to set their own standards, and it’s certainly true that some have dumbed-down their exams to meet the law’s requirements. The latest national standards effort is intended to correct this practice and ensure high-quality standards across all 50 states.
However, national standards won’t tell us anything we don’t already know about underperforming states. The U.S. already has a mandatory federal test in place—the National Assessment of Educational Progress exam (NAEP)—to expose states with weak standards. Mississippi may claim that 89% of its fourth graders are proficient in reading, according to the state test. But when NAEP scores show this is true of only 18% of fourth graders, Mississippi education officials aren’t fooling anyone.
It’s true that some countries with uniform standards (Singapore, Japan) outperform the U.S., though other countries with such standards (Sweden, Israel) do worse. On the 2007 eighth-grade TIMSS test, an international math exam, all eight countries that scored higher than the U.S. had national standards. But so did 33 of the 39 countries that scored lower. The U.S. is also commonly regarded as having the best higher education system in the world, though we lack national standards for colleges and universities.
National standards won’t magically boost learning in the U.S., and if this debate distracts attention from more effective reforms, then public education will be worse off. State and local educators don’t need more top-down control from Washington. They need the freedom and authority to close bad schools, recruit better teachers and pay them based on effectiveness rather than tenure.
Most important, families need more educational choices. Some 2,000 high schools are responsible for half of all drop-outs in America, and forcing those schools to compete for students and shape up or shut down is the main chance. Higher standards will be the fruit of such reforms, not the driver.
3 Comments | school accountability | Tagged: national standards | Permalink
Posted by Jay P. Greene
The Democrats’ War on Education Science
March 17, 2010
“I keep my brain in this pickle jar for safe-keeping.”
It was perfectly predictable but still sad to watch. The U.S. Senate voted 55-42 yesterday against continuing the DC voucher program. Among Republicans only Olympia Snowe voted against the program. Among Democrats (or Independents), Feinstein, Lieberman, Nelson, and Warner voted for the program.
Normally we hear that Republicans are engaging in a war on science — opposing stem cell research, questioning global warming claims, etc… But judging from the arguments that opponents made in yesterday’s debate, Democrats are also engaged in a war on science, at least a war on education science. They couldn’t be bothered to fully or accurately reference the U.S. Department of Education’s evaluation of the program that found significant benefits for voucher recipients after 3 years.
Instead, the quality of the opponents’ scientific reasoning was exemplified by Sen. Byron Dorgan of South Dakota. As you can see in this link to CSPAN coverage (starting around minute 21), he argues that there is no need for vouchers because our public school system is doing a great job. And we know this because graduates of American public schools were the people who put a man on the moon. I’m not sure what public school Wernher von Braun attended.
Dorgan goes on to reference the U.S. Department of Ed evaluation, but he leaves out the positive main finding and focuses only on a sub-group analysis of students who came from very low performing public schools. The point estimate for that sub-group analysis is positive but the sample is small and so the effect is not statistically significant.
I know I’m using big words that may be a little hard for the likes of Sen. Dorgan to grasp, but the blatant disregard for scientific evaluations of government programs demonstrated by Dorgan, Durbin and the rest of the program opponents shows that they are the ones engaging in a war on science.
UPDATE — Maybe my brain has been picked because Dorgan is from ND, not SD. Oops.
7 Comments | politics, vouchers | Permalink
Posted by Jay P. Greene
Peterson on Charters
March 16, 2010Paul Peterson has an excellent piece on charter schools and the merits of competition on education in today’s WSJ.
Here’s the money quote:
[Ravitch] offers a naïve and static view of markets. “It is in the nature of markets that some succeed, some are middling, and others fail,” she wrote.
Twentieth century economist Joseph Schumpeter saw it another way. In his view, it is in the nature of markets that middling firms are “creatively” destroyed by good firms, which are themselves eventually eliminated by still better competitors. Ignoring this basic economic principle, critics of charter schools and other forms of school choice see no hope for competition in education. These critics ask us to leave public schools alone apart from creating voluntary national standards—speed zones without traffic tickets, as it were.
Yet few doubt that public schools today are troubled, as the president noted on Saturday. What the president left out is that the performance of American high school students has hardly budged over the past 40 years, while the per-pupil cost of operating the schools they attend has increased threefold in real dollar terms. If school districts were firms operating in the market place, many would quickly fall victim to Schumpeter’s law of creative destruction.
Ms. Ravitch and other critics of school choice reverse causation by blaming the sad state of public schools on events that occurred long after schools had stagnated. They point, for example, to President Bush’s No Child Left Behind law (enacted in 2002), mayoral governance of schools recently instituted in some cities, and the creation of a small number (4,638) of charter schools that serve less than 3% of the U.S. school-age population.
To uncover what is wrong with American public schools one has to dig deeper than these recent developments in education. One needs to consider the impact of restrictive collective bargaining agreements that prevent rewarding good teachers and removing ineffective ones, intrusive court interventions, and useless teacher certification laws.
And then he delivers the evidence:
To identify the effects of a charter education, a wide variety of studies have been conducted. The best studies are randomized experiments, the gold standard in both medical and educational research. Stanford University’s Caroline Hoxby and Harvard University’s Thomas Kane have conducted randomized experiments that compare students who win a charter lottery with those who applied but were not given a seat. Winners and losers can be assumed to be equally motivated because they both tried to go to a charter school. Ms. Hoxby and Mr. Kane have found that lottery winners subsequently scored considerably higher on math and reading tests than did applicants who remained in district schools.
In another good study, the RAND Corp. found that charter high school graduation rates and college attendance rates were better than regular district school rates by 15 percentage points and eight percentage points respectively.
Instead of taking seriously these high quality studies, charter critics rely heavily on a report released in 2004 by the American Federation of Teachers (AFT). The AFT is hardly a disinterested investigator, and its report makes inappropriate comparisons and pays insufficient attention to the fact that charters are serving an educationally deprived segment of the population. Others base their criticism of charters on a report from an ongoing study by Stanford’s Center for Research on Education Outcomes (Credo), which found that there are more weak charter schools than strong ones. Though this report is superior to AFT’s study, its results are dominated by a large number of students who are in their first year at a charter school and a large number of charter schools that are in their first year of operation.
Credo’s work will be more informative when it presents findings for students in charters that have been up and running for several years. You can’t judge the long-term potential of schools that have not amassed a multi-year track record.
To identify the long-term benefits of school choice, Harvard’s Martin West and German economist Ludger Woessmann examined the impact of school choice on the performance of 15-year-old students in 29 industrialized countries. They discovered that the greater the competition between the public and private sector, the better all students do in math, science and reading. Their findings imply that expanding charters to include 50% of all students would eventually raise American students’ math scores to be competitive with the highest-scoring countries in the world.
What makes charters important today is less their current performance than their potential to innovate. Educational opportunity is about to be revolutionized by powerful notebook computers, broadband and the open-source development of curricular materials (a la Wikipedia). Curriculum can be tailored to the level of accomplishment each student has reached, an enormous step forward.
If American education remains stagnant, such innovations will spread slowly, if at all. If the charter world continues to expand, the competition between them and district schools could prove to be transformative.
1 Comment | charter schools | Tagged: Paul Peterson | Permalink
Posted by Jay P. Greene
It Was the Best of Administrations, It Was the Worst…
March 15, 2010
I can’t decide what to think about the Obama administration on education policy. This administration has said some of the best things about education reform I have heard out of any administration, but they have also said some of the worst things.
Take for example the plans for reauthorizing (or replacing) NCLB that came out over the weekend. Obama/Duncan have the good idea of getting rid of the unrealistic goal of universal proficiency in basic skills by all groups by 2014. But they have the bad idea of setting an even more unrealistic goal of universal college-readiness by 2020. (Mental note — be sure to set deadline for unrealistic goals several years after end of one’s possible era of responsibility. That way you are never responsible for failure. : ) )
They favor the good idea of focusing on growth in student achievement rather than percent proficient, but they endorse the bad idea of making the measures of achievement so mushy as to be useless, like including “learning environment” (whatever that is) in the measure and by wanting portfolio assessments.
They say they want to end micro-managing of schools from DC (not that this is really happening), but then they want national standards that would ultimately lead to a national curriculum, national testing, and national micro-managing.
They want to identify the worst schools and reorganize those schools, including firing bad teachers. They also want to use test data to identify and reward the best teachers and schools. This is all great! But they don’t spell out any details in their proposals and want to leave drafting of legislation to Congress where these good things will almost certainly be removed or made impotent.
They deplore racial disparities in educational outcomes, but rather than empower low income minority families with vouchers, they want to empower them to sue their schools. This sounds like No Lawyer Left Behind. Giving disadvantaged groups legal power rather than market power hasn’t worked well for special ed and it won’t work well for low income minorities.
So, which one is it? Is this the best education administration or the worst? Or is it somehow both? I can’t completely make up my mind, but one thing I do know — the good stuff they want is much less likely to happen than the bad stuff. In that case, I guess I’d rather have a federal government that did as little as possible with education.
[UPDATED] I left out the great idea in the new O’Duncan proposal that we get rid of “highly qualified” teacher requirements, which are understood as credentialing requirements and replace it with teacher quality assessments based on growth in test scores. Of course, this was one of the ideas that has made the unions come out strongly against the proposal.
10 Comments | politics | Permalink
Posted by Jay P. Greene
National Standards Nonsense
March 10, 2010
The national standards train-wreck is pulling into the station, again. This time it is a completely voluntary set of national standards in the same way that complying with a 21-year-old drinking age is completely voluntary for states to receive federal highway money. States had to commit to a rushed and largely secretive national standard setting process as part of the Race to the Top application.
Well, now the draft standards have been released for a hurried public comment period before they try to cram them into place. In the end they’ll probably fail to get all the states on board for anything meaningful, but it won’t be for lack of arm-twisting. The Gates Foundation has sprinkled money on just about every education policy organization to ensure their support or at least muted opposition.
Even people and groups that should have no interest in these national standards and even expressed skepticism of them in the recent past are now embracing them. Barely two weeks ago Checker Finn wrote:
This is enormously risky and, frankly, hubristic, since nobody yet has any idea whether these standards will be solid, whether the tests supposed to be aligned with them will be up to the challenge, or whether the “passing scores” on those tests will be high or low, much less how this entire apparatus will be sustained over the long haul.
But today he is quoted in the New York Times expressing his enthusiastic support:
I’d say this is one of the most important events of the last several years in American education… Now we have the possibility that, for the first time, states could come together around new standards and high school graduation requirements that are ambitious and coherent. This is a big deal.
What gives? Nothing in the draft standards should have put Checker at ease about their rigor. And nothing has happened that has addressed his earlier concerns about aligning tests, setting high cut scores, or sustaining rigor over time.
Similarly the folks over at Core Knowledge have decided to drink the Kool-Aid. Just a few months ago I expressed frustration with national standards advocates:
Every decade or so we have to debate the desirability of adopting national standards for education. People tend to be in favor of them when they imagine that they are the ones writing the standards. But when everyone gets into the sausage-making that characterizes policy formulation, it generally becomes clear that no one is going to get what they want out of national standards. What’s worse is that the resulting mess would be imposed on everyone. There’d be no more laboratory of the states, just uniform banality. Of course, some people always hope that they’ll somehow manage to sneak their preferred vision into place without having to go through the meat grinder.
At the time Core Knowledge’s Robert Pondiscio linked to that post and added “I’m inclined to agree.” But today he is the press contact for a statement from Core Knowledge declaring that the new draft national standards are a “not-to be-missed opportunity for American education.”
What’s even more amazing is that the draft national standards are being guided by the same 21st Century Skills nonsense articulated by Tony Wagner. Core Knowledge supporters should recoil in horror at this approach unless they fantasize that they will “somehow manage to sneak their preferred vision into place” without the edublob noticing and blocking them. Good luck.
I’ve seen this movie before and it doesn’t end well. The standards will inevitably be diluted and made even more 21st century skill-like to gain sufficiently broad support. The standards-based reformers at Fordham and Core Knowledge will end up renouncing the final product, but will continue to believe that if only the right standards were adopted all would be well. And we’ll start this all over again in about a decade.
Wash. Rinse. Repeat.
30 Comments | school accountability | Tagged: Core Knowledge, Fordham Institute, national standards | Permalink
Posted by Jay P. Greene
RTTT… Yawn
March 4, 2010
The Race to the Top finalist states were announced today. 15 states are in the hunt for some portion of $2.3 billion, which is less than one-half of a percent of annual K-12 education spending. It is rounding error.
The contest may shape state and local education policy debates where something might actually happen, but no one should be fooled into thinking that this money is going to have any significant, direct effect.
But it will certainly keep the chattering class busy with excitement. My reaction is… Yawn.
12 Comments | money myth | Permalink
Posted by Jay P. Greene
Feds And Research Shouldn’t Mix
March 2, 2010 
As head of a department that has received and may wish to continue receiving federal research funds, it is completely contrary to my self-interest to say this: the federal government should not be in the business of conducting or funding education policy research. The federal government should facilitate research by greatly expanding the availability of individual student data sets stripped of identifying information. But the federal government is particularly badly positioned to conduct or fund analyses based on those data.
The reasons for keeping the federal government out of education policy research should be obvious to everyone not blinded by the desire to keep eating at the trough. First, the federal government develops and advocates for particular education policies, so it has a conflict of interest in evaluating those policies. Even when those evaluations are outsourced to supposedly independent evaluators, they are never truly independent. The evaluation business is a repeat-play game, so everyone understands that they cannot alienate powerful political forces too much without risking future evaluation dollars. The safe thing to conclude in those circumstances is that the evidence is unclear about the effectiveness of a policy but future research is needed, which, not surprisingly, is what many federally funded evaluations find.
Unfortunately, political influence in education policy research is often more direct and explicit than the implicit distortions of a repeat-play game. Every federally funded evaluation with which I am familiar has been subject to at least some, subtle political influence.
I can’t mention most without breaking confidences, but I can briefly describe my own experience with a What Works Clearinghouse (WWC) panel on which I served (which was managed by a different firm than the one that currently manages WWC). On that panel we were supposed to identify what was known from the research literature on how to turn around failing schools. As we quickly discovered, there was virtually nothing known from rigorous research on how to successfully turn around failing schools. I suggested that we should simply report that as our finding — nothing is known. But we were told that the Department of Education wouldn’t like that and we had to say something about how to turn around schools. I asked on what basis we would draw those conclusions and was told that we should rely on our “professional judgment” informed by personal experience and non-rigorous research. So, we dutifully produced a report that was much more of a political document than a scientific one. We didn’t know anything from science about how to turn around schools, but we crafted a political answer to satisfy political needs.
In addition to being politically influenced, federally funded research is almost always overly expensive. The cost of federal education policy research is many-fold more expensive than that research has to be. There are several federal evaluations where the cost of the evaluation rivals the annual cost of the program being evaluated.
Beyond being politically distorted and cost-inefficient, a whole lot of federally funded research is really awful. In particular, I am thinking of the work of the federally funded regional research labs. For every useful study or review they release, there must be hundreds of drek. The regional labs are so bad that the Department of Education has been trying to eliminate them from their budget for years. But members of Congress want the pork, so they keep the regional labs alive.
Being politically distorted, cost-inefficient, and often of low quality is not a good combination. Let’s get the feds out of the research business. They can still play a critically important role of providing data sets to the research community, but they should not be funding evaluations or research summaries. We need the feds to help with data because privacy laws are too great of a barrier for individual researchers. But once basic data is available, the cost of analyzing the data should be quite low — just the time of the researchers and some computer equipment, perhaps supplemented with additional field data collection. And if there is no “official” evaluation or “official” summary of the research literature, the research community is free to examine the evidence and draw its own conclusions. Yes, there will be disagreement and messiness, but the world is uncertain and messy. Freedom is uncertain and messy. The solution is not to privilege over-priced, often lousy, politically driven federally funded work.
(edited for typos)
18 Comments | methodology, politics | Tagged: What Works Clearinghouse | Permalink
Posted by Jay P. Greene
Jay P. Greene's Blog 