Merit Pay Bust

September 22, 2010

For some time now I have expressed disillusionment with merit pay as an ed reform strategy. In a paper Stuart Buck and I produced last spring for a Harvard conference on performance incentives we wrote:

All of this leads us to measured skepticism about the merit of merit pay, unless coupled with other reforms such as competition between schools. After all, merit pay boils down to an attempt to recreate a market system within a tightly controlled state monopoly. This is an objective fraught with peril. Even if wise and benevolent state actors manage to get the incentives right at a particular moment in time in a particular place, their actions can always be undone by immediate successors. Those successors may well be more influenced by the powerful special interests that want to block merit pay, loosen the standards, or even to call a system “merit pay” while rewarding behavior that has no relation to actual achievement.

Now we have additional reasons for skepticism.  A well-designed random-assignment experiment led by Vanderbilt’s Matt Springer found:

While the general trend in middle school mathematics performance was upward over the period of the project, students of teachers randomly assigned to the treatment group (eligible for bonuses) did not outperform students whose teachers were assigned to the control group (not eligible for bonuses).

Keep in mind that this experiment only tests whether financial incentives increase teacher motivation, resulting in higher student achievement.  It does not address whether merit pay might change the composition of the teacher labor force, attracting and retaining more effective teachers.

Still, color me even more skeptical about the promise of merit pay as an ed reform strategy.  It may well be that the current crop of teachers we have believe that they are doing their best, so offering them money for trying harder doesn’t result in a significant change in effort.  And given the political and organizational barriers to merit pay, I hold out little hope that a well-designed program can be sustained long enough to effect the composition of the teacher labor market.

In the last week, I hope ed reformers have learned that we can’t really improve the school system by maintaining the same centralized system while trying to sneak a reformer into the control-room (a la Michelle Rhee).  And I also hope we’ve learned that we can’t tinker with the incentives within that same centralized system ( a la merit pay).  The key to effective reform is decentralization of control via school choice, including charters, vouchers, tax credits, weighted student funding, etc…
(edited for typos)

7 Comments | competitive effects, research reports, vouchers | Tagged: , , , | Permalink
Posted by Jay P. Greene

New Heritage Brief on the Racial Achievement Gap

September 18, 2010

 

(Guest Post by Matthew Ladner)

Back in 1997, Professor Lawrence Stedman wrote:

Twelfth-grade black students are performing at the level of middle school white students. These students are about to graduate, yet they lag four or more years behind in every area including math, science, writing, history, and geography. Latino seniors do somewhat better than 8th-grade white students in math and writing but, in other areas, are also four years behind white 12th graders…. Schools and society remains divided into two different worlds, one black, one white, separate and unequal.

Thirteen years later, sadly not much has changed with the national numbers, but some states have proven that far-reaching policy changes can reduce achievement gaps.

Lindsey Burke and I sing a new duet celebrating Florida’s reduction of the racial achievement gap  in a new Heritage brief.  Let’s just say the evidence from Florida is fairly compelling:

From the brief:

If trends since 1998 were to hold nationally, it would be about 33 years before we could expect Hispanics to close the gap with their white peers. In Florida, however, black students could catch up in half that time, and Hispanics could exceed the national average for white students as early as 2011.

This is just the sort of progress that the “Broader-Bolder” crowd would like us to believe is not possible without a vast expansion of the welfare state.

OOOOOOOOPS! Do you think we’re stupid Hans? It is accountability with teeth, real transparency and expanded parental choice that is making this happen. Cue the slo-mo fall scene-and please try not to make too big of a mess on the sidewalk.

 

2 Comments | competitive effects, research reports | Tagged: , , | Permalink
Posted by matthewladner

Next?

July 1, 2010

 (Guest Post by Matthew Ladner)

So despite the odd note of protest here and there, Florida’s McKay Scholarship program continued to march across the country this year with the birth of twin offspring programs in Oklahoma and Louisiana. There are many reasons for this progress, including information like this:

For those of you squinting at your IPAD (I wish I was as cool as you, by the way) the pretty chart that I can’t get the blog to make bigger shows the reading scores for Florida and the National average on NAEP for public school children with disabilities. Back in 1998, both Florida and the nation had only 24% of children with disabilities reading at Basic or Better. In the most recent 2009 test, the national average had improved to 34%, but the Florida average had improved to 45%. That means that a child with a disability is approximately 26% more likely to be reading by 4th grade than the national average. It’s worth mentioning that the national figure would look a bit worse if it were possible to exclude the Florida numbers.

As I have mentioned before, Florida’s progress has multiple sources, but we do have evidence linking the program to improvement in scores for disabilities in public schools. McKay helped, and certainly didn’t hurt.

So now the fun part: who will be next?

Florida, Ohio, Utah, Georgia, Arizona, Oklahoma and Louisiana have jumped in with private choice programs for children with disabilities. The water is fine! There are a number of other reforms to special education that states should undertake, including universal screening, but none of these are mutually exclusive with the McKay approach. States need to focus like a laser on early literacy skills, remediate children who are behind, get the diagnosis correct, and give the maximum amount of choice to children with special needs.

My guess is that the next state or states will be in Big 10 country. Indiana, Wisconsin or Ohio with an expanded program (Ohio currently has a voucher program for children with Autism). Maybe all of the above.

Make your prediction now for 2011. Winner gets a coveted JPGB No Prize!

5 Comments | competitive effects, special ed, vouchers | Tagged: , , | Permalink
Posted by matthewladner

Tampa Tribune: Competition Boosts Public Schools

June 21, 2010

(Guest Post by Matthew Ladner)

Check it out.

Leave a Comment » | competitive effects | Tagged: , | Permalink
Posted by matthewladner

Burke and Ladner Sing the real “Empire State of Mind” Duet on NRO

June 9, 2010

Now you’re in New York FLOR-I-DA!  Our minority children outscore your WHOLE STATE! There’s nothing we can’t do! 

(Guest Post by Matthew Ladner)

The Heritage Foundation’s Lindsey Burke and I hit National Review Online on Florida’s K-12 success in raising minority academic achievement.

In California, Meg Whitman won the Republican nomination for governor in overwhelming fashion on Tuesday. As you can see on her campaign site, Whitman wants to bring Florida reforms to California, which desperately needs them. California is a gigantic state that scores like an urban school district on NAEP. Without large improvements in California, it is unlikely that we will see the United States even begin to close the academic gap with European and Asian nations.

3 Comments | competitive effects, instructional reform, vouchers | Tagged: , , , , | Permalink
Posted by matthewladner

New Study Links Tax Credit to Florida Public School Gains

June 3, 2010

(Guest Post by Matthew Ladner)

A new study by David Figlio links higher gains among Florida public schools with higher levels of competition from the Step Up for Students tax credit program. You can read the St. Pete Times story by Ron Matus here.  Matus wrote:

Figlio emphasized the boost was significant, but modest.

“Anybody looking for a silver bullet has to keep looking,” he said. “What we find is certainly positive and statistically strong, but it’s not like public schools are revolutionizing overnight because of this, either.”

So it turns out that the public school gains associated with a state program with an initial statewide cap of $50m in a state with a multi-billion dollar public school budget were statistically significant but modest. Would it be reasonable to expect anything more from such a modest program? I suggest we scale this public school improvement program up to say a cool billion per year and then measure the impact.

My favorite line in the story comes from a hostile academic:

Another researcher remained skeptical. Stanford labor economist Martin Carnoy, who has studied the impact of vouchers and reviewed the latest study, said Figlio and Hart did “an honest job with the data.”

“But here is the real story: even after several years the effect size is TINY,” he wrote in an e-mail. “They are so small that even small downside effects would nullify them, leaving vouchers as mainly an ideological exercise.”

This is one of the more unintentionally hilarious statements I have read in some time. The field of education reform battle is covered with the dead bodies of reforms that show nothing in the way of a statistically significant impact. Increasing per pupil funding, Head Start, teacher certification, almost everything studied by the “What Works” clearinghouse so far, etc. All of these failures cost a great deal of money and deliver nothing in the way of sustained academic gains.

So the state of Florida passes a small law that actually saves the state money and shows a statistically significant and small result of improving public schools, and we are supposed to wring our hands and despair because something bad could come along and nullify the gains? Ummmmm, no.

First of all, nothing bad did come along and nullify the gains- quite the opposite. This program was only a part of the strategy to increase parental choice in Florida. That strategy also includes charter schools, McKay vouchers and virtual schooling- all of which either already are or soon will be much larger programs than Step Up for Students.

Second, the parental choice strategy was itself a part of a larger effort to improve Florida public schools. Parental choice reinforced the central K-12 reform of grading schools A-F. Transparency, rewards for success, consequences for failure formed the core of the Florida strategy.

Did it work?

The Step Up for Students program played a contributing role in Florida’s symphony of success rather than “destroying public education.”  This is what Milton Friedman argued all along. Bravo- the obvious conclusion to draw is to push both parental choice and public school reform still further in Florida and elsewhere.

3 Comments | competitive effects, tax-credit scholarships | Tagged: , , , | Permalink
Posted by matthewladner

New Florida Study Makes It 18-0

June 3, 2010

(Guest post by Greg Forster)

David Figlio’s study (with Cassandra Hart) on how the Florida Tax-Credit Scholarship program impacts public schools is finally out. Guess what? His detailed statistical analysis finds that competition from school choice improves public schools. (Here’s some local news coverage.)

But that was no surprise to anyone who’s been following the research. Early last year I counted up the studies and here’s what I got:

Removing the double-count for studies that had findings in multiple locations, that made it 16 studies finding school choice improves public schools to zero finding they hurt public schools. (The one null finding was in DC, where the program pays enormous cash bribes to the public system – apparently on the princple that children are the chattel property of the government school system – in order to deliberately neutralize its effect on public schools.)

After that, Jay came out with yet another study finding that Milwaukee vouchers have improved public schools. That brought it up to 17-0.

Now Figlio and Hart in Florida, adding the first study that looks at tax-credit scholarships rather than vouchers, have made it 18-0.

Oh, and in case you’re wondering, the Florida tax-credit program also dramatically improves education for the students who are using it.

As always, critics are trying to make hay out of the fact that in the Figlio/Hart study, a tiny, population-limited, regulation-cramped choice program produces only moderate-sized benefits. Well, geniuses, if the benefits of a tiny, population-limited, regulation-cramped program are too small for you, can you think of any way you might make the program’s impact bigger?

1 Comment | competitive effects, tax-credit scholarships, vouchers | Tagged: , , , , | Permalink
Posted by Greg Forster

Why Do Miami Kids Read a Grade Level Better than Oregon Kids?

June 2, 2010

(Guest Post by Matthew Ladner)

The NAEP released the 2009 Urban District NAEP results recently, which of course was an invitation to go exploring the data. I thought it would be interesting to look at the results for 4th grade reading.

So Charlotte, Miami and Austin come out looking pretty good among urban districts. Oh, and Oregon too. Silly me, I must have accidentally slipped the statewide average for all kids in Oregon into the comparison of urban school districts. When you throw in all the rich kids in Oregon into the mix, they look like a decent urban school district, although not, I will note, the best urban school district.

Perhaps a bit of control for demographic differences between these jurisdictions is in order. After all, some districts like Austin (and I suspect Charlotte) have quite a few affluent kids attending them. So in the next chart, I only look at free and reduced lunch eligible children in the districts for more of an apples to apples comparison.

So Miami wins overall with a score of 215 for FRL kids, followed closely by NYC at 214. Both of these scores exceed several statewide averages for all students- such as California’s. Miami not only was the low-income reading champion for 4th grade, but the both the low-income and the overall reading champion for 8th grade.

Oregon low-income kids perform **ahem** like a mid-tier urban district despite the inclusion of suburban kids, and approximately a grade level behind both Miami and NYC.  Some might also find it interesting that the Miami school district is 91% minority, while Oregon is 72% Anglo. 

I certainly do. Quite a bit actually.

When I read Bernard Lewis’ book What Went Wrong? about how the Islamic world went from being the premier civilization to an economic backwater, it seemed to me that Lewis had asked the wrong question. Most of the world, after all, is a backwater. The real question is What Went Right? with the West more than what went wrong in the Islamic world.

It behooves us to ask both questions in this case: what in the world is wrong with Oregon, and what is going right in Miami? I have a very good idea of what is going right in Miami. Good standards and testing, transparency, letter grade rankings for schools, parental choice, alternative certification, curtailment of social promotion. I don’t know what Oregon has been doing, but it looks to me like they should make some rather dramatic changes.

Leave a Comment » | competitive effects, instructional reform | Tagged: , , | Permalink
Posted by matthewladner

DC NAEP Scores-Where is the Death Spiral?

May 13, 2010

 

(Guest Post by Matthew Ladner)

No one seems to be taking much note of it, but some Washington DC has some very favorable trends in their NAEP scores.

To be sure, the District’s scores still reflect widespread academic failure on an inexcusable level for a district blowing through $20k per child per year. The positive trend predates Michelle Rhee’s tenure, which is good, as I think we are likely to see further (badly needed) progress. It is still too early to judge whether Rhee will accelerate this rate of progress, but I’d be willing to bet she will.

If you go to the NAEP page for DC and look at the 4th grade reading scores, you will find that the catastrophically low score of 188 in 1992 fell to an even more pathetic 179 in 1994.   That’s almost a grade level drop from an already low base. A score of 179 makes me wonder what the score would be if we simply gave every child in DC a library card and hoped for the best. Mind you, that wouldn’t work well either, but it couldn’t work that much worse than DCPS circa 1994. Since 1994, however, scores have climbed 23 points. The percentage scoring basic or better increased from 24% in 1994 to 44% in 2009. Math improvement has also been impressive and shows the same trend- progress after the mid 1990s.

One blindingly obvious cause for the improvement: the 100 charter schools operating in the district educating over 30,000 children. DC’s charter law passed in 1996 (near the bottom of DC performance) and the opening of schools has been very strong. In 1996-7, DCPS had 78,648 students enrolled. In 2007-08 it had dropped to 58,191.

This is no doubt why DCPS spending per pupil has spiralled to such absurdly high levels. No on apparently thought that it might be appropriate to cut the budget for a district that is 20,000 fewer students, but I digress. DC’s scores still stink, but in the progress department they have clobbered all states other than Delaware and Florida.

I’m not willing to celebrate a district that spends over $20k per student per year and has 56% of 4th graders illiterate. I am however willing to celebrate progress, and DC has momentum. If they would like to accelerate that progress, parental choice policies that would be helpful would be to reverse the shameful decision of the NEA robots majority of the Democratic caucus to kill the DC Opportunity Scholarship Program. The program merits not only renewal but a large expansion.

In addition, DC should institute a McKay Scholarship program with children with disabilities, if they would like to stop paying for the 5th homes and country club memberships of the attorneys endlessly battering DCPS on failure to provide FAPE under IDEA. Both the kids and the district budget would win big from such a program.

The enemies of parental choice have always painted the nightmare scenario of an academic death spiral for the children “left behind” in the district. Perhaps these same folks would like to explain to us now how it is that DCPS lost a quarter of their students since the mid 1990s and watched their reading scores improve by 23 points. Where is the death spiral? Oh, I mean in DCPS scores.  The death spiral for the credibility of choice opponents is impossible to miss.

Leave a Comment » | charter schools, competitive effects, vouchers | Tagged: , , , , | Permalink
Posted by matthewladner

Reformer’s Disease

April 14, 2010

I think I’ve discovered a new medical disorder that I call Reformer’s Disease.  Good and smart people involved in education reform can easily be stricken with this disorder in which they visualize a desirable reform policy and then imagine that they can simply impose that policy on our education system and that it will come out as they want.

In particular, I’ve noticed instances of Reformer’s Disease in discussions with folks over national standards as well as in Mike Petrilli’s recent post on Flypaper about teacher tenure reform. Advocates for national standards tend to imagine that the national standards that will be adopted are the ones they prefer.  And they further imagine that people whose vision of national standards they oppose will never take control of the standards in the future.  National standards advocates don’t seem to have any theory about how political systems operate, what kinds of standards those systems are likely to adopt, or how those systems are likely to alter standards in the future.  Instead, these victims of Reformer’s Disease have grown tired of politics and simply imagine that they will be the puppeteers who will get the educational system to do the right things without having to think about how the incentives and structure of that system may well thwart or pervert their efforts.

Similarly, Mike Petrilli shows signs of Reformer’s Disease in his post on teacher tenure reform.  He asks, “Rather than use choice to set in motion a chain reaction that ends with the removal of bad teachers from the classroom, why not go right at the bad teachers themselves?”  Why focus on structures, incentives, and politics when we can just get schools to do the right thing — remove bad teachers, adopt the right standards and curricula, etc…?

Perhaps Mike’s question can best be answered by transplanting this discussion to a different industry.  Why should we bother with all of this choice and competition among restaurants when we can just get right at ensuring that bad chefs are removed?  Why have all of these different restaurants with their varying style and quality when we can just ensure quality through national restaurant standards?

Of course, when we transplant the discussion to restaurants the answer to Mike’s question seems obvious.  We need choice and competition because it helps impose the proper incentives on decision-makers within the educational system to make the right choices.  With stronger choice and competition bad teachers are more likely to be removed because keeping bad teachers would harm the interests of their bosses by causing schools to lose students and revenue.  The main barrier to removing bad teachers is not tenure, per se; it is the lack of incentives to remove bad teachers that allows the tenure system to be adopted and continue.  Just removing tenure would not rid the system of bad teachers because principals, superintendents, and others up the chain have little to no incentive to fire bad teachers.

Yes, schools need to get rid of bad teachers and the tenure that protects them.  Yes, schools need solid standards and curricula.  But people need to avoid Reformer’s Disease and remember that they can’t simply impose solutions on an unwilling system governed by perverse incentives.  Choice and competition are not at odds with tenure reform or standards reform.  Competition is a necessary part of how one actually accomplishes and sustains those other reforms.

1 Comment | competitive effects, instructional reform, vouchers | Tagged: , , | Permalink
Posted by Jay P. Greene

« Previous Entries
Next Entries »