The Determined Pessimism of Rick and Mike

September 23, 2010

(Guest Post by Matthew Ladner)

My friends Mike Petrilli and Rick Hess have been either (a) cautioning people about becoming overly optimistic about Waiting for Superman  and our ability to improve K-12 outcomes or (b) ridiculing the idea completely.

Hmmmm…

Let me begin by saying that I am no starry-eyed naif when it comes to the possible impact of the film. I wrote the other day that I am starting to entertain the idea that it might be a big deal. Union reactionaries do find themselves increasingly isolated in K-12 policy discussions, and many of their catspaws will be turned out of office in November.

Let me say in advance however that the unions are not going anywhere. They still control hundreds of millions of dollars, legions of organized activists and all the lobbyists that they care to employ. I’m not claiming that a tipping point has been achieved and it is all downhill from here for them, merely that they are in for what could prove to be a sizeable rough patch.

Where I seem to differ with Mike and Rick is with their seeming determined pessimism regarding the realm of the possible for improvement. Rick and I appeared on a panel together at the State Policy Network a couple of weeks ago, and discussed the same issue.

Readers of this blog find themselves subjected to my battering away with Florida’s NAEP scores on a regular basis. I won’t bother going into the litany because you already know it, so let’s take a couple of other examples where real reform agendas have been instituted, and what has been going on with their NAEP scores.

I pick a couple because, well, only a few exist. You have to be in a position to roll the establishment to do these things, and keep them rolled. Very few have pulled that off. However, the results are encouraging.

I am encouraged that New York City now outscores some statewide averages on NAEP, despite a student body that is 84 percent minority and 85% FRL eligible. NYC kids scored 217 on 4th grade reading in 2009, only 206 in 2002. That’s a meaningful difference, and should embolden Chancellor Klein.

Likewise, DCPS is still an academic blight, but has made substantial progress since the mid 1990s. When my coauthors and I tracked the learning gains of general education low-income students for the 50 states and DC from 2003 to 2009 in all four main NAEP subjects, Florida came in with the biggest gains and DC came in with the second largest gains. Coincidence? I doubt it- both Florida and DC have engaged in far-reaching reforms.

MA is justifiably proud of having the nation’s highest NAEP scores accompanying their standards-led reforms. It has been mentioned before that the usual suspects fiercely opposed their adoption.

Notice that there is no one path up the mountain here-but there are some common threads to the reforms: testing, accountability, choice. So maybe I’m like Ronald Reagan and I just think that there has got to be a pony somewhere in all that manure. It seems to me, however, that there is a pattern here: in the limited number of instances when jurisdictions take control of policy away from the reactionaries, keep it away from them for a sustained period, and implement reforms that they hate, NAEP scores make substantial improvement.

My own experience in interacting with lawmakers, candidates and philanthropists around the country is that they almost all like substantial improvement in NAEP scores. It doesn’t matter whether they are on the right or left or center. The funny thing is that everyone but those directly benefiting from the status-quo seem to not only want improvement, many of them are willing to fight for it.

So have we “cracked the code.” Yes, as a matter of fact, I think a few places have done so. Yes with fantastic difficulty and always imperiled sustainability. The success of reformers is limited and fragile, but very real. If the third largest state in the union doesn’t represent “results at scale” then what pray tell does? 

We have learned a great deal over the past 20 years. Our decisions are being guided less by theory and more by experience. Less and less this is less about “Assume a can opener” and more and more about “You know, they did something like that in X, let’s see what we can learn about the results.”

If throwing money at schools, lowering class sizes, expanding preschool, open classrooms, whole language or <fill in the blank here> had produced these types of results, this blog would not exist. There would be no need for an education reform cottage industry, and no one would donate to it. They failed. It’s too bad, because I would much rather be spending my life an executive at Rhino Records putting together compliation CD’s of punk rock bands covering all of Dean Martin’s greatest hits. The cover would have a guy in a tux holding up a martini above a violent mosh pit.

A’int Love a Kick in the Head? Oi….let me demonstrate! But I digress…

Our ideas have barely been tried, and very rarely in sustained concert with each other. Unless someone is able to demonstrate the Florida NAEP, the DCPS NAEP, and the Trial Urban District Assessment NAEP for NYC and Miami have all been cooked, the only reasonable conclusion to reach is that unions hate policies that succeed in substantially improving the education of children.

There have been and will continue to be misteps. There will be gains and losses along the way. This is a war, and war is hell. The unions are not going away, but neither are we nor the evidence of our successes. As Dino’s pally Frank used to say, the best is yet to come.


Merit Pay Bust

September 22, 2010

For some time now I have expressed disillusionment with merit pay as an ed reform strategy. In a paper Stuart Buck and I produced last spring for a Harvard conference on performance incentives we wrote:

All of this leads us to measured skepticism about the merit of merit pay, unless coupled with other reforms such as competition between schools. After all, merit pay boils down to an attempt to recreate a market system within a tightly controlled state monopoly. This is an objective fraught with peril. Even if wise and benevolent state actors manage to get the incentives right at a particular moment in time in a particular place, their actions can always be undone by immediate successors. Those successors may well be more influenced by the powerful special interests that want to block merit pay, loosen the standards, or even to call a system “merit pay” while rewarding behavior that has no relation to actual achievement.

Now we have additional reasons for skepticism.  A well-designed random-assignment experiment led by Vanderbilt’s Matt Springer found:

While the general trend in middle school mathematics performance was upward over the period of the project, students of teachers randomly assigned to the treatment group (eligible for bonuses) did not outperform students whose teachers were assigned to the control group (not eligible for bonuses).

Keep in mind that this experiment only tests whether financial incentives increase teacher motivation, resulting in higher student achievement.  It does not address whether merit pay might change the composition of the teacher labor force, attracting and retaining more effective teachers.

Still, color me even more skeptical about the promise of merit pay as an ed reform strategy.  It may well be that the current crop of teachers we have believe that they are doing their best, so offering them money for trying harder doesn’t result in a significant change in effort.  And given the political and organizational barriers to merit pay, I hold out little hope that a well-designed program can be sustained long enough to effect the composition of the teacher labor market.

In the last week, I hope ed reformers have learned that we can’t really improve the school system by maintaining the same centralized system while trying to sneak a reformer into the control-room (a la Michelle Rhee).  And I also hope we’ve learned that we can’t tinker with the incentives within that same centralized system ( a la merit pay).  The key to effective reform is decentralization of control via school choice, including charters, vouchers, tax credits, weighted student funding, etc…
(edited for typos)

New Heritage Brief on the Racial Achievement Gap

September 18, 2010

 

(Guest Post by Matthew Ladner)

Back in 1997, Professor Lawrence Stedman wrote:

Twelfth-grade black students are performing at the level of middle school white students. These students are about to graduate, yet they lag four or more years behind in every area including math, science, writing, history, and geography. Latino seniors do somewhat better than 8th-grade white students in math and writing but, in other areas, are also four years behind white 12th graders…. Schools and society remains divided into two different worlds, one black, one white, separate and unequal.

Thirteen years later, sadly not much has changed with the national numbers, but some states have proven that far-reaching policy changes can reduce achievement gaps.

Lindsey Burke and I sing a new duet celebrating Florida’s reduction of the racial achievement gap  in a new Heritage brief.  Let’s just say the evidence from Florida is fairly compelling:

From the brief:

If trends since 1998 were to hold nationally, it would be about 33 years before we could expect Hispanics to close the gap with their white peers. In Florida, however, black students could catch up in half that time, and Hispanics could exceed the national average for white students as early as 2011.

This is just the sort of progress that the “Broader-Bolder” crowd would like us to believe is not possible without a vast expansion of the welfare state.

OOOOOOOOPS! Do you think we’re stupid Hans? It is accountability with teeth, real transparency and expanded parental choice that is making this happen. Cue the slo-mo fall scene-and please try not to make too big of a mess on the sidewalk.

 


Next?

July 1, 2010

 (Guest Post by Matthew Ladner)

So despite the odd note of protest here and there, Florida’s McKay Scholarship program continued to march across the country this year with the birth of twin offspring programs in Oklahoma and Louisiana. There are many reasons for this progress, including information like this:

For those of you squinting at your IPAD (I wish I was as cool as you, by the way) the pretty chart that I can’t get the blog to make bigger shows the reading scores for Florida and the National average on NAEP for public school children with disabilities. Back in 1998, both Florida and the nation had only 24% of children with disabilities reading at Basic or Better. In the most recent 2009 test, the national average had improved to 34%, but the Florida average had improved to 45%. That means that a child with a disability is approximately 26% more likely to be reading by 4th grade than the national average. It’s worth mentioning that the national figure would look a bit worse if it were possible to exclude the Florida numbers.

As I have mentioned before, Florida’s progress has multiple sources, but we do have evidence linking the program to improvement in scores for disabilities in public schools. McKay helped, and certainly didn’t hurt.

So now the fun part: who will be next?

Florida, Ohio, Utah, Georgia, Arizona, Oklahoma and Louisiana have jumped in with private choice programs for children with disabilities. The water is fine! There are a number of other reforms to special education that states should undertake, including universal screening, but none of these are mutually exclusive with the McKay approach. States need to focus like a laser on early literacy skills, remediate children who are behind, get the diagnosis correct, and give the maximum amount of choice to children with special needs.

My guess is that the next state or states will be in Big 10 country. Indiana, Wisconsin or Ohio with an expanded program (Ohio currently has a voucher program for children with Autism). Maybe all of the above.

Make your prediction now for 2011. Winner gets a coveted JPGB No Prize!


Tampa Tribune: Competition Boosts Public Schools

June 21, 2010

(Guest Post by Matthew Ladner)

Check it out.


Burke and Ladner Sing the real “Empire State of Mind” Duet on NRO

June 9, 2010

Now you’re in New York FLOR-I-DA!  Our minority children outscore your WHOLE STATE! There’s nothing we can’t do! 

(Guest Post by Matthew Ladner)

The Heritage Foundation’s Lindsey Burke and I hit National Review Online on Florida’s K-12 success in raising minority academic achievement.

In California, Meg Whitman won the Republican nomination for governor in overwhelming fashion on Tuesday. As you can see on her campaign site, Whitman wants to bring Florida reforms to California, which desperately needs them. California is a gigantic state that scores like an urban school district on NAEP. Without large improvements in California, it is unlikely that we will see the United States even begin to close the academic gap with European and Asian nations.


New Study Links Tax Credit to Florida Public School Gains

June 3, 2010

(Guest Post by Matthew Ladner)

A new study by David Figlio links higher gains among Florida public schools with higher levels of competition from the Step Up for Students tax credit program. You can read the St. Pete Times story by Ron Matus here.  Matus wrote:

Figlio emphasized the boost was significant, but modest.

“Anybody looking for a silver bullet has to keep looking,” he said. “What we find is certainly positive and statistically strong, but it’s not like public schools are revolutionizing overnight because of this, either.”

So it turns out that the public school gains associated with a state program with an initial statewide cap of $50m in a state with a multi-billion dollar public school budget were statistically significant but modest. Would it be reasonable to expect anything more from such a modest program? I suggest we scale this public school improvement program up to say a cool billion per year and then measure the impact.

My favorite line in the story comes from a hostile academic:

Another researcher remained skeptical. Stanford labor economist Martin Carnoy, who has studied the impact of vouchers and reviewed the latest study, said Figlio and Hart did “an honest job with the data.”

But here is the real story: even after several years the effect size is TINY,” he wrote in an e-mail. “They are so small that even small downside effects would nullify them, leaving vouchers as mainly an ideological exercise.”

This is one of the more unintentionally hilarious statements I have read in some time. The field of education reform battle is covered with the dead bodies of reforms that show nothing in the way of a statistically significant impact. Increasing per pupil funding, Head Start, teacher certification, almost everything studied by the “What Works” clearinghouse so far, etc. All of these failures cost a great deal of money and deliver nothing in the way of sustained academic gains.

So the state of Florida passes a small law that actually saves the state money and shows a statistically significant and small result of improving public schools, and we are supposed to wring our hands and despair because something bad could come along and nullify the gains? Ummmmm, no.

First of all, nothing bad did come along and nullify the gains- quite the opposite. This program was only a part of the strategy to increase parental choice in Florida. That strategy also includes charter schools, McKay vouchers and virtual schooling- all of which either already are or soon will be much larger programs than Step Up for Students.

Second, the parental choice strategy was itself a part of a larger effort to improve Florida public schools. Parental choice reinforced the central K-12 reform of grading schools A-F. Transparency, rewards for success, consequences for failure formed the core of the Florida strategy.

Did it work?

The Step Up for Students program played a contributing role in Florida’s symphony of success rather than “destroying public education.”  This is what Milton Friedman argued all along. Bravo- the obvious conclusion to draw is to push both parental choice and public school reform still further in Florida and elsewhere.


New Florida Study Makes It 18-0

June 3, 2010

(Guest post by Greg Forster)

David Figlio’s study (with Cassandra Hart) on how the Florida Tax-Credit Scholarship program impacts public schools is finally out. Guess what? His detailed statistical analysis finds that competition from school choice improves public schools. (Here’s some local news coverage.)

But that was no surprise to anyone who’s been following the research. Early last year I counted up the studies and here’s what I got:

Removing the double-count for studies that had findings in multiple locations, that made it 16 studies finding school choice improves public schools to zero finding they hurt public schools. (The one null finding was in DC, where the program pays enormous cash bribes to the public system – apparently on the princple that children are the chattel property of the government school system – in order to deliberately neutralize its effect on public schools.)

After that, Jay came out with yet another study finding that Milwaukee vouchers have improved public schools. That brought it up to 17-0.

Now Figlio and Hart in Florida, adding the first study that looks at tax-credit scholarships rather than vouchers, have made it 18-0.

Oh, and in case you’re wondering, the Florida tax-credit program also dramatically improves education for the students who are using it.

As always, critics are trying to make hay out of the fact that in the Figlio/Hart study, a tiny, population-limited, regulation-cramped choice program produces only moderate-sized benefits. Well, geniuses, if the benefits of a tiny, population-limited, regulation-cramped program are too small for you, can you think of any way you might make the program’s impact bigger?


Why Do Miami Kids Read a Grade Level Better than Oregon Kids?

June 2, 2010

(Guest Post by Matthew Ladner)

The NAEP released the 2009 Urban District NAEP results recently, which of course was an invitation to go exploring the data. I thought it would be interesting to look at the results for 4th grade reading.

So Charlotte, Miami and Austin come out looking pretty good among urban districts. Oh, and Oregon too. Silly me, I must have accidentally slipped the statewide average for all kids in Oregon into the comparison of urban school districts. When you throw in all the rich kids in Oregon into the mix, they look like a decent urban school district, although not, I will note, the best urban school district.

Perhaps a bit of control for demographic differences between these jurisdictions is in order. After all, some districts like Austin (and I suspect Charlotte) have quite a few affluent kids attending them. So in the next chart, I only look at free and reduced lunch eligible children in the districts for more of an apples to apples comparison.

So Miami wins overall with a score of 215 for FRL kids, followed closely by NYC at 214. Both of these scores exceed several statewide averages for all students- such as California’s. Miami not only was the low-income reading champion for 4th grade, but the both the low-income and the overall reading champion for 8th grade.

Oregon low-income kids perform **ahem** like a mid-tier urban district despite the inclusion of suburban kids, and approximately a grade level behind both Miami and NYC.  Some might also find it interesting that the Miami school district is 91% minority, while Oregon is 72% Anglo. 

I certainly do. Quite a bit actually.

When I read Bernard Lewis’ book What Went Wrong? about how the Islamic world went from being the premier civilization to an economic backwater, it seemed to me that Lewis had asked the wrong question. Most of the world, after all, is a backwater. The real question is What Went Right? with the West more than what went wrong in the Islamic world.

It behooves us to ask both questions in this case: what in the world is wrong with Oregon, and what is going right in Miami? I have a very good idea of what is going right in Miami. Good standards and testing, transparency, letter grade rankings for schools, parental choice, alternative certification, curtailment of social promotion. I don’t know what Oregon has been doing, but it looks to me like they should make some rather dramatic changes.


DC NAEP Scores-Where is the Death Spiral?

May 13, 2010

 

(Guest Post by Matthew Ladner)

No one seems to be taking much note of it, but some Washington DC has some very favorable trends in their NAEP scores.

To be sure, the District’s scores still reflect widespread academic failure on an inexcusable level for a district blowing through $20k per child per year. The positive trend predates Michelle Rhee’s tenure, which is good, as I think we are likely to see further (badly needed) progress. It is still too early to judge whether Rhee will accelerate this rate of progress, but I’d be willing to bet she will.

If you go to the NAEP page for DC and look at the 4th grade reading scores, you will find that the catastrophically low score of 188 in 1992 fell to an even more pathetic 179 in 1994.   That’s almost a grade level drop from an already low base. A score of 179 makes me wonder what the score would be if we simply gave every child in DC a library card and hoped for the best. Mind you, that wouldn’t work well either, but it couldn’t work that much worse than DCPS circa 1994. Since 1994, however, scores have climbed 23 points. The percentage scoring basic or better increased from 24% in 1994 to 44% in 2009. Math improvement has also been impressive and shows the same trend- progress after the mid 1990s.

One blindingly obvious cause for the improvement: the 100 charter schools operating in the district educating over 30,000 children. DC’s charter law passed in 1996 (near the bottom of DC performance) and the opening of schools has been very strong. In 1996-7, DCPS had 78,648 students enrolled. In 2007-08 it had dropped to 58,191.

This is no doubt why DCPS spending per pupil has spiralled to such absurdly high levels. No on apparently thought that it might be appropriate to cut the budget for a district that is 20,000 fewer students, but I digress. DC’s scores still stink, but in the progress department they have clobbered all states other than Delaware and Florida.

I’m not willing to celebrate a district that spends over $20k per student per year and has 56% of 4th graders illiterate. I am however willing to celebrate progress, and DC has momentum. If they would like to accelerate that progress, parental choice policies that would be helpful would be to reverse the shameful decision of the NEA robots majority of the Democratic caucus to kill the DC Opportunity Scholarship Program. The program merits not only renewal but a large expansion.

In addition, DC should institute a McKay Scholarship program with children with disabilities, if they would like to stop paying for the 5th homes and country club memberships of the attorneys endlessly battering DCPS on failure to provide FAPE under IDEA. Both the kids and the district budget would win big from such a program.

The enemies of parental choice have always painted the nightmare scenario of an academic death spiral for the children “left behind” in the district. Perhaps these same folks would like to explain to us now how it is that DCPS lost a quarter of their students since the mid 1990s and watched their reading scores improve by 23 points. Where is the death spiral? Oh, I mean in DCPS scores.  The death spiral for the credibility of choice opponents is impossible to miss.


Follow

Get every new post delivered to your Inbox.

Join 940 other followers