New Score: Tom Vander Ark 3, New Gates PLDD Strategy 0

March 19, 2013

The research score continues to run-up in favor of the old Gates education reform strategy of creating small schools of choice rather than the new Gates PLDD strategy of centrally determining what students should be taught (Common Core) and how teachers should be evaluated (Measuring Effective Teachers).  When Tom Vander Ark led the Gates education effort, they had a winning strategy.

The new evidence comes from another paper that was presented at the Association for Education Finance and Policy conference.  Two of my excellent students, Anna Jacob Egalite and Brian Kisida, received a data award from the Kingsbury Center to analyze the effect of school size on student achievement using NWEA test results.  They used a student-fixed-effects research design to see if students improved or worsened their academic achievement when they switched to a school of a different size.  And the results:

We found consistent negative effects of large school size on student math and reading achievement, especially in secondary schools that enroll more than 540 students. In grades 6-10, for example, math achievement declined by -.043 SD (standard deviation) and reading achievement declined by -.024 SD.

If a student moved from a largest quintile school in grades 6-10 to a school in the smallest quintile for those grades, we would expect a 6.4% of a standard deviation improvement in math performance.  Of course, a student fixed effects study is not quite as strong methodologically as the random-assignment evaluation of small high schools in New York City, but it is pretty darn good.  And this study has the advantage of using a large data base of more than 2 million students from across the country.  It’s pretty clear that students would benefit significantly from a reduction on school size — especially junior high and high school students.

Bring back Tom Vander Ark.


Bring Back Tom Vander Ark

March 18, 2013

Under Tom Vander Ark‘s leadership the Gates Foundation pursued an education reform strategy focused on creating smaller high schools.  The theory was that smaller high schools would create tighter social bonds between schools and students, preventing students from slipping through the cracks and increasing the likelihood that they would graduate and go on to college.  Smaller high schools could also be more varied in their approaches and offerings, allowing students to choose schools that best fit their needs.

But around the same time Vander Ark left the Gates Foundation at the end of 2006, the reform strategy shifted.  Rather than fostering small, diverse schools of choice, the Gates Foundation now wanted to build centralized systems of what everyone should be taught (Common Core) and centralized systems of evaluating, training, and promoting teachers (Measuring Effective Teachers).  As I’ve written before, the shift in Gates strategy was not prompted by research.  In fact, the high quality random-assignment study that Gates had commissioned to evaluate the small high school strategy showed strong, positive results.  But the post-Vander Ark leadership at Gates couldn’t wait for the evidence.  The knew the truth without any pesky research and had abandoned the small high schools strategy in favor of their new centralization approach years before those results were released.

Well, the evidence continues to pile up that the Gates strategy under Tom Vander Ark was effective and the new strategy is a failure.  A new National Bureau of Economic Research study by Lisa Barrow, Amy Claessens, and Diane Whitmore Schanzenbach examines the effects of small schools in Chicago.  Just like the earlier random-assignment study of small high schools in New York City had found and like the Vander Ark theory of reform had suggested, small schools promote high school graduation:

We find that small schools students are substantially more likely to persist in school and eventually graduate. Nonetheless, there is no positive impact on student achievement as measured by test scores. The finding of no test score improvement but a strong improvement in school attainment is consistent with a growing literature suggesting that interventions aimed at older children are more effective at improving their non-cognitive skills than their cognitive skills.

As Vander Ark had expected, smaller schools have non-achievement effects, like creating stronger social bonds, that help students go further in their schooling.  And as numerous studies have shown, higher educational attainment is strongly predictive of a host of good outcomes for students later in their lives. Score: Tom Vander Ark 2, New Gates PLDD Strategy 0.

Meanwhile I witnessed further confirmation of the failure of the new Gates approach during a panel at the Association for Education Finance and Policy conference last weekend.  The Gates folks spent more than an hour presenting their Measuring Effective Teachers work.  That stuff may win over gullible policymakers and journalists, but the researchers at AEFP were not impressed.

Tim Sass had the first question and essentially repeated my concerns about how MET fails to provide evidence for the use of multiple measures.  Value-added test scores are predictive of later life earnings, as Chetty, et al have shown, he said, but why should we believe that classroom observations measure anything we care about?  The Gates folks didn’t really have an answer.  Jane Hannaway then articulated my concern that effective teaching may be too context-dependent to lend itself to a single formula for effective practices.  The Gates folks responded that there are probably some basic skills for effective teaching that are useful in all contexts.  They may have a point but that does not address whether MET is getting at those basic skills or not.  The weak correlations of everything in the study suggest they are not finding approaches that are commonly effective.

A third questioner wondered about the cost of adopting the MET approach, especially given the need for multiple observations by multiple, trained raters.  The response was that schools are already spending money on classroom observations but of course that does not address the extra costs of the multiple rater/observations approach.

And lastly William Mathis asked about the generally weak correlations between classroom observations and value-added test scores.  Doesn’t this suggest that these are distinct dimensions of effective teaching that shouldn’t be combined in a single measure, he wondered.  The Gates response surprised me.  They said that the weak correlations were good news.  It is precisely because classroom observations and value-added measures capture different dimensions of effective teaching that we need to combine them in an overall measure.

Of course, this ignores the Sass question about whether we know that the classroom observations are measuring any important aspect of effective teaching.  But more problematic was the “heads I win, tails you lose” nature of their response.  If earlier arguments defending MET were based on how classroom observations and VAM were correlated, then how could the lack of correlation also be presented as proof of MET’s success?  I went up to the Gates presenters after the panel (and chatted with some of them later) and asked them what MET could have found that would have led them to conclude that combining the measures was a bad idea.  They were stumped.  Maybe negative correlations would have dissuaded them from advocating a combined measure, but they weren’t confident about that either.

Essentially, they admitted that the MET policy recommendation is a non-falsifiable claim.  No research finding would have dissuaded them from it.  The ability to falsify a claim is at the heart of science.  MET is not science; it is just politics.  You should have seen the discomfort of the Gates researchers as they pondered why they were presenting non-falsifiable claims as research findings at an academic conference.  This sort of thing corrupts science and has reputational consequences for the researchers who lend their credibility to the new evidence-free Gates reform strategy.

There is a solution to the rot at Gates — Bring back Tom Vander Ark.  At least his ideas were supported by rigorous research.


Where Have You Gone Mr. Moynihan? Our Nation Turns its lonely eyes to you

March 12, 2013

(Guest Post by Matthew Ladner)

The lovely Mrs. Ladner gave me Daniel Patrick Moynihan: A Portrait in Letters of an American Visionary for Christmas. Once I started reading it, a found it quite difficult to put down the 702 page tome. DPM’s career as advisor to Presidents Kennedy, Johnson, Nixon, Ambassador to India and the United Nations and United States Senator is interesting in itself. The real value of the book however lies in the letters of the scholarly Senator providing a first hand account of how Washington elites wrestled with the issues that still consume us today.

Despite the fact that Johnson viewed DPM somewhat suspiciously as an Irish Catholic “Kennedy man” Moynihan’s memoranda to President Johnson especially jump off the page. Moynihan laid out in detail why the path to the Civil Rights Act was comparatively easy compared to what faced the nation going forward. Americans broadly support equality of opportunity he explained but the reform coalition would inevitably fragment over attempts to provide equality of condition.

Citing data from the United States Armed Services exam, he attempted to steel the resolve of the Johnson administration that American Blacks had been so profoundly damaged by the legacy of slavery and Jim Crow that they were poorly situated to thrive in an atmosphere of free competition. DPM estimated that the nation faced a 50 year project to right this wrong, and that it would be enormously controversial, but that the administration may as well get on with it.  You can judge for yourself whether this was a wise course to follow, but Moynihan’s clear-eyed prescience regarding everything leading up to the Civil Rights Act having been the easy part of the struggle is breathtaking in its stark clarity.

During this same period however Moynihan began to sound the alarm concerning the dissolution of the Black family, predicting that the trend would ultimately undermine anti-poverty efforts. DPM gained lifelong enemies on the left for having dared to speak such a truth, only some of whom ever apologized to him years and even decades after it had become abundantly clear that he had been quite correct.

DPM’s cross-species jump to become a prominent policy advisor in the Nixon White House is amusing to watch. Many of DPM’s memos contain dark warnings concerning the New Left campus radicals. While DPM clearly viewed such people as illiberal thugs, as someone born in 1967 I can’t quite decide whether he genuinely viewed such people as a threat to democracy or whether he was simply manipulating a group of gullible Republicans.  Perhaps both and someone who lived through this era should let me know what they think. Regardless, Moynihan paints a vivid picture of a deeply chaotic and misguided America which I am quite happy to have left to others to endure.

DPM’s writings of the time reveals the Nixon administration as deeply concerned with issues of poverty and race along with Moynihan’s growing disenchantment with the Vietnam War. Repeatedly he warns Nixon that what had been Johnson’s War in the public’s perception steadily transforming into Nixon’s burden. DPM pushed Milton Friedman’s proposal for a negative income tax hard, hoping to defuse what he saw as a catastrophic cycle of resentment over welfare programs.

Welfare reform remained Moynihan’s white whale throughout his career in the Senate. He championed meaningful reform legislation in 1989, and led a desperate effort to get the Clinton administration to address the subject in the early 1990s. Moynihan seemed to carry the Hillarycare bill primarily to leverage stronger action on welfare reform, and eventually he turned on the administration. At one point he even set up a meeting between Hillary Clinton and economist William Baumol in an effort to explain the doom of the approach. Later to her credit Senator Clinton graciously admitted that she ought to have listened.

In the end, DPM passionately opposed the Personal Responsibility Act of 1996, coming across as misguided in the process to this reader. Unfortunately it does not come across clearly in the book exactly what sort of welfare reform DPM would have pushed in 1993 if given the chance. In any case, the country will once again be facing these sort of issues as we face the painful task of putting the nation’s fiscal house in order.

It is impossible to read this book without yearning for Moynihan’s trained mind, keen intellect and above all moral courage to return to our national politics.  Moynihan’s lifelong passionate support of parental choice served as simply one example of these towering qualities. Just in case no one else is going to suggest it, a Daniel Patrick Moynihan Institute dedicated to pursuing the still unfinished business of the Senator’s career could greatly enrich our public debate.

Whether you agreed or disagreed with him, Moynihan was one of the great figures of the United States Senate. When will we see another?


I. Must. See. It.

March 7, 2013

Joss Whedon has a movie of Much Ado About Nothing coming out in general release.  I. Must. See. It.


Heads You Win, Tails You Still Win

March 5, 2013

(Guest Post by Matthew Ladner)

I had the opportunity to testify to the Senate Education committee in Texas today on the experience with parental choice programs for special needs children. One of the items of discussion was the following chart:

McKay Texas 1This phenomenon is often discussed regarding special education, but seldom quantified. In 2004 however officials from Education Service Center 20 (a regional body roughly covering school districts in the San Antonio area) provided the following chart to quantify the additional cost per special education student in a number of school districts. There were costs above and beyond those covered by state funding, and thus represented in effect a transfer from district general funds into special education funds on a per special education student basis.

 

Stanford economist Caroline Hoxby also testified to this interim committee in 2004, and she made the point that since school districts have been complaining that states don’t cover the full costs of special education for decades, that they have no cause to complain about students leaving with their (inadequate) funding. Districts can either keep these funds in the general education effort, or spend more on their remaining special education students (approximately 5% of Florida special education students directly utilize McKay but far more benefit from it) but either way they benefit.

 

 


Jason Bedrick Defeats Darth Strauss Over Tax Credit Scholarships

March 1, 2013

Jason Bedrick has an excellent post on Education Next rebutting Valerie Strauss’s ridiculous column on Tax Credit Scholarships.

People should keep their eye on Jason.  The force is strong with that one.


Alabama Lawmakers Pass Tax Credits for Children Attending Failing Public Schools

March 1, 2013

(Guest Post by Matthew Ladner)

Wow– Alabama lawmakers are officially the Cinderella kids, coming out of nowhere to shock the school choice world with an unexpected victory with the creation of two new tax credit programs for children attending failing public schools.

Here is a link to the bill, with the tax credit programs starting on page 14.


%d bloggers like this: