New Score: Tom Vander Ark 3, New Gates PLDD Strategy 0

March 19, 2013

The research score continues to run-up in favor of the old Gates education reform strategy of creating small schools of choice rather than the new Gates PLDD strategy of centrally determining what students should be taught (Common Core) and how teachers should be evaluated (Measuring Effective Teachers).  When Tom Vander Ark led the Gates education effort, they had a winning strategy.

The new evidence comes from another paper that was presented at the Association for Education Finance and Policy conference.  Two of my excellent students, Anna Jacob Egalite and Brian Kisida, received a data award from the Kingsbury Center to analyze the effect of school size on student achievement using NWEA test results.  They used a student-fixed-effects research design to see if students improved or worsened their academic achievement when they switched to a school of a different size.  And the results:

We found consistent negative effects of large school size on student math and reading achievement, especially in secondary schools that enroll more than 540 students. In grades 6-10, for example, math achievement declined by -.043 SD (standard deviation) and reading achievement declined by -.024 SD.

If a student moved from a largest quintile school in grades 6-10 to a school in the smallest quintile for those grades, we would expect a 6.4% of a standard deviation improvement in math performance.  Of course, a student fixed effects study is not quite as strong methodologically as the random-assignment evaluation of small high schools in New York City, but it is pretty darn good.  And this study has the advantage of using a large data base of more than 2 million students from across the country.  It’s pretty clear that students would benefit significantly from a reduction on school size — especially junior high and high school students.

Bring back Tom Vander Ark.

Bring Back Tom Vander Ark

March 18, 2013

Under Tom Vander Ark‘s leadership the Gates Foundation pursued an education reform strategy focused on creating smaller high schools.  The theory was that smaller high schools would create tighter social bonds between schools and students, preventing students from slipping through the cracks and increasing the likelihood that they would graduate and go on to college.  Smaller high schools could also be more varied in their approaches and offerings, allowing students to choose schools that best fit their needs.

But around the same time Vander Ark left the Gates Foundation at the end of 2006, the reform strategy shifted.  Rather than fostering small, diverse schools of choice, the Gates Foundation now wanted to build centralized systems of what everyone should be taught (Common Core) and centralized systems of evaluating, training, and promoting teachers (Measuring Effective Teachers).  As I’ve written before, the shift in Gates strategy was not prompted by research.  In fact, the high quality random-assignment study that Gates had commissioned to evaluate the small high school strategy showed strong, positive results.  But the post-Vander Ark leadership at Gates couldn’t wait for the evidence.  The knew the truth without any pesky research and had abandoned the small high schools strategy in favor of their new centralization approach years before those results were released.

Well, the evidence continues to pile up that the Gates strategy under Tom Vander Ark was effective and the new strategy is a failure.  A new National Bureau of Economic Research study by Lisa Barrow, Amy Claessens, and Diane Whitmore Schanzenbach examines the effects of small schools in Chicago.  Just like the earlier random-assignment study of small high schools in New York City had found and like the Vander Ark theory of reform had suggested, small schools promote high school graduation:

We find that small schools students are substantially more likely to persist in school and eventually graduate. Nonetheless, there is no positive impact on student achievement as measured by test scores. The finding of no test score improvement but a strong improvement in school attainment is consistent with a growing literature suggesting that interventions aimed at older children are more effective at improving their non-cognitive skills than their cognitive skills.

As Vander Ark had expected, smaller schools have non-achievement effects, like creating stronger social bonds, that help students go further in their schooling.  And as numerous studies have shown, higher educational attainment is strongly predictive of a host of good outcomes for students later in their lives. Score: Tom Vander Ark 2, New Gates PLDD Strategy 0.

Meanwhile I witnessed further confirmation of the failure of the new Gates approach during a panel at the Association for Education Finance and Policy conference last weekend.  The Gates folks spent more than an hour presenting their Measuring Effective Teachers work.  That stuff may win over gullible policymakers and journalists, but the researchers at AEFP were not impressed.

Tim Sass had the first question and essentially repeated my concerns about how MET fails to provide evidence for the use of multiple measures.  Value-added test scores are predictive of later life earnings, as Chetty, et al have shown, he said, but why should we believe that classroom observations measure anything we care about?  The Gates folks didn’t really have an answer.  Jane Hannaway then articulated my concern that effective teaching may be too context-dependent to lend itself to a single formula for effective practices.  The Gates folks responded that there are probably some basic skills for effective teaching that are useful in all contexts.  They may have a point but that does not address whether MET is getting at those basic skills or not.  The weak correlations of everything in the study suggest they are not finding approaches that are commonly effective.

A third questioner wondered about the cost of adopting the MET approach, especially given the need for multiple observations by multiple, trained raters.  The response was that schools are already spending money on classroom observations but of course that does not address the extra costs of the multiple rater/observations approach.

And lastly William Mathis asked about the generally weak correlations between classroom observations and value-added test scores.  Doesn’t this suggest that these are distinct dimensions of effective teaching that shouldn’t be combined in a single measure, he wondered.  The Gates response surprised me.  They said that the weak correlations were good news.  It is precisely because classroom observations and value-added measures capture different dimensions of effective teaching that we need to combine them in an overall measure.

Of course, this ignores the Sass question about whether we know that the classroom observations are measuring any important aspect of effective teaching.  But more problematic was the “heads I win, tails you lose” nature of their response.  If earlier arguments defending MET were based on how classroom observations and VAM were correlated, then how could the lack of correlation also be presented as proof of MET’s success?  I went up to the Gates presenters after the panel (and chatted with some of them later) and asked them what MET could have found that would have led them to conclude that combining the measures was a bad idea.  They were stumped.  Maybe negative correlations would have dissuaded them from advocating a combined measure, but they weren’t confident about that either.

Essentially, they admitted that the MET policy recommendation is a non-falsifiable claim.  No research finding would have dissuaded them from it.  The ability to falsify a claim is at the heart of science.  MET is not science; it is just politics.  You should have seen the discomfort of the Gates researchers as they pondered why they were presenting non-falsifiable claims as research findings at an academic conference.  This sort of thing corrupts science and has reputational consequences for the researchers who lend their credibility to the new evidence-free Gates reform strategy.

There is a solution to the rot at Gates — Bring back Tom Vander Ark.  At least his ideas were supported by rigorous research.

Chingos and West on Florida’s Pension Reforms

February 21, 2013

Matt Chingos and Marty West have a new paper published by Fordham examining pension reforms in Florida.  Specifically, Florida offered its new teachers the option of choosing between a defined benefit and a defined contribution retirement plan.  The defined benefit plan is the type most commonly found for teachers and defined contribution is more commonly found for private sector workers.

Defined benefit plans have some unusual characteristics that may push some teachers out of the workforce before they really should leave and may keep others as teachers longer than they should.  These defined benefit plans also reward long-serving and immobile teachers at the expense of shorter-serving and more mobile teachers, like those most commonly found in charter schools.  And defined benefit plans shift all of the risk for achieving sufficient investment returns to the government, which given recently weak investment returns, government under-funding of plans, and overly generous promised benefits is putting many states in serious financial trouble.

So states like Florida are considering shifting more teachers to defined contribution plans, which are more like 401k plans where the employer and employee each contribute money to an investment account and then the employee bears the risk of investment returns.

Matt and Marty addressed four questions in their study: 1) What portion of new teachers have chosen the defined contribution (DC) option? 2) What kinds of new teachers were more likely to make that selection? 3) Did the teachers who chose DC more likely to be effective teachers? and 4) Is there a difference in attrition between new teachers who choose DC or defined benefits (DB)?

The quick answers are 1) Between a quarter and a third of new teachers chose DC.  This is a surprisingly large share choosing DC, especially given that DB was the default option.  2) Teachers with more advanced degrees and degrees in math and science (presumably those with the most attractive options outside of teaching) were more likely to choose DC.  3) There was relatively little relationship between whether a teacher chose DC and their later effectiveness as measured by value-added scores. 4) New teachers who chose DC were more likely to leave their teaching positions.

Check out the full report to see all the details.

Jeb Bush and Clint Bolick on Immigration in the WSJ

January 25, 2013

(Guest Post by Matthew Ladner)

Jeb Bush and Clint Bolick are teaming up for a book on immigration that will come out in March, and previewed their thinking today in the Wall Street Journal. Read the article here. Basic thesis:

In some conservative circles, the word “comprehensive” in the context of immigration reform is an epithet—a code word for amnesty. People who oppose such reform declare that securing the United States border must come before moving toward broader reform.

Such an approach is shortsighted and self-defeating. Border security is inextricably intertwined with other aspects of immigration policy. The best way to prevent illegal immigration is to make sure that we have a fair and workable system of legal immigration. The current immigration system is neither.

The immigration system is like a jigsaw puzzle. If one or more pieces are out of whack, the puzzle makes no sense. To fix the system, Congress must make sure all of the pieces fit together, logically and snugly.

The Implications of a Blue Texas

January 17, 2013

(Guest Post by Matthew Ladner)

So I have been thinking about the talk of a “Blue Texas.” Texas has experienced a profound shift in partisan dominance within our lifetimes, and demographic changes in the state portend that it may happen again. Texas moved out of being part of the “Solid South” starting in the 1970s with the slow but steady rise of the Texas Republican party. Republicans had captured all of the statewide elected offices by the 1990s. Finally, the Republicans overcame Democratic gerrymandering to capture a majority in the Texas House and Senate in 2003.

A profound demographic shift has placed an expiration date upon the control of the Texas legislature by conservative Anglos. Conservatives may or may not remain ascendant in Texas but the days of the political dominance of conservative Anglos are certainly numbered.

One can see this trend coming in the ethnic distribution of the Texas school population. In 2011-12, Hispanics comprised 50.8% of children enrolled in the Texas public school system. Anglos comprised only 30.5 percent, and African-Americans only 12.8 percent. You can also get a sense of the scale and the growth in Texas by looking at public education statistics. With nearly five million students, Texas educates nearly as many public school students as the twenty smallest states combined. Texas may soon have twice as many public school students as Florida-despite the fact that Florida has the 4th largest public school population.  Texas has been adding a public school population roughly equal in size to the public school system of Wyoming every 14 months or so. Texas was the only state to gain 4 new Congressional seats after the 2010 Census- a small number of other states gained two, no one else gained either 3 or 4.

In 2012, Texas Hispanics comprised 25 percent of the electorate and favored Barack Obama over Mitt Romney by a margin of 62% to 37%. That’s a more balanced result than the national numbers, but hardly reassuring if you are a Texas Republican. Each passing year will see older Republicans passing on, and more young Hispanic voters entering the electorate. Some forecasters predict a “Blue Texas” by 2020- although it could happen either later or earlier or never depending upon a variety of factors.

Let’s start with the electoral college map. Republicans haven’t been very good at getting to 270 lately even with the now 38 Texas electoral votes in the bag. Without them states like Florida and Ohio could become mere style points for the Democratic nominee rather than crucial swing states. One could imagine other states trending Republican to counteract a Blue Texas, but it seems imaginary indeed.

For someone of modestly libertarian politics like myself, the most alarming scenario would be for a Blue Texas that becomes in effect a second California- a gigantic state in which organized public sector groups play an incredibly strong role in state policy making. I would expect that might blunt this momentum rather decisively:

Or perhaps not-predictions are hard, especially about the future. Some of you of course will be excited by the idea of a Blue Texas, others horrified by the prospect. Regardless the implications of a Blue Texas stretch far beyond Presidential politics. We can discuss some of those in future posts.

For now let’s keep an eye on this to see what happens next…

Starring Matt Ladner as the Difference Principle!

January 16, 2013

Hippies on stage

(Guest post by Greg Forster)

Are you ready for this? A Theory of Justice: The Musical!

No, really:

In order to draw inspiration for his magnum opus, John Rawls travels back through time to converse (in song) with a selection of political philosophers, including Plato, Locke, Rousseau and Mill. But the journey is not as smooth as he hoped: for as he pursues his love interest, the beautiful student Fairness, through history, he must escape the evil designs of his libertarian arch-nemesis, Robert Nozick, and his objectivist lover, Ayn Rand. Will he achieve his goal of defining Justice as Fairness?

Wait, I thought they already made that show. It was called Hair.

Here’s a publicity photo from the production – Matt Ladner in costume for his co-starring role as “The Difference Principle”:


HT David Koyzis

Head Start Revealed

January 14, 2013

Despite the obvious effort to delay and conceal the disappointing results from the official and high quality evaluation of Head Start, the Wall Street Journal shines the light on the issue in today’s editorial.  DC’s manipulating scumbags might want to take note that efforts to hide negative research might just draw more attention.  It’s comforting to see that the world may sometimes look more like Dostoevsky’s Crime and Punishment than Woody Allen’s Crimes and Misdemeanors.

The Journal reveals that Head Start supporters have not only ignored the latest study, but they are trying to sneak an extra $100 million for Head Start into the relief package for victims of Hurricane Sandy.  They also note that the most recent disappointing Head Start result is just the latest in a string of studies failing to find benefits from the program despite a cumulative expenditure of more than $180 billion.

And then the Journal finishes with this:

The Department of Health and Human Services released the results of the most recent Head Start evaluation on the Friday before Christmas. Once again, the research showed that cognitive gains didn’t last. By third grade, you can’t tell Head Start alumni from their non-Head Start peers.

President Obama has said that education policy should be driven not by ideology but by “what works,” though we have to wonder given his Administration’s history of slow-walking the release of information that doesn’t align with its agenda.

In 2009, the Administration sat on a positive performance review of the Washington, D.C., school voucher program, which it opposes. The Congressionally mandated Head Start evaluation put out last month was more than a year late, is dated October 2012 and was released only after Republican Senator Tom Coburn and Congressman John Kline sent a letter to HHS Secretary Kathleen Sebelius requesting its release along with an explanation for the delay. Now we know what was taking so long.

Like so many programs directed at the poor, Head Start is well-intentioned, and that’s enough for self-congratulatory progressives to keep throwing money at it despite the outcomes. But misleading low-income parents about the efficacy of a program is cruel and wastes taxpayer dollars at a time when the country is running trillion-dollar deficits.

A government that cared about results would change or end Head Start, but instead Congress will use the political cover of disaster relief to throw more good money after proven bad policy.

[UPDATE: And here is a good follow-up op-ed on the study by Lindsey Burke on the Fox News web site.]


Get every new post delivered to your Inbox.

Join 2,567 other followers