Under Tom Vander Ark‘s leadership the Gates Foundation pursued an education reform strategy focused on creating smaller high schools. The theory was that smaller high schools would create tighter social bonds between schools and students, preventing students from slipping through the cracks and increasing the likelihood that they would graduate and go on to college. Smaller high schools could also be more varied in their approaches and offerings, allowing students to choose schools that best fit their needs.
But around the same time Vander Ark left the Gates Foundation at the end of 2006, the reform strategy shifted. Rather than fostering small, diverse schools of choice, the Gates Foundation now wanted to build centralized systems of what everyone should be taught (Common Core) and centralized systems of evaluating, training, and promoting teachers (Measuring Effective Teachers). As I’ve written before, the shift in Gates strategy was not prompted by research. In fact, the high quality random-assignment study that Gates had commissioned to evaluate the small high school strategy showed strong, positive results. But the post-Vander Ark leadership at Gates couldn’t wait for the evidence. The knew the truth without any pesky research and had abandoned the small high schools strategy in favor of their new centralization approach years before those results were released.
Well, the evidence continues to pile up that the Gates strategy under Tom Vander Ark was effective and the new strategy is a failure. A new National Bureau of Economic Research study by Lisa Barrow, Amy Claessens, and Diane Whitmore Schanzenbach examines the effects of small schools in Chicago. Just like the earlier random-assignment study of small high schools in New York City had found and like the Vander Ark theory of reform had suggested, small schools promote high school graduation:
We find that small schools students are substantially more likely to persist in school and eventually graduate. Nonetheless, there is no positive impact on student achievement as measured by test scores. The finding of no test score improvement but a strong improvement in school attainment is consistent with a growing literature suggesting that interventions aimed at older children are more effective at improving their non-cognitive skills than their cognitive skills.
As Vander Ark had expected, smaller schools have non-achievement effects, like creating stronger social bonds, that help students go further in their schooling. And as numerous studies have shown, higher educational attainment is strongly predictive of a host of good outcomes for students later in their lives. Score: Tom Vander Ark 2, New Gates PLDD Strategy 0.
Meanwhile I witnessed further confirmation of the failure of the new Gates approach during a panel at the Association for Education Finance and Policy conference last weekend. The Gates folks spent more than an hour presenting their Measuring Effective Teachers work. That stuff may win over gullible policymakers and journalists, but the researchers at AEFP were not impressed.
Tim Sass had the first question and essentially repeated my concerns about how MET fails to provide evidence for the use of multiple measures. Value-added test scores are predictive of later life earnings, as Chetty, et al have shown, he said, but why should we believe that classroom observations measure anything we care about? The Gates folks didn’t really have an answer. Jane Hannaway then articulated my concern that effective teaching may be too context-dependent to lend itself to a single formula for effective practices. The Gates folks responded that there are probably some basic skills for effective teaching that are useful in all contexts. They may have a point but that does not address whether MET is getting at those basic skills or not. The weak correlations of everything in the study suggest they are not finding approaches that are commonly effective.
A third questioner wondered about the cost of adopting the MET approach, especially given the need for multiple observations by multiple, trained raters. The response was that schools are already spending money on classroom observations but of course that does not address the extra costs of the multiple rater/observations approach.
And lastly William Mathis asked about the generally weak correlations between classroom observations and value-added test scores. Doesn’t this suggest that these are distinct dimensions of effective teaching that shouldn’t be combined in a single measure, he wondered. The Gates response surprised me. They said that the weak correlations were good news. It is precisely because classroom observations and value-added measures capture different dimensions of effective teaching that we need to combine them in an overall measure.
Of course, this ignores the Sass question about whether we know that the classroom observations are measuring any important aspect of effective teaching. But more problematic was the “heads I win, tails you lose” nature of their response. If earlier arguments defending MET were based on how classroom observations and VAM were correlated, then how could the lack of correlation also be presented as proof of MET’s success? I went up to the Gates presenters after the panel (and chatted with some of them later) and asked them what MET could have found that would have led them to conclude that combining the measures was a bad idea. They were stumped. Maybe negative correlations would have dissuaded them from advocating a combined measure, but they weren’t confident about that either.
Essentially, they admitted that the MET policy recommendation is a non-falsifiable claim. No research finding would have dissuaded them from it. The ability to falsify a claim is at the heart of science. MET is not science; it is just politics. You should have seen the discomfort of the Gates researchers as they pondered why they were presenting non-falsifiable claims as research findings at an academic conference. This sort of thing corrupts science and has reputational consequences for the researchers who lend their credibility to the new evidence-free Gates reform strategy.
There is a solution to the rot at Gates — Bring back Tom Vander Ark. At least his ideas were supported by rigorous research.
Two points: Tom’s ideas about smaller high schools were supported with rigorous research after the fact–the NYC and Chicago studies only recently were released–though one could argue that his ability to identify the promising direction itself is a positive.
And the Chicago research found that graduation rates are higher in smaller schools but not test scores (the DC voucher study comparing students in smaller private schools to students in larger public schools had a similar finding). The MET study looked at correlations between measures of teaching and test scores, so it and most value added studies will be blind to these other outcomes.
Which leads to another consideration. Maybe noncognitive outcomes are amenable to improvement and possibly more important for the life course. This finding would have implications for the current environment of increasing test-score accountability. It may be oriented around a number that is less informative than believed, and graduation and college going may be more informative.
Or, perhaps, once graduation & college-going will become the variables to watch, they will get corrupted even more than they are today, like test-scores probably became over the last decade plus. And then they will also stop being indicative of most anything meaningful, right Mark? Except that this refocusing will corrupt college acceptance even more than it has been already.
Yes, I think Campbell’s Law would apply to measures of graduation and college-going. I was suggesting that recent research points to these indicators as ones that educators should be focusing on rather than test scores, but to use Campbell’s term, they are ‘corruptible.’ Trying to identify education indicators that are both useful and not corruptible is an interesting problem.
Did Vanderark “lead” the Gates Foundation or just direct its education program?
Doesn’t common sense say small schools are better than large schools for getting at the big challenges many students face? This isn’t really a Gates concept.
The problem is that developing and sustaining small schools is VERY expensive, at a time when public support for education is strained and being CUT. Gates used grant money to start thousands of small schools that struggled to survive, and many didn’t, the minute the Gates money was gone. Gates gave no thought to or help with sustaining them or building their capacity for that.
The real successes with small schools were/are in New York City where Annenberg Foundation started them, Gates money came in, and Annenberg money came BACK after Gates left to sustain those schools with continuing millions to New Visions for Public schools.
Maybe rather than bring Vanderark back, Gates could just put its careless, poorly implemented billions into something else for a while, and give education a rest?
You are right that Vander Ark led the education effort, not the whole foundation.
But I think you are mistaken in suggesting that small schools are not financially sustainable. There is no evidence of cost-efficiencies from scale in K-12 education. And small schools of choice have been growing rapidly over the last decade despite receiving less funding — they’re called charter schools.
Seriously?? Small Schools is the idea of Theodore Sizer: PROGRESSIVE EDUCATOR. Gates did try the small schools idea and it proved to be a waste of money.
Anyone who’s stepped into a small school with lousy curriculum and teachers who do not know their content, understand a small school does NOT make for a great education.
It’s the CURRICULUM…STUPID 🙂
Seriously. The two areas that fail schools the most:
1)The lousy curriculum and teaching methods (think fuzzy math and Conctructivism)
2)Teachers who are not experts in the academic content.
Now had Gates any clue as to why schools are failing, he would have focused his $$ on giving scholarships to poor children. Get them out of these failing schools.
One way to reform the schools of ed? States need to set the bar HIGH on giving teachers a license. Similar to what is required of Nurses.
Mass. did just that and found many of the teachers failed basic math.
Small Schools?? Just another wasted effort that ignores the real problems in education.
Remember that the Gates people also think it is is the curriculum. They just have a different one than you want. Rather than everyone trying to impose their vision of curriculum on all schools, why not let schools vary in their curriculum and compete with each other? They are likely to gravitate away from silly ideas since that will disadvantage them in a competitive market.
I’ll bet you’ll get more schools you like if we have small schools of choice that are free to adopt their own curriculum than if we have a single where one curriculum is imposed from the center. The odds are strongly against your curriculum being the one imposed from the center.
“I went up to the Gates presenters after the panel (and chatted with some of them later) and asked them what MET could have found that would have led them to conclude that combining the measures was a bad idea. They were stumped. Maybe negative correlations would have dissuaded them from advocating a combined measure, but they weren’t confident about that either.”
Wow. Just . . . wow.
I’m too stunned to BOOOOOOOOOOOOOOOOOOM.