Understanding the Gates Foundation’s Measuring Effective Teachers Project

January 9, 2013

If I were running a school I’d probably want to evaluate teachers using a mixture of student test score gains, classroom observations, and feedback from parents, students, and other staff.  But I recognize that different schools have different missions and styles that can best be assessed using different methods.  I wouldn’t want to impose on all schools in a state or the nation a single, mechanistic system for evaluating teachers since that is likely to be a one size fits none solution.  There is no single best way to evaluate teachers, just like there is no single best way to educate students.

But the folks at the Gates Foundation, afflicted with PLDD, don’t see things this way.  They’ve been working with politicians in Illinois, Los Angeles, and elsewhere to centrally impose teacher evaluation systems, but they’ve encountered stiff resistance.  In particular, they’ve noticed that teachers and others have expressed strong reservations about any evaluation system that relies too heavily on student test scores.

So the folks at Gates have been trying to scientifically validate a teacher evaluation system that involves a mix of test score gains, classroom observations, and student surveys so that they can overcome resistance to centrally imposed, mechanistic evaluation systems.  If they can reduce reliance on test scores in that system while still carrying the endorsement of “science,” the Gates folk imagine  that politicians, educators, and others will all embrace the Gates central planning fantasy.

Let’s leave aside for the moment the political reality, demonstrated recently in Chicago and Los Angeles, that teachers are likely to fiercely resist any centrally imposed, mechanistic evaluation system regardless of the extent to which it relies on test scores.  The Gates folks want to put on their lab coats and throw the authority of science behind a particular approach to teacher evaluation.  If you oppose it you might as well deny global warming.  Science has spoken.

So it is no accident that the release of the third and final round of reports from the Gates Foundation’s Measuring Effective Teachers project was greeted with the following headline in the Washington Post: “Gates Foundation study: We’ve figured out what makes a good teacher,”  or this similarly humble claim in the Denver Post: “Denver schools, Gates foundation identify what makes effective teacher.”  This is the reaction that the Gates Foundation was going for — we’ve used science to discover the correct formula for evaluating teachers.  And by implication, we now know how to train and improve teachers by using the scientifically validated methods of teaching.

The only problem is that things didn’t work out as the Gates folks had planned.  Classroom observations make virtually no independent contribution to the predictive power of a teacher evaluation system.  You have to dig to find this, but it’s right there in Table 1 on page 10 of one of the technical reports released yesterday.  In a regression to predict student test score gains using out of sample test score gains for the same teacher, student survey results, and classroom observations, there is virtually no relationship between test score gains and either classroom observations or student survey results.  In only 3 of the 8 models presented is there any statistically significant relationship between either classroom observations or student surveys and test score gains (I’m excluding the 2 instances were they report p < .1 as statistically significant).  And in all 8 models the point estimates suggest that a standard deviation improvement in classroom observation or student survey results is associated with less than a .1 standard deviation increase in test score gains.

Not surprisingly, a composite teacher evaluation measure that mixes classroom observations and student survey results with test score gains is generally no better and sometimes much worse at predicting out of sample test score gains.  The Gates folks trumpet the finding that the combined measures are more “reliable” but that only means that they are less variable, not any more predictive.

But “the best mix” according to the “policy and practitioner brief” is “a composite with weights between 33 percent and 50 percent assigned to state test scores.”  How do they know this is the “best mix?”  It generally isn’t any better at predicting test score gains.  And to collect the classroom observations involves an enormous expense and hassle.  To get the measure as “reliable” as they did without sacrificing too much predictive power, the Gates team had to observe each teacher at least four different times by at least two different coders, including one coder outside of the school.  To observe 3.2 million public school teachers for four hours by staff compensated at $40 per hour would cost more than $500 million each year.  The Gates people also had to train the observers at least 17 hours and even after that had to throw out almost a quarter of those observers as unreliable.  To do all of this might cost about $1 billion each year.

And what would we get for this billion?  Well, we might get more consistent teacher evaluation scores, but we’d get basically no improvement in the identification of effective teachers.  And that’s the “best mix?”  Best for what?  It’s best for the political packaging of a centrally imposed, mechanistic teacher evaluation system, which is what this is all really about.  Vicki Phillips, who heads the Gates education efforts, captured in this comment what I think they are really going for with a composite evaluation score:

Combining all three measures into a properly weighted index, however, produced a result “teachers can trust,” said Vicki Phillips, a director in the education program at the Gates Foundation.

It’ll cost a fortune, it doesn’t improve the identification of effective teachers, but we need to do it to overcome resistance from teachers and others.  Not only will this not work, but in spinning the research as they have, the Gates Foundation is clearly distorting the straightforward interpretation of their findings: a mechanistic system of classroom observation provides virtually nothing for its enormous cost and hassle.  Oh, and this is the case when no stakes were attached to the classroom observations.  Once we attach all of this to pay or continued employment, their classroom observation system will only get worse.

I should add that if classroom observations aren’t useful as predictors, they also can’t be used effectively for diagnostic purposes.  An earlier promise of this project is that they would figure out which teacher evaluation rubrics were best and which sub-components of those rubrics that were most predictive of effective teaching.  But that clearly hasn’t panned out.  In the new reports I can’t find anything about the diagnostic potential of classroom observations, which is not surprising since those observations are not predictive.

So, rather than having “figured out what makes a good teacher” the Gates Foundation has learned very little in this project about effective teaching practices.  The project was an expensive flop.  Let’s not compound the error by adopting this expensive flop as the basis for centrally imposed, mechanistic teacher evaluation systems nationwide.

(Edited for typos and to add links.  To see a follow-up post, click here.)

How the Gates Foundation Spins its Research

January 7, 2012

The Gates Foundation has released the next installment of reports in their Measuring Effective Teachers Project.  When the last report was released, I found myself in a tussle with the Gates folks and Sam Dillon at the New York Times because I noted that the study’s results didn’t actually support the finding attributed to it.  Vicki Phillips, the education chief at Gates,  told the NYT and LA Times that the study showed that “drill and kill” and “teaching to the test” hurt student achievement when the study actually found no such thing.

With the latest round of reports, the Gates folks are back to their old game of spinning their results to push policy recommendations that are actually unsupported by the data.  The main message emphasized in the new round of reports is that we need multiple measures of teacher effectiveness, not just value-added measures derived from student test scores, to make reliable and valid predictions about how effective different teachers are at improving student learning.

This is the clear thrust of the newly released Policy and Practice Brief  and Research Paper and is obviously what the reporters are being told by the Gates media people.  For example, Education Week summarizes the report as follows:

…the study indicates that the gauges that appear to make the most finely grained distinctions of teacher performance are those that incorporate many different types of information, not those that are exclusively based on test scores.

And Ed Sector says:

The findings demonstrate the importance of multiple measures of teacher evaluation: combining observation scores, student achievement gains, and student feedback provided the most reliable and predictive assessment of a teacher’s effectiveness.

But buried away on p. 51 of the Research Paper in Table 16 we see that value-added measures based on student test results — by themselves — are essentially as good or better than the much more expensive and cumbersome method of combining them with student surveys and classroom observations when it comes to predicting the effectiveness of teachers.  That is, the new Gates study actually finds that multiple measures are largely a waste of time and money when it comes to predicting the effectiveness of teachers at raising student scores in math and reading.

According to Table 16, student achievement gains correlate with the underlying value-added by teachers at .69. If the test scores are combined (with an equal weighting) with the results of a student survey and classroom observations that rate teachers according to a variety of commonly-used methods, the correlation to underlying value-added drops to be between .57 and .61.  That is, combining test scores with other measures where all measures are equally weighted actually reduces reliability.

The researchers also present the results of a criteria weighted combination of student achievement gains, student surveys, and classroom observations based on the regression coefficients of how predictive each is of student learning growth in other sections for the same teacher.  Based on this the test score gains are weighted at .729, the student survey at .179, and the classroom observations at .092.  This tells us how much more predictive test score gains are than student surveys or classroom observations.  Yet even when test score gains constitute 72.9% of the combined measure, the correlation to underlying teacher quality still ranges between .66 and .72, depending on which method is used for rating the classroom observations.  The criteria-weighted combined measure provides basically no improvement in reliability over using test score gains by themselves.

And using multiple measures does not improve our ability to distinguish between effective and ineffective teachers.  Using test scores alone the difference between the top quartile and bottom quartile teacher in producing  student value-added is .24 standard deviations in math learning growth on the state test.  If we combine test scores with student surveys and classroom observations using an equal weighting, the difference between top and bottom quartile teachers shrinks to be between .19 and .21.  If we use the criteria weights, where test scores are 72.9% of the combined measure, the gap between top and bottom teacher ranges between .22 and .25.  In short, using multiple measures does not improve our ability to distinguish between effective and ineffective teachers.

The same basic pattern of results holds true for reading, which can be seen in Table 20 on p. 55 of the report.  Combining test score measures of teacher effectiveness with student surveys and classroom observations does improve a little our ability to predict how students would answer survey items about their effort in schools as well as how they felt about their classroom environment.  But unlike test scores, which have been shown to be strong predictors of later life outcomes, I have no idea whether these survey items accurately capture what they intend or have any importance for students’ lives.

Adding the student surveys and classroom observation measures to test scores yields almost no benefits, but it adds an enormous amount of cost and effort to a system for measuring teacher effectiveness.  To get the classroom observations to be usable, the Gates researchers had to have four independent observations of those classrooms by four separate people.  If put into practice in schools that would consume an enormous amount of time and money.  In addition, administering, scoring, and combing the student survey also has real costs.

So, why are the Gates folks saying that their research shows the benefits of multiple measures of teacher effectiveness when their research actually suggests virtually no benefits to combining other measures with test scores and when there are significant costs to adding those other measures?  The simple answer is politics.  Large numbers of educators and a segment of the population find relying solely on test scores for measuring teacher effectiveness to be unpalatable, but they might tolerate a system that combined test scores with classroom observations and other measures.  Rather than using their research to explain that these common preferences for multiple measures are inconsistent with the evidence, the Gates folks want to appease this constituency so that they can put a formal system of systematically measuring teacher effectiveness in place.  The research is being spun to serve a policy agenda.

This spinning of the findings  is not just an accident or the results of a misunderstanding.  It is clearly deliberate.  Throughout the two reports Gates just released, they regularly engage in the same pattern of presenting the information. They show that the classroom observation measures by themselves have weak reliability and validity in predicting effective teachers.  But if you add the student survey and then add the test score measures, you get much better measures of effective teachers.  This pattern of presentation suggests the importance of multiple measures, since the classroom observations are strengthened when other measures are added.  The only place you find the reliability and validity of test scores by themselves is at the bottom of the Research Paper in Tables 16 and 20.  If both the lay-version and technical reports had always shown how little test scores are improved by adding student surveys and classroom observations, it would be plain that test scores alone are just about as good as multiple measures.

The Gates folks never actually inaccurately describe their results (as Vicki Phillips did with the previous report).  But they are careful to frame the findings as consistently as possible with the Gates policy agenda of pushing a formal system of measuring teacher effectiveness that involves multiple measures.  And it worked, since the reporters are repeating this inaccurate spin of their findings.


(UPDATE — For a post anticipating responses from Gates, see here.)

Gates Foundation — Release the MET Results

October 25, 2011

A sketch of the $500 million new Gates Foundation headquarters

Bill and Melinda Gates mentioned again in the Wall Street Journal the Measuring Effective Teachers (MET) project that their foundation is orchestrating.  Bill and Melinda may want to check on the status of the MET research they’ve been touting since full results were promised in the spring of 2011 and have yet to be released.

Just to review… In an earlier interview with the Journal, MET was described as follows:

the Gates Foundation’s five-year, $335-million project examines whether aspects of effective teaching, classroom management, clear objectives, diagnosing and correcting common student errors can be systematically measured. The effort involves collecting and studying videos of more than 13,000 lessons taught by 3,000 elementary school teachers in seven urban school districts.

The motivation, re-iterated in the new piece by Bill and Melinda Gates is to identify  what “works” in classroom teaching to develop systems that train and encourage other teachers to imitate those practices:

It may surprise you—it was certainly surprising to us—but the field of education doesn’t know very much at all about effective teaching. We have all known terrific teachers. You watch them at work for 10 minutes and you can tell how thoroughly they’ve mastered the craft. But nobody has been able to identify what, precisely, makes them so outstanding….

The intermediate goal of MET is to discover what we are able to measure that is predictive of student success. The end goal is to have a better sense of what makes teaching work so that school districts can start to hire, train and promote based on meaningful standards.

As I’ve argued before, using research to identify “best practices” in teaching only makes sense if the same teaching approaches would be desirable for the vast majority of teachers and students, regardless of the context.  And as I’ve also  suggested before, I don’t believe this effort is likely to yield much in education.  Effective teaching is like effective parenting — it is highly dependent on the circumstances.  Yes, there are some parenting (and teaching) techniques that are generally effective for almost everyone, but those are mostly known and already in use.

This doesn’t mean we are completely unable to measure effective teaching (or parenting).  It just means that we have to judge it by the results and cannot easily make universal statements about the right methods for producing those results.  To make a sports analogy, there is no single “best practice” for hitters in baseball.  There are a variety of stances and swings.  The best way to judge an effective hitter is by the results, not by the stance or swing.  And if we tried to make all hitters stand and swing in the same way, we’d make a lot of them worse hitters.

It is because of this heterogeneity in effective teaching practices that I think the MET project is doomed to disappoint.  And according to inside sources, I’ve heard that results are being delayed because they are failing to produce much of anything.

According to the MET web site, the full results for the 1st year should have been released in the spring:

 In spring 2011, the project will release full results from the first year of the study, including predictors of teaching effectiveness and correlation with value-added assessments.

It is almost November and we have not seen these results.  I understand that in very large and complicated projects, like MET, things can take much longer than originally planned.  If so, it would be nice to hear that explanation.  It would be even nicer if the Gates Foundation released results if they have them, even if those results were not what they had hoped they would find.

Some inquisitive reporters should start asking Gates officials and members of the research team about the status of the MET results.  Reporters should go beyond talking to the media flacks at Gates HQ and actually talk to individual members of the team confidentially.  If they do that, they may confirm what I have been hearing: MET results have been delayed because they aren’t panning out.

(UPDATE:  Gates responds.

The Gates Foundation and the Rise of the Cool Kids

July 28, 2011

(Guest Post by Matthew Ladner)

Jay and Greg have been carrying on an important discussion concerning the Gates Foundation and education reform. I wanted to add a few thoughts.

Rick Hess and others have noted the “philanthropist as royalty” phenomenon in the past. Any philanthropist runs the danger of only hearing what they want to hear from their supplicants, and Gates as the largest private foundation runs the biggest risk. The criticism of the Gates Foundation I had seen in the past emanated from the K-12 reactionary fever swamp, hardly qualifying as constructive.

The challenge faced by philanthropists: how do you challenge your own assumptions and evaluate your own efforts honestly? Do you hire formidable Devil’s advocates to level their most skeptical case against your efforts?

I don’t know the answer to these questions, just that if I were Bill Gates I would be terrified of everyone telling me how right my thinking is because they want my money. This is however the best sort of problem to have…

Jay’s central critique of the Gates Foundation strategy seems to be that they have put too much faith in a centralized command and control strategy. They would be wise to entertain this thought. If command and control alone were the solution, then we wouldn’t have education problems-district, state and federal governance have all failed to prevent widespread academic failure for decades.

The Gates strategy does however embrace decentralization. Over the years they have supported charter schools, and fiercely opposed the worst one-size fits all policy of all: salary schedules and automatic/irrevocable tenure. Riley’s WSJ article makes clear that Gates understands the benefits of private school choice, but that he falls for the Jay Mathews fallacy of thinking it is just too politically difficult.

Sigh…perhaps next year Greg can make a dinner bet with Bill.

Gates is also the primary backer of Khan Academy. This new article on Sal Khan in Wired magazine makes clear that Khan understands the danger of being swallowed by school systems and that he is not going to allow it to happen. Khan academy is both radically decentralized and is in the early stages of being used by people within the centralized school system to improve outcomes.

Whatever the mistakes to date, the Gates Foundation has in my mind has succeeded in serving as a counter-weight to the NEA, mostly through funding the efforts of a myriad network of reform organizations collectively known as the Cool Kids. Today, there is a struggle for power going on within the Democratic Party over K-12 policy and the Gates Foundation deserves some credit in my mind for supporting  the ideas behind the “Democrat Spring” on education policy. This spring is following more of the Syrian than the Egyptian model thus far, but it is happening, and it is very important.

Does that mean that they are the “good guys” and Jay should lay off of them? Of course not-reasoned critiques of large philanthropists are in short supply for all of the factors cited above. Jason Riley wished that Gates were bolder in embracing decentralization reforms, but noted that in the end that it was the Gates rather than the Riley Foundation. This is absolutely true, but it doesn’t make the royalty problem go away, and leaves a continuous question of how the emperor gets feedback on his new clothes.

I don’t agree with the Cool Kids about everything. The next time I hear someone ask a question about having Common Core replace NAEP (the very pinnacle of naive folly) for instance I may pull out entire tufts of my graying, thinning hair in utter exasperation. Reformers of all stripes need to be on guard against the ship-wheel conceit, which is to imagine that if only my strong hands steered the ship, we’d sail through the rocky shoals of ed reform without a hitch.

The East Germans ran a much better economy than the North Koreans, much to the benefit of Germans and to the detriment of Koreans. This is real and important in human terms- I do not make this point glibly. I never heard about an East German famine decimating the population, but food shortages have even soldiers starving to death in North Korea (pity the women and children). Better quality management is good and desirable, but…it will only take you so far. Today, Chinese apparatchiks are noisily crediting themselves for the tremendous economic progress in China without the slightest hint of irony. Without the market forces Deng introduced and with more apparatchiks, China would revert back to a starving backwater. With fewer apparatchiks, her progress would almost certainly accelerate.

As Sara Mead correctly noted in this guest post at Eduwonk, today’s education debate largely involves a mixture of technocratic and market-based reforms (neo-liberals) on one side and a group of reactionaries lacking realistic solutions on the other. A third of our 4th graders can’t read and have been shoved into the dropout pipeline. We need both technocratic and market based reforms, and we need stronger reforms of both sorts than those fielded to date.

Jay’s critique concerns the right mix of reforms within the bounds of the neo-liberal consensus. This of course is a matter of debate, and debate is the path to deeper understanding. The sheer size of the Gates Foundation has the potential to stifle such debate as it relates to their efforts, even passively, and reformers should recognize the danger in allowing it to do so. This isn’t about them so much as it is about us.

Gates Foundation Follies (Part 2)

July 26, 2011

A sketch of the $500 million new Gates Foundation headquarters

In Part 1 of this post, I described how the Gates Foundation came to recognize the importance of using political influence to reform the education system rather than focusing on reforming one school at a time in the hopes that school systems would see and replicate successful models.  No private philanthropist has enough money to buy and sustain widespread adoption of an effective approach and the public school system has little incentive to identify and spread effective approaches on their own.

Faced with the unwillingness of the public school system to reproduce successful models (assuming that Gates could even offer one), the Foundation was left with two solutions to encourage innovation: 1) identify the best practices themselves and impose them from the top down, or 2) encourage choice and competition so that schools would have the proper incentive to identify, imitate, and properly implement effective approaches.

The Gates Foundation made the wrong choice.  Their top-down strategy cannot work for the following reasons:

1) Education does not lend itself to a single “best” approach, so the Gates effort to use science to discover best practices is unable to yield much productive fruit;

As I’ve explained before, there are many different “best” techniques for different kinds of teachers with different kinds of students in different situations with different available resources.  There are some practices that are universally beneficial in education, but they tend to be pretty obvious and are already well known (e.g. it is bad to beat kids, it is better when teachers know the material they are teaching, it is helpful to break down ideas into their essential components, etc…).

The difficulty of discovering universally beneficial  practices that are not already well-known, especially with the blunt tools available to researchers probably helps explain why the Measuring Effective Teachers (MET) project, on which the Gates Foundation is spending $335 million has yet to produce any meaningful results despite entering its third year of operation.

2) As a result, the Gates folks have mostly been falsely invoking science to advance practices and policies they prefer for which they have no scientific support;

Despite having nothing to show for the $335 million they are spending on MET, the Gates folks nevertheless claim that it “proves” the harmfulness of teachers engaging in “drill and kill.” The fact that the research showed no such thing did not deter them from telling the NY Times and LA Times that it did.  Even when I pointed out the error, the Gates folks refused to issue a correction (although the LA Times ran one on their own).

Similarly, the Gates-orchestrated effort to push national standards, curricular materials, and assessments is advancing without any scientific evidence of the desirability of these approaches.  Gathering a group of Checker Finn’s friends (er, I mean, “a panel of experts”) to attest that the Common Core standards are better is not science.  It is the false invocation of science to manipulate people into compliance with their agenda.

3) Attempting to impose particular practices on the nation’s education system is generating more political resistance than even the Gates Foundation can overcome, despite their focus on political influence and their devotion of significant resources to that effort;

Opponents of centralized control of education have begun to mobilize against the Gates-orchestrated effort to establish national standards, curricular materials, and assessments.  But the bulk of the political resistance to the Gates strategy will come from the teacher unions.  They don’t want anyone to infringe on their autonomy or place their interests in jeopardy with a nationalized accountability system.  They may play along with Gates for a while and take their money, but when push comes to shove the unions can only tolerate one dictator in education — the unions.  Of course, those of us who don’t want anyone centrally-controlling the nation’s education system will oppose both Gates and the teacher unions.

We already have a taste of the kind of resistance teacher unions will put up against the Gates nationalization effort in the slogans emanating from Diane Ravitch and Valerie Strauss’ Twitter feed, supported by their Army of Angry Teachers.  Falsely claiming that MET proved that drill and kill is harmful did not mollify these folks at all.

The teacher unions derive far more power and money from the status quo than Gates can ever offer them, unless of course Gates builds a nationalized system and cedes control to the unions, which is not part of the Gates plan.  Nothing in the Gates strategy weakens the unions and would force them to make significant concessions, so in the end the unions will either hijack the Gates strategy for their own benefit or block it.  Even Gates does not have the resources to beat the unions without first diminishing their power.

4) The scale of the political effort required by the Gates strategy of imposing “best” practices is forcing Gates to expand its staffing to levels where it is being paralyzed by its own administrative bloat; 

Over the last decade the Gates Foundation has roughly doubled its assets but increased its staffing by about 10-fold.  The Foundation is now huge, which is part of why it needs the Education Pentagon pictured above to house everyone.  The Foundation has gotten huge because it is trying to buy political influence as it buys people.  Gates has been snapping up or funding just about every advocacy group, researcher, or education journalist they can find.  Getting all of these people on board for a nationalized education system (or at least mute their dissent) involves paying an enormous number of people and organizations.

Gates can buy a lot of folks, but they can’t buy everyone and they can’t keep the folks they do pay in line for very long.  It’s like herding cats. (I should note that I’ve received Gates Funding in the past).

And the sheer size of their staff and funded allies along with the focus on controlling the political message is so overwhelming that it is significantly hindering their ability to do anything.  People inside the organization have told me that they are suffering from a bureaucratic gridlock with endless meetings, conference calls, and chains of approvals.  Notice that Gates is paying a ton of researchers and yet virtually no research is coming out.  Very curious.

5) The false invocation of science as a political tool to advance policies and practices not actually supported by scientific evidence is producing intellectual corruption among the staff and researchers associated with Gates, which will undermine their long-term credibility and influence.

As noted above, the need to advance a particular political message has led Gates to mischaracterize their own research (for example, claiming that MET proves that drill and kill is harmful when the research does not show that).  But the intellectual corruption extends much farther.  I had a highly respected and accomplished researcher employed by Gates tell me that Vicki Phillips’ mischaracterization of the MET results was not so far off because there isn’t a big difference between a low correlation and a negative one.  He also defended comparing the magnitude of a series of pair-wise correlations to determine the relative influence of different variables.  To hear someone who knows better twist the truth to avoid contradicting the education boss at Gates was just sad.

Unfortunately, too many advocates, researchers, and others are being similarly corrupted.  In most cases the Gates folks don’t have to exert any explicit pressure on people to keep them in line; they just anticipate what they think would serve the Gates strategy.  But I am aware of at least one case in which a researcher’s findings were at odds with the desired outcome and that person suffered for it.

I’ve heard another story from someone involved in the MET project that the delay in releasing any results from the analyses of classroom videos even as the project enters its third year is explained by their inability to find any meaningful results.  Perhaps another year of data will make something turn up that they can finally tout for their $335 million investment.  The fact that the initial MET report with basically no useful findings was released on a Friday just before Christmas suggests that the Gates folks are working hard to shape their message.

The national standards, curriculum, and testing campaign is rife with intellectual corruption.  For example, people are twisting themselves into knots to explain how the effort is purely voluntary on the part of states when it is manifestly not, given federal financial “incentives,” offers of selective exemptions to NCLB requirements for states that comply, and the threat of future mandates.  There is so much spin around Gates that it makes one dizzy.


Let me be clear, most of the folks affiliated with Gates are good and smart people.  The problem is that when your reform strategy requires a top-down approach, these good and smart people are put under a lot of stress to have a unified vision of the “best” that will be imposed from the top.  And whenever an organization starts sprinkling millions of dollars on researchers and advocacy groups unaccustomed to that kind of money, there are temptations that are hard for the most virtuous to resist.

But the good and smart people at Gates can stop the counter-productive strategy that the Foundation is pursuing.  The Foundation changed course once before and it can do it again.


UPDATE — For my suggestions of what the Gates Foundation could do instead, see this post.

Gates Foundation Follies (Part 1)

July 25, 2011

A sketch of the $500 million new Gates Foundation headquarters

Jason Riley’s interview with Bill Gates in the Wall Street Journal was not as great as Riley’s interview with me last week (shameless plug for my new mini-book), but it was still very illuminating.  In particular, the Gates interview confirmed two things about the Foundation’s education efforts: 1) they’ve realized that the focus of their efforts has to be on the political control of schools and 2) they are uninterested in using that political influence to advance market forces in education. Instead, the basic strategy of the Gates Foundation is to use science (or, more accurately, the appearance of science) to identify the “best” educational practices and then use political influence to create a system of national standards, curricular materials, and testing to impose those “best practices” on schools nationwide.

The Gates Foundation came to understand the necessity of political influence over schools with the failure of their previous small schools strategy.  Under that strategy they tried to achieve reform by paying school districts to break-up larger high schools into smaller ones.  The problem with that strategy is that even the Gates Foundation does not have nearly enough money to buy systemic reform one school at a time.

School districts currently spend over $600 billion per year and the Gates Foundation only has $34 billion in total assets.  With the practice of spending only about 5% of assets each year and given the large (and effective) efforts the Foundation makes in developing country health-care, Gates only spends a couple hundred million dollars on education reform each year. Given the small share of total education spending Gates could offer, most public districts refused to entertain the Gates strategy of smaller schools, others took the money but failed to implement it properly, and others reversef the reform once the Gates subsidies ended.

The way I described the situation in my chapter “Buckets into the Sea” in the 2005 book, With the Best of Intentions, edited by Rick Hess is:

Philanthropists simply don’t have enough resource to reshape the education system on their own; all their giving put together amounts to only a tiny fraction of total education spending, so their dollars alone can’t make a significant difference.  In order to make a real difference, philanthropists must support programs that redirect how future public education dollars are spent.

And in 2008 I repeated this claim, saying: “total private giving to public education is a tiny portion of total spending on schools.  All giving, from the bake sale to the Gates Foundation, makes up less than one-third of 1% of total spending.  It’s basically rounding error.”

I don’t know whether the Gates Foundation was influence by my writing or whether they arrived at the same conclusions independently, but they are now articulating those same conclusions, often with the same exact words:

“It’s worth remembering that $600 billion a year is spent by various government entities on education, and all the philanthropy that’s ever been spent on this space is not going to add up to $10 billion. So it’s truly a rounding error.”

This understanding of just how little influence seemingly large donations can have has led the foundation to rethink its focus in recent years. Instead of trying to buy systemic reform with school-level investments, a new goal is to leverage private money in a way that redirects how public education dollars are spent.

While the focus of the Gates Foundation on influencing education policy is sensible, the particular political approach they have chosen is doomed to fail and attempting it is likely to be counter-productive.  In Part 2 of this post I will explain how the new strategy Gates has decided to pursue is flawed.

To give you a taste of what is coming in Part 2, the arguments can be summarized as: 1) Education does not lend itself to a single “best” approach, so the Gates effort to use science to discover best practices is unable to yield much productive fruit; 2) As a result, the Gates folks have mostly been falsely invoking science to advance practices and policies they prefer for which they have no scientific support; 3) Attempting to impose particular practices on the nation’s education system is generating more political resistance than even the Gates Foundation can overcome, despite their focus on political influence and their devotion of significant resources to that effort; 4) The scale of the political effort required by the Gates strategy of imposing “best” practices is forcing Gates to expand its staffing to levels where it is being paralyzed by its own administrative bloat; and 5) The false invocation of science as a political tool to advance policies and practices not actually supported by scientific evidence is producing intellectual corruption among the staff and researchers associated with Gates, which will undermine their long-term credibility and influence.

Tune in for Part 2.


UPDATE — For my suggestions of what the Gates Foundation could do instead, see this post.

Changing the Conversation is Not the Same as Changing the World

January 20, 2015

Last week I noted that attention is not influence.  When foundations and others reward reform organizations for clicks, tweets, hits, etc… they are not actually rewarding influence.  If foundations really want to influence policy they have to reward actions that really lead to policy change.

Today I am extending that argument by emphasizing that changing the conversation is not the same as changing the world.  There are times when everyone around us seems to agree on something and we are liable to feel a sense of accomplishment.  “We did it,” we think to ourselves.  “We won.”

But having people around you change what they say is not the same as accomplishing it in the world.  Perhaps it is a result of pervasive post-modern thinking, but our representation of the world is not the world.  Mass rallies with #bringbackourgirls or #jesuischarlie did not free girls in Nigeria or establish the right to produce images of Muhammad.  There really is a world out there and what we say about it does not necessarily lead to changing it.  We also have to do something to make our talk real.

In education reform the conversation has been dominated by discussion of Common Core.  It was amazing how easy it was for supporters to accomplish this.  Getting a bunch of DC-based organizations to write some reports, hold conferences, and engage in advocacy for Common Core doesn’t take all that much money or effort.  It’s not that these organizations are saying things that contradict their beliefs.  They just don’t have a lot of deeply held beliefs and are eager to remain relevant and active on whatever everyone else is talking about.

It wasn’t even that hard to get state boards of education to endorse Common Core.  While they don’t say it out loud, many state officials (rightly) see standards as a bunch of vague and empty words in a document that have little effect on what really happens in schools.  If someone is offering them grants from the Gates Foundation and the possibility of millions from Race to the Top in the midst of an economic crisis, why not declare fealty to these standards?  In addition, state officials regularly get drawn into fights over standards.  Why waste political capital on something that hardly matters?  It’s much easier to just join the Common Core crowd and hide behind the skirts of “experts,” national organizations, and the federal government than to defend and constantly revise their own crappy state standards.  Besides, they could always change their mind later when it came to actually doing something, like adopting tests or imposing consequences on schools and teachers based on their actual implementation of Common Core.  Even state officials who embraced Common Core understood, on some level, that what they were doing was just talk.

Don’t get me wrong.  Standards could matter.  Determining what students should learn and when could have a profound effect on education.  And the difference between excellent and lousy schools has a lot to do with whether they have high expectations for their students and seek to teach worthy content.  The problem is that in a large and diverse society we  have little agreement on what constitutes worthy content or appropriate expectations.  To obtain democratic support for state (let alone national) standards, they have to be written at such a level of generality that they are largely meaningless.  That is why multiple studies show no relationship between the judged quality of standards and academic outcomes.  And to the extent that standards actually stand for anything, they draw opposition from those who disagree.  In a democratic country that opposition has plenty of opportunities to block, dilute, or co-opt standards, preventing the “talk” of Common Core from becoming reality.

I’ve been making this point that Common Core “talk” will not result in real educational change for years now.  When I do, I hear things like, “At last count, 1 state out of 45 has repealed the standards.”  And DC-based folks take comfort from the fact that everyone they meet at receptions agrees that Common Core opposition is crazy, paranoid, hysterical, political,  [insert your preferred empty pejorative here].  They all falsely believe that they have won the conversation and therefore have won the policy.  They continue to hold their hashtag signs.

But since I am not a post-modern and still believe that there is a world out there that is not changed simply by our words, I have developed a wager with Morgan Polikoff as an imperfect indicator of whether Common Core really is changing the world:

In ten years, on April 14, 2024, I bet Morgan that fewer than half the states will be in Common Core.  We defined being in Common Core as “shared standards with shared high stakes tests-even if split between 2 tsts.”  Given 51 states and DC, Morgan wins if 26 or more states have shared standards and high stakes tests and I win if the number is 25 or less.  The loser has to buy the winner a beer (or other beverage).

Well, it didn’t take long but I think am already ahead on that bet.  Mississippi just voted to withdraw from using PARCC, one of the two Common Core-aligned tests.  In addition, Chicago is refusing to administer PARCC to all of its students.  And governor Walker in Wisconsin just re-iterated  his desire to withdraw the state from Common Core standards and testing.  A bill to that effect failed last legislative session, but the dike will only hold for so long.  It’s hard for me to find a current count of what tests states are using, but I believe we have dropped below half using one of the two Common Core tests.  If nothing changes over the remainder of our 10 year bet, I will win.  But I expect more states will abandon Common Core standards and/or tests.  The talk was easy.  The implementation is hard.


July 31, 2014

Ancient mystics believed that one could have the magical power to create reality simply by uttering certain words.  This is the origin of “magical words” like abracadabra, which means “I create as I speak” in Aramaic.  But the belief in using magical words to create reality continues to this day, and not just among cheesy stage illusionists.  The Gates Foundation and their various grant recipients have “in a series of strategy sessions in recent months… concluded they’re losing the broader public debate [over Common Core] — and need to devise better PR.

Common Core supporters haven’t considered the possibility that their political strategy is flawed because they are trying to impose a top-down reform on a hostile and well-organized opposition of teachers and affluent parents.  Nope.  It must be that they just aren’t using the right words.  In particular, they think they need to shift from talking so much about “facts” and “evidence” and start using more “emotional” words.  If only they say the right words, people’s interests will change and the opposition will melt.  Abracadabra!

This faith in magical words is a symptom of a larger disease.  Education reformers have invested way too much in people who do almost nothing except craft political messages.  They try to coin just the right soundbite to fit in their dozens of daily tweets.  But they don’t just repeat these soundbites on Twitter, they use this “messaging” at policy conferences, in essays, and in conversations with each other.  They have put so much energy into perfecting the Twitter-bite that they can no longer think in any way other than in short bursts of spin.  It is rotting their brains.

Unfortunately, I think the rot starts at the top.  The Gates Foundation not only funds a large amount of this messaging nonsense, but engages in this type of slogan-speak themselves.  I’ve been reviewing their own descriptions of the purposes of their grants and have found poetry, like “to support organizations in a strategic visioning engagement to develop their innovative professional development theory of action and implementation strategies” or “to bring together a coalition of thought leaders, policy-makers, consultants and practitioners as part of the Global Education Leaders’ Program (GELP) and support them through a convening.”  Ugh.  

Here on JPGB we’ve been warning about the abuse of the English language in education reform for a while now.  And Rick Hess has joined the party, alerting readers to common phrases that should raise alarms with your BS-detector.  As Orwell understood, the problem with slogan-speak is not just that it muddles debates by obscuring the substance of what people are really saying.  And the problem is also not limited to the fact that degrading policy discourse with this gibberish undermines the credibility of future attempts at serious policy discussion.

The worst problem of slogan-speak may be that it is distorting the thinking of the ed reformers themselves.  They are usually completely sincere when they spout this slogan-speak.  They believe it.  And so their analysis of education reform issues is stunted and superficial.  They can’t think through an issue much more than how it sounds in a Twitter post.  And perhaps this is why they are doubling-down on a top-down standards reform that has no political logic to it.  They just can’t think it through.  So, when it runs into trouble they revert to what they know — more messaging.

Common Core Political Naivete and the Enemies List

July 2, 2014

The entire Common Core enterprise has been characterized by shocking political naivete and over-reach.  Despite investing a fortune in political operatives and holding weekly conference calls “directed by Stefanie Sanford, who was in charge of policy and advocacy at the Gates Foundation,” the folks pushing Common Core did not anticipate that the Unions would betray them and oppose the implementation of Common Core as soon as it suited their purposes.  They did not anticipate that there was no authentic constituency for the proper implementation of the new standards and aligned high stakes tests.  They did not anticipate that the combined forces of the Unions and conservative opponents of centralized control would overwhelm the largely paid mercenaries they had on their side.  For people who imagine themselves politically sophisticated they look like a pack of amateurs.

And as the Common Core effort crumbles, its supporters are not just failing, but losing ground on previous accomplishments.   If you liked accountability testing, Common Core has done more to set back your efforts than Randi Weingarten ever could have done on her own.  As Rick Hanushek points out in the Wall Street Journal, the Unions are using Common Core not only to block new tests, but to eliminate high stakes testing altogether.  Several states will soon have no high stakes testing while they adopt a moratorium on stakes in their supposed transition to new tests.  The Gates Foundation has backed a two year delay in the hopes of rescuing their effort from collapse.  Like a retreating army suggesting a cease fire, they will find their opponents have little reason to keep the delay temporary.

In the hopes of achieving a total victory (changing standards and testing everywhere), the Common Core folks are going to end up with weaker testing and standards in many places.  As I suggested in my post on the Paradoxical Logic of Ed Reform Politics, seeking total victory often produces stunning defeat.

The other unintended side-effect of Common Core crumbling is that it is producing abusive efforts by its supporters to rescue it.  The whole enterprise depended on putting it into place quickly so that anyone who opposed the fait accompli could be dismissed as a kook or extremist.  The standards were adopted rapidly, but implementation of the high stakes tests has taken long enough for strong opposition to materialize.  Common Core may have captured Nijmegen, but the Arnhem of high stakes testing has proved a bridge too far.

This has not stopped the attempt to characterize opponents as kooks and extremists.  To be fair, some opponents are kooks and extremists, but many are not and Common Core supporters have had a bad habit of avoiding substantive debate by trying to dismiss their opponents as crazy.  There is something vaguely authoritarian about trying to centralize all education standards and testing, so not surprisingly Common Core supporters have also resorted to authoritarian tactics.  Taking a page from Tricky Dick, they have begun to use the power of the government to identify and punish opponents.

No, I’m not just talking about the threat that NCLB waivers and RTTP money would be more available to those who played ball with Common Core.  I’m talking about going after individuals who dissent.  Check out this story about  Brad McQueen, a teacher in Arizona, who published an op-ed against Common Core.

The state’s Associate Superintendent, Kathy Hrabluk, alerted her subordinates to this teacher’s dissent and asked them to “check your list of teacher teams (from which teachers are selected to work on tests at the Dept of Education)” so that he would not be involved in future teacher workgroups on state tests and other matters.  McQueen had been on those workgroups for the previous five years for which he received extra compensation.  No more.  As the Deputy Associate Superintendent for Assessments, Irene Hunting, replied to her boss, “We have made a note in his record.”  Another state official replied, “This was such a surprise for Arizona as Brad has been on many committees…  Let’s make sure he is not going to Denver later this month [to work on the new tests]. Please remove Brad McQueen from the list.”

Another Arizona education official, displaying all of the political sophistication of the Common Core movement, then replied on her government email, saying: “What a f*cktard.”

State education officials, doing their best to be the Common Core equivalent of the White House Plumbers, then proceeded to work on identifying one of McQueen’s fellow teachers to lend his or her name to a rebuttal op-ed that they would ghost write.  The bureaucrat in charge of PARCC for Arizona also called McQueen in his classroom to challenge him on why he opposed her test and quiz him about whether he was teaching the required standards.  McQueen feared they were fishing for grounds to terminate him and got off the call feeling like he has been threatened by a senior state official.

It’s an ugly story.  But this is what happens when you flirt with authoritarian reforms of education.  You start acting like an authoritarian.

(updated as described in comments)

Shakespeare’s Birthday and the Death of Humanities

April 23, 2014

Today is being recognized as the 450th anniversary of Shakespeare’s birth.  Harold Bloom helpfully suggests that our continued interest in Shakespeare has something to do with Shakespeare’s particular insight into what it means to be a human being: “Shakespeare not only invented the English language, but also created human nature as we know it today.”

This may also help explain the declining interest in Shakespeare in schools and among some of the more prominent ed reform movements — they don’t really care about teaching children about what it means to be a human being (otherwise known as “the humanities”).  They increasingly view school as a mechanism for improving students’ economic prospects.  And of course, training students to earn a living is an important component of school, but it is not the only or even most important element of education.

We aren’t gorillas, for whom zoo-keepers seek to optimize food, shelter, and longevity.  Unlike gorillas we are inclined to reflect on what our existence means and try to give that existence purpose.  Education should help guide us in doing that, not just train us to optimize food, shelter, and longevity by becoming the best future workers we can be.  To reflect on what it means to be a human being we need to learn the humanities, including history, literature, and art.

Who is against the humanities?  Few will say it out loud, but it is the dominant thrust in the 21st Century Skills movement, which is backed by the same people who gave us Common Core, with its shift away from literature to “informational texts.”  When confronted with their manifest disinterest in the humanities, 21st Century Skills folks tend to respond that of course they are also for art, history, and all that stuff.  But I challenge you to find where the humanities are in their “framework for 21st century learning.”  See if you can find it in this graphic they say represents the “key elements of 21st century learning“:


Did you find the humanities?  Is it in in “Life and Career Skills”?  Does poetry fit in “Information, Media, and Technology Skills”?  It can’t be in the “4Cs” or “3Rs” because history doesn’t start with an R or C.  Anyone who thinks that alliteration constitutes a persuasive argument is likely to be an uncultured barbarian.

Remember that Microsoft and the Gates Foundation are important supporters of the “Partnership for 21st Century Skills.”  And Bill Gates himself seems to have a low opinion of the art and humanities, or at least museums devoted to those subjects:

“Quoting from an argument advanced by moral philosopher Peter Singer, for instance, [Gates] questions why anyone would donate money to build a new wing for a museum rather than spend it on preventing illnesses that can lead to blindness. ‘The moral equivalent is, we’re going to take 1 per cent of the people who visit this [museum] and blind them,’ he says. ‘Are they willing, because it has the new wing, to take that risk? Hmm, maybe this blinding thing is slightly barbaric.'”

To which Terry Teachout, the Wall Street Journal’s art and theater critic, replied masterfully.  Let me take the liberty of quoting him at length:

Where to start sifting through the nonsense? For openers, Mr. Gates would do well to find a better guru than Mr. Singer, whose greatest-good-for-the-greatest-number approach to moral philosophy (if you want to call it that) has led him to advocate, among other horrific things, what he politely calls “permissible infanticide.” It strikes me that Mr. Gates might possibly want to be a bit more careful about the intellectual company that he keeps.

More to the point, though, it seems clear to me that Mr. Gates thinks it immoral for rich people to give money to museums instead of medical projects, presumably those that have received the official Bill Gates Seal of Moral Approval. To be sure, he deserves full credit for putting his own money where his mouth is: The Bill & Melinda Gates Foundation gives away some $4 billion a year, much of which is used to support health-related initiatives in developing countries, including a world-wide initiative to stamp out polio.

Good for him—but when it comes to art, he’s got it all wrong, and then some.

It almost embarrasses me to restate for Mr. Gates’s benefit what most civilized human beings already take to be self-evident, which is that art museums, like symphony orchestras and drama companies and dance troupes, make the world more beautiful, thereby making it a better place in which to live. Moreover, the voluntary contributions of rich people help to ensure the continued existence of these organizations, one of whose reasons for existing is to make it possible for people who aren’t rich to enjoy the miracle that is art. If it weren’t for museums, you wouldn’t get to see any of the paintings of Rembrandt and Monet and Jackson Pollock (and, yes, Francis Bacon). Instead they’d be hanging in homes whose owners might possibly deign to open their doors to the public once a year. Maybe.

As long as folks who have little appreciation for the arts and humanities are dominating ed reform discussions, we are unlikely to make much progress in reviving those topics in schools.  We may be celebrating Shakespeare’s birth, but what he stood for is dying.


Get every new post delivered to your Inbox.

Join 2,779 other followers