Gates Foundation — Release the MET Results

A sketch of the $500 million new Gates Foundation headquarters

Bill and Melinda Gates mentioned again in the Wall Street Journal the Measuring Effective Teachers (MET) project that their foundation is orchestrating.  Bill and Melinda may want to check on the status of the MET research they’ve been touting since full results were promised in the spring of 2011 and have yet to be released.

Just to review… In an earlier interview with the Journal, MET was described as follows:

the Gates Foundation’s five-year, $335-million project examines whether aspects of effective teaching, classroom management, clear objectives, diagnosing and correcting common student errors can be systematically measured. The effort involves collecting and studying videos of more than 13,000 lessons taught by 3,000 elementary school teachers in seven urban school districts.

The motivation, re-iterated in the new piece by Bill and Melinda Gates is to identify  what “works” in classroom teaching to develop systems that train and encourage other teachers to imitate those practices:

It may surprise you—it was certainly surprising to us—but the field of education doesn’t know very much at all about effective teaching. We have all known terrific teachers. You watch them at work for 10 minutes and you can tell how thoroughly they’ve mastered the craft. But nobody has been able to identify what, precisely, makes them so outstanding….

The intermediate goal of MET is to discover what we are able to measure that is predictive of student success. The end goal is to have a better sense of what makes teaching work so that school districts can start to hire, train and promote based on meaningful standards.

As I’ve argued before, using research to identify “best practices” in teaching only makes sense if the same teaching approaches would be desirable for the vast majority of teachers and students, regardless of the context.  And as I’ve also  suggested before, I don’t believe this effort is likely to yield much in education.  Effective teaching is like effective parenting — it is highly dependent on the circumstances.  Yes, there are some parenting (and teaching) techniques that are generally effective for almost everyone, but those are mostly known and already in use.

This doesn’t mean we are completely unable to measure effective teaching (or parenting).  It just means that we have to judge it by the results and cannot easily make universal statements about the right methods for producing those results.  To make a sports analogy, there is no single “best practice” for hitters in baseball.  There are a variety of stances and swings.  The best way to judge an effective hitter is by the results, not by the stance or swing.  And if we tried to make all hitters stand and swing in the same way, we’d make a lot of them worse hitters.

It is because of this heterogeneity in effective teaching practices that I think the MET project is doomed to disappoint.  And according to inside sources, I’ve heard that results are being delayed because they are failing to produce much of anything.

According to the MET web site, the full results for the 1st year should have been released in the spring:

 In spring 2011, the project will release full results from the first year of the study, including predictors of teaching effectiveness and correlation with value-added assessments.

It is almost November and we have not seen these results.  I understand that in very large and complicated projects, like MET, things can take much longer than originally planned.  If so, it would be nice to hear that explanation.  It would be even nicer if the Gates Foundation released results if they have them, even if those results were not what they had hoped they would find.

Some inquisitive reporters should start asking Gates officials and members of the research team about the status of the MET results.  Reporters should go beyond talking to the media flacks at Gates HQ and actually talk to individual members of the team confidentially.  If they do that, they may confirm what I have been hearing: MET results have been delayed because they aren’t panning out.

(UPDATE:  Gates responds.

8 Responses to Gates Foundation — Release the MET Results

  1. George Mitchell says:

    “inquisitive reporters”?

    In education?


  2. Greg Forster says:

    Sounds like like MET is really MEH.

    (Measuring Effective Heterogeneity)

  3. Niki Hayes says:

    Well written comments that make sense, as yours do on this subject, are not understood by the average reporter. I can say this after having spent 17 years in journalism, outside of 28 years in public education as a teacher, counselor, and principal.

    I like the business model being applied to much of education, particularly at the administrative level, because so few in administration have business knowledge. (As a former math teacher, I was asked by a director if the principals knew how to handle their budgets. I said “no,” because most are afraid of anything that smacks of math.)

    Good teaching begins with a good selection of candidates and solid, practical training, which is an ugly concept in education because it reflects repetition of practice, and God knows we can’t have boring activities such as repetition OR practice. Selection of candidates and training by schools of education are awful. (If you can pay the tuition, you’re in the ed program.)

    The Gates’ efforts, as in most educational ventures, are based on new “visions” that use our children as guinea pigs to determine what might work. When parents are required to sign agreements for experimental curricula or methods being used on their kids, then we might begin to see some real questioning that must have real answers.

  4. Daniel Earley says:

    A longitudinal study of best practices in Jedi training indicates that optimal mentoring occurs when instructors reverse subject-verb sequencing and hold weaponry with only three digits.

  5. GGW says:

    Hmm. Not with you on this one. You make a number of points. I’ll tackle one.

    You say most of the effective teaching techniques are already “known” and “in use.” I disagree.

    We’d probably agree that Ed Schools suffer from a number of market failures. So these institutions, which train teachers, probably often disseminate them the WRONG stuff. Stuff they claim is “known.” (Because of faulty research). But it’s not known.

    Let’s set aside the question of whether MET will be able to generate empirical data on “what works.” Let’s set aside whether MET can further disaggregate in a way that’s helpful (there’s heterogeneity but it’s not infinite — a method that seems to work with 10% of the population is still 5 million kids).

    In fact, what works is probably not known, and CERTAINLY not in use (or we wouldn’t have the results we do).

    Furthermore, you say “The best way to judge an effective hitter is by the results, not by the stance or swing.” For all the MET faults real or imagined, that’s precisely the whole approach! They’re mapping back from imperfectly measured RESULTS. Almost every other “Teacher researcher” focuses only stance or swing. Preferably project-based stances and portfolio swings.

  6. Daniel Earley says:

    Please read carefully and completely.

    • George Mitchell says:

      Anecdotal evidence: One of my daughters teaches theater to elementary age children. I have seen a couple of her productions. Amazing stuff. From an early age she was a “herder” of neighborhood kids. She has what I have seen in other outstanding teachers, i.e., an intangible ability to command attention and respect. She “learned” none of that at the respected midwest university where she received a BA in education. The idea that some global initiative financed by a mega-grant is going to filter down and revolutionize teaching strikes me as not credible. What evidence is there from other large industries or enterprises that the approach contemplated by Gates will work?

  7. The problem is the results, and MET thinks standardized test scores are adequate indicators of ‘results’. They are not, they are at best weak proxies for some of the useful things society seems to have as goals for schools. Just to use the baseball batter analogy, note with computers the evolution of complicated studies of results. We used to have batting average, rbi, slugging came along fairly early, of course homers vs singles, number of walks and strikeouts etc. Now these are combined to try to find more ‘powerful’ stats to indicate the ‘best’ hitters, what combo of power and on-base, etc, describes the most desirable hitter. But of course a team needs a variety of good types to have an effective offense, just as a society needs a variety of skills, knowledge, temperaments, etc. — only society is vastly more complicated than baseball. MET’s effort to determine who best boost scores on mostly multiple-choice tests is pretty akin to deciding that the best singles hitter is the best batter. Unfortunately, this pseudo-science will likely stampede an even greater effort to teach to the test, if that is possible.

Leave a Reply to George Mitchell Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s