Murray Misses the Mark

The New York Times features a piece by Charles Murray arguing that choice has failed to improve test scores.  In general, Murray doesn’t think schools can do much to improve test scores.  He says:

This is true whether the reform in question is vouchers, charter schools, increased school accountability, smaller class sizes, better pay for all teachers, bonuses for good teachers, firing of bad teachers — measured by changes in test scores, each has failed to live up to its hype.

It should come as no surprise. We’ve known since the landmark Coleman Report of 1966, which was based on a study of more than 570,000 American students, that the measurable differences in schools explain little about differences in test scores. The reason for the perpetual disappointment is simple: Schools control only a small part of what goes into test scores.

Cognitive ability, personality and motivation come mostly from home. What happens in the classroom can have some effect, but smart and motivated children will tend to learn to read and do math even with poor instruction, while not-so-smart or unmotivated children will often have trouble with those subjects despite excellent instruction. If test scores in reading and math are the measure, a good school just doesn’t have that much room to prove it is better than a lesser school.

Murray wants to be clear that he still favors choice, but not to improve test scores.  Instead, he favors choice because it satisfies the diversity of preferences about how schools teach and what they teach.  Standardized test scores impose a uniform concept of higher achievement on students, and so cannot capture the improved satisfaction of the diversity of tastes that choice can more efficiently satisfy.

There is a kernel of truth in Murray’s argument.  We should support school choice simply because it allows us the liberty of providing our children with the kind of education that we prefer.

But Murray is completely mistaken in asserting that choice cannot (and has not) produced improved outcomes on standardized measures.  The only research he references is the recently released, non-random assignment evaluation of the effect of Milwaukee’s voucher program on students receiving vouchers.  This ignores the 10 superior, random research designed studies summarized here.  Importantly, it also ignores the effects of expanding choice and competition on achievement in entire school systems.

Especially with regard to a large and mature voucher program, like the one in Milwaukee, the relevant thing to focus on is systemic effects, not participant effects.  Almost everyone in Milwaukee has access to expanded choice, so everyone is receiving the treatment — school choice.  The difference between voucher participants and non-participants is where they chose to go to school, not the difference between having access to choice or not. And if you look at the systemic effects study in Milwaukee it shows significant gains in student achievement as choice and competition are expanded.

It is irritating to have to repeat this discussion of the evidence each time Charles Murray, Sol Stern, or Diane Ravitch selectively cite (or ignore) the research literature and claim that choice has no effect.  It’s also puzzling why “conservative” activists feel the need to denounce choice and competition in order to promote their pet reform idea.

Murray may well be right that schools face serious constraints in improving student achievement, but you don’t have to trash the gains that have been realized to make that point.  (And I think the constraints are less severe than he suggests).

Stern may well be right that even schools in more competitive markets have to make good decisions with regard to curriculum and pedagogy to produce significant improvement.  But choice and competition facilitate schools making good decisions about curriculum and pedagogy by providing negative consequences for those who choose foolishly (as well as giving schools the freedom to try more effective instructional techniques).  And Ravitch may be right about … well, maybe she isn’t right about very much.

Are conservative activists so starved for attention that they are willing to feed the New York Time’s preferred strategy of promoting conservative in-fighting, just so they can get into the pages of the Grey Lady?

(Edited to add link)

19 Responses to Murray Misses the Mark

  1. Patrick says:

    And we shall call this the French Education Reform Plan, or “Give Up and Surrender Now”

  2. Professor Greene,

    “Are conservative activists so starved for attention that they are willing to feed the New York Time’s preferred strategy of promoting conservative in-fighting, just so they can get into the pages of the Grey Lady?”

    That would not be my explanation (mostly).

    Unlike Ravitch, Murray is a free marketeer. Ravitch qualifies as conservative in that she favors a traditional classical education. She would use State power to force her vision on other people’s kids and to compel taxpayers to support it. The NEA/AFT/AFSCME cartel will take the support of traditionalists for their system, and then deny them a role in operation and curriculum formation.

    Unlike Ravitch, Murray does specialize in education, and I expect he has hypnotized himself into a mild genetic determinism. I suspect that Murray has convinced himself that some gap in performance is inevitable, and has not seen the evidence that improved curricula, improved methods of instruction, and closer tailoring of curricula and methods to individual students can boost performance for all students.

    The 1996 TIMSS reported a Singapore 5th (fifth) percentile score (8th grade Math) above the US 50th (fiftieth) percentile score. If this were IQ, The US “average” student would qualify as severely retarded in Singapore. Nobody reports a Asian/White IQ gap that large. Curriculum has to make the difference.

    • KDeRosa says:

      Malcolm, In Real education Murray makes the claim that we might expect to see an improvement of about a quarter of a standard deviation in the long run due to improved curricula. Part of the rerason is that we start to get fade out effects when the students hit adolescence and part of the reason is becasue we don’t have many long term interventions that have worked. Murray’s takes seems to be a fair (though perhaps overly-conservative)reading of the extant research.

      • The brain is an evolved organ. Parents roll a bucket of dice when they put their kids together, and some kids come up snake-eyes. Different regional varieties (races) of parents draw their buckets from different pools. The idea that the brain is uniquely exempt from selective pressure and genetic drift is intuitively unlikely. That’s the abstract argument.

        Against the conclusion, drawn largely from that abstract argument, that the potential achievement gains from improved curricula are as limited as Murray says (admission: I have read neither The Bell Curve nor Real Education, just reviews), I offer my experience as a teacher and tutor that the current US K-12 school system is a huge waste of time. I expect that, with coherent instruction and appropriate incentives (mom’s love and approval, in early years, for example; that’s one reason why homeschooling works), most humans of any regional variety could get through the usual undergraduate Math curriculum by age 18. Probably age 14 if parents wanted to do this, and started early enougfh. I hope to see “Your Toddler Can Compute Triple Integrals In Polar Coordinates” on the market before I die. Someone really ought to start work on it.

  3. Well said, Malcolm.

  4. Thanks. My point about Murray makes sense if you understand that, while Murray has studied intelligence and academic performance, I intended “Murray does not specialize in education”, meaning comparative education systems, and this is why he has not seen the evidence.

    Ravitch deserves credit for her scholarship and for the work she put into ploughing through heaps of academic educationese in her research on Left Back. Likewise, Murray deserves credit for the effort that he put into Losing Ground, and the courage it took to put The Bell Curve into circulation. I’m disappointed to see Murray’s failure to appreciate what markets can accomplish in education.

  5. Greg Forster says:

    I don’t think I would describe Murray’s genetic determinism as “mild.”

  6. I have to differ with you on your praise for Bell Curve. The “courageous” things in it were really just mostly mistaken.

    • KDeRosa says:

      Which claims were mistaken, Jay?

      Some were (or at least overreached) and some of the methodology was flawed. But the Bell Curve book is not the only source of research for some of the claims associated with that book.

      Greg, why not? What’s your take on the issue and why is Murray’s view not mild, implying a normative judgment for what is an empirical question for which the research is not dispositive either way as of yet.

      • The controversial claim in Bell Curve was about IQ differences across racial/ethnic groups. Of course, this was only a side-point to the main argument about how more efficient IQ sorting of the population would lead to greater IQ inequities.

        I have no issue with Murray’s main point, which does not really require the differences across racial group argument. But on the differences across racial groups, I think Murray is mistaken because he has excessive confidence in the ability of an socio-economic status control variable in a regression model to sort out environmental from genetic differences.

      • KDeRosa says:

        jay, yes, that is one of the methodological flaws in the analysis, but pointing out that flaw doesn’t end the debate, just like poitnig out one of the methodological flaws in one of the voucher studies doesn’t end the debate.

        There is significantly more research on the issue. Rushton and Jensen catalog it in theor 2005 paper: Thirty years of research on Black-White differences in cognitive ability. Psychology, Public Policy, & the Law, 11, 235-294. (Link)

        Morever, in his next book, Income Inequality and IQ, Murray responded to the criticism you noted by adopting a sibling study design.

  7. Greg Forster says:

    To say that Murray’s genetic determinism is not “mild” does not imply a normative judgment about the empirical question. It does not imply anything about the empirical question at all. It is a characterization of Murray’s evaluation of the empirical question.

    If (by your own admission) the data on the empirical question are not yet dispositive, Murray should stop talking as though they are dispositive. That’s one reason I wouldn’t characterize his genetic determinism as “mild.”

    • KDeRosa says:

      Indeed, that was one of the methodological flaws. However, there are numerous other studies that have found essentially the same thing.

      It’s like when public school apologists ignore all the voucher studies that Greg cites below.

      Rushton and Jensen have a thirty year survey of the relevant research, most of which is not so easily discounted, if at all.

  8. KDeRosa says:

    Greg, I’ll give you the benefit of the doubt on the normative question, but there are certainly more neutral ways of saying “not supported by the evidence.”

    What was wrong with Murray’s analysis of the empirical question (not to be confused with policy prescriptions)? He seems to follow the hereditarian theory which posits that about 50% – 70% of cognitive ability is hereditary, with the rest the result of envuironmental effects. That’s hardly deterministic. And, more importantly, is supported by ample indirect evidence, though we have no direct causal proof.

    Let me tell you another thing that has lots of indirect evidence and no direct causal proof yet — school vouchers and charter schools. And yet the members of this blog (one of the best education blogs out there) have no trouble advocating for vouchers and school charters despite the absence of direct causal proof and more than occiasionally neglect to temper the advocay with scientifically prudent privisos. (Not that I think they are needed in a publication for a general audience, but if you insist on holding Murray to that standard, you should at least play fairly).

    It is ironic that this blog has to fight so valiantly, on an almost daily basis, against those who miscite and ignore the evidence on vouchers and charters, and then fails to offer the same consideration for a fellow “scholar” whose views they do not agree with, but with which they share the same evidentiary boat.

  9. KDeRosa says:

    There are too many uncontrolled variables in those studies to conclude a direct causal effect for vouchers alone. They are good studies but not so good to make that claim without some serioud disclaimers.

  10. I did not mean to bring the race and IQ argument here. It’s raging at the Volokh Conspiracy and the most temperate opinions come from the least emotionally involved. Seems to me the least involved devote their emotional committment to freedom, since a free marketeer will suppose that the policies which effectively address differences in scholastic aptitude within geographic varieties (races) of humans will be the same as those that address differences in scholastic aptitude within geographic varieties (races) of humans.

    One large and under-apprecieted cost of the current State-monopoly system is the opportunity cost to society of the lost innovation which a competitive market in education services would generate. Stuart Buck recently wrote an essay on human capital misallocation. This is just one area where society at large and students in particular could benefit from improvements over the current one-size-fits-all policy, which restricts parents’ options for the use of the taxpayers’ age 6-18 education subsidy to schools operated by dues-paying members of the NEA/AFT/AFSCME cartel.

  11. Sorry. Another typo. “Within” races, then “between” races. Need coffee.

  12. concerned says:

    I would call it opportunity lost.

    For everyone interested in improving student achievement and increasing opportunities for students, I would highly recommend “Teach Like A Champion”

    It truly gets to the heart of the issues raised here.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: