Jay P. Greene's Blog

Actually, it was Friedrich Hayek who said this, but this image seemed a lot funnier.  Hayek — the economist with the funny mustache, not the movie star — was briefly a professor at the University of Arkansas before going on to the University of Chicago and winning a Nobel Prize.  Go Hogs!  As far as I know, Salma Hayek has never taught at the University of Arkansas and has also never won a Nobel Prize.  I think we can all see the causal connection.

While we are on the topic of the difficulty of drawing causal connections and being reminded of how little we know about what we imagine we can design, I’d like to add another piece of evidence to my previous post on “The Over-Confidence of Portfolio Management.” The central idea of portfolio management is to identify and close under-performing schools while opening and expanding higher quality schools.

The problem I noted in my previous post, is that portfolio management (PM) does not have a reliable tool for identifying which are the “good” and “bad” schools.  In New Orleans they identify “bad” schools based on the state’s school grading system, which is determined almost entirely by the level of student achievement on math and reading tests (and almost not at all on the growth in learning).  There is no reason to believe that schools with low levels of scores are “bad” schools, so there is no reason to believe that portfolio management is effectively identifying and closing  bad schools.  And if PM in NOLA can’t effectively distinguish between good and bad schools, there is no reason to believe that whatever progress has been made in New Orleans is due to portfolio management.

But this is not holding back PM supporters from adopting portfolio management as the “best practice” and hoping to implement it in Detroit and elsewhere.  They’re confident that they can identify bad schools for closure by focusing on growth in standardized test results, even if that was not what was done in NOLA.  The problem I noted in my last post is that even annual growth in test performance does not seem to be strongly predictive of later-life success.  Looking at 7 rigorous studies of long-term outcomes for charter and private school choice programs, we find that programs that produce large test score gains produce little or no gain in educational attainment while other programs with modest or no gains in test scores make large gains in high school graduation, college attendance, and even workplace earnings.

Well, now we have an 8th rigorous study and it fits the same pattern as the previous 7.  Joshua M. Beauregard’s Harvard doctoral thesis examines college outcomes for students applying by lottery for admission to High Tech High (HTH) School in San Diego.  High Tech High is a widely-lauded charter school that receives significant financial support from many of the big education reform foundations.  It’s a model of the kind of school that portfolio managers are supposed to open and expand.

So what is the effect of HTH on whether its students attend college?  Beauregard finds: “I detect no impact on the probability of enrollment in college of any type, which includes two-year and four-year colleges, for my aggregate sample. My estimates are consistent when including baseline demographics and ELA scores.” (p. 28) Similar to Angrist, et al’s results from studying “no excuses” charter schools in Boston, Beauregard does find that attending HTH seems to shift college-going students from two to four-year colleges, but it has no overall effect on post-secondary education.  He describes the increase in four-year college attendance as “statistically significant” although the results in Table 5 show that this is only at p<.1.  And we have no idea whether shifting students from 2 to 4 year colleges is good if students are unable to complete a 4 year degree. Beauregard also reports no increase in attendance at “competitive” four-year colleges as a result of attending High Tech High.

With HTH we have another example of a school of choice with impressive test score performance that does not produce commensurate improvements in long-term outcomes for students.  The central flaw of portfolio management is that the tool it uses to identify which schools should be closed and which should be opened and expanded is not well-connected with long-term success for students.  PM can’t improve outcomes for students by closing “bad” schools if it can’t accurately determine which are the bad schools.

Maybe Salma Hayek should explain to them “how little they really know about what they imagine they can design.”

This entry was posted on Saturday, November 14th, 2015 at 10:24 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.