Jay P. Greene's Blog

(Guest Post by Jason Bedrick)

Yesterday, Jay took apart the factually challenged hit piece the Gray Lady ran on charter schools in Detroit. Matt added his two cents and I piled on as well. Additionally, Tom Gantert of the Mackinac Center for Public Policy identified another dozen or so errors, distortions, or omissions of key facts.

But the most illuminating part of this whole imbroglio was the following Twitter exchange with the NYT reporter:

Jay and others, including Prof. Martin West of Harvard, explained how the article misrepresented the findings of CREDO’s research on charter schools. The NYT reporter, Kate Zernike, defended her piece by noting that she contacted CREDO about the charter she highlighted, which CREDO had found to be low performing. However, Jay pointed out that she cherry-picked that one bad charter and failed to note that CREDO found overall positive results.

Zernike  apparently forgot the First Law of Holes: when you find yourself in one, stop digging.

Jay also took her to task for relying only on “claims and anecdotes” instead of data, so she cited data from Excellent Schools Detroit.

However, as Jay explained, those data do not allow for direct comparisons. Zernike is right that the data show the citywide averages in each sector, but looking at the averages is misleading. Zernike seemed to think Jay was objecting to the standardized tests, but the real problem he identified was not the tests themselves but rather how she tried to compare the students taking those tests.

As I noted in my blog post:

The charter schools tend to be mission-based schools that open in the toughest areas and serve the most at-risk students. Comparing city-wide averages fails to take that into account. It would be like comparing the New England Patriots against a championship high school team and concluding that the teenagers are superior athletes because they scored more touchdowns per game.

The appropriate comparison is between the charters and the district schools that serve the same or similar student populations. That is what the CREDO study attempted to do by matching students with similar characteristics and initial test scores in each sector, then tracking and comparing them.

Amazingly, Zernike then claimed that CREDO did not find that Detroit charters had a positive effect overall. In fact, that’s exactly what CREDO found in both its 2013 report on charters in Michigan and its 2015 nationwide charter report.

As Matt noted, the 2013 CREDO study on Michigan found: “on average, charter students in Michigan gain an additional two months of learning in reading and math over their [traditional public school] counterparts. The charter students in Detroit gain over three months per year more than their counterparts at traditional public schools.” [emphasis added]

Nevertheless, Zernike persisted in claiming that CREDO found results that it didn’t find.

That’s right. Zernike says CREDO “does not consider Detroit stellar” — but page 33 of the 2015 CREDO report calls Detroit’s charter sector “a model to other communities.” See for yourself:

“models to other communities”

But apparently some reporters don’t like it when people point out that what they wrote is factually incorrect. Here was her response to my Tweet correcting her:

Blocked!

If the New York Times has any integrity, it will issue a correction.

“The creatures outside looked from pig to man, and from man to pig, and from pig to man again; but already it was impossible to say which was which.”

When I became involved in education reform more than two decades ago, the movement was about empowering parents to make choices for their own children rather than having choices made for them by well-meaning but distant bureaucrats and professionals.  At its heart, ed reform was about decentralization of control.

In the last quarter-century this effort to expand choice in education has been amazingly successful.  We’ve gone from the first charter school in 1990 to more than 4% of all students enrolled in charters.  We’ve gone from two, century-old voucher programs in Maine and Vermont to having private school choice in more than half of the states.  And the beauty of expanding school choice is that it generates its own advocates as families that benefit from these programs lobby to protect and expand their choices.We are almost at the point where ed reform organizations don’t have to do very much other than to coordinate choice families pushing for more choices.

But just as choice is achieving escape velocity, a groupthink gang of petty little dictators are grabbing the reins of ed reform organizations to advocate for greater restrictions and regulations on choice. They are beginning to make arguments and advocate policies that are essentially the same as the ones favored by the traditional education establishment. Like their rivals in the traditional ed establishment, this new ed reform establishment mistrusts parents to make choices.  Parents, in their view, are not capable of making good choices without the guidance and restrictions imposed by experts and policymakers. And children need to be protected by regulations and bureaucrats against the errors and abuses of their parents or schools.

It has gotten to the point where, like in Animal Farm, it is difficult to tell the difference between the nanny-statism of the old ed establishment and the new ed reform establishment.  The new ed reformers are no longer fighting for parental empowerment, they are just struggling with the old establishment over who will be in control.  Will it be the smart and righteous reformers, as they imagine themselves, or the stupid and self-interested old establishment, as they imagine the unions and their allies? The reformers are convinced they can do it better, but the arrangements they favor are not all that different from those championed by the old guard.

Reformers are currently gathered in a groupthink frenzy over the need to regulate how charter schools discipline their students.  You know who else issues detailed policies on school discipline? Traditional school districts.  Last year they were in a frenzy over the need to force charter schools to “backfill” so that they can take more students in more grades that are assigned to them. You know who else is pre-occupied with filling seats in schools with assigned students? Traditional school districts.

It is currently the fashion among reformers to favor portfolio management, in which a single super-regulator would control which schools open, which close, and issue policies regarding transportation, special education, discipline and other matters.  We’ve even heard proposals recently to have the entity responsible for opening, closing, and regulating schools be elected democratically.  Let’s see if you can guess what all of this sounds like.  That’s right — traditional school districts.  They are also democratically elected.  They also decide which schools should open and close.  They also issue policies regarding transportation, special education, discipline and other matters.  I have looked from portfolio management to districts, and from districts to portfolio management, and from portfolio management to districts again; but already it is impossible to say which is which.

The advocates of portfolio management or democratically elected authorizers say that the difference is that traditional districts actually operate schools, while their proposed entities only concern themselves with opening, closing, and regulating while avoiding interference in operational matters.  We were assured that things would be fine with portfolio management in New Orleans despite the takeover of that role by the Orleans Parish School Board because the district is prohibited from interfering with school operations.

I may not be able to read the continually revised commandments on the barn wall much better than Boxer, but I’m pretty sure that issuing policies with respect to school discipline, special education, admissions, and transportation necessarily interfere with school operations.  And it is only a hop, skip, and jump from telling schools whether they can suspend kids to telling them which methods best teach reading or how many minutes they should be on the playground.  Anyone who is not hypnotized by the reform groupthink would recognize that school boards do not “operate” schools any more than portfolio managers do.  Boards just develop policies to govern schools, just like portfolio managers do.  They contract with others to operate schools under those regulations, just like portfolio managers do.  And they decide which schools should be opened and which should be closed, just like portfolio managers do.

The ed reform crowd enamored with portfolio management and issuing a host of regulations dictating how schools must operate and what parents may choose has become almost indistinguishable from the traditional education crowd with whom they are vying for control.  I say a pox on both their houses.  I got into this line of work because I was excited about empowering parents to make decisions, not imposing my superior brand of control on them.

(edited for typos)

In today’s Wall Street Journal  Paul Peterson makes a personal and compelling argument for why deference should be given to parents over the false-expertise of distant bureaucrats.

Paul’s adult son is autistic and sometimes engages in potentially life-threatening self-injurious behavior, such as banging his head and shoving his hands down his throat.  After many years and much searching, Paul and his family found an effective intervention that checks this self-injurious behavior and enables his son to live a better and safer life. The intervention involves skin electric shocks, which essentially grab his son’s attention and interrupt his self-injurious activities.

The shocks do hurt, but the pain is relatively mild and fleeting relative to the serious harm his son might do to himself otherwise.  And his son receives plenty of positive rewards for avoiding self-injury.  The shocks are a back-stop when things get out of control.

But bureaucrats at the FDA think they know better and want to prohibit the type of intervention benefiting Paul’s son.  They believe that there are drug therapies that are more effective.  Unfortunately, even if those drug therapies work on average, there is a distribution of results and Paul’s son has tried the drugs without success. No matter, declares the FDA, everyone should get what we think works best for the average person even if your circumstances differ.

You should read Paul’s entire piece.  When doing so keep in mind that this isn’t just about the FDA and people with autism.  This is about who knows best.  Should families and care-providers who possess more contextual information decide what to do or should distant bureaucrats impose on everyone.  And, of course, this is the heart of my support for choice in education.  Who should decide what is best for children — their families and the educators they select or regulators, portfolio managers, and other over-bearing bureaucrats?

The problem: too many choices?

(Guest Post by Jason Bedrick)

The abundance of choices among competing grocery stores is “clogging” our roads, according to an Australian academic who advocates assigning people to government-run grocery stores based on the location of their home:

Almost 60 per cent of Melbourne shoppers are bypassing their local grocery store, according to world-renowned academic John Hattie.

Professor Hattie said ‘food choice’ had led to a “clogging of the motorways” as shoppers avoided their neighbourhood grocery store in pursuit of alternatives.

Food choice has also fuelled unhealthy competition between grocery stores, he told a packed lecture theatre at the University of Melbourne on Tuesday evening.

“Nearly all this choice is based on hearsay, the flavor of the food, and rarely on whether the food is or isn’t adding nutritional value to the shoppers’ health,” he said.

All right, all right. If you’re a regular reader of the JPG blog, you’ve probably already deduced that I changed the words “school” and “students” to “grocery stores,” “food,” and “shoppers.” Here’s what Professor Hattie actually argued:

Almost 60 per cent of Melbourne students are bypassing their local school, according to world-renowned academic John Hattie.

Professor Hattie said school choice had led to a “clogging of the motorways” as students avoided their neighbourhood school in pursuit of alternatives.

School choice has also fuelled unhealthy competition between schools, he told a packed lecture theatre at the University of Melbourne on Tuesday evening.

“Nearly all this choice is based on hearsay, the nature of the students, and rarely on whether the school is or isn’t adding value to the students’ learning,” he said.

Frankly, it’s no less absurd than the modified version.

So-called “experts” demonstrate their own hubris and contempt when they proclaim from their ivory towers that they know better than the little folks below, despite lacking any local knowledge about a particular student’s aptitude, interests, or learning needs. Aside from the fact that the best international evidence suggests that school choice boosts student outcomes (i.e., on average, the parents outperform the central planners), educational choice is an end in and of itself.

Moreover, although I don’t think education policy should be used as a lever to promote environmental goals, research from Professor Bart Danielson suggests that school choice policies are more environmentally friendly than district-based systems. Countless parents have fled urban areas to rescue their children from failing district schools–the “white flight” that Hattie decries–which contributes greatly to suburban sprawl. According to Danielson’s research, parents are more likely to remain in urban areas when they have more educational options, thereby reducing their carbon footprint.

In short: the “clogged roads” argument against choice is as spurious as it is silly.

I’ve written several times recently about how short term gains in test scores are not associated with improved later life outcomes for students.  Schools and programs that increase test score quite often do not yield higher high school graduation or college attendance rates.  Conversely, schools and programs that fail to produce greater gains in test scores sometimes produce impressive improvements in high school graduation and college attendance rates, college completion rates, and even higher employment and earnings.  I’ve described at least 8 studies that show a disconnect between raising test scores and stronger later life outcomes.

Well, now we have a 9th.  Earlier this month MDRC quietly released a long-term randomized experiment of the effects of the SEED boarding charter school in Washington, DC.  Because SEED is a boarding school, there was a lot of hope among reformers that it might be able to make a more profound difference for very disadvantaged students by having significantly more time to influence students and structure their lives.  Of course, boarding schools also cost significantly more — in this case roughly twice as much as traditional non-residential schools.

While the initial test score results are very encouraging, the later life outcomes are disappointing.  After two years students admitted to SEED by lottery outperformed those denied admission by lottery by 33% of a standard deviation in math and 23% in reading.  If we judged the quality of schools entirely based on short term changes in test scores, as many reformers would like to do, we’d say this school was doing a great job.

In fact, SEED may be doing a great job in a variety of ways, but when we look at longer term outcomes for students on a variety of measures the evidence demonstrating SEED’s success disappears or even turns negative.  Of the students accepted by lottery to SEED 69.3% graduate from high school after four years compared to 74.1% for the control group, a difference that is not statistically significant.  And when asked about their likelihood of attending college, there was no significant difference between the two groups.  SEED students also score significantly higher on a measure of engaging in risky behavior and lower on the grit scale.

We’ve seen this pattern before.  Research by Marty West and colleagues of no excuses charter schools in Boston found large gains in test scores but also significantly lowered student performance on noncognitive measures.  And Josh Angrist and colleagues found that those schools actually decrease four year high school graduation rates despite large gains in test scores.  In their words:

Perhaps surprisingly given the gains in test score graduation requirements reported in column 2 of table 4, the estimates in column 4 of this table suggest not. In fact, charter attendance reduces the likelihood a student graduates on time by 14.5 percentage points, a statistically significant effect.

It’s time that people start paying a lot more attention to this pattern of a disconnect between short term test score gains and long term life outcomes.  We can’t just dismiss this pattern as fluke.  And the reduction in noncognitive skills may be important for explaining this pattern.  Reduced grit scores may not just be the product of reference group bias.  It appears that certain types of charter schools that are able to produce large test score gains also lower character skills and fail to yield long term improvements in life outcomes.  Conversely other types of charter and private schools in choice programs fail to improve test scores but yield large gains in later life outcomes.

If we think we can know which schools of choice are good and ought to be expanded and which are bad and ought to be closed based primarily on annual test score gains, we are sadly mistaken.  Various portfolio management and “accountability” regimes depend almost entirely on this false belief that test scores reveal which are the good and bad schools.  The evidence is growing quite strong that these strategies cannot properly distinguish good from bad schools and may be inflicting great harm on students.  Given the disconnect between test scores and later life outcomes we need significantly greater humility about knowing which schools are succeeding.