Education Reform 2003 to 2017: Modest Success/Epic Failure so What’s Next?

July 23, 2018

(Guest Post by Matthew Ladner)

Having had some time to reflect upon the 2017 NAEP, let’s take stock of things. In this we should keep in mind our broad ignorance between policy changes and state NAEP trends- and the same goes for average school quality. NAEP gives regular 4th and 8th grade scores in math and reading, and all 50 states have participated since 2003. 8th grade scores are more likely to reflect school quality than 4th grade scores in my opinion, as the students have more years of schooling. I’m not sure what to make of positive 4th grade score trends that do not result in higher 8th grade scores for instance. So this in essence a window into what we have to show for American K-12 reform 2003 to 2017 in 8th grade math and reading by state:

So what to make of the above chart? The below chart eliminates a lot of clutter by only including the states with statistically significant gains in both math and reading 2003-2017:

 

So 19 out of 50 states demonstrate statistically significant gains in both 8th grade math and reading. Notice also the absence in the second chart of mega-states Illinois, New York and Texas (although it is good to see California and Florida making it in). Texas has as many K-12 students as the 20 smallest states combined and annually adds approximately a Wyoming public school system sized number of new students. Florida has half as many students as Texas and California is still larger than Texas.

Since we don’t know the relationship between policy and academic trends, we are limited in the conclusions we can draw with confidence. Having said that, policies that have been broadly applied across all 50 states apparently suffer from severe limitations in their ability to move the needle academically. All 50 states for instance have adopted state academic standards and accountability exams, but most states have failed to move the needle on 8th grade scores. Even if we were feeling incredibly generous and made the wild assumption that none of the second chart gains would have happened in the absence of testing, a failure rate of 62% after 14 years is a far cry from leaving no child behind.

Mike Petrilli and Peter Cunningham recently offered up “where do we go from here” think pieces. I think Mike has some interesting ideas, but Peter’s call for a vast increase in spending is broadly unrealistic imo given the nation’s trillions of dollars in unfunded pension and entitlement liabilities, 10k Baby Boomers per day reaching the age of 65, etc. In normal times, Mike’s incremental adjustments might make a lot of sense, but we don’t live in either normal times, or in times that are going to allow some Great Society on Steroids increase in K-12 spending.

A much more difficult scenario may loom whereby the district system continues to resist reform, reformers continue to push reforms the public does not care for, and severe funding needs for increased health care spending leads to a broad reduction in per pupil spending.  State constitutions guarantee K-12 funding, but whether or not they will be creating schools that the vast majority of parents will continue to entrust with their children, I don’t feel as confident about. There are hopeful signs in the NAEP from state charter sectors:

…but the rate of charter growth has slowed substantially nationwide. Of particular disappointment are the last several state charter laws to pass that produce very few charter schools. Even states with relatively fast growing sectors have large wait lists. There are alas limits to what we can realistically hope for from a charter movement that has to a large degree lost its way by prioritizing cartel behavior over the interests of children on wait lists imo.

The private choice movement enjoyed a strong run earlier in the decade, but has since ran into political headwinds. Many private choice programs exist, but most remain modest in scale. The case for private choice remains as strong as ever, and the need will continue to grow, but the looming state funding crisis is coming fast. In four years, half of the Baby Boom generation will have reached the age of 65, and by 2030 all of them will be there. They have called dibs in advance on all plausible funding increases and a whole lot more.

So what is next? An increasingly likely scenario in my mind is that state district systems retain their flaws but loses a significant part of their funding and that choice systems continue to fail to meet existing much less expanded demand. In such a scenario an increasing percentage of families may decide to fend for themselves. Call them home-schools, home-school co-ops or micro-schools, my spidey-sense tells me that we should expect to see a great many more of them in the years ahead. I’ll write more about this in a follow-up post.

Advertisements

Pre-K Helps Test Scores in Short Run But Hurts Them Later

July 16, 2018

Image result for jerry lewis professor

The Arnold Foundation’s Straight Talk On Evidence web site provides a very useful summary of a recently published large RCT on a state-funded pre-K program in Tennessee.  Consistent with a previous, nationally representative RCT of Head Start, this study found that students given access to government-funded pre-school by lottery initially score higher than those who lose the lottery on standardized test scores but then fare worse later.

In the TN study, treatment students score higher at the end of pre-K.  But, as the Arnold summary puts it:

At the end of third grade, the study found statistically-significant adverse effects on student math and science achievement. In math, the VPK group scored 0.12 standard deviations lower than the control group, which equates to roughly 13 percent less growth in math achievement than would be expected in the third grade year.[ii] In science, the VPK group scored 0.09 standard deviations lower than the control group, which equates to roughly 23 percent less growth in science achievement than would be expected in the third grade year.[iii]

In an effort to explain the negative longer-term result, the authors suggest that special education may be to blame.  Students admitted to the government-funded pre-K program were more likely to be labeled as needing special education services and that designation may have lowered academic expectations.  But this explanation is inconsistent with Hanushek, Kain, and Rivkin’s finding that special education tends to improve test score results.  Straight Talk at least considers the possibility that children being with family or in a non-government-funded pre-school may just be academically superior.

The hard reality is that the process of human development is complex and highly varied, so we just don’t know the optimal arrangements for all children.  Andy Smarick has an excellent piece along these lines in the Weekly Standard, suggesting that education policy experts suffer from a Hayekian information problem.  And this was also the subtext of my post last week on how parents are smarter than Technocrats.  Even when Technocrats are armed with the best science, they generally do not have enough information to centrally plan the lives of others.  This doesn’t mean that we never regulate anything.  It just means that if we do regulate we should do so with great caution and large dollops of humility because the experts are typically missing a lot of important information that the individuals they are regulating are more likely to posses.

But caution and humility are no fun, so the Arnold Foundation’s Straight Talk chooses instead to double-down on Technocracy by suggesting that the disappointing results of pre-school as shown in RCTs of both Head Start and the TN program be remedied by identifying which subset of pre-schools seem to be more effective and regulating programs toward imitating those schools:

The above findings and observations, we believe, underscore the need to reform programs such as VPK and Head Start by incorporating (i) rigorous evaluations aimed at identifying the subset of local approaches that are effective, and (ii) once such approaches are identified, strong incentives or requirements for other local program sites to adopt and faithfully implement them on a larger scale.

Keep in mind that the TN program already has regulations in place meant to ensure quality, including requiring at least 5.5 hours of instructional time per day, a cap of 20 students per classroom, a licensed teacher in each classroom, and the requirement that schools choose among a state approved set of curricula.  Also keep in mind that short-term test scores, which are the most common tool by which regulators monitor quality, showed positive results.

If these regulatory practices are insufficient to avoid harming students over the medium term, why would Straight Talk believe that doubling down on the Technocratic approach would make things better?  It would be nice if they at least considered the possibility that we are suffering from a Hayekian information problem and may be unable to devise optimal arrangements for education.


Parents are Smart. Technocrats are Dumb

July 12, 2018

Image result for jerry lewis professor

The technocratic brand of ed reform that is currently dominant is based on the premise that policy elites, guided by science, need to ensure school quality.  Parents should have choices, but they should only choose among quality options.  Mostly using test scores, technocrats believe they can identify quality schools and quality-promoting educational practices, which should over-ride parental preferences about which schools and practices offer a quality education.

A new study by Diether W. Beuermann and C. Kirabo Jackson suggests that parents may be better at detecting which schools promote long-term positive outcomes for their children than technocrats guided by short-term test scores.  They examine the school system in the Barbados in which parents seek admission for their children into schools they prefer, but those schools use test-score cut-offs to determine which students gain admissions.  The cut-offs create a discontinuity that allows for a rigorous causal identification of whether students who barely gain admission to a desired school have different outcomes than those with barely lower lower test scores who are denied admission.

They find that test score gains are no greater for students who were admitted to the schools their parents preferred than those not admitted.  For boys there are some signs that the effect on test score gains may actually be negative.  But when they look at longer-term outcomes, including educational attainment, employment, and earnings, they find significant benefits for students who were admitted to the schools the parents preferred.  These positive effects were driven mostly by gains for girls.  When they explore mechanisms for why these gains occurred, they find a significant reduction in teen motherhood for girls admitted to preferred schools, which contributed to their educational attainment and later employment and earnings.  They also found that both boys and girls experienced significant long-term health benefits as measured by a healthy BMI, regular exercise, and dental check-ups if they gained admission to the schools their parents preferred.  The researchers conclude: “This suggests that preferred schools may promote productive habits and attitudes that are not measured by test scores but contribute to overall well-being. This may represent a significant, previously undocumented, return to school quality.”

So, parents, on average, could detect important aspects of school quality that technocrats guided by test scores would get wrong.  Technocrats would conclude that the schools that parents prefer do nothing to improve student outcomes because test scores don’t rise or even go down when students get into the school their parents want.  But parents are smarter than the technocrats.  They prefer schools that improve long-term outcomes for their children.  Specifically parents seem to be able to choose schools that are more effective in developing the “character” of their children, making the students less likely to get pregnant as teens and more likely to be engaged in positive health behaviors later.  For boys this may not make a big difference in the labor market (although it does not harm those outcomes), but for girls these health improvements seem to drive higher educational attainment, employment, and earnings.

This study is consistent with a long line of research that finds a disconnect between short-term test score outcomes and long-term life outcomes, as described in a recent meta-analysis by my colleagues, Mike McShane, Pat Wolf, and Collin Hitt.  It’s amazing to me how champions of the technocratic approach continue to have faith that they have access to scientific tools to identify school quality that less well-informed parents lack despite the growing body of scientific evidence that demonstrates the very real defects of the technocratic approach.  Despite their daily hymns of praise to science, the technocrats don’t seem very scientific at all.

 


Craig Harris Throws His Other Shoe at the Charter Telescreen

July 12, 2018

(Guest Post by Matthew Ladner)

GOLDSTEIN! Er CHARTER SCHOOLS!

In our last two-minute hate, the Arizona Republic’s Craig Harris fretted about the founders of Basis buying a condo in NYC. In this exciting episode, Harris throws his shoe at charter school facility procurement.

So let me start by saying I’m not sure how other states handle charter school facility procurement laws, and I am curious about it. It is however worth noting that Arizona provides the best access to charter schools by zip code, per this Hamilton Project map from Brookings:

Hmm, maybe other states should be doing what Arizona is doing. Also worth noting is the fact that Arizona charters are not only more proximate than other states, they are also crushing the ball academically (statewide averages in blue, statewide charter averages where available in red):

Arizona just might be doing something right in this space, so my elephant says “Careful about ‘fixing’ something that is not broken.”

Here is the missing context from this article- construction firms make profit from both the construction of charter and district schools. In recent years districts in the state of Arizona have been spending at a half-billion per year annual clip on facilities despite having relatively flat enrollment growth in aggregate.

Let’s put it on the table from the outset that this number could never be zero (air-conditioners die, roofs leak etc.) Moreover, some of this spending involves the districts who are big winners in open enrollment making seats to meet demand. I can’t make any complaint about this portion of the building spree-I like it.

Having said that, $500,000,000 per year is a lot of money (enough to pay every teacher in the state $10,000 per year more) and there are hundreds of thousands of empty seats in Arizona districts, which were badly overbuilt during the boom. The Arizona School Facilities Board lists 1.4m square feet of vacant district space (approximately 35% of the total) and strangely enough almost none of this space is suitable for a charter school (in the estimation of district officials).

The word on the street here in PHX is that a handful of the big construction firms find that $500,000,000 profitable enough to invest in bond and override campaigns. So…we continue building space, including in districts with declining enrollment like Scottsdale. Maybe a reporter should look into that…

Now back to the current Harris piece. It is lacking in context, giving no information about the relative profit margins in charter versus district construction. There are no non-profits building schools in Arizona to my knowledge. Moreover, the profits in this case only come about because of the demand for the school model. Without demand, the CMO in question would lack the funds to buy the buildings-ergo no profit. Contrast this with a district system with 1.4 million empty seats but continuing to build more of them despite flat aggregate demand and at an enormous annual cost despite huge spare capacity and the costs for vacant buildings drawing money out of the classroom.

So all in all, where does charter school construction profits rank in a list of school facility issues in your opinion?

 

 

 

 

 


Religious Left Baptizes the Blob

July 5, 2018

classroom-2787754_640

(Guest post by Greg Forster)

In my latest for OCPA, I look at how Oklahoma’s religious left is baptizing the blob, including support for the teacher strike.

It’s not my position that religious leaders should have nothing to say about education policy:

It’s true that America’s great experiment in religious freedom implies our public policy can be based on shared moral commitments even if we disagree about the ultimate cosmic basis of those commitments. But as George Washington rightly pointed out in his farewell address, we can’t talk only about the morals of public policy and ignore the religious foundations of the morality upon which we draw. For if the foundations are neglected, the building collapses.

But if religious leaders are going to speak about education policy, they should make a serious theological argument and not just parrot the political talking points of secular special-interest groups. Otherwise they end up captive to political manipulators. This is exactly what happened to the religious right:

As a matter of fact, I’ve spent almost 10 years speaking out against the ideological captivity of the religious right. I appreciate that the fight for the sanctity of human life and other issues has accomplished some good. But the larger effect of the religious right movement was to push churches to become voter registration offices of the Republican Party. As it became clear what was going on, this did incalculable damage to the religious credibility of the churches involved. We are still living in the disastrous aftermath, as huge portions of our culture have disconnected themselves from faith entirely.

So I’m only playing fair when I say that I see the same dangerous sellout in the efforts of Oklahoma’s religious left to baptize the blob. The pronouncements of Oklahoma’s religious left on education don’t bring any theological light to the public policy questions. They’re not saying anything the secular left isn’t saying. They’re just pasting Bible verses on self-interested interest group politics. Organizing events and statements to support a secular special interest’s demand for money, parroting its secular talking points, doesn’t become a spiritual discipline because you do it with a clerical collar on—quite the reverse.

There are, in fact, serious theological arguments to be made on education policy. I’ve participated in some of them, including my response to theological arguments from the religious left as well as theological arguments from the religious right. So I welcome – though I often disagree on the merits – real theological arguments from the religious left and right. What’s alarming is when religious leaders make themselves tools of secular selfishness in the name of, yet to the detriment of, better schools for kids.


The Michigan Charter Lion Sleeps Happily Tonight with a Belly Full of Unjust Criticisms

July 3, 2018

(Guest Post by Matthew Ladner)

The world has a funny way of not behaving according to expectations. Michigan charters came under withering fire last year despite the fact that basically every bit of formal research available found that they produce better learning gains. Never you mind that whole “outcomes” business, Michigan charters were “Wild West” in nature and thus not to be trusted. Louisiana charters have been touted as a national model- properly gardened and/or quarterbacked etc. Some but certainly not all fans of Louisiana charter policies were also critics of Michigan charter policies.

The chart below constitutes the longest period that NAEP has data for LA charters on all four tests. Instead of the customary state flags, we’ll use the NFL logos of the Detroit Lions and New Orleans Saints:

Back in 2011 Louisiana charters tied Michigan charters on one of the four tests and had higher scores on the other three. The 2017 scores for MI charters on 4m, 8m, 4r and 8r were: 232, 272, 218 and 259 respectively. The 2017 scores for LA charters on the same exams for Louisiana: 214, 264, 197 and 254. Whether based upon scores or over time improvement, it seems odd indeed to hold that Michigan has awful policies to avoid at all cost while simultaneously holding that the rest of the country should drop what they are doing to emulate Louisiana.

The NAPCS dashboard that keeps track of such things is currently down but last I checked both LA and MI charters had tough student demographic profiles. We cannot know what role average school quality plays in these trends, so it is barely possible that statistical noise is consistently bouncing the way of Michigan charters over and over again (the 2015 exams also favor them) and consistently bounce against Louisiana charters repeatedly (the 2015 results were also trending down). Multiple formal studies of state scores in Michigan showing positive results leans heavily against such an already unlikely conclusion, as do similar negative trends in state charter scores in Louisiana.

I’m open to the fact that the world is complex. Perhaps there is some complicated reason why Michigan charters appear to be improving steadily, and some equally complex reason why Louisiana charter scores appear have declined. Just maybe Michigan charters deserved some of the criticisms they received, and perhaps Louisiana charter policies are not quite as terrible as the state and NAEP scores would seem to indicate. Occam’s razor may cut against such explanations, but no one is making an effort to offer them at all.

Using the Jonathan Haidt framework, my elephant is inclined to believe that “Wild West” is under-rated, and technocratic gardening is over-rated. My elephant believes that like the Dauphin of Shakespeare’s Henry V, opponents of relatively free-wheeling charter sectors “come over us with our wilder days, not measuring what use we made of them.” The rider of my elephant continues to bring me further reasons to believe this. He’s good at that. He’s also pretty good at finding flaws in the arguments of opponents, but Haidt has persuaded me that he is not to be fully trusted.

I could be wrong, but if so it will require the hate Michigan/love Louisiana tribe to poke holes in my theory/evidence as solid reasoning is a communal activity, not to be left to mere individuals.

 

 

 

 


Pass the Popcorn: That’s My Girl

July 2, 2018

Incredibles-2-sneak-peek-still-Helen-Bob-and-Lucius

(Guest post by Greg Forster)

This post contains mega major spoilers. I mean it. You have been warned.

Okay, let’s get the tiresome part out of the way first. Pass the Popcorn hereby presents:

EVERYTHING WRONG WITH INCREDIBLES II

  • Ridiculously patronizing fan service: A visit to Edna, with the conveyor belt and everything, is shoehorned in; Frozone gets yelled at by Honey
  • Act I totally undermines the psychological climax of the first movie: They put on their masks to fight the Underminer – they are now supers again, and proud of it! But then we needed to make another movie, so suddenly they’re not.
  • The visual look of this movie is just different enough from the original that you constantly notice it – especially Dash, did no one go back and review what he looked like in the first movie? – but not so different that your brain accepts it as different. It’s the animation version of the uncanny valley.
  • The villain’s back story is insultingly contrived: It would have been sufficient if she’d just hated supers for the reasons given in the big Screenslaver speech, which would have made her a really interesting political/ideological villain with a megalomaniacal vision of reshaping the world by force; but no, it all had to go back to a ham-handed story about a personal trauma, because we’re all babies now.
  • Mind control is always bad: It negates the only part of any story that’s really interesting: the characters’ choices and struggles.

What bugs me is not that these are huge problems, because they aren’t, but that they would have been so easy to fix if the studio had respected our intelligence just a wee bit more. I think these issues collectively made the difference between I2 being a really good movie (which it is) and a really great movie, on par with the original (which it isn’t, alas).

Whew! Now let’s get to the fun part.

16incredibles-2-1-articleLarge

I2 continues the Pixar/New Disney tradition of goring our cultural sacred cows, but doing it in just the right way so people will take it. In this case, as in some others (see: Frozen) it’s done by giving us a real encounter with the reason people believe in those sacred cows – the other side gets a full airing of its case before the movie pulls the rug out.

The Incredibles franchise, here as in the first movie, takes on two big cultural dimensions at once. Which is what would have made I2 a really great movie if they hadn’t fumbled too much of the small stuff, since taking on just one is tough enough.

The first dimension is the male ego. In the first movie this was simply (simply!) the conflict between Mr. Incredible’s longing for the glory days and the prosaic task of being a father. He must learn that parenting is heroic. “You are my greatest adventure, and I almost missed it!”

That theme is echoed in I2 by having Mr. Incredible become a full-time dad to three unruly kids with superpowers, the comedic value of which is expertly milked. And it leads, halfway or so through the movie, to the same conclusion – parents are the real heroes.

But now there is a new twist. The male ego comes up against the female ego.

cycle

Mr. Incredible is not just invested in his own professional success – his glory days. Now he’s also deeply threatened by the possibility that his wife might outshine him.

All civilizations always needed to mortify the male ego sufficiently to make men into good fathers. But in the new social order that is emerging after feminism, the male ego must be further mortified to make room for female competition in the workplace – without the compensatory satisfaction of the paternal role being valued in the same way the maternal role is valued. Whether that is sustainable is an open question, but if it is, it will only be so if men learn heroic virtue similar to the heroic female virtue Tocqueville praised as the foundation of the American regime. Tocqueville said (in substance) that the American experiment in constitutional democracy avoids degeneration into atomistic indiviudalism only because its women had not demanded the equal rights to which they were clearly entitled under the governing principles of the regime; if they ever did demand those rights, he warned, the regime could not deny them, but the result would be the collapse of the traditions by which the family rather than the atomized indiviudal is the basic unit of society. That, in its way, is one of the lessons of I2 – only men of heroic moral virtue can sustain the new social order feminism has catalyzed.

You see what I mean about goring sacred cows? But we’re not done yet.

The female ego comes in for a subtle but no less sharp skewering in I2. From the moment they meet, Evelyn begins stroking Helen’s female ego – her sense of resentment and exclusion in a man’s world – in order to get under her defenses and take advantage of her. And Helen falls for it hook, line and sinker. The two of them spend half the movie just stroking each other’s female egos, right up to the point where Evelyn plunges in the knife. Like Iago worming his way into Othello’s trust by flattering his male ego and then twisting that very ego to his own purposes, Evelyn has used Helen’s feminist pride to destroy her.

The other big social topic is of course the role of superheroes.

INCREDIBLES 2

In the first movie, Syndrome spoke for the envy and resentment of all those who hate heroes – and they have been a prominent and influential voice throughout the modern period of history. So pervasive is this resentment that Helen herself parrots it without really thinking – “Everyone is special, Dash.” The modern period could almost be defined as the period during which it became plausible to say that it is evil to admire heroes.

I2 tries to pick that thread back up. In the Screenslaver speech, the critique of superheroes is even broadened into a critique of the Big Media and Internet culture for which superhero franchises are a sort of proxy. “You don’t talk, you watch talk shows; you don’t play games, you watch game shows.”

The makers of I2 have seen that under the surface of the standard-issue snobbery about mass media we constantly hear is an aristocratic (or worse) contempt for democracy and egalitarianism. Evelyn’s attack on bourgeois society strikes the note of every totalitarian ideology: You’re all SHALLOW and WEAK and LAZY. That’s why mass media is nothing but a tool of social control – you’re all so easily controlled because you’re pathetic and worthless. WAKE UP, SHEEPLE!

This is all botched, however, because it does not remain at the center of the villain’s character and motivation. The contrast with Syndrome is instructive. Syndrome has a personal trauma to provide resentment and a motive for megalomania, yes. But Syndrome’s revenge consists of remaking the world according to a new political vision, a vision whose principles he understands and articulates as a coherent ideology. This is precisely what makes him interesting. “When everyone is super, no one will be!” If Evelyn’s big “monologuing” scene with Helen in the frozen chamber had been a further elaboration of that political vision and not the recitation of a mind-numbingly boring series of convolutions designed to give her a stronger personal trauma, she could have ranked with the best of the Bond villains.

incredibles2_1

But in spite of this misfire, I2 does remember to pull the rug out from under Evelyn’s ideology. And the character who does it is Violet.

Let’s face it, there’s a great deal to be said for the critique of bourgeois society as shallow and morally undeveloped. But the bourgeois society is a little like Winston Churchill’s democracy – it’s the worst kind of society, except for all the others that have been tried from time to time.

The self-appointed superiors who look down upon the shallow complacency of bourgeois society never actually rise to moral heights. Time after time, in arrogance and contempt, they sink to moral depths.

Who is it that learns real humility? The supers, who – precisely because they have so much power – must learn to use it rightly or face catastrophe.

Violet spends the whole movie wanting to be a superhero. Even when she says she doesn’t, she only says it because she really does – and the sacrifices are so painful. And then, in the climactic moment, she chooses to stay out of the fray in order to protect her baby brother rather than seek glory and adventure.

And her father says, “that’s my girl.”