Jay P. Greene's Blog

The Arnold Foundation’s Straight Talk On Evidence web site provides a very useful summary of a recently published large RCT on a state-funded pre-K program in Tennessee.  Consistent with a previous, nationally representative RCT of Head Start, this study found that students given access to government-funded pre-school by lottery initially score higher than those who lose the lottery on standardized test scores but then fare worse later.

In the TN study, treatment students score higher at the end of pre-K.  But, as the Arnold summary puts it:

At the end of third grade, the study found statistically-significant adverse effects on student math and science achievement. In math, the VPK group scored 0.12 standard deviations lower than the control group, which equates to roughly 13 percent less growth in math achievement than would be expected in the third grade year.[ii] In science, the VPK group scored 0.09 standard deviations lower than the control group, which equates to roughly 23 percent less growth in science achievement than would be expected in the third grade year.[iii]

In an effort to explain the negative longer-term result, the authors suggest that special education may be to blame.  Students admitted to the government-funded pre-K program were more likely to be labeled as needing special education services and that designation may have lowered academic expectations.  But this explanation is inconsistent with Hanushek, Kain, and Rivkin’s finding that special education tends to improve test score results.  Straight Talk at least considers the possibility that children being with family or in a non-government-funded pre-school may just be academically superior.

The hard reality is that the process of human development is complex and highly varied, so we just don’t know the optimal arrangements for all children.  Andy Smarick has an excellent piece along these lines in the Weekly Standard, suggesting that education policy experts suffer from a Hayekian information problem.  And this was also the subtext of my post last week on how parents are smarter than Technocrats.  Even when Technocrats are armed with the best science, they generally do not have enough information to centrally plan the lives of others.  This doesn’t mean that we never regulate anything.  It just means that if we do regulate we should do so with great caution and large dollops of humility because the experts are typically missing a lot of important information that the individuals they are regulating are more likely to posses.

But caution and humility are no fun, so the Arnold Foundation’s Straight Talk chooses instead to double-down on Technocracy by suggesting that the disappointing results of pre-school as shown in RCTs of both Head Start and the TN program be remedied by identifying which subset of pre-schools seem to be more effective and regulating programs toward imitating those schools:

The above findings and observations, we believe, underscore the need to reform programs such as VPK and Head Start by incorporating (i) rigorous evaluations aimed at identifying the subset of local approaches that are effective, and (ii) once such approaches are identified, strong incentives or requirements for other local program sites to adopt and faithfully implement them on a larger scale.

Keep in mind that the TN program already has regulations in place meant to ensure quality, including requiring at least 5.5 hours of instructional time per day, a cap of 20 students per classroom, a licensed teacher in each classroom, and the requirement that schools choose among a state approved set of curricula.  Also keep in mind that short-term test scores, which are the most common tool by which regulators monitor quality, showed positive results.

If these regulatory practices are insufficient to avoid harming students over the medium term, why would Straight Talk believe that doubling down on the Technocratic approach would make things better?  It would be nice if they at least considered the possibility that we are suffering from a Hayekian information problem and may be unable to devise optimal arrangements for education.

The technocratic brand of ed reform that is currently dominant is based on the premise that policy elites, guided by science, need to ensure school quality.  Parents should have choices, but they should only choose among quality options.  Mostly using test scores, technocrats believe they can identify quality schools and quality-promoting educational practices, which should over-ride parental preferences about which schools and practices offer a quality education.

A new study by Diether W. Beuermann and C. Kirabo Jackson suggests that parents may be better at detecting which schools promote long-term positive outcomes for their children than technocrats guided by short-term test scores.  They examine the school system in the Barbados in which parents seek admission for their children into schools they prefer, but those schools use test-score cut-offs to determine which students gain admissions.  The cut-offs create a discontinuity that allows for a rigorous causal identification of whether students who barely gain admission to a desired school have different outcomes than those with barely lower lower test scores who are denied admission.

They find that test score gains are no greater for students who were admitted to the schools their parents preferred than those not admitted.  For boys there are some signs that the effect on test score gains may actually be negative.  But when they look at longer-term outcomes, including educational attainment, employment, and earnings, they find significant benefits for students who were admitted to the schools the parents preferred.  These positive effects were driven mostly by gains for girls.  When they explore mechanisms for why these gains occurred, they find a significant reduction in teen motherhood for girls admitted to preferred schools, which contributed to their educational attainment and later employment and earnings.  They also found that both boys and girls experienced significant long-term health benefits as measured by a healthy BMI, regular exercise, and dental check-ups if they gained admission to the schools their parents preferred.  The researchers conclude: “This suggests that preferred schools may promote productive habits and attitudes that are not measured by test scores but contribute to overall well-being. This may represent a significant, previously undocumented, return to school quality.”

So, parents, on average, could detect important aspects of school quality that technocrats guided by test scores would get wrong.  Technocrats would conclude that the schools that parents prefer do nothing to improve student outcomes because test scores don’t rise or even go down when students get into the school their parents want.  But parents are smarter than the technocrats.  They prefer schools that improve long-term outcomes for their children.  Specifically parents seem to be able to choose schools that are more effective in developing the “character” of their children, making the students less likely to get pregnant as teens and more likely to be engaged in positive health behaviors later.  For boys this may not make a big difference in the labor market (although it does not harm those outcomes), but for girls these health improvements seem to drive higher educational attainment, employment, and earnings.

This study is consistent with a long line of research that finds a disconnect between short-term test score outcomes and long-term life outcomes, as described in a recent meta-analysis by my colleagues, Mike McShane, Pat Wolf, and Collin Hitt.  It’s amazing to me how champions of the technocratic approach continue to have faith that they have access to scientific tools to identify school quality that less well-informed parents lack despite the growing body of scientific evidence that demonstrates the very real defects of the technocratic approach.  Despite their daily hymns of praise to science, the technocrats don’t seem very scientific at all.

(Guest post by Greg Forster)

In my latest for OCPA, I look at how Oklahoma’s religious left is baptizing the blob, including support for the teacher strike.

It’s not my position that religious leaders should have nothing to say about education policy:

It’s true that America’s great experiment in religious freedom implies our public policy can be based on shared moral commitments even if we disagree about the ultimate cosmic basis of those commitments. But as George Washington rightly pointed out in his farewell address, we can’t talk only about the morals of public policy and ignore the religious foundations of the morality upon which we draw. For if the foundations are neglected, the building collapses.

But if religious leaders are going to speak about education policy, they should make a serious theological argument and not just parrot the political talking points of secular special-interest groups. Otherwise they end up captive to political manipulators. This is exactly what happened to the religious right:

As a matter of fact, I’ve spent almost 10 years speaking out against the ideological captivity of the religious right. I appreciate that the fight for the sanctity of human life and other issues has accomplished some good. But the larger effect of the religious right movement was to push churches to become voter registration offices of the Republican Party. As it became clear what was going on, this did incalculable damage to the religious credibility of the churches involved. We are still living in the disastrous aftermath, as huge portions of our culture have disconnected themselves from faith entirely.

So I’m only playing fair when I say that I see the same dangerous sellout in the efforts of Oklahoma’s religious left to baptize the blob. The pronouncements of Oklahoma’s religious left on education don’t bring any theological light to the public policy questions. They’re not saying anything the secular left isn’t saying. They’re just pasting Bible verses on self-interested interest group politics. Organizing events and statements to support a secular special interest’s demand for money, parroting its secular talking points, doesn’t become a spiritual discipline because you do it with a clerical collar on—quite the reverse.

There are, in fact, serious theological arguments to be made on education policy. I’ve participated in some of them, including my response to theological arguments from the religious left as well as theological arguments from the religious right. So I welcome – though I often disagree on the merits – real theological arguments from the religious left and right. What’s alarming is when religious leaders make themselves tools of secular selfishness in the name of, yet to the detriment of, better schools for kids.

(Guest Post by Jason Bedrick)

One of the many issues surfaced in Max Eden’s recent exposé on a New York City high school was how city officials’ insistence on keeping suspension rates down created incentives for lower-level bureaucrats to hide problems rather than address them. In an earlier report, he noted that under the DeBlasio’s administration’s new discipline policies, the NYC School Survey showed that “teachers report less order and discipline, and students report less mutual respect among their peers, as well as more violence, drug and alcohol use, and gang activity.” Despite this, DeBlasio declared that the city’s district schools experienced “the safest [year] on record.” What accounts for the disparity? The answer appears to be juked statistics.

In the wake of Eden’s exposé, many questioned how widespread this problem is. That’s a question researchers should set out promptly to address, but evidence from New Hampshire suggests that New York is far from an anomaly:

As told by schools’ self-reported statistics, the story of bullying in New Hampshire’s public schools is one of great progress. Since the signing of a landmark anti-bullying law, the number of incidents recorded by schools has dropped by more than half, from 5,561 in the 2010-2011 school year to 2,233 in 2016-2017, according to Department of Education data.

But advocates and state officials say those numbers belie the reality for Skylar and other students. More than a fifth of Granite State high-schoolers, for example, reported in a 2017 survey that they were bullied on school property during the previous year.

The Youth Risk Behavior Survey is administered annually to students across the country. Since 2009, the rate at which New Hampshire high school students say they have been bullied has stayed the same – between 21 and 22 percent – even as schools report more than 50 percent reductions in claims of bullying.

Yet again, it appears that school officials are working harder to hide incidents of bullying than address them:

The rate at which schools investigate students’ claims and find actual incidents of bullying has also dropped dramatically at the high school level. In 2010-2011, high schools confirmed bullying in 58 percent of reported incidents. Seven years later, it has dropped to 29 percent.

Some schools have put a lot of effort into stopping bullying, advocates say, but they believe the discrepancies in the data are evidence that some schools are exploiting weaknesses in the state’s law to under-report and underinvestigate claims of bullying.

The new spotlight on juked bullying statistics comes in response to two cases of student suicides over bullying in a state that has a smaller population than many cities. In both cases, parents argued that the schools didn’t do enough to protect their children from bullying.

Education officials in New Hampshire should work swiftly to correct perverse incentives and produce more accurate accounts of the level of bullying in the district school system, and schools should step up their efforts to combat bullying. In the meantime, bullied students should get access to educational choice options to provide an escape hatch from their tormentors.