Jay P. Greene's Blog

Today AEI released a systematic review by Collin Hitt, Mike McShane, and Pat Wolf on the relationship between changes in test scores and changes in later educational attainment in rigorous studies of school choice programs.  I’ve been writing and talking about this for some time now, inspired to a large degree by informal conversations with the authors of this new report.  Now they have made the point more systematically.

They examined every study of school choice programs with both test score and attainment effects, consisting of “39 unique impact estimates across studies of more than 20 programs.”  They examine whether the direction and significance of the estimated effects of those programs on test scores are consistent with the direction and significance on attainment.  They are not.

They find: “Across the studies we examine, there is no significant or meaningful association between school choice impacts on math scores and high school graduation or college attendance. Nor are ELA impacts meaningfully associated with high school graduation rates. Under some tests, the relationship between ELA impacts and college attendance are significant—but the relationship is weak in magnitude, and the sample of studies is far narrower for college attainment than for high school graduation.”

Keep in mind that the policy relevant question is not whether individual changes in test scores are correlated with individual changes in attainment.  There is some research that has found this relationship (see for example Chetty, et al), but a surprising number of studies find no or only a weak relationship between individual gains on these near-term and later-term measures of success.  But none of them directly address the policy relevant question of whether aggregate test score changes at the school or program level are predictive of aggregate changes in attainment.

If we are going to judge schools or programs as good or bad based on changes in test scores, then those aggregate measures (not individual results) should be predictive of later success.  The fact that they are not, at least when judging school choice programs and schools, suggests that there is something fundamentally wrong with how we have approached public regulation (wrongly called “accountability”) of those programs.  You can’t regulate the quality of schools and programs if you can’t predict their quality.

(Guest Post by Matthew Ladner)

A 7-year veteran teacher teaching in Arizona’s Paradise Valley Unified School District has created a social media firestorm by posting her pay stub on social media- $35,621. Before writing anything else, I like anyone else want to pay dedicated teachers much better than this. Teacher compensation decisions however are made at the district/campus level, and an examination of PVUSD finances just makes things look worse. Much worse.

The Arizona Auditor General reports on district finances. Paradise Valley received total revenue of $10,143 per pupil in 2017. This figure is above the state-wide average.  A class of 30 students produces over $300,000 in revenue. Even with benefit costs included, teacher compensation will not even sniff 20% of the revenue generated by this class of 30. Where did the rest of the money go?

This chart from the Heritage Foundation tracks national data, and provides a big part of the answer- American schools have seen a vast increase in the hiring of non-teachers. This is not to say that any school can ever do without non-teachers, but at one point we had 2 teachers for every non-teacher in American schools. These days it looks more like 1 to 1. Despite a substantial increase in inflation adjusted spending per pupil since the 2-1 days, the large increase in non-teaching staff places a limit on teacher salaries.

Ms. Milich works for a district that receives over $10,000 per pupil in revenue. That district has both an elected school board and a chapter of the Arizona Education Association that is very active in the politics of the district. A teacher with seven years under her belt getting paid $35,000 does not suggest that either the district or the association has been placing a priority on teacher compensation.

Impossible? Just take another look at the chart. American schools have spent decades placing a priority on increasing school district employment, with a much stronger focus on non-teachers. What we refer to as “teacher unions” are actually “school district employee unions” and just as a quick mental exercise close your eyes and ask yourself: do you think the chart above would look that way if the NEA and AFT opposed these trends? What if they had been supporting them for decades?

Ok open your eyes now. Get it? Good.

This teacher has every right to feel frustrated. I hope the district decides to compensate her in a fashion commensurate with her contribution to her school. I strongly suspect that contribution is far greater than a low double digit percentage of the revenue her class generates.

Yes state spending and taxing decisions also play into this- but we should never forget that Arizona is not a wealthy state, has an unusually small working age population, and decides on school funding levels through direct and indirect democracy. In 2012 statewide voters declined to raise taxes for schools in Prop. 204 by a very wide margin. A few years ago voters (very narrowly) chose to increase school funding by increasing the payout rate from state land trust.

The lopsided loss in Prop. 204 and the narrow margin on the Prop. 123 vote both relate to a deep skepticism on the part of the public that increased funding will reach teachers like this one. This skeptical attitude is entirely justified. For instance, between fiscal year 2016 and 2017 per pupil revenue increased in PVUSD by $664 per pupil (from $9,497 to $10,143) and this teacher received a raise of $131.25 and this was after completing professional development.

Arizona voters decide school funding democratically, and they will have the opportunity to increase funding again in the future. Under the current district setup, the bucket looks to have multiple large leaks if the aim is to increase teacher compensation. Some of my tribe on the center right here in Arizona tends to think that these spontaneous teacher protest movements are secretly the work of the Arizona Education Association. I don’t believe this is the case. Although much of this frustration is misdirected at choice programs, things like this social media paystub are indicators of genuinely felt grievances.

It would be a huge mistake however to continue avoiding questions about district priorities.

My student, Ian Kingsbury, will be presenting a paper next week at the annual meeting of the Association for Education Finance and Policy examining factors that help explain which applications to operate charter schools are more likely to be approved.  He is still in the early stages of this project and I’m sure will benefit from feedback on how to improve the work, but he has already analyzed nearly 400 applications to operate charter schools in 7 states.  His basic findings, which seem unlikely to change as he gets feedback, should surprise no one but should shock everyone interested in charter schools — the more burdensome the regulatory environment for approving charters, the less likely charters led by minority applicants are to be approved by authorizers.

Using the score that the National Association of Charter School Authorizers (NACSA) gives to each state’s charter policies as a proxy for regulation, Ian finds that for each 1 point increase in a state’s NACSA score (on a scale from 0 to 33), African-American and Hispanic-led charter applications are 1.7 percentage points less likely to be approved.  Given that one state included in the study has had a NACSA score as low as 9 and other states, like Indiana and Nevada, have a 33, the variation in regulatory environments Ian observed is associated with about a 41 percentage point difference in the probability that minority-led charter applications would be approved.

Of course, the charter regulations favored by NACSA, the National Alliance of Public Charter Schools, and most of the charter establishment are meant to promote quality.  Unfortunately, there is no evidence that these regulations are in fact associated with higher quality charter schools. But they clearly make it harder for charter applications to get approved, especially for minority-led charter applicants. Even when Ian controls for whether the minority-led charter applicants attended more selective colleges or were affiliated with CMOs/EMOs, which might be proxies for the quality of those applications, minority-led applicants were still 1.7 percentage points less likely to get approved for each 1 point increase in NACSA score.  In other words, if the purpose of regulations is quality control, those regulations still tend to keep out minority-led charter schools even after adjusting for reasonable proxies of the quality of those proposed charter schools.

This pattern of regulations in the name of quality posing a disproportionate barrier to minorities without actually being related to quality should sound familiar to anyone who has paid attention to the issue of occupational licensure.  A variety of groups, from the Obama White House to the Institute for Justice, have noted that raising requirements to enter many occupations has been an important barrier to opportunity, especially for disadvantaged groups.  Requiring people to spend 2,100 hours and about $22,000 to obtain a cosmetology license before they can braid hair has little to do with quality but is an important obstacle to opportunity.  The same can be said of the type of regulations favored by NACSA and the charter establishment — they have little to do with quality but seem to be large obstacles to minority operated charter schools.

Keeping out minority-led charter schools has potentially serious educational and political implications.  There is some evidence that minority students fare better when educators are of their same race/ethnicity.  Minority-led charter schools may be more likely to provide this type of educational benefit for minority students.  In addition, excluding minority leaders of charter schools severely damages the political prospects for charter schools by making minority community leaders significantly less invested in the growth and success of the charter sector.

If any of you will be at the Association for Education Finance and Policy conference next week, I would encourage you to stop by Ian’s panel on Thursday (March 15) at 10:15.  While Ian’s project is not finished, the evidence is becoming clear enough that NACSA and the rest of the charter establishment need to explain why the policies they favor have such a negative effect on minority-led charter schools.  And if they are going to defend that negative effect by claiming that the policies they favor promote quality, they need to provide evidence to support that claim.  The way it looks now, the types of regulations favored by NACSA and others seem to just keep minorities out without producing any increase in quality.

Update — I’ve added a link to the paper, which is available here.

(Guest post by Greg Forster)

RealClearInvestigations carries a well reported and persuasive article by Paul Sperry documenting how Broward County has for years been among the most enthusiastic adopters of initiatives to stop the “school to prison pipeline” by . . . not arresting students when they commit crimes.

A surer way to make sure a kid stays in the pipeline to prison would be harder to imagine.

“If you commit crimes, there will be no consequences.” Yes, that’s exactly the lesson we want to teach at-risk youth. That’s going to work out just great for them in the long run.

Not to mention everyone else. The failure of police to take any action against the kid who went on to become the mass shooter, even as he committed crime after crime after crime and was never arrested – so he had a clean record when the time came – suddenly makes much more sense in this light.

What we need to look into now is the extent to which the whole system of having police officers assigned to schools has been subverted to serve purposes other than school safety. The more comes to light about the Broward Coward, Scot Peterson, the more it looks like being a school officer is something other than a legit police assignment. Is it merely a cushy job for lousy cops who want to draw a salary for no real work, which would be bad enough? Or is something more sinister going on – is there an organized system for placing officers in schools who know that they’re not supposed to make arrests?

Why did Peterson refuse to share information with a Department of Children and Families investigator about an incident involving the kid who went on to become the shooter? Is it normal for a cop to withhold information from investigators?

It is when they have something to hide.

HT David French