(Guest Post by Matthew Ladner)
The double-plus good propaganda generators are at it again here in the Cactus Patch- in the latest edition of the two-minute hate, the Grand Canyon Institute claims each private choice student is costing the state general fund more money than the public school students. In the instant-classic turn of phrase by Robert Pondiscio however, it is important not to make vast decisions based on half-vast data.
You can watch a public affairs show interview with the author here. In the author states that the analysis supports abolishing private choice programs. Just how half-vast is the Grand Canyon Institute analysis? Well they’ve basically done a two variable regression analysis with private school enrollment as the dependent variable, and they’ve hung their entire estimate on one of the two regression coefficients. The first problem, which sinks the entire analysis by itself, is that their dependent variable for Arizona private school attendance is unreliable.
GCI uses a federal source for private school attendance by state. That source however does not have anything like a 100% response rate (see the appendix) and lists 320 private schools in Arizona. If one however goes to the Arizona Department of Revenue report on private school tax credits, you find tax credit scholarship tax credits going to 362 different private schools. The list of 362 may or may not be exhaustive. So CGI has a flawed dependent variable on their hands- the reality is that we don’t know the number of students attending Arizona private schools- we don’t even know the number of schools.
Second off a regression analysis with two independent variables (% of students attending charters and private choice dollars, respectively) would likely a grade of incomplete in an Introduction to Statistics class-if the instructor wanted to encourage the student and was feeling generous. The universe of other factors which influence private school attendance. A quick google search for instance revealed an analysis testing 14 separate variables– many of which were statistically significant. Arizona’s economy crashed during the period examined in the CGI analysis for instance, and the above analysis finds family income explains variance in private school attendance. Considering that the entire GCI analysis rests on a regression coefficient, failure to control for more than two variable would be deeply suspicious even if they had a valid dependent variable, which they don’t.
In short, the Grand Canyon Institute’s elephant went in search of a way to claim that Arizona’s private school choice programs were bad, and sure enough they (predictably) found it. The analysis reveals much more about CGI than private choice programs in the end.
Jay P. Greene's Blog 
So what is the cost per student?
When you pontificate without ever answering the question, I have to ask why you dodged the question.
Give us some facts. Math. Data.
I didn’t dodge the question. There have been a series of fiscal note estimates on expansions of private choice programs and the last one estimated a savings to the state general fund, many others have been neutral, but the reality is that no one can can be sure.
Matt,
It’s easy to do a shoot down of a study by saying variables are imperfect–every study faces limitations. Our motivating question is how many students would be in private school in Arizona if there were no regular education private school taxpayer based subsidy programs.
Face validity is whether the ultimate result makes sense. We have about 46,000 students in private school in Arizona–Private schools would still exist if there were no public subsidy programs for them–just as they did before the subsidy programs existed. The study estimates just over 13,000 of them are in private school due to the subsidy programs. So otherwise, enrollment in private schools in Arizona would be about 33,000. That to me seem pretty plausible in terms of face validity. This is especially true when you realize that we had between 44 and 45 thousand students in private school in Arizona when the programs began–and today it’s around 46,000–that’s not controlling for any variables–but it doesn’t look like a program that is accomplishing its objective.
You suggest we can’t have confidence in the federal survey of private school enrollment. I disagree. Enrollment in private schools is done as best as we can expect it. The federal data to be unreliable would need to be systematically irregular over the full 20+ years evaluated in the biannual survey…There are nonresponse rates, but, they make statistical adjustments for non-response rates-check for yourself. And to the degree any school are missed in this adjustment process–they are very small schools accounting for a very small fraction of enrollment. It’s the best data source we have.
You are correct the analysis has only two dependent variables and we are constrained by the limit that a biannual private school census imposes. But the variables do control for more than two things and most importantly they statistically match up with a huge portion of the variation in the dependent variable. Normally a poor regression fails because it has a low F value/ low Rsquared, leaving lots “unexplained.” So while you could add more variables–they likely wouldn’t aid much –and given the limit in the private school enrollment, it’s better to make sure the variables control for other factors. This model has high statistical relationships for its dependent variables with statistical significance at the 99% level.
The Dependent Variable is AZ/US Private School Enrollment Ratio. This is meant to create a measure of how Arizona is varying from the US with structural factors controlled for as best as can be reasonably done. So the West has fewer private schools, when recessions hit private school enrollment drops, etc. The ratio controls for that. Of course, there can be some variation occasionally as the recession was harder in Arizona than in the US–but that’s only modestly impacting one data point in a biannual survey.
The independent variables are the portion of charter school enrollment in Arizona–which negatively impacts private schools as expected. (so effectively the paper is suggesting we have two school choice programs competing with each other–and that the charter school program has been far more successful than the private school one)
The other is the real tax dollars per private school student for regular education subsidy programs–and the coefficient here drives the results.
I just wanted to make sure any readers also got a full rebuttal that was easy to access. Given the large and growing outlay in these private school subsidy programs–GCI believes they deserve to be scrutinized as to whether or not they are cost-effective.
The full study is here: http://grandcanyoninstitute.org/wp-content/uploads/2018/09/GCI_Policy_Private_School_Program_Costs_2018_Sept_5_2018.pdf