Yesterday marked the second anniversary of the tragically early passing of Andrew J. Coulson, the brilliant and (in the words of his beloved wife, Kay) “happy, effusive, relentlessly upbeat” education reformer, policy analyst, and director of the Cato Institute’s Center for Educational Freedom.
IMHO, the best tribute we can pay to Andrew is to reflect on his ideas. Although he didn’t live to see it, PBS ran his magnum opus documentary, School, Inc., about how and why our education system lacks the progress, innovation, and efficiency gains seen in nearly every other industry. Last year, the Friedmanesque three-part series won the Anthem Film Festival’s award for Excellence in Filmmaking – Documentary Feature, and now Free to Choose Media is making the documentary available to view for free online.
Andrew’s voice is greatly missed in today’s debates over education policy, but as Neal McCluskey wrote, “Thankfully, his ideas remain, and they will always illuminate the pathway forward.”
NOTE: This post has been updated to clarify that it is Free to Choose Media that is making School, Inc. available to watch free online.
The 2017 NAEP is due to be released in a few weeks, so I thought it would be a good time to review a brief history of where we’ve been. The above table lists all of the available Math cohort gains by jurisdiction for the entire period all states have been giving NAEP. These cohort gains are calculated by subtracting the 4th grade scores of a cohort of students from their 8th grade scores. NAEP math and reading tests were specifically scaled and timed in such a way to allow for such comparisons.
Now…just a minute…stop staring at your state’s results and pay attention…oh okay fine go stare at your state’s results and then come back.
Right, now that you are done with that, allow me to draw your attention to the AVERAGE row at the bottom. This is a simple average between states, and it appears to be in slow but steady decline. Notice for instance Maryland’s transformation from a reform super-hero to a state that appeared to forget to teach mathematics to kids in 6th grade. Notice that the top gains from the 2009-2013 and 2011-2015 periods (Arizona) would have not been the top gainers in the golden age of 2003-2007. Arizona winds up coming top in recent years because they remained consistently pretty high while other states declined.
It should be noted that factors other than the quality of instruction could be at play here. For instance, inclusion rates for students with disabilities and ELL learners may have varied over time, creating the appearance of a decline. To test this, the below table runs the same math cohort gains but this time only for general education students:
Overall the story does not change a great deal- we still see a declining trend, and Maryland forgot to teach math to both general ed students about as much as everyone else. I will also note that Arizona owes its status as the math gains champ for 2009-13 and 2011-15 to gains among special education and/or ELL students, which as someone who worked on choice programs for special needs students in Arizona for a decade and a half, warms my heart:
My guess is that reformers picked the low-hanging fruit of education reform in the early aughts. The introduction of standards and testing in the early days seems to have produced a bump in achievement. Over time however this effect may be fading. Political Science 101 teaches that organized interests defeat diffuse interests 99 times out of a hundred, so the ability of states to employ a cat o’ nine tails and whip schools into improvement has limits. Dozens of decisions taken daily in the musty basements of State Departments of Education and obscure measures voted on by State Boards of Education can slowly but surely defang and/or subvert state accountability systems.
If there are two things that the organized employee interests of adults working in schools are expert at it is passive resistance and bureaucratic infighting. In my book, much of the reform crowd chose to fight their opponents on ground they did not choose wisely, and upon which they have little chance to prevail. Things fall apart, the center cannot hold.
Mike Petrilli recently and correctly imo noted that the 2017 NAEP would be a pretty definitive test on the efficacy of the Obama year projects- promoting Common Core and teacher evaluation, student discipline reform. Top down directives have a funny way of not working out, even backfiring. Let’s see what happens next.
Opponents of school choice frequently claim that the sky will fall if a school choice program is enacted. For example, the School Superintendents Association (AASA) claims that choice programs would “‘starve’ public education of critical funding” because so many students would leave their assigned district school in favor of alternatives.
Leaving aside how such concerns treats kids as mere funding units, choice opponents making this claim fail to take into account why families want other alternatives. Indeed, as the Cato Institute’s David Boaz has pointed out, such arguments reveal “the contempt that the [education] establishment has for its own product.”
Perhaps it is a new modicum of self-awareness that prompted a spokesperson of the AASA to abruptly (if unpersuasively) reverse course in a recent interview with EdWeek:
“Conservative think tanks are trying to solve problems that families and communities aren’t asking them to solve through school choice,” said Sasha Pudelski, the advocacy director at AASA, the School Superintendents Association. “When you talk to stakeholders, you don’t hear, ‘Please provide our families with more educational options;’ they want their own schools to be better. They are not looking for an alternative. It’s a solution without a problem.”
So which is it? Either no one is looking for alternative education options, in which case educational choice programs pose no threat because no one will use them anyway, or a significant number of families are looking for alternatives, in which case the district school establishment needs to explain why they shouldn’t have any (or, at least, why those options should only be open to children whose parents can afford them).
While we’re on the subject of the Arizonan wild west, here’s my latest for OCPA’s Perspective on some very non-wild-west Oklahoma school overregulation:
The more prominent strategy, the one that got the most attention and funding, was toward greater centralized control. If schools are given more inputs and they fail to use them to produce better outcomes, then the schools are clearly working to enrich themselves. They can’t be trusted to carry the ball for fixing education.
Who could be trusted? Why, the reformers, of course.
Some interesting information from a state think tank:
The 1889 Institute’s database of public school regulations is the cumulative legacy of these earlier forces and the dramatic increase of regulations in the last generation. It runs to 610 entries. Schools are required to track every individual student’s progress in financial literacy education and every individual teacher’s professional development “points,” spend at least a certain minimum amount on their libraries, and meet test score targets or be subject to sanctions. They must also master obscure laws governing everything from inter-district transfers to the nutritional value of diet soda…
Very few of the regulations in the 1889 Institute’s database deal with issues that really need to be handled at the district level, never mind the state. I honestly think that the nutritional value of diet soda might not even need to be managed by schools at all. But if it does, why not let the principal hire lunchroom staff who are up to the job?
If you want to let me know what you think, the comment section below is not overregulated!
Yours truly joins Marty West for the Ed Next Podcast on charter schools in the Wild West. My favorite bit is our discussion of Marty’s study using 2012 data showing meh results for Arizona charters. I’m confident that this result was accurate. In fact the 2013 NAEP also showed lower 8th grade scores in both Math and Reading for AZ charters than AZ districts. What gives?
In 2012 the Philadelphia Eagles went 4-12, but earlier this week they won the Superbowl. This doesn’t shock us much in sports as we understand that player turnover in sport is high and one year’s team can be very different from the previous year’s squad. Likewise in a charter sector as dynamic as Arizona’s you literally have had hundreds of charters open and close since 2012. Also during this period you had a large number of young schools mature (the survivors the crucible of their formative stage). The Great Recession was a period of rapid charter school growth in Arizona as many high quality CMOs seized the opportunity to obtain bargain priced properties. That also however meant lots of young schools going through their shakedown cruise periods.
If the Eagles had been playing a large number of rookies in 2012, their record would look bad, but come back a few years later and those former rookies have grown into grizzled vets. The guys who couldn’t cut it are off the squad. So too in 2015 Arizona charter students crushed the ball on all six NAEP exams, and their AZMerit scores have improved subsequently improved in both 2016 and 2017 along with the scores of districts.
I wish I had seen the above Brookings map before writing the Ed Next piece, as it kind of sums up the four corner charter phenomenon in a nice visual. The higher percentage of kids that have access the charters, the more likely it is that your suburban districts will participate in open enrollment. We you have access to suburban (and/or private schools) your willingness as a parent to put up with a dysfunctional charter school moves closer to zero and they get very quick on the draw. Result: Yippie kai yay!
In a fascinating newstudy by Robert W. Fairlie and Peter Riley Bahr, they examine the effects of an experiment in which some community college students received free computers and others did not by lottery. Comparing these randomly assigned treatment and control groups, the researchers found that computer skills rose among students who were given computers, but those skills did not translate into higher college enrollment, employment, or earnings for the treatment group.
These results are particularly important because many politicians have focused on improving computer skills as the key to improving educational outcomes. In Arkansas, the main education policy initiative championed by the governor is a law that requires all public schools to offer computer science classes. Texas has adopted a similar policy. Leaving aside all of the obvious practical concerns, like whether schools have or can develop staff qualified to teach computer science, this new research raises questions about the aim of these policies. How important is increasing computer skills for the vast majority of students? No one doubts that most workers have to use computers, but many students may already possess the skills they need and it seems doubtful that raising average computer skills would lead to significant changes in employment outcomes — and that’s assuming we can improve computer skills in a meaningful way.
The new study is also incredibly useful in that it reminds us of how important it is to rely on randomized experiments rather than studies that use matching or controls for observables. They conclude:
Importantly, our null effect estimates from the random experiment differ substantially from those found from an analysis of CPS data, raising concerns about the potential for selection bias in non-experimental estimates of returns. Estimates from regressions with detailed controls, nearest-neighbor models, and propensity score models all indicate large, positive, and statistically significant relationships between computer ownership and earnings and employment, in sharp contrast to the null effects of our experiment. It may be that non-experimental estimates overstate the labor market returns to computer skills.
It is simply false that matching studies are just as good or almost as good as randomized experiments. Sometimes you get the same result in a matching and RCT study, but that could simply be because selection did not bias the result in that case or you were just lucky. Sometimes a coin flip will also give you the same result. Theoretically, we know that selection bias is a serious concern, which means that we can never have strong confidence in research designs that assume selection issues don’t exist.
Matt Ladner and Max Eden have observed that the National Alliance for Public Charter Schools (NAPCS) as well as the National Association of Charter School Authorizers (NACSA) give their highest ratings to states that barely have any charter schools. As Max noted of the NAPCS rating: “Half the states in the top ten have fewer than 10 charter schools.” He added: “three of your top 10 states have produced 20 schools in 20 years.”
It’s nice that Robin Lake is noticing that charter growth has stalled and that Portfolio Management is beginning to block more charter schools in Denver, but somehow neither she nor Paul Hill, nor any of the other charter intelligentsia, seem to be able to connect the dots and trace the problem to the types of burdensome regulatory schemes that they all prefer. These folks back burdensome regulations with the claim that they help promote charter quality and forestall political problems, even though there is no evidence that they accomplish either of these goals. But the evidence is becoming quite clear that what these burdensome regulatory schemes accomplish is the creation of very few charter schools and even fewer led by minority members of local communities.
It’s as if the charter intelligentsia thinks that the best charter school is the one that isn’t there. This reminds me of the Girlfriend in Canada trope. The best girlfriend (charter school) is the one who isn’t there. She’s really great and I wish you could meet her, but she lives far away. Avenue Q captured this trope nicely, so I’ve modified the lyrics a bit:
I wish you could go to my charter school
My charter school that’s placed in Canada
The scores couldn’t be higher, I swear I’m not a liar
My charter school that’s placed in Canada
Its leadership is Ivy League, too bad they’re all lily-white
Competitors are not in sight, no one can put up a fight
They test kids every single day, just to make sure that everything’s okay
It’s a pity the school’s so far away in Canada
Last year we reported the highest grad rate
Too bad it’s because we chose to inflate
It’s so sad, that doesn’t mean we’re not great
Our discipline’s progressive and our politics transgressive
I wish you could go to my charter school
But you can’t because it’s in Canada
I know I’m persistent, even if it’s non-existent
That’s why I favor district schools… er, I mean charter schools
Darn, I really want district schools to create more charter schools
It’s the best charter school, my wonderful charter school
Yes, I have a charter school that’s placed in Canada
And I can’t wait to give kids more choices
The National Alliance for Public Charter Schools released their new rankings of charter school laws. I gave NACSA grief for their rankings, so I need to be an equal opportunity offender in the interests of consistency I need to do the same for NAPCS. Both sets of rankings rate state charter laws against a model bill, and as a consequence, both wind up ranking relatively weak charter laws in their top 10. There is an obvious problem here- we may not know what a good charter law is, and this may be especially the case given the diversity of needs and cultures across states. Best then to judge charter laws by their outcomes, and here is where you find difficult to justify patterns in the model bill rankings.
Both NACSA and NAPCS have ranked Indiana as the top ranked charter law. There’s a problem with this however as Indiana’s charter school law has not produced many “charter schools.” The Brookings Hamilton project provided this handy map that shows the percentage of students per state that have a charter school operating in their zip code:
A reasonable way to judge the quality of a charter law in my book would be some combination of the following factors-how many charter school seats has your law produced, what does the average achievement of charter school students look like, and the degree of competitive pressure is being generated on the district system. Indiana appears meh on all three fronts, except with regards to academics where they have been too small to show up in the NAEP samples thus far.
So if a judging against a model bill puts Indiana on top, is it possible that there is something wrong with the model bill itself? Yesterday during a lively conversation on social media Max Eden made the following observations regarding states landing in the top 10 of the NAPCS rankings:
Mississippi is 5 years old. There are two charter schools there.
Maine is 8 years old. There are nine schools there.
Washington state’s law is six years old. They have 8 schools. And what, then, are the rational grounds for believing that these newer laws are superior?
So, three of your top 10 states have produced 20 schools in 20 years. There is no rational case for why this is a better approach. If state policymakers follow your model law, charter growth will be strangled.
You need to be very innocent with lots of solid alibis if Max Eden is prosecuting a case against you, and well, things are looking pretty grim for the model bill approach. Referencing the handy Brookings map above we see only 4.3% of Maine students have a charter school operating in their zip code. Less than one percent in Washington, and then you Mississippi…zero point zero.
This map is from 2015 so things are somewhat better now, but not much and the point remains. Kentucky passed a charter law, and it also landed in the NAPCS top 10. I read a late (not final) version of the bill, and based upon that reading I would say we should expect the sort of pace that Eden noted in Maine, Mississippi and Washington. The draft I read greatly empowered school districts to interfere with the operations of charter schools to an extent that struck me as likely to dissuade rational actors from coming in from out of state, and all but the most gung-ho in-state operators.
If charter school laws that fail to produce charter schools are topping your rankings, it is time to reexamine the bill. There looks to be something or somethings deeply wrong with it. In meantime, congratulations to Indiana for winning what amounts to a technocratic beauty pageant.
OMG! I knew that one year default closure provision would impress the judges!
If you’d like to see the states whose students are actually benefiting from their state’s charter laws, the Brookings map is a good place to start. The first duty of a charter school law is to provide charter school seats- any charter law is somewhere on the boutique curiosity to abject failure spectrum without them. Summing up it is best to always remember:
Posted by Jason Bedrick 
Jay P. Greene's Blog 