2016: The Year in Edu-Review

December 29, 2016

(Guest Post by Matthew Ladner)

So what did we learn in 2016?

Tom Loveless reported in an analysis performed this year that the adoption of Common Core had resulted in less than a point of average improvement in NAEP scores, and that we probably already got the partial point. Meanwhile states continued revising their standards and tests. Another K-12 master plan bites the dustbin of history.

Speaking of tests, one of the subtle trends that continued in 2016 was the enhancement of academic transparency by NGOs. Non-profit firms have built platforms that analyze state testing data into digestible ratings and in addition collect parent reviews. Thus even when states adopt phony “trophies for everyone” school rating systems, a private platform like Greatschools gives parents a more realistic skinny on academic performance, and user reviews to boot. Already the amount of web traffic these sites generate dwarf those of state departments of education websites, and Greatschools has competition.

Similar to Mark Perry explaining that your television didn’t cost $6,200 because people figured things out over time, perhaps a bit more diversity could help the cause of academic transparency. The best case scenario on testing likewise imo would be to give schools more flexibility regarding the standards and testing used. The private platforms have already been dealing with “trophies for everyone” and can further take on the task of digesting a more diverse testing data for parents so long as state officials make some basic efforts regarding comparability and avoid opt-out provisions. Accountability could then take the form of parents choosing schools with decent information, which is strongly preferable to the accountability-free accountability bureaucratic compliance systems practiced in most states today. States can of course choose to keep things as they have been, but quite frankly its a bit difficult to see much benefit in so doing. Arizona lawmakers struck a deal to increase testing flexibility in preference to an opt-out bill in 2016. Perhaps other states will follow in 2017.

When we look back at 2016, the most important research may prove to be a Harvard study of the Georgia Tech MOOC Master’s degree program. Cliff’s Notes: the inexpensive Masters program in Computer Science program is competing against non-consumption. In other words, in the absence of the online GT program, the participating students would simply not pursue graduate level training in the field. As the study explained:

Demand for the online option is driven by mid-career Americans. By satisfying large, previously unmet demand for mid-career training, this single program will boost annual production of American computer science master’s degrees by about eight percent. More generally, these results suggest that low cost, high quality online options may open opportunities for populations who would not otherwise pursue education.

Crunchy education traditionalists like Jay will doubtlessly harumph that we don’t yet know what the job market will make of such a degree as yet. They will alas be right, but competing against non-consumption moves the question from “is this as good as a normal GT Masters” to “hey is this worth $7,000 and my time?”

On the parental choice legislative front, 2016 does not rank among the legendary years for progress, in part because it was an election year. Speaking of the election year, the Massachusetts ballot issue on charter schools ought to serve as a wake up call.  Writing on the Presidential election in MA, New York Times reported “You could drive a full 30 miles through the leafy suburbs northwest of Boston before reaching a town where Mr. Trump hit 20 percent of the vote.” Note however that these same wealthy and progressive voters slapped down more charter schools for inner city Boston kids on the same ballot. AFDC makes a poor role model for the parental choice movement, while the example of Social Security suggests a way forward.

Finally the biggest K-12 story of 2016 doesn’t have much to do with K-12. One of the two American catch-all parties commands a dominant position at the state level where the vast majority of K-12 action lies. Since President Obama took office, the Democratic Party has seen their collection of state legislative seats shrink by almost a quarter, and experienced a net decline in governors as well. Republicans will hold “trifectas” (the Governor and legislative majorities in both chambers of the legislature) in 25 states in 2017, while Democrats hold trifectas in six states in their coastal strongholds (and Hawaii). The sea-change at the state level occurred in November of 2010, but many may have been expecting the trend to moderate in 2016, but the voters made other plans for the time being.

Even before the election of 2016, the tyrants seemed to be defeating the street in the ongoing Arab Spring of the center-left debate over education policy-NAACP, Democratic Party Platform, etc. I agree with Greg that the choice movement has cultivated ties with progressives and can ill-afford to squander them. Has however the loss of almost a thousand state legislative seats moved Democratic caucuses hopelessly to the left on K-12 issues?  Can progressives keep clear about the benefits of choice to disadvantaged communities even though Donald Trump professes an affection for it?*

There is only one way to find out- let’s get 2017 started.


*If not I will set my stopwatch to await progressive opposition to infrastructure spending and federal family leave legislation, as “I hate anything Trump likes” will be about the level of analysis utilized.




Nine Lemons Dancing

December 24, 2016


(Guest post by Greg Forster)

Via Byron York, this delicious datum from the Milwaukee Journal Sentinel regarding the Milwaukee teachers’ union:

After Act 10’s passage, the 4,500-member Milwaukee union took a hard look at the local’s work and found that 85% of its time went toward litigating disputes and misconduct cases involving 2% of its members.

But remember, the unions are all about delivering a better education to your kids, rubber rooms and lemon dances are a myth, and voluntary unionism is a threat to the republic.

Merry Christmas and Happy New Year to all!

Give Pluralism a Chance Sol

December 19, 2016

(Guest Post by Matthew Ladner)

Over at City Journal Sol Stern revives his complaint that billionaires should be busily imposing his favorite curriculum on as many people as possible rather than supporting parental choice. I’ve explained previously how choice mechanisms serve as the main vehicle for advancing this sort of curriculum in my neck of the cacti. The dream of the one true way however apparently dies hard.

I actually share Sol’s curricular preferences, but I’m happy to note that AZ’s charter school 8th graders went to the wire with Massachusetts on the three 8th grade NAEP exams while spending far less per pupil and with a far more diverse student body than MA public schools. Attendance at such schools is delightfully voluntary, with the only problem being long wait lists at high demand schools- something that billionaires could help with far more easily that wresting control of curriculum away from the Blob. Arizona charter schools are still growing, MA’s widely admired academic standards alas are no longer with us. I may be missing something here, but it looks to me like an embrace of pluralism represents a more effective strategy than yearning for benevolent technocrats who share your preferences indefinitely despite the unrelenting hostility of powerful incumbent interests.

Sol’s piece is silent on just how the bad curriculum is to be overthrown, but giving people more choice seems like a swell method to me, albeit one that needs speeding up. Sol relates a tale of the famously centralized French K-12 system adopting damaging progressive curriculum to the detriment of students. This sounds like par for the course with central planning to these ears. The fact that well-meaning people like Sol want E.D. Hirsch curriculum means little-Sol is almost always going to find himself outnumbered in the universe of people deciding these sort of things.

The fact that parents beat down the doors to get a classical education when offered through choice mechanisms however represents a viable path forward imo.

Can a Market in Education Work?

December 16, 2016


(Guest Post by Jason Bedrick)

At the risk of arousing Greg’s ire, I’d like to talk about the market for education.

In a recent interview, Professor Joshua Goodman of Harvard thoughtfully expressed skepticism that markets could improve quality in education. Whereas I’ve found other attempts to make the “education is different!” argument to be unpersuasive, Goodman makes a few solid points that choice advocates would do well to consider:

I became an economist in part because of Milton Friedman’s argument that vouchers could improve schooling through market forces. At the time, this argument struck me as both revolutionary and obviously right. Competition improves supermarkets, restaurants — why shouldn’t this model apply to schools? It seemed to me that anyone who denied this idea didn’t understand basic economics.

But the more I read, the more I realized that the empirical evidence for choice and market forces improving educational outcomes is thin at best. I found that disappointing and also puzzling, and I have spent some time thinking about why that theory doesn’t match current reality.

Regarding the evidence, it’s certainly true that none of the catastrophes that school choice critics have predicted have come true. But neither have the policies produced the transformative changes that supporters had promised. Instead, we’ve had a consistent batch of random-assignment studies finding positive but relatively small effects (setting aside the high-regulation disaster in Louisiana). The effects tend to be larger for minorities, parents are much happier, and they cost less per pupil, so I think they’re still worth pursuing even on purely consequentialist grounds, but school choice programs have not been transformative. Why not? Goodman continues:

Here’s what I think the biggest problem in thinking of schools as a classical market. Econ 101 models assume consumers observe product quality. But schools are complicated goods, and quality, particularly a school’s long-run quality, is hard to judge for many parents. It takes a lot of time to figure out whether this school and these teachers are serving my child well. Unlike restaurants or supermarkets, where quality can be judged at the moment of the purchase, school quality reveals itself later. […]

Observing school quality is quite costly, and in many settings there is no credible way to inform future students about the quality of education they are getting. The for-profit college sector is a perfect example of this. Market forces fail to discipline for-profit colleges because for an individual student there’s no repeated game here. Students enroll and only much later realize lousy labor market outcomes. In particular, that students must enroll for a while to see long-run outcomes limits the power of the market to provide discipline. The time it takes to learn that a school is low quality damages the student, and that student’s information may not be transmissible to other students in a systematic or credible way.

Goodman concludes that he does believe “schools could use a bit of market pressure to counteract inertia” because district “monopolies clearly allow many schools to coast on their current trajectories without trying new approaches or investing more effort,” but he is “much more skeptical now than I was before that market forces are some sort of panacea, as they appear to be in some industries.” In the end, he thinks there will “always be a need for public accountability, if only to lower information costs to parents and students.”

I agree with much of this analysis. As I wrote at EdNext today:

There is no panacea. There is no perfect information just as there are no perfect bureaucrats or, for that matter, perfect parents. The question before us is how to design a system with imperfect people and imperfect information that will come as close as possible to providing every child with access to a high-quality education.

Getting parents the information they need to make good decisions is indeed a challenge. As Goodman notes, it is harder to identify a quality education than it is to identify a good meal or a good car. There is a great degree of subjectivity and the payoff is usually far in the future.

The question then becomes what sort of institutions are better equipped to address this challenge. As Lindsey Burke and I argue in a new report published by the Heritage Foundation and the Texas Public Policy Foundation, the technocratic approach of holding schools accountable on metrics that are easily observable and standardized creates perverse incentives that narrow the curriculum, stifle innovation, and can drive away quality schools from participating in the choice program. Instead, what we need are a variety of different third-party reviewers and platforms for parents and students to share their personal experiences in order to provide parents with the information they need.

I won’t rehash the entire argument here. You can either read the report on the short version at EdNext. The main point I wanted to highlight here was that the market is more likely than the government to overcome the information challenge — but for that to happen, there needs to be a large enough market. As I noted:

The K-12 education sector has historically lacked high-quality sources of information about school performance, but to a large extent that is because the vast majority of students attend their assigned district school. With little to no other educational options, there has been little parental need for information to compare competing options. And without much in the way of competition, existing private schools don’t feel great pressure to be forthcoming about performance data. However, as states implement educational choice policies, the demand for information will increase and schools that refuse to share their data will be at a competitive disadvantage. We are already seeing parents to turn organizations like GreatSchools.org and Niche.com to find information about schools they are considering and we should expect to see more organizations emerge as demand increases.

School choice programs that merely fill empty seats are not enough. They won’t bring the transformative change that Milton Friedman and others predicted. Modest choice programs produce modest results. It’s time to go bold.

Trump and School Choice

December 14, 2016


(Guest post by Greg Forster)

I was grateful to be included in this Washington Post article on Trump and school choice yesterday. My post on Trump’s racism and illiberalism gets a mention, but the Post is right that another division is also important:

Free-market purists believe that parents know best, that they can choose the best schools for their children without intervention, something that could force poor-quality schools to close. On the other end of the spectrum are those who believe that intensive oversight and regulation are necessary to ensure that the schools from which parents are choosing are high-quality.

As long as Mike is taking his lumps out in the wild, wild west of Arizona, maybe he could rethink which side of this unavoidable civil war – unavoidable because opponents of parent choice have made it so – he really wants to be on.

Another point: I don’t blame the Post for describing advocates of parent choice as “free-market purists” while describing opponents of parent choice more neutrally. It is we in the parent choice camp who have chosen to make deep investments in “free market” ideological rhetoric. Everything we’re saying about markets is in fact true, but it’s a bad idea for us to make “markets” and “competition” the main points in favor of choice.

This was one of the main arguments of my recent series on “the next accountability.” As I wrote at the end of the series:

Markets and competition as drivers of efficiency and performance are important. But they do not provide the moral norms and narratives needed to inform the next accountability. The best case for universal school choice does not center on them. These should be secondary, not primary themes.

We should develop ways of articulating these principles as the basis of the next accountability:

  • The purpose of education is to help children develop the knowledge, skills and virtues they need to live a good life—achieving and appreciating the true, good and beautiful—and to live as good citizens of a community where we disagree about what is good.
  • To cultivate these, we need teachers who are wise professionals (possessing the qualities they seek to instill, and guided by an independent professional ethic) and schools that are free communities (where shared purpose, not the arbitrary dictates of distant authorities, shape a shared life).
  • Teachers and schools can educate the individual student for free pursuit of the good life as he or she sees it, and also for good citizenship and respect for others’ rights in a diverse community, because of what we share in common as human beings and as fellow Americans.
  • Teachers and schools should be held accountable to do this by parents and local communities—the more local the better—because they are in the closest moral and social connection to schools, and can therefore hold them accountable in ways that support their social fabric rather than disrupting it.

Is this too much to ask of a highly polarized education reform movement, strongly committed to moral narratives that center on either markets or test scores? I’m looking forward to finding out.

Anyone want to bet against Arizona for the 2017 NAEP?

December 13, 2016

(Guest Post by Matthew Ladner)

So Lisa Graham Keegan and I finally had the opportunity to collect on our bet with Mike Petrilli on the 2015 NAEP.  You may recall that Mike bet us before the release of the 2015 NAEP results for Reading and Math that Arizona’s NAEP scores would decline. Using our spidey-sense, LGK and I bet Mike that they would be going up, not down.  Arizona’s NAEP scores did go up. Mike was a good sport and quite appropriately paid his debt to us in copper cups (one of the state nicknames is the Copper State).

Depending upon how you examine the data Arizona is either near or else is at the actual top on gains. Measured by student cohort over time, Arizona’s 4th grade class of 2009 made more progress on Math and Reading between 4th and 8th grade scores in 2013 than any other state. Arizona’s 4th grade class of 2011 achieved the same pinnacle in their 2015 scores as 8th graders. (NAEP Math and Reading exams are both scaled and timed to allow such comparisons). The gains for Arizona charter school students dwarf those of Arizona as a whole, or any other state.

So anyhoo, the term “Wild West” is being thrown around as if it is a term of derision by some of those uncomfortable with the selection of Betsy DeVos as Secretary of Education. Here in the actual Wild West we wear the term with pride. The Arizona charter school sector has a majority minority student population, scored like a New England state on all six NAEP exams, and shows consistent results on the state PARCC exams.

Let me know when your state pulls something like that off, because I will be happy to celeNAEP with you. In the meantime, NAEP will be giving state level exams in Reading, Math and Writing in just a few weeks! Let’s see what happens next…

When Evidence and Science are Really Just Assumptions and Ideology

December 5, 2016

Doug Harris has a new post that attempts to reply to the many critics of his New York Times op-ed, including me.  In the NYT piece Harris claimed that “one well-regarded study found that Detroit’s charter schools performed at about the same dismal level as its traditional public schools.” He also claimed that the relatively light regulatory approach to Detroit’s charter schools has led to “the biggest school reform disaster in the country.”  He prefers instead the heavy-regulation approach adopted in New Orleans, which he says has produced “impressive” results.  He then suggests that failing to believe these claims is “a triumph of ideology over evidence.”

Several critics, including me, noted that the “well-regarded study” Harris cites actually finds that Detroit charter schools are producing significantly greater gains than traditional public school alternatives — gains that are only slightly smaller than those in New Orleans and greater than in another high-regulation darling, Denver.  But Harris wants to continue posturing as the person backed by science and evidence, while describing his opponents as ideologues.  In his reply to critics in Education Next he uses the word “evidence” 16 times.

So, his defense of the claim that Detroit charters perform “at about the same dismal level as its traditional public schools” must be based on evidence, right? Actually, no.  He provides four arguments to rescue his assertion that “the failure of Detroit charter schools to improve student outcomes” is true despite the fact that the CREDO evidence he cited to make that claim shows otherwise.  Each one of those four arguments is based on assumptions, not evidence.  If you believe all of his assumptions, you might believe his claim that Detroit charter schools really are a disaster. But drawing conclusions that depend completely on assumptions and are contrary to the evidence he cites is certainly not science.  It seems more like a faith-based or ideological exercise.

Let’s consider his four arguments.  First he says, “given the lack of oversight in Detroit and evidence from other cities that some charter schools cherry-pick their preferred students, these results may make Detroit’s charter schools look better than they are.”  Got that?  He has no evidence that Detroit charters are cherry-picking students at all, let alone that they are doing so at a higher rate than in New Orleans, but he nevertheless posits that “if it’s happening, then the charter effects on achievement [in Detroit] are inflated.”

Second, given Harris’ assumed concerns about cherry-picking in Detroit charter schools and the inability of the CREDO study to account for that, he examines evidence from the urban NAEP test and finds that the city of Detroit has experienced below average growth in those scores in recent years.  Using NAEP results from the entire city to draw conclusions about Detroit’s charter schools requires a host of assumptions.  He’d have to assume that test results from all schools, charter and traditional, somehow speak to the effectiveness of charter schools.  He’d have to assume that demographic and other non-school factors in Detroit do not affect the comparison of test growth in Detroit relative to other cities.

A number of education analysts have coined a term — misNAEPery —  to capture how unreasonable it is to make the assumptions required to use NAEP to compare policies across jurisdictions.  Oddly, some of those analysts who like to accuse others of misNAEPery, like Morgan Polikoff and Matt Barnum, have somehow failed to denounce Harris’ use of misNAEPery in both the NYT op-ed and in the Ed Next reply to critics.  Both have even “retweeted” Harris’ new post, so we can assume they’ve read it.

And just to anticipate concerns about my own consistency on the use of NAEP, I think comparisons using NAEP that control for observed demographics are about as convincing as the CREDO results, which also rely on comparisons controlling for some observed characteristics.  Information from NAEP or CREDO can be interesting or suggestive, which is why I say that Arizona charters “appear” to be doing very well, even if I am not convinced that any of this is causal.  At the very least, it is useful to offer a disclaimer that neither NAEP nor CREDO provides convincing causal evidence, even if confession does not assure absolution.  Harris does not offer any disclaimer and instead uses his NAEP comparisons to bolster his assumption that Detroit charters may be cherry-picking.

Third, Harris builds on the observation that Detroit city has very low NAEP test scores to assume that “the extraordinarily low standing of the city as a whole, to the degree it is caused by low performance of traditional public schools, should make it easier to improve student outcomes when trying something new.”  It is also quite plausible — perhaps more plausible — that a city with extraordinarily low test scores also has severe social and economic problems that are outside of the control of schools, which would make it harder for charters to improve outcomes.

Lastly, Harris argues that even if Detroit charters have produced gains, the gains are smaller than those in New Orleans, so we should prefer the regulatory approach used in New Orleans to the one in Detroit. But this assumes that any greater gains produced by New Orleans charter schools are caused by the regulatory approach in that city.  In fact, we have no idea whether New Orleans’ regulations helped, hurt, or had no effect on how large the gains in that city were. For all we know, the gains made by New Orleans charters are largely attributable to the importation of top-notch human capital from elite colleges and a huge increase in per pupil spending, and that these gains were made despite the hindrance of burdensome regulations.  The heavy regulations in Denver somehow failed to produce gains as large as those in Detroit.

Other than these four-assumption-dependent arguments, Harris offers very little to defend the strong claims he made in the NYT that Detroit charter schools have been a “failure” and a “disaster.”  He does cite some national “charter-friendly” organizations, like CRPE and NACSA,  as being critical of Detroit charters.  But that is an argument from authority — not evidence.

He also falsely suggests that Detroit has failed to close failing charter schools.  In fact, 30% of Detroit’s charter schools have been shuttered.  And another national charter-friendly organization, NAPCS, gives Michigan higher marks on closure than Louisiana, noting that Michigan has closed more charter schools than Louisiana, 47 to 26.

Harris also cites two voucher studies with negative results — one of which is an RCT and the other not — as proof that lower regulation approaches are less effective.  Leaving aside the distinct possibility that the negative RCT result for Louisiana was actually a function of the over-regulation of that program, it’s important to note that Harris fails to cite the entire literature of rigorous studies on the effects of private school choice programs, which overwhelmingly shows positive outcomes.

I agree with Doug that ideology, which could more kindly be described as “principles” or “values,” has an important role to play in policymaking.  I just disagree with him that the evidence clearly shows the superiority of a high-regulation approach to school choice.  I think it’s more accurate to say that the evidence is unclear on this matter, which means that we may — appropriately — need to rely more on our values, principles, and broader ideology when deciding how to proceed.

[Edited to remove tangential comment that was poorly phrased.]