On Taking Ignorance Seriously

August 16, 2018


(Guest post by Greg Forster)

I have a post at OCPA on taking ignorance seriously:

Some states managed to have success in some cases—Massachusetts’ standards reforms and Florida’s mix of school choice, exit exams, and incentives to raise test scores across demographic groups are notable examples. But the overall story was failure. We just couldn’t take these good ideas to scale…

Why was school choice the only winner? Because it takes our ignorance seriously. It doesn’t try to generalize the content of education across millions of unique children.

Throwback to my review of the evidence on Pre-K included at no extra charge!


Masters in Someone Else’s Home is No Way to Go Through Life

August 15, 2018

(Guest Post by Matthew Ladner)

In the film Gandhi a crucial scene involves a meeting with British colonial overlords. At one point a British official plays what seems to be an ultimate trump card- in essence that His Majesty has millions of Muslim subjects in India, and without British administration a civil war would break out. Gandhi’s response: yes this is a problem, but it is our problem, not yours.

This scene came to mind when I read this Houston Chronicle article detailing the Houston Independent School District narrowly avoiding a state takeover of the district. Money quote from the article:

HISD and civic leaders are expected to gather for a celebration Wednesday at Worthing High School, which has suffered dramatic academic declines in recent years amid constant leadership turnover, persistent concerns about safety and a drain of students to school choice options.

One could spend a long time just unpacking that sentence, but I for one am happy that students at this school had the opportunity to seek a different setting, making the “drain of students” frame simply mind-blowing. There is also something deeply perverse about “celebrating” at Worthing given the state of affairs there. We get to keep things the same- hoorah!?

But in the end, kind of, yes in a sad but important way.

The Texas legislature should feel no small degree of wariness about a statute they passed that might find the Texas Education Agency taking over districts and/or closing schools. I’ve seen K-12 focus groups address the closure issue and people came across as uniformly and passionately against the entire notion of government led closures based on test scores.

On district takeovers, if not for the manifest flaws of school district democracy, we could all be doing something else with our time. School district elections are low-turnout/information affairs that sadly lend themselves readily to regulatory capture by organized employee/contractor interests. The word on the street for instance is that the AFT swept the last round of HISD school board elections.

There may be ways to improve the quality of school district democracy that could be implemented from the state level. I don’t however believe that suspending democracy, even a deeply flawed one, is one of those better ideas. No not even if it is “temporary” nor even if it is “for their own good.” Winston Churchill noted “Many forms of Government have been tried, and will be tried in this world of sin and woe. No one pretends that democracy is perfect or all-wise. Indeed it has been said that democracy is the worst form of Government except for all those other forms that have been tried from time to time.”

Churchill, an old-school imperialist, probably did not have many notions in common with Gandhi but when the Venn diagrams between them overlap it is probably best to pay attention. In the end it gets back to build new, don’t reform old. A district takeover is like a nuclear artillery piece- which used to be a thing– overpowered and a danger to the person those firing it.


Political Bias in Education Policy Research

August 13, 2018

Image result for political bias in academia

Education policy research is not really a scientific enterprise.  If it were, the field would be equally open to accepting research of equal rigor regardless of the findings.  That is simply not the case.  Research with preferred findings is more easily published in leading journals and embraced by scholars than research supporting less favored results.

There are countless examples of this, but here is one to illustrate the point…

The Journal of Policy Analysis and Management, a top journal in our field, has just published an analysis of vouchers in Indiana based on a matching research design.  Despite the fact that matching is normally intended to produce treatment and comparison groups that are nearly identical on observed characteristics, in this study the treatment group differed significantly from the control group in their pre-treatment measure of math performance.  Specifically, the treatment group has significantly higher scores on math tests.  And the one negative effect observed by the study was on math test scores, which was roughly comparable in magnitude to the amount by which the treatment group was higher on math scores pre-treatment.  So, basically the treatment group reverted to having about the same math scores as the control group once treatment began.  This negative effect, which was really the equalizing of the matched groups, was detected the first time students enrolled in a private school and did not grow in magnitude as students persisted in private school.  One might think that if private schools really harmed math scores, that harm might compound over time, but that did not occur.

These results certainly deserve publication and ought to inform the school choice policy debate despite the obvious limitations of the matching design that failed to make the groups comparable on the one outcome measure for which a negative outcome was observed.  While worthy of publication and discussion, it is questionable whether this article deserves publication in one of the field’s top journals and even more doubtful that it should be given as much credence as some folks in the field seem willing to give it.

Corey DeAngelis and Pat Wolf have a similar school choice study based on a matching research design with similar imperfections.  It examines whether students enrolled in the Milwaukee voucher program were more likely to be accused or convicted of a crime in later years than comparable students who had attended Milwaukee’s public schools.  Students in the treatment group were matched to public school students on a number of observable characteristics, including the neighborhood in which they lived.  Despite that matching effort,  the treatment and control groups were significantly different, with the treatment group having higher reading scores and more likely to be female.  Unlike the JPAM study, neither of these variables were the same as the outcome for which they observed effects.  Controlling for observable student and parental characteristics, students who had enrolled in Milwaukee’s voucher program were significantly less likely to be accused of a crime in later years.

The defects of Corey and Pat’s study are similar to those of the JPAM study.  It also uses a matching research design, and as I have said many times before, I don’t think we should have much confidence in matching designs to produce causal inferences.  And like the other study, Corey and Pat’s matching fails to produce treatment and control groups that are similar on all observed characteristics.  But unlike the other study, Corey and Pat’s research is not being published in JPAM.  In fact, JPAM desk rejected Corey and Pat’s study, deeming it unworthy even of being sent out for review.  A number of other journals did the same and they are now struggling to get it published in any journal.  I’m convinced that if only they had found that vouchers increased criminal behavior, their piece would already be in print in a respected journal.  But because they found a positive result for vouchers, the bar is higher and editors and reviewers can rightly note the defects in the study to justify rejection.

All research has limitations that might be invoked to support rejection or overlooked to support publication.  The double-standard used when judging voucher studies with favorable or unfavorable findings is a function of political bias and is an indication that our field is much less scientific than we would like to imagine.

It’s a shame that education policy researchers are largely uninterested in this problem of political bias.  Despite considerable energy devoted to promoting many dimensions of diversity within our field, there is virtually no effort to promote ideological diversity.  My department has a few researchers who would describe themselves as conservatives (while we also have had two faculty members who describe themselves as socialists), but I suspect most departments don’t have any self-described conservatives while others have no more than one or two.

It is interesting to note that despite having a department with six endowed chair holders, half of whom have Harvard doctorates, and all of whom have impressive research records, none of us have ever been asked to serve on the editorial boards of any journals (excluding the Journal of School Choice that my colleague, Bob Maranto, edits).  We’ve tried to play a part in governing our profession, but because we are branded (sometimes incorrectly) as conservatives we have been shunned.  The composition of editorial boards shapes who reviews submissions, which shapes what is published in those journals, which shapes what people in the field imagine the research consensus to be on various issues.

There are consequences to this political bias in our field.  First, the scientific quality of research is harmed by an increasing groupthink that fails to critically examine the key assumptions, methods, and implications of much of the work being produced.  Second, research in the field has diminished credibility and policy influence because others increasingly look at the field as more ideological and less scientific.  Some of the leading people in our field regularly take to Twitter to deride policymakers and the public for failing to heed what they believe research has to say. But why should policymakers obey “science” when it is being produced by an increasingly insular group of researchers who may confuse their political agenda for science? Third, frustrated conservatives are likely to give up trying to be accepted by the dominant professional associations and journals and instead build their own parallel institutions.  The Bar Association drove out conservatives who built the Federalist Society, which now seems to be thriving more than the “mainstream” organization at exercising policy influence.

I don’t expect this piece to alter this state of affairs.  Leading scholars in our field seem quite adept at defending their prior convictions, sometimes in remarkably unscholarly ways on social media, rather than critically examining their own beliefs and behaviors.  As far as I’m concerned they can rail away, but they will be left with the kind of nasty, unscientific, and irrelevant field they seem determined to build.

AZMerit 2018 release and Johnny Rotten says RELAX (80s Extended Dance Mix)

August 8, 2018

(Guest Post by Matthew Ladner)

Another hope feeds another dream 

Another truth installed by the machine.

A secret wish, a marrying of lies

Today comes true what common sense denies.


Some predictions of doom age better than others. The “Arizona Public Schools are heading for destruction” mantra falls squarely in the post-Pistols Johnny Rotten odd rap/disco experiment quality side of things imo. Since 2016 the not-so-secret wish of some in Arizona has been to make the 2018 elections all about K-12 and its terrible failure. And well…

…the election year ain’t over yet so expect more double-plus good destructo speak. Sigh.

Meanwhile, back in mere reality the reports of Arizona public education’s demise appear to have been greatly exaggerated, given that the academic results continue to rise. Recently the Arizona Department of Education has released the 2018 AZMerit exam results. Both district and charter scores continued to improve.

The Arizona Republic greeted the news glumly:

Many Arizona schools kicked off the new school year this week with some bleak news: The majority of their students last year failed the state AzMERIT test. 

State testing data shows the same trend lines in students’ performance that schools have seen every year since the AzMERIT test debuted four years ago. There was incremental growth in overall reading and math scores, but the majority of kids failed.

This is a bit much. The passing bar on AZMerit is set high. Only half the kids in Massachusetts score proficient on NAEP, but you don’t see them wringing their hands over it, and with good reason. Massachusetts has Japan like scores in a NAEP/PISA equating study:

Arizona charter schools have Massachusetts-like NAEP scores, so let’s call it in Japan’s neighborhood. Plus the districts are getting better as well. Must….resist…urge…to…include…more….bad…80s…music…resistance….futile!

That 2011 study had Arizona as a whole between Latvia and Greece and alas in the real world you don’t leap over Latvia one day and put South Korea to shame the next. Put me in the “keep the incremental progress coming and let’s count our blessings” camp si vous plait.

My advice is to ignore the propaganda and relax, the election year silly season won’t last forever, it will only seem that way.


Now Would be the Time to Add That Position

August 2, 2018

(Guest Post by Matthew Ladner)

As longtime JPBG blog readers may or may not be aware, one of my hobbies is college football, specifically Longhorn football. One of my favorite posters on the Orangebloods site goes by the handle Orbea and focuses on financial information in his posts, and this recent post in particular struck me:

This is a post about sentiment. This is a post of how frequently it is the case that when investment management companies close funds, fire managers, and change strategies it marks the end of a trend and the start of a new trend in the other direction. 

But first some prior examples. These are done from memory. Because of that I likely have some of the details a little off, which in no way is a detraction as the gist of the story is basically correct. 

Tom Jackson
Tom Jackson was a deep value equity manager with an impeccable track record. In the late 80’s or early 90’s he was hired by Prudential Mutual Funds to manage their flagship equity mutual fund, the Prudential Equity Fund. He went on a tear for the first half of the 1990’s, won a couple of awards, and was on the cover of investment magazines (that in and of itself is a sign that a trend is over). Then in 1996 Technology stocks took off and Jackson started to lag badly. By 1998/99 Jackson was holding a third of the fund in cash because there were no deep value stocks to buy. In early 2000 Prudential fired Jackson and revamped the strategy of the fund to be a large cap growth fund. 

Of course, Prudential fired Jackson at the wrong time. Large cap growth tanked over the next three years while deep value went up.

Michael Metz
Michael Metz was the Chief Investment Officer for Oppenheimer (the brokerage firm, not the mutual fund company). In the first half of 1998 Metz recommended selling stocks because valuations were ridiculous and buying the 10 year Treasury. The official investment position of the firm was not to own equities but to own the 10 year. Unfortunately, stocks went on a tear to the upside for the next two years and bonds went down in value. In the early 2000 Oppenheimer sacked Metz and recommended selling bonds and buying stocks.

Of course, Metz was ultimately correct. Over the 5 year period from 1998 to 2003 the 10 year Treasury handily outperformed the S&P 500

Third Avenue Focused Credit Fund
Third Avenue management company was found in 1986 by Martin Whitman. Whitman was often called a revolutionary deep value manager with a speciality in small cap stocks. Often times deep value involved buying bankrupt bonds or distressed bonds. In order to expand their mutual fund offerings, sometime after the last recession, Third Avenue decided to open a junk bond fund. Since the bias of the company was in deep value and distressed securities the fund was chock full of the worst bonds imaginable. The fund did well for a few years, and then credit spread started to widen. Credit spread really widened on low tier debt (which was basically all the fund held). From the summer of 2014 through December 2015 the fund cratered. In December 2015 Third Avenue, in a surprise move, decided to close the fund and liquidate the holdings.

Of course, the decision by Third Avenue to close the fund came within 45 days of the bottom in junk bonds. 

A tip of the hat to @mm1966 who pointed out to me that the name of the fund was the Third Avenue Focused Credit Fund and not the Third Avenue High Yield Fund. 

Andy Hall
Hall was referred to as the God of the energy and oil markets. He rose to fame when he earned a $100 million bonus in 2008 as an energy trader at Citi. He went on to start his own hedge fund. For a number of years Hall racked up huge gains in the energy markets. Then the 2014 oil price collapse happened. Hall looked to be recovering from the collapse until the 2017 decline in oil prices happened and his fund lost 30% of its value in the first half of 2017. In July 2017 Hall shut down his primary hedge fund stating that $50 a barrel oil was the new normal.

Of course, the fund was shut down within weeks of the 2017 bottom and oil is now at $70.

Vanguard Changes The Strategy Of Its Precious Metals Fund
Vanguard just announced that it is changing the investment direction of its Precious Metal Fund. The new name of the fund is Global Capital Cycles Fund (whatever that is). The rationale for this change was to (and I quote) “to broaden the fund’s mandate and diversify the portfolio”.

Over the last seven years the fund has lost 60% of its value, which was right in line with the GDX (the gold miners ETF). 

This a recurring story with investment manage firms. By the time an investment firm throws in the towel on an asset class, then the bottom in that asset class is not far away. 

So, if you don’t have a small position in precious metals and miners – now would be the time to add that position.

Why post this on an education policy blog? I’m not sure but I’ll just leave this here in case anyone wants to consider the possibility that they sold their “Let a Thousand Flowers Bloom” stock prematurely:

The Way of the Future: Self-Reliance (with equity?)

August 1, 2018

(Guest Post by Matthew Ladner)

In a previous post I basically made the case that a majority of American states failed to show much academic progress between 2003 and 2017 and that the nation’s fiscal problems are closing in fast. While state constitutions guarantee K-12 funding, making it about as close to a permanent institution as you get in American life, the looming crunch in state budgets between K-12 and Medicaid/Pension shortfalls does not look to be pretty. For instance, consider these projections from the Texas Comptroller’s Office:

I won’t bother to dig up the likely increased size of the Texas K-12 population that they will be attempting to educate with a smaller percentage of the budget, but you get the point: austerity is a safe assumption, and there aren’t any other states that would sit as the world’s third largest oil producer if ranked as an independent country. Things may be generally bigger in Texas, but the problems may be even bigger elsewhere, making austerity a safe bet. Austerity isn’t necessarily bad for student outcomes but the same can’t be said for politics:

Nevertheless, state funding austerity seems very likely. So where does this leave the future? Jay used the phrase “hybrid homeschooling” years ago to describe practices by upper-income families to supplement the educate of their children. These parents enroll their children in schools, but then pay out of pocket for a variety of tutors, Mathnasium, Kumon, club sports and various other educational/cultural enrichment activities. The United States certainly does not have the after-school-school culture of East Asia, but the well to do have been using a multi-provider approach to education for a long time, and in fact it is ubiquitous to the point of seeming unremarkable.

Then let’s revisit the 2015 article from Wired documenting the rise of home-schooling in Silicon Valley Money quote:

“There is a way of thinking within the tech and startup community where you look at the world and go, ‘Is the way we do things now really the best way to do it?’” de Pedro says. “If you look at schools with this mentality, really the only possible conclusion is ‘Heck, I could do this better myself out of my garage!’”

Check out Cottageclass.com. Cottage class will help you find anything from a summer camp to a nanny-share to a private tutor or a micro-school. It’s well worth some of your time to click on various options. I’ve heard it described as Airbnb for education, and it is fascinating. Crucially, it includes user reviews to help parents navigate what is a growing universe of options. If all of this sounds reminiscent of ESAs, it is only because it is. For instance, Adam Peshek wrote in a Fordham Wonkathon:

An ESA doesn’t require committee hearings to decide where funds should be sent. It doesn’t require a school board meeting to vote on whether or not to cut the music program. Feel like your child isn’t exposed to enough music education? Pay for it. Little Stevie is falling behind in math? Get a tutor. If you’re in high school, use an ESA to pay for Advanced Placement courses to get a leg up in college, or use it to earn an industry certification so that you can graduate high school with an employable skill. An education savings account allows personalized learning to move from catch-word to reality.

That’s why we cannot implement these programs with the mindset of standardization. The ESA program needs to be seen as an innovative way to bring new options to K–12 parents—one designed to allow parents to maximize each child’s unique learning abilities by offering the educational path that suits them best.

Increasing numbers of families seem to be taking the a la carte approach, and like the Silicon Valley feature, they are doing it with their own money.

North Carolina for instance is an interesting place to keep an eye on. North Carolina has adopted both large charter and private choice programs, but thus far they can’t keep up. North Carolina keeps statistics on homeschooling, and it is interesting that other choice options can’t keep up with it. In 2007 there were 71,566 home-schooling students. In 2017-18, the figure was 135,749. This increase came despite the state taking a cap off of charter schools in 2010, and a doubling of charter enrollment. North Carolina lawmakers also created a statewide voucher program and (very recently) an ESA program for special needs students in the intervening years, but homeschooling is more prevalent than either charter or private school attendance in the state. Charters and private choice programs are growing, but they alone are not scratching the itch. District enrollment meanwhile has been flat for years despite rapid population growth.

The most interesting space these days lies within the space between a home-school co-op and a micro-school private school. The rumblings about the demand for micro-schools grow increasingly audible. Justin Cohen quoted Andy Calkins of the Next Generation Learning Challenges:

“It wouldn’t surprise me if, 5 to 10 years from now, everyone looks at this and thinks, ‘That grew a whole lot faster than I thought it could,’” he said. “There is a slice of the market that is not being served by public education. They’re saying, ‘The public schools don’t work, [and] I can’t get into the charter schools.’”

Predictions are tough, especially when they are about the future. The last decade and a half has broadly disappointed in improving public school outcomes, and the next decade and a half looks tougher rather than easier. A broad trend towards self-reliance and multiple service providers rather than one stop shopping in K-12 is already underway. Parents are not waiting on policy innovation, but policy innovation will be necessary to address equity concerns. Micro-schools often cost less than traditional private schools, and offer more options, but they do cost money.

States like North Carolina, with a statewide voucher program for low-income students and two different private choice programs to help students with disabilities, are ahead of the curve on the equity front. Florida likewise has a tax-credit program for low-income children and two programs for children with disabilities. States with these policy mixes create the possibility of economically integrated private schools options. States without them, not so much.

Let’s see what happens next.



Education Reform 2003 to 2017: Modest Success/Epic Failure so What’s Next?

July 23, 2018

(Guest Post by Matthew Ladner)

Having had some time to reflect upon the 2017 NAEP, let’s take stock of things. In this we should keep in mind our broad ignorance between policy changes and state NAEP trends- and the same goes for average school quality. NAEP gives regular 4th and 8th grade scores in math and reading, and all 50 states have participated since 2003. 8th grade scores are more likely to reflect school quality than 4th grade scores in my opinion, as the students have more years of schooling. I’m not sure what to make of positive 4th grade score trends that do not result in higher 8th grade scores for instance. So this in essence a window into what we have to show for American K-12 reform 2003 to 2017 in 8th grade math and reading by state:

So what to make of the above chart? The below chart eliminates a lot of clutter by only including the states with statistically significant gains in both math and reading 2003-2017:


So 19 out of 50 states demonstrate statistically significant gains in both 8th grade math and reading. Notice also the absence in the second chart of mega-states Illinois, New York and Texas (although it is good to see California and Florida making it in). Texas has as many K-12 students as the 20 smallest states combined and annually adds approximately a Wyoming public school system sized number of new students. Florida has half as many students as Texas and California is still larger than Texas.

Since we don’t know the relationship between policy and academic trends, we are limited in the conclusions we can draw with confidence. Having said that, policies that have been broadly applied across all 50 states apparently suffer from severe limitations in their ability to move the needle academically. All 50 states for instance have adopted state academic standards and accountability exams, but most states have failed to move the needle on 8th grade scores. Even if we were feeling incredibly generous and made the wild assumption that none of the second chart gains would have happened in the absence of testing, a failure rate of 62% after 14 years is a far cry from leaving no child behind.

Mike Petrilli and Peter Cunningham recently offered up “where do we go from here” think pieces. I think Mike has some interesting ideas, but Peter’s call for a vast increase in spending is broadly unrealistic imo given the nation’s trillions of dollars in unfunded pension and entitlement liabilities, 10k Baby Boomers per day reaching the age of 65, etc. In normal times, Mike’s incremental adjustments might make a lot of sense, but we don’t live in either normal times, or in times that are going to allow some Great Society on Steroids increase in K-12 spending.

A much more difficult scenario may loom whereby the district system continues to resist reform, reformers continue to push reforms the public does not care for, and severe funding needs for increased health care spending leads to a broad reduction in per pupil spending.  State constitutions guarantee K-12 funding, but whether or not they will be creating schools that the vast majority of parents will continue to entrust with their children, I don’t feel as confident about. There are hopeful signs in the NAEP from state charter sectors:

…but the rate of charter growth has slowed substantially nationwide. Of particular disappointment are the last several state charter laws to pass that produce very few charter schools. Even states with relatively fast growing sectors have large wait lists. There are alas limits to what we can realistically hope for from a charter movement that has to a large degree lost its way by prioritizing cartel behavior over the interests of children on wait lists imo.

The private choice movement enjoyed a strong run earlier in the decade, but has since ran into political headwinds. Many private choice programs exist, but most remain modest in scale. The case for private choice remains as strong as ever, and the need will continue to grow, but the looming state funding crisis is coming fast. In four years, half of the Baby Boom generation will have reached the age of 65, and by 2030 all of them will be there. They have called dibs in advance on all plausible funding increases and a whole lot more.

So what is next? An increasingly likely scenario in my mind is that state district systems retain their flaws but loses a significant part of their funding and that choice systems continue to fail to meet existing much less expanded demand. In such a scenario an increasing percentage of families may decide to fend for themselves. Call them home-schools, home-school co-ops or micro-schools, my spidey-sense tells me that we should expect to see a great many more of them in the years ahead. I’ll write more about this in a follow-up post.