The Achievement-Attainment Disconnect Strikes Again!

April 2, 2019

I’ve lost track of how many studies with rigorous causal identification find that improving test scores is not associated with improving later life outcomes, like graduating high school, attending and completing college, and getting a job and earning higher salaries.  In the area of school choice alone we now have dozens of such studies, but you can also find this achievement-attainment disconnect in rigorous studies of pre-school and other interventions.

You can add to this list the latest study from Mathematica on the long-term effects of attending a charter middle school. Mathematica examined lotteries at a sample of charter middle schools around the nation, comparing outcomes for lottery winners to those of lottery losers.  When the original results were completed almost a decade a ago, they found no overall effect on test scores.  But when they disaggregated results, they found that urban charters had a significantly positive effect while suburban charters had the opposite result.  This prompted leaders in the charter and ed reform worlds to conclude that we should focus our efforts on urban charters, since those were the ones that “worked.”

Now Mathematica has followed-up on those students to see whether they eventually attended and completed college.  The new results show that attending a charter middle school has no effect on attending or completing college, just as it had no overall effect on test scores.  But — and this is the important part for our discussion — they also concluded:

The success of an individual charter middle school in improving college outcomes was not related to its success in improving middle school achievement. The study schools that improved middle school achievement were not consistently more successful than others in boosting college enrollment and completion.

So, all of those technocratically-minded ed reformers who thought we should focus on urban charters because test scores showed they “worked” were guilty of mis-judging long-term success based on unreliable short-term measures.  The evidence shows that changing test scores is not a particularly good indicator of schools that will improve their students’ lives.

City Fund, NACSA, and others who support portfolio management, harbor-mastering, quarterbacking, or whatever marketing term they are using nowadays are once again left trying to explain exactly how they intend to distinguish the “good” charter schools from the “bad” ones better than parents can.  But take comfort, their political ineptitude matches their technocratic inclinations so that I expect they will burn through their $200 million without successfully installing and maintaining any multi-sector portfolio management systems. So children will be safe from their falsely-guided superior judgement.

Richard Henry Pratt for the Higgy

March 31, 2019

(Guest Post by Matthew Ladner)

In Arizona and elsewhere there has been a very difficult history regarding the education of Native American children, and a good portion (but not all) of the blame for this can be laid at the feet of Richard Henry Pratt. Born in 1840, Pratt had a long military career which included service in the Union Army during the Civil War and in later military action against Native Americans during the Reconstruction era. Pratt is best known however for championing the forced abduction of Native American children into distant boarding schools in order to “assimilate” them. “Kill the Indian, and Save the man” Pratt is famous for saying.

Pratt’s terrorizing of Native American families lasted for decades but his paternalistic non-sense victims also  included African-Americans, Puerto Ricans, Mexicans, Latinos, Pacific Islanders, Asian-Americans, and Mormons.

Pratt’s zeal for benevolent assimilation cultural genocide had more than a faint echo of the effort to make Catholics into “real Americans” during this same period. In the 1920s the Ku Klux Klan and their fellow travelers had a state law passed to make it illegal for a family to attend a private school, which was thankfully struck down by the United States Supreme Court. Pratt however seemed to look on the paternalistic zeitgeist with approving envy, noting “Indian schools are just as well calculated to keep the Indians intact as Indians as Catholic schools are to keep the Catholics intact. Under our principles we have established the public school system, where people of all races may become unified in every way, and loyal to the government; but we do not gather the people of one nation into schools by themselves, and the people of another nation into schools by themselves, but we invite the youth of all peoples into all schools. We shall not succeed in Americanizing the Indian unless we take him in exactly the same way.”

Let me put this through the translato-meter “Our profoundly illiberal attempts to homogenize Catholics are awesome so let’s quadruple down and force it down the unwilling throats of Native Americans at the point of a gun.”

Pratt founded the first “Indian School” in Pennsylvania and the folly spread around the country. Children were forcibly abducted from their families, transported vast distances away, beaten when they spoke their native language, forced to cut their hair etc.  The damage done by these practices was huge past the point of quantification and needless to say things did not turn out the way Pratt planned. Today Native American students have on average the lowest levels of academic achievement to be found in the NAEP. The only good thing I can find to say about Pratt is that he was a sharp critic of the reservation system and argued (correctly imo) that the system was producing a damaging dependence on the federal government. Fair enough but Pratt’s alternative vision seems to have been to place Native American children in internment camp schools against their will-even worse.

It will only be a matter of time until you next read some misguided soul attempting to wax poetic about how allowing people to choose education for themselves is somehow a threat to democracy and the common good. “Kids need to attend their zoned public school you see, after all, it’s for their own good,” will be the gist of it. When that happens, think of Richard Henry Pratt and the horrors he caused for decades. Then, in the most polite way possible, invite the fool and/or villain pushing this line of thought to go to hell, go directly to hell, to not pass “Go” and not to collect $200. If they are unfortunate enough to actually get there, they can say hello to RHP, who I am happy to nominate for a William Higginbotham Inhumanitarian Award.

It’s Time for “The Higgy”

March 31, 2019

William Higginbotham

It is time once gain to solicit nominations for the William Higinbotham Inhumanitarian Award.  Below I reproduce portions of the first announcement of “The Higgy” in 2012, so you have an understanding of the historic significance and criteria for this dishonor.


As someone who was recognized in 2006 as Time Magazine’s Man of the Year, I know a lot about the importance of awards highlighting people of significant accomplishment.  Here on JPGB we have the Al Copeland Humanitarian Award, but I’ve noticed that “The Al” only recognizes people of positive accomplishment.  As Time Magazine has understood in naming Adolf Hitler, Joseph Stalin, and Ayatullah Khomeini as Persons of the Year, accomplishments can be negative as well as positive.

(Then again, Time has also recognized some amazing individuals as Person of the Year, including Endangered Earth, The Computer, Twenty-Five and Under, and The Peacemakers, so I’m not sure we should be paying so much attention to what a soon-to-be-defunct magazine does.  But that’s a topic for another day when we want to talk about how schools are more likely to be named after manatees than George Washington.)

Where were we?  Oh yes.  It is important to recognize negative as well as positive accomplishment.  So I introduce “The Higgy,” an award named after William Higinbotham, as the mirror award to our well-established “Al.”

Just as Al Copeland was not without serious flaws as a person, William Higinbotham was not without his virtues.  Higinbotham did, after all  develop the first video game.  But Higinbotham dismissed the importance of that accomplishment and instead chose to be an arrogant jerk by claiming that his true accomplishment was in helping found the Federation of American Scientists and working for the nonproliferation of nuclear weapons.  I highly doubt that the Federation or Higinbotham did a single thing that actually advanced nonproliferation, but they sure were smug about it…

I suspect that Al Copeland, by contrast, understood that he was a royal jerk.  And he also understood that developing a chain of spicy chicken restaurants really does improve the human condition.  Higinbotham’s failing was in mistaking self-righteous proclamations for actually making people’s lives better in a way that video games really do improve the human condition.

So, “The Higgy” will not identify the worst person in the world, just as “The Al” does not recognize the best.  Instead, “The Higgy” will highlight individuals whose arrogant delusions of shaping the world to meet their own will outweigh the positive qualities they possess.

We will invite nominations for “The Higgy” in late March and will announce the winner, appropriately enough, on April 15.  Thanks to Greg for his suggestions in developing “The Higgy.”

Setting the Record Straight on Choice

March 28, 2019


(Guest post by Greg Forster)

OCPA carries my article fisking an especially sloppy smear job attacking school choice:

I call it a strange document because it’s trying to present itself as some sort of scholarly press release. It’s published by something calling itself Scholars Strategy Network, and the byline is from two academics, with their academic affiliations and emails listed at the top. But it’s not a work of scholarship, nor is it informed by scholarship. It’s two pages of emotive bullet points, unsubstantiated bumper-sticker assertions, shoddy reasoning, and deceptive characterizations of the empirical research. An impressively long list of “sources,” formatted to look like scholarly citations, is supplied at the end in the desperate hope of simulating gravitas.

Come for the blatant dishonesty about easily checkable facts:

Assaulting private school choice, the authors appear to be afraid to make their own assertions, but quote someone else’s claim that no “independent studies” have ever found that students using private school choice in Milwaukee, Cleveland, or Washington, D.C. performed better than children who remained in public schools. But the official study in Washington, D.C. looked at exactly this question, using random-assignment methods (the gold standard), and found huge increases in high school graduation and college attendance rates. No doubt the report being cited here doesn’t count this study as “independent,” because it was a federal program and the study was federally funded as part of the program. That’s blatantly dishonest cherry-picking. And what is their excuse for leaving out the two—two!—gold-standard studies of this question finding academic improvements in Milwaukee?

Not to mention that it’s cherry-picking to include only selected cities. Across all private-choice programs, there have been a total of 18 gold-standard studies (no cherry-picking). Of these, 14 found academic improvements, two found no visible effect, and two (both examining a poorly designed program in Louisiana) found negative results.

Stay for the outrageous ideological claptrap!

The authors complain that schools in choice programs are not “transparent.” But parents have the power to demand whatever information they think important, or not attend the school. This is why private schools are already more transparent, by orders of magnitude, than organizations are typically required to be when participating in other kinds of government programs. Look at the reams of hard data on named, particular private schools on or in the U.S. Department of Education’s Common Core of Data. Then try getting comparable data on apartment buildings that take Section 8 housing vouchers, or grocery stores that take food stamps.

Send me a scholarly press release letting me know what you think!

New Arts Studies Lost in Busy Week

March 15, 2019

Image result for lost in the shuffle

It’s been a busy week with the publication of an op-ed by me and Rick Hess in the Wall Street Journal and a study in Education Next documenting the monolithic partisan composition of education reform advocates and those who conduct research on those efforts. One might think that folks committed to evidence-based decision-making would be very interested in facts about their field, but their social media response has generally been counter-productive and fact-free.  Those responses have focused on how they are not to blame, how Republicans are icky anyway, and how many of their best friends are Republicans.  I’m not bothering to link to those responses because there really is no point.  If folks are happy with a uniformly Democratic movement, then they are welcome to keep it… as long as someone continues to be willing to pay for this party.  Given the groupthink and political ineffectiveness that is likely to result from this lack of heterodoxy, I can only wonder why and for how long funders will subsidize it.

Lost in the shuffle of this busy week, some graduate students and I released two new studies of the medium-term effects of students receiving multiple arts-focused field trips to the Woodruff Arts Center in Atlanta.  We randomly assigned school groups to a treatment that involved three field trips per year to visit an art museum, see live theater, and listen to the symphony, or to a control condition.  Among the treated students, some received 3 experiences over 1 year and some received 6 experiences over 2 years.

We split the analyses into two separate reports.  The first, led by Heidi Holmes Erickson, found that these arts-focused field trips improved school engagement, as measured by disciplinary infractions and survey responses, as well as increased standardized test scores in math and reading. These benefits persisted even one year after treatment ended for the first cohort in the study.

The second study, led by Angela Watson, examined social-emotional outcomes.  It found that exposure to multiple arts-focused field trips increased social perspective taking and tolerance.  It also found evidence of an improvement among treated female students in their conscientiousness, as measured by survey effort.

Heidi and Angela will be presenting these results at the Association for Education Finance and Policy conference next week.  Please attend their sessions to learn more about this research and to provide suggestions for improving their papers.  And if folks at AEFP are also interested in engaging in a productive discussion of how to improve the intellectual and ideological diversity of the organization, that would also be wonderful.

Sympathy for Teachers and Taxpayers

March 8, 2019

(Guest Post by Matthew Ladner)

I was 19 when I first heard this Rolling Stones song. Yes I know late to the party but whatevs I was born in 1967 so the Stones were not exactly on point for my generation, and to this day I hurl a trident at anyone who recognizes any Stones album besides Some Girls as their finest work.

Coo Coo!

So anyway I’m 19 and during the final exam period my dorm had something known as “yell spell” where you could make as much noise as you wanted for 15 minutes. One of my fellow residents somehow had two closet sized speakers in his broom closet sized dorm room and played the original version of this song at an unimaginably loud level. Every “Pleased to meet you!” would rattle your teeth. RAs from other floors rushed down and yelled at our RA, but it didn’t matter because no one could hear you scream. As my eardrums began to rupture, I was struck by the following lyrics:

I shouted out
Who killed the Kennedys?
When after all
It was you and me

Anyhoo this all came back to mind as I began looking into some of the factors influencing teacher pay in Arizona in a piece for the Chamber Business News:

The teacher who posted her paystub on social media however received an annual pay increase of $131.25 despite the override vote, and average salaries dropped by an average of $7,885 between 2017 and 2018. In addition, the fund balance of this district stood at $52 million on July 1, 2016 according to the Superintendent’s Financial Report but at $173 million on June 30, 2018. One cannot discern what the district plans to do with these balance funds from the report but thus far it does not seem to involve improving teacher pay.

I searched the employee association websites looking for signs of displeasure regarding the drop in teacher pay, or the percentage of funds devoted to teacher salary. I didn’t find any. Not only do teachers have reason to be frustrated with this, so do district taxpayers who supported the override.


Picture Yourself in a State by the Ocean with Really Low Scores and Nothing to Lose

March 4, 2019

(Guest Post by Matthew Ladner)

Believe it or not 20 years have passed since Jeb Bush kicked off the Florida K-12 reforms during the 1999 legislative session. In a kickoff post at Redefined for a series of articles I lay out the reasoning behind the Florida Supreme Court’s decisive rejection of a recently concluded challenge to these reforms.

A highly coveted Jayblog “No-Prize” goes to whomever leaves the best Beatles lyric pun in the comment section that hasn’t already been used on social media.