Jay P. Greene's Blog

(Guest post by Robert Costrell)

Background:   The essay below was written by invitation for Education Week, to be published along with other commentaries, in conjunction with the release of Rick Hess’ annual “edu-scholar” rankings this month.   The invitation was to respond to the prompt below.  Ed Week accepted the essay for publication, but then pulled it when I declined to submit to their extensive requested revisions.

Ed Week’s Prompt:  “It’s no great secret that the American professoriate tilts to the left, particularly in the social sciences and humanities. The disjuncture between the academic mainstream and a large swath of the American public has been especially evident during this year’s heated presidential campaign and in the course of the Trump transition. What should public-minded academics make of this? Is it a problem if academic sentiment generally aligns with one side of the political spectrum? Does it create challenges for the academy or limit the ability of academics to offer policy ideas or engage in a more robust public debate? What, if anything, should publicly-engaged academics try to do about any of this?”

My Essay:

Education Policy Scholarship a Bright Spot in a Depressing Political and Academic Era

In a depressing era for politics, policy, and academia, our little field of education policy scholarship stands as a bright spot – for now, at least.  The current presidential transition actually illustrates the point.  Unlike other nominations, the debate over Betsy DeVos has been well informed by policy scholars, regarding Michigan’s charter schools, a central element of Ms. DeVos’ record.  Our colleagues have debated what the evidence says or does not say and how the policy choices in Michigan may or may not have worked.  Yes, some political posturing has crept into that debate, even among our public scholars, but compared to what we have seen during and beyond the long presidential campaign, and over many decades in academia, our field’s contribution to this current episode is not bad.

It was not always thus.  Thirty years ago, education policy scholarship was a backwater.  With some notable exceptions, it was non-disciplinary, as opposed to inter-disciplinary.  Since then, high-caliber economists, political scientists, legal scholars, and others have brought their disciplines into a weak field and, on the whole, have strengthened it. Consider the transformation of our main professional organization, the Association for Education Finance and Policy (AEFP).  AEFP founded an impressive new journal (Education Finance and Policy, MIT Press) and our annual conferences feature growing numbers of well-trained, bright young scholars, interacting with policy practitioners.  Yes, political predispositions often lurk beneath the surface, and sometimes erupt, but in general they are well-contained.  By contrast, just look at those fields – including some major ones – where annual conferences are hijacked by destructive elements seeking to academically boycott their colleagues from Israel.  That kind of thing does not happen at AEFP.

Looking more broadly to the question posed above, regarding the current impact of academia’s leftward tilt, there are definitely reasons for concern.  Ideally, personal political convictions of the faculty should not matter, if we all adhered to the historical norm for the academic temperament, checking our political, religious, and other personal passions at the university door.  But over the nearly four decades since I entered the professoriate, that norm has been increasingly abandoned.  Far too many of our colleagues want our academic freedom, but not the “corresponding duties” of scholarly restraint emphasized by the AAUP a century ago.  Moreover, many universities now face student activists born and bred in this environment, who exacerbate matters by demanding ever-more politicized compliance on our part with specified stances and speech, abetted in too many cases by pusillanimous administrators.

A politically lop-sided professoriate will obviously have reduced influence on public policy when its side is out of power.  Yes, there is a role for the loyal scholarly opposition.  Indeed, historically Democratic and Republican scholars would simply switch places, between government and academia, when elections so determined.  But, what now?  Where are Republican state governments to turn for academic policy expertise after their dramatic ascendancy across the country since 2009?  Moreover, when a State House switches power again, where will the few Republican academics who served those administrations find an academic home, from which to provide their loyal scholarly opposition?

This is not only a matter of the service we can provide our governments, but conversely, the immense benefit we can derive as scholars from a stint of public service.  I speak from personal experience of the insights – and humility – acquired during seven years of serving three Republican governors in Massachusetts facing veto-proof Democratic legislatures.  Ironically, I found far more political tolerance and dialogue in that divided State House than in the academic environment from which I had come at the University of Massachusetts.

The recent presidential vote – populist in result – suggests the problems in academia may run deeper than a simple lack of balance between right and left.  I agree with New York Times columnist David Brooks and the Brookings Institutions’ William Galston, writing in the Wall Street Journal, that our national problem has been the collapse of the broad center.   If we are to have political identifications in academia (not my first-best solution), we should try to restore the presence of and dialogue between the center-right and center-left.

As for education policy scholarship, to stay relevant, we should learn from the disastrous examples in so many other parts of academia.   We should resist the encroachment of identity politics, which has proven so corrosive, both to academia and the comity of our body politic.   We should also resist the hubris to which policy scholars can succumb.  When a publicly-engaged scholar like MIT economist Jonathan Gruber is repeatedly caught on video bragging that he put one over on the “stupid” American public in his design features for Obamacare, should we be surprised when large swaths of that public vote against the whole idea of policy expertise?

Let us borrow Benjamin Franklin’s famous warning after the Constitutional Convention, much quoted in the aftermath of our recent election.  We in education policy scholarship have a good field, “if we can keep it.”

——————————————————————

Robert Costrell is professor of education reform and economics at the University of Arkansas.  He served MA Governors Paul Cellucci, Jane Swift, and Mitt Romney, 1999-2006, as policy research director, chief economist, and education advisor.

Doug Harris has a new post that attempts to reply to the many critics of his New York Times op-ed, including me.  In the NYT piece Harris claimed that “one well-regarded study found that Detroit’s charter schools performed at about the same dismal level as its traditional public schools.” He also claimed that the relatively light regulatory approach to Detroit’s charter schools has led to “the biggest school reform disaster in the country.”  He prefers instead the heavy-regulation approach adopted in New Orleans, which he says has produced “impressive” results.  He then suggests that failing to believe these claims is “a triumph of ideology over evidence.”

Several critics, including me, noted that the “well-regarded study” Harris cites actually finds that Detroit charter schools are producing significantly greater gains than traditional public school alternatives — gains that are only slightly smaller than those in New Orleans and greater than in another high-regulation darling, Denver.  But Harris wants to continue posturing as the person backed by science and evidence, while describing his opponents as ideologues.  In his reply to critics in Education Next he uses the word “evidence” 16 times.

So, his defense of the claim that Detroit charters perform “at about the same dismal level as its traditional public schools” must be based on evidence, right? Actually, no.  He provides four arguments to rescue his assertion that “the failure of Detroit charter schools to improve student outcomes” is true despite the fact that the CREDO evidence he cited to make that claim shows otherwise.  Each one of those four arguments is based on assumptions, not evidence.  If you believe all of his assumptions, you might believe his claim that Detroit charter schools really are a disaster. But drawing conclusions that depend completely on assumptions and are contrary to the evidence he cites is certainly not science.  It seems more like a faith-based or ideological exercise.

Let’s consider his four arguments.  First he says, “given the lack of oversight in Detroit and evidence from other cities that some charter schools cherry-pick their preferred students, these results may make Detroit’s charter schools look better than they are.”  Got that?  He has no evidence that Detroit charters are cherry-picking students at all, let alone that they are doing so at a higher rate than in New Orleans, but he nevertheless posits that “if it’s happening, then the charter effects on achievement [in Detroit] are inflated.”

Second, given Harris’ assumed concerns about cherry-picking in Detroit charter schools and the inability of the CREDO study to account for that, he examines evidence from the urban NAEP test and finds that the city of Detroit has experienced below average growth in those scores in recent years.  Using NAEP results from the entire city to draw conclusions about Detroit’s charter schools requires a host of assumptions.  He’d have to assume that test results from all schools, charter and traditional, somehow speak to the effectiveness of charter schools.  He’d have to assume that demographic and other non-school factors in Detroit do not affect the comparison of test growth in Detroit relative to other cities.

A number of education analysts have coined a term — misNAEPery —  to capture how unreasonable it is to make the assumptions required to use NAEP to compare policies across jurisdictions.  Oddly, some of those analysts who like to accuse others of misNAEPery, like Morgan Polikoff and Matt Barnum, have somehow failed to denounce Harris’ use of misNAEPery in both the NYT op-ed and in the Ed Next reply to critics.  Both have even “retweeted” Harris’ new post, so we can assume they’ve read it.

And just to anticipate concerns about my own consistency on the use of NAEP, I think comparisons using NAEP that control for observed demographics are about as convincing as the CREDO results, which also rely on comparisons controlling for some observed characteristics.  Information from NAEP or CREDO can be interesting or suggestive, which is why I say that Arizona charters “appear” to be doing very well, even if I am not convinced that any of this is causal.  At the very least, it is useful to offer a disclaimer that neither NAEP nor CREDO provides convincing causal evidence, even if confession does not assure absolution.  Harris does not offer any disclaimer and instead uses his NAEP comparisons to bolster his assumption that Detroit charters may be cherry-picking.

Third, Harris builds on the observation that Detroit city has very low NAEP test scores to assume that “the extraordinarily low standing of the city as a whole, to the degree it is caused by low performance of traditional public schools, should make it easier to improve student outcomes when trying something new.”  It is also quite plausible — perhaps more plausible — that a city with extraordinarily low test scores also has severe social and economic problems that are outside of the control of schools, which would make it harder for charters to improve outcomes.

Lastly, Harris argues that even if Detroit charters have produced gains, the gains are smaller than those in New Orleans, so we should prefer the regulatory approach used in New Orleans to the one in Detroit. But this assumes that any greater gains produced by New Orleans charter schools are caused by the regulatory approach in that city.  In fact, we have no idea whether New Orleans’ regulations helped, hurt, or had no effect on how large the gains in that city were. For all we know, the gains made by New Orleans charters are largely attributable to the importation of top-notch human capital from elite colleges and a huge increase in per pupil spending, and that these gains were made despite the hindrance of burdensome regulations.  The heavy regulations in Denver somehow failed to produce gains as large as those in Detroit.

Other than these four-assumption-dependent arguments, Harris offers very little to defend the strong claims he made in the NYT that Detroit charter schools have been a “failure” and a “disaster.”  He does cite some national “charter-friendly” organizations, like CRPE and NACSA,  as being critical of Detroit charters.  But that is an argument from authority — not evidence.

He also falsely suggests that Detroit has failed to close failing charter schools.  In fact, 30% of Detroit’s charter schools have been shuttered.  And another national charter-friendly organization, NAPCS, gives Michigan higher marks on closure than Louisiana, noting that Michigan has closed more charter schools than Louisiana, 47 to 26.

Harris also cites two voucher studies with negative results — one of which is an RCT and the other not — as proof that lower regulation approaches are less effective.  Leaving aside the distinct possibility that the negative RCT result for Louisiana was actually a function of the over-regulation of that program, it’s important to note that Harris fails to cite the entire literature of rigorous studies on the effects of private school choice programs, which overwhelmingly shows positive outcomes.

I agree with Doug that ideology, which could more kindly be described as “principles” or “values,” has an important role to play in policymaking.  I just disagree with him that the evidence clearly shows the superiority of a high-regulation approach to school choice.  I think it’s more accurate to say that the evidence is unclear on this matter, which means that we may — appropriately — need to rely more on our values, principles, and broader ideology when deciding how to proceed.

[Edited to remove tangential comment that was poorly phrased.]

School choice is on the march.  Even before Trump’s election and his selection of Betsy DeVos as Secretary of Education, charter and private school choice have been steadily expanding every year.  Expanding school choice, like almost all of education reform, occurs in the states, so who is in charge in DC will not make too much of a difference other than turning a headwind into a tailwind.

But there is a certain crowd whose lives and thoughts revolve around what is happening in DC, so they have become all aflutter with either hope or dread about the prospects of a Trump presidency for school choice.  Those motivated by dread have fired first in the looming Regulation War, arguing that choice only works — if it works at all — within a heavy regulatory framework to ensure quality.  If choice can’t be stopped at least it can be controlled and that control can improve the choices parents will make.

The problem with the pro-regulation argument is that you can’t ensure quality if you don’t have the ability to predict quality.  It’s true that parents have a hard time anticipating how different schools might affect long-term outcomes and are quite likely to make mistakes in choosing schools.  And some emphasize this fact to justify a strong role for regulators in controlling the range and type of options from which parents can choose.  But what they rarely consider is whether regulators are any better at anticipating how schools will affect long-term outcomes.  After all, regulators can’t protect families against making mistakes if they are equally or more likely to make those mistakes themselves.

I’ve written previously about the disconnect between near-term test score gains and changes in later life outcomes.  If we can’t reliably use rigorously identified test score gains to predict later life outcomes, then on what basis will regulators be able to judge quality to protect families against making bad choices?  And the situation is even worse because most regulators making decisions about what choice schools should be opened, expanded, or closed are not relying on rigorously identified gains in test scores — they just look primarily at the levels of test scores and call those with low scores bad.  Of course, all this does is punish and discourage schools from attempting to serve the most challenging students since the level of scores is more a reflection of student background than school quality.

But let’s say you don’t believe me about the weak predictive power of test score gains and are determined to use tests as the main indicator of school quality.  We are still left with the question of whether regulators are any good at identifying which schools will contribute to test score gains.  Fortunately, we have a recent study that examined whether the criteria used by regulators in New Orleans are predictive of test score growth — even if we accept test gains as a reliable indicator of quality.  The bottom line is that none of the factors used by authorizers to open or renew charter schools in New Orleans were predictive of how much test score growth these schools could produce later on.

In particular, the study examines ratings derived from criteria favored by the National Association of Charter School Authorizers (NACSA) to see if they are predictive of test score growth or enrollment growth.  Neither the NACSA aggregate rating nor individual factors, like school governance features, educational plans, or financial and enrollment characteristics were predictive.  As the study concluded, “None of the factors, however, predict the main dependent variables of interest: SPS, value-added, enrollment, and enrollment growth, as direct measures of future school performance. ”

If regulators are unable to predict which schools will be good (assuming, falsely, that test score gains are a reliable indicator of good schools), how are they supposed to “protect” parents from making bad choices about schools?  The case for regulation is often expressed as a tautology: people will make better choices if they only have good schools from which to choose. But this presupposes that regulators can identify and eliminate bad choices.  And if regulators really could do this, why have school choice at all?  Why not just ensure that every student succeeds by forbidding any school from being bad?

No one should be surprised that NACSA’s criteria have no relationship to their own metric for school quality — test score growth — given how well Arizona charter schools appear to be doing even while NACSA gives the state a very low score for charter quality.  What might be more surprising is that the recent study I’ve been referencing that finds no connection between the criteria regulators use and future test score growth was co-authored by none other than Doug Harris.  Yes, that’s the same Doug Harris who wrote the NYT op-ed claiming that the lack of regulation in Detroit contributed to “the biggest school reform disaster in the country,” while New Orleans’ more heavily regulated approach produced “impressive” results.

As Ramesh Ponnuru has noted, using the same CREDO research that Doug cites, Detroit charter schools produce significantly better outcomes than the traditional public school alternatives.  In fact, the test score gains achieved by Detroit charters exceed those in heavily regulated Denver and only slightly lag those in New Orleans.  (See the table above for a graphic presentation of results across cities).

Mind you, I do not believe the CREDO results because that research only matches on observed characteristics and therefore does not have a rigorous identification of causal effects.  But Doug seems to believe that research even though it undermines the very argument he was making.  And one can only assume that Doug believes his own research showing that regulators lack effective tools to identify and predict good schools.  Doug accuses DeVos of being guilty of “a triumph of ideology over evidence,” but it seems like there is a lot of that going around.

Over the last few years I have developed a deeper skepticism about the reliability of relying on test scores for accountability purposes.  I think tests have very limited potential in guiding distant policymakers, regulators, portfolio managers, foundation officials, and other policy elites in identifying with confidence which schools are good or bad, ought to be opened, expanded, or closed, and which programs are working or failing.  The problem, as I’ve pointed out in several pieces now, is that in using tests for these purposes we are assuming that if we can change test scores, we will change later outcomes in life.  We don’t really care about test scores per se, we care about them because we think they are near-term proxies for later life outcomes that we really do care about — like graduating from high school, going to college, getting a job, earning a good living, staying out of jail, etc…

But what if changing test scores does not regularly correspond with changing life outcomes?  What if schools can do things to change scores without actually changing lives?  What evidence do we actually have to support the assumption that changing test scores is a reliable indicator of changing later life outcomes?

This concern is similar to issues that have arisen in other fields about the reliability of near-term indicators as proxies for later life outcomes.  For example, as one of my colleagues noted to me, there are medicines that are able to lower cholesterol levels but do not reduce — or even may increase — mortality from heart disease.  It’s important that we think carefully about whether we are making the same type of mistake in education.

If increasing test scores is a good indicator of improving later life outcomes, we should see roughly the same direction and magnitude in changes of scores and later outcomes in most rigorously identified studies.  We do not.  I’m not saying we never see a connection between changing test scores and changing later life outcomes (e.g. Chetty, et al); I’m just saying that we do not regularly see that relationship.  For an indicator to be reliable, it should yield accurate predictions nearly all, or at least most, of the time.

To illustrate the un-reliability of test score changes, I’m going to focus on rigorously identified research on school choice programs where we have later life outcomes.  We could find plenty of examples of disconnect from other policy interventions, such as pre-school programs, but I am focusing on school choice because I know this literature best.  The fact that we can find a disconnect between test score changes and later life outcomes in any literature, let alone in several, should undermine our confidence in test scores as a reliable indicator.

I should also emphasize that by looking at rigorous research I am rigging things in favor of test scores.  If we explored the most common use of test scores — examining the level of proficiency — there are no credible researchers who believe that is a reliable indicator of school or program quality.  Even measures of growth in test scores or VAM are not rigorously identified indicators of school or program quality as they do not reveal what the growth would have been in the absence of that school or program.  So, I think almost every credible researcher would agree that the vast majority of ways in which test scores are used by policymakers, regulators, portfolio managers, foundation officials, and other policy elites cannot be reliable indicators of the ability of schools or programs to improve later life outcomes.

With the evidence below I am exploring the largely imaginary scenario in which test scores changes can be attributed to schools or programs with confidence.  Even then, the direction and magnitude of changing test scores does not regularly correspond with changing later life outcomes.  I’ve identified 10 rigorously designed studies of charter and private school choice programs with later life outcomes.  I’ve listed them below with a brief description of their findings and hyperlinks so you can read the results for yourself.

Notice any patterns? Other than the general disconnect between test scores and later life outcomes (in both directions), I notice that the No Excuses charter model that is currently the darling of the ed reform movement and that New York Times columnists have declared as the only type of “Schools that Work” tend not to fare nearly as well in later outcomes as they do on test scores.  Meanwhile the unfashionable private choice schools and Mom and Pop charters seem to do much better on later life outcomes than at changing test scores.  I don’t highlight this pattern as proof that we should shy away from No Excuses charters.  I only mention it to suggest ways in which over-relying on test scores and declaring with confidence that we know what works and what doesn’t can lead to big policy mistakes.

Here are the 10 studies:

1. Boston charters (Angrist, et al, 2014) – Huge test score gains, no increase in HS grad rate or postsecondary attendance. Shift from 2 to 4 yr
2. Harlem Promise Academy (Dobbie and Fryer, 2014) – Same as Boston charters
3. KIPP (Tuttle, et al, 2015) – Large test score gains, no or small effect on HS grad rate, depending on analysis used
4. High Tech High (Beauregard, 2015) – Widely praised for improving test scores, no increase in college enrollment
5. SEED Boarding Charter (Unterman, et al, 2016) – same as Boston charters
6. TX No Excuses charters (Dobbie and Fryer, 2016) – Increase test scores and college enrollment, but no effect on earnings
7. Florida charters (Booker, et al, 2014) – No test score gains but large increase in HS grad rate, college attendance, and earnings
8. DC vouchers (Wolf, et al, 2013) – Little or no test score gain but large increase in HS grad rate
9. Milwaukee vouchers (Cowen, et al, 2013) – same as DC
10. New York vouchers (Chingos and Peterson, 2013) – modest test score gain, larger college enrollment improvement

Every donor, every foundation or advocacy staffer, every academic, or anyone else who cares about having an intelligent strategy for improving education should watch this video.  It’s as if he’s been reading this blog and stealing our thoughts, but Katzman puts it all together in an incredibly compelling way.

The bottom line is improving schools is going to require more markets and choice and less testing and accountability.  And you don’t have to worry too much about testing and accountability because it is so politically unpopular that it will mostly destroy itself.

My former students, Dan Bowen and Albert Cheng, have a new study that was just published in the Journal of Catholic Education on how religious priming may affect student character or non-cognitive skills.  They find that priming students to think about religion increases students’ willingness to delay gratification as well as their political tolerance.  No effects are observed if students are instead primed to think of secular success.  This work suggests that there may be particular benefits from religiously-based education that are more difficult to produce in a secular context.  Abandoning private religious education for secular charter schools may come at a cost to these character skills.

These results come from an experiment we conducted at the Arkansas School for Mathematics, Sciences and the Arts (ASMSA), a public boarding school in Hot Springs.  In the experiment we randomly assigned 180 students to one of three conditions.  All students were asked to work on a sentence scramble exercise in which there are ten sets of five words.  Students were asked to look at each set of five words, drop one word, and then make a sentence out of the remaining four words.

In 5 of the sentences one word was altered, changing only five of the 50 words across the three conditions.  One group was primed to think about religion by having the words “worship, preacher, heaven, devotion, and commandments” included in the sentence scramble.  A second group was primed to think about secular equivalents: “honor, leader, success, commitment, and expectations.”  And a third group saw neutral words: “eat, path, man, cabbage, and numerous.”

Prior research had found that students primed in this way to think about religion demonstrated higher levels of self-regulation.  The idea of this experiment was to attempt to replicate those previous findings while exploring whether secular equivalents could produce similar effects.  Several observers have noted that KIPP and other high-achieving charter schools appear to simulate the religious rituals of Catholic schools but replace religious rhetoric with talk of secular success and achievement.  The question this study explores is whether talk of secular achievement appears to be as motivational for students as religious rhetoric.

Dan and Albert find that something is lost when we substitute secular aspirations for religious ones.  Students exposed to the religious priming expressed a stronger feeling of religiosity.  So, the priming worked in getting students to think about religion, even though changing only 5 words out of 50 is very subtle and the students were not consciously aware of the nature of the manipulation.

Students exposed to this religious priming experienced an increase in delayed gratification in that they were more willing to receive $6 the following week as compensation for participating in the study rather than $5 right then.  Students in the religious priming condition were also more likely to express political tolerance on the Sullivan scale, which measures people’s willingness to allow disliked groups to engage in political activities, like holding rallies, having books in the library, or running for office.  But students exposed to the secular success priming were no different from the neutral priming control group in that both were less likely to delay gratification or express political tolerance.

This was a small scale experiment and the effects were observed at p<.1, so there are limits to how confident we should be about the results.  But given that the results are consistent with prior research, we should have some concerns about dropping religious school choice in favor of secular charter schools.  This is especially so given that the No Excuses charter model that has become the darling of ed reformers often comes up short at improving later life outcomes, while private school choice programs seem to fare better at improving high school graduation, college enrollment, and even earnings.

This year’s set of Al Copeland Humanitarian Award nominees was particularly strong, making selection of a winner exceptionally difficult.  As Greg noted in a comment, “this is clearly a ‘political’ year for The Al, in the sense that we’re all nominating witnesses against injustice rather than the creative entrepreneurs who usually dominate.”

Well, almost all.  Matt, as is his habit, nominated the entrepreneurs, Tim and Karrie League, who developed the Alamo Draft House chain of movie theaters.  The Alamo Draft House is one of the greatest places on earth.  The theaters carefully select movies, audience activities, food, and drink to create a completely engaging and entertaining experience.  Some people give hundreds of millions of dollars to art museums that fail to package their offerings nearly as well as Tim and Karrie League do.  And the Alamo does it without any donations while making a profit.  Improving the human condition while also making profit is a quintessential characteristic of winners of The Al.  And I almost slected Tim and Karrie League for this honor.

But as Greg said this seems like a political year in which selecting a traditional entrepreneur-type as the winner just didn’t seem right.  All of the other nominees fell in the “witnesses against injustice” category and with so much injustice all around us, I felt like I should choose one of them.  I could have chosen Jason’s excellent nominee, Remy Munasifi, whose musical parodies expose and help rebut oppression, hypocrisy, and other types of foolishness.  I could also have chosen my own nominee, Yair Rosenberg, whose trolling of neo-Nazis and other anti-Semites on Twitter deprives these bullies of the sense of power that drives much of their behavior.

Instead, I have chosen Master Sergeant Roddie Edmonds over Remy Munasifi and Yair Rosenberg because Edmonds was more than a witness to injustice.  He actively took steps, at enormous danger to himself, to promote justice in the world.  By refusing to comply with Nazi orders to separate Jewish POWs and insisting that he and all of the soldiers under his command were Jewish, Edmonds risked being shot to defy the Nazi’s hateful and murderous plans against Jews.

I hesitated for a moment in selecting Edmonds only because The Al does not typically go to people who have been widely recognized elsewhere, like Steve Jobs or John Lasseter, and Edmonds was recently honored as one of the Righteous Among the Nations at Yad Vashem, Israel’s Holocaust museum.  Unfortunately, being honored by Yad Vashem does not constitute being widely recognized, so drawing more attention to Edmonds seems important and fitting.

As much as I love Remy Munasifi and Yair Rosenberg, mocking injustice on the internet just isn’t enough.  As Ken M, last year’s winner of The Al, taught us, social media is a pretty useless forum for trying to improve the world. So, that silly video you shared on Facebook or that sly remark you made on Twitter doesn’t really do much other than amuse you.

There’s nothing wrong with some amusement. After all, that is the Prime Directive of this blog — to amuse ourselves rather than to change the world.  And being amusing is a lot better than those insufferable political rants or self-righteous internet petitions, which are all talk and no action.

If you want to fight injustice you can’t really do much with a blog, Twitter, or Facebook.  You need to find real injustices, not trumped-up (pardon the pun) minor slights like:

And then you need to follow the example of Master Sergeant Roddie Edmonds and take action that might even put yourself at risk.  Evil will always remain in the world, but we will suffer less from it if we have more people like Master Sergeant Roddie Edmonds.

When we examine the results of standardized test scores we typically think we are seeing evidence of what students know.  As it turns out, that is only partially right.  Test scores capture both what students know as well as their willingness to exert effort to show us what they know.

A new paper by my colleagues, Gema Zamarro, Collin Hitt, and Ildefonso Mendez uses multiple, novel techniques to demonstrate that between 32% and 38% of the variation in PISA test performance across countries can be explained by how much effort students are willing to exert rather than what they know.  The implications of this finding for ed reform are huge.  When we see low test score performance we are often misdiagnosing the problem as poor content instruction when it may in fact be insufficient development of student character skills.  If we focus all of our energy on the former without addressing the later, we’ll fail to make as much progress.

So, how do Gema, Collin and Ildefonso know that between 32% and 38% of variation in PISA test performance across countries is explained by effort?  They used three different methods to measure the influence of effort.  First, they took advantage of the fact that the order of questions without the PISA was randomly ordered.  They then compared how well students performed on the first set of items relative to the last.  Because the order of items was randomized the first and last questions were, on average, of equal difficulty.  The decline in getting items correct from the start to the end of the exam is therefore a function of the decline in effort students are willing to exert, not the difficulty of the items.

If you compare performance in the US and Greece (as can be seen in the figure above), students in the two countries do about as well at the beginning of the test.  That means that students in Greece and the US know about the same amount of stuff.  But students in Greece decline much more rapidly across the test, which means that those students are less willing to exert consistent effort.  When we compare PISA results from the US and Greece we wrongly conclude that content instruction in Greece must be much worse.  In reality, Greek students know as much as students in the US but simply exert a lot less effort.

A second way the paper measures effort is by examining responses students gave to a survey that was administered at the same time as the PISA.  Using novel techniques that have been validated in previous research, they measure the extent to which students skip answers (or say “don’t know”) as well as the extent to which students give careless answers as proxies for their effort.  Both skipped answers and careless answers yield very similar results to what they find from the decline across the test.

Some people have expressed skepticism about the focus on “non-cog” or character in education research because they believe that these capture personality traits that are largely inherited and immutable.  This research contradicts that claim.  Unless we think there are big and important genetic differences across countries, the variation in effort across countries has to be explained by factors that are social constructed and, at least in theory, could be changed.  In addition, great work by Gema and Albert Cheng has found that student effort can actually be changed when students are randomly assigned to different teachers who themselves possess different character skills.

The evidence is becoming clear that character matters and is subject to influence by the education system.

Massachusetts voters will be deciding in a few weeks whether to expand charter schools in the state. By all rights, the measure should be winning by a landslide.  Rigorous evaluations of existing Boston charters show large test score gains.  Charter supporters are spending millions to blanket the airwaves with ads. And following what appears to be the new ed reform ideal model, Massachusetts charters predominantly serve highly disadvantaged communities, so they have positioned themselves as the progressive promoters of social justice.

Despite all of this, Question 2 is trailing by double digits in recent polls and appears headed for defeat.  Why?  As I’ve written recently, ed reformers appear to have become so obsessed with social justice virtue-signaling that they’ve forgotten how politics actually works.  Narrowly targeting programs toward disadvantaged communities leaves programs politically vulnerable to harmful regulation, restriction, or repeal.  As much as disadvantaged communities desperately need education improvement, they tend to be poorly positioned to advocate for those efforts politically.

If you want to help the poor, you should design programs that include the middle and upper-middle classes.  This is the political genius of Social Security.  It is extremely effective at alleviating poverty among low-income seniors because high income seniors, who tend to be better positioned for political advocacy, also get it.  This is the political genius of many college subsidy efforts — the poor can benefit from them because wealthier families are also eligible.

I understand that Question 2 is attempting to expand charters so that they can include more middle class and upper-middle class families, but those voters are unaware of how charters might benefit them because already existing Massachusetts charters have largely failed to serve them.  And the unions and their local suburban school officials are doing a great job of scaring suburbanites about how a charter expansion might harm the relatively good arrangements they currently enjoy.

Charters in Massachusetts would have been better positioned politically if they had not previously neglected to benefit more middle and upper-middle class families.  Then more politically-advantaged families could have learned about benefits from their own experience and through networks of family and friends.  Ed reform needs to win by convincing middle and upper-middle class families that they can benefit themselves from creating and expanding ed reform programs.  Trying to obtain their support by arguing that poor and minority families who live somewhere else would benefit is a losing strategy.  Most political decisions are driven by crude calculations of self-interest, not high-minded appeals or guilt.

But ed reformers appear to have taken such a strong aversion to the rough political realities of self-interest that guilt is their dominant political message.  For example Richard Whitmire attempted to shame suburban voters, tweeting to an approving chorus of progressive reformers: “All comes down to: Will well-off suburbanites deny better schools to urban parents?”  The fact that Question 2 is trailing significantly in the polls provides the obvious answer to Whitmire’s query — of course suburbanites will deny better schools to urban parents if they think they might lose something and have little to gain.  That’s how politics works.

Until ed reformers temper this antipathy towards more advantaged families, abandon guilt-driven political appeals, and embrace the political realities of self-interest they should expect to continue suffering a series of political defeats.

The 2016 presidential election is a reminder of just how horrible the world can be.  Like Ben Shapiro, who was a star reporter at Breitbart before their pro-Trump-putsch, I used to think that tales of right-wing anti-Semites, neo-Nazis, and other backwoods fascists were largely ghost stories the Democratic party told Jews to keep them from venturing out of the party’s tent.  I imagined that maybe 10% of the population adhered to these fringe beliefs on the right and perhaps another 10% believed in the left-wing version of Jew-hating conspiracy craziness.

Like Shapiro I now have to admit that I was wrong.  This election has brought the crazies out of the woodwork and it is clear they are not a tiny fringe.  America is not as moderate and sensible as I always believed.  We can see evidence of this not only from the huge spike in “alt-right” activity in social media, but also from the large number of voters supporting hateful nonsense in the primaries and in general election polls.

America has a real fascism problem. So, what are we supposed to do about this?  One essential weapon in the anti-fascist arsenal is humor.  Charlie Chaplin mocked Hitler in The Great Dictator. Stanley Kubrick ridiculed Soviet despots as weepy drunks in Dr. Strangelove.  Latin American dictators in the mold of Che Guevera and Fidel Castro were portrayed as silly madmen in Woody Allen’s Bananas and in Alan Arkin’s The In-Laws.

Part of the attraction of these dangerous despots to their crazed followers is the appearance of strength and stature.  And part of the attraction of fascist movements to the stupid and weak is the illusion that they may be part of something great and powerful.  Mocking fascists undermines this appeal by revealing how ridiculous they actually are.

This is why my nominee for the 2016 Al Copeland Humanitarian Award is Yair Rosenberg.  Rosenberg is a journalist who writes mostly for Tablet Magazine, but his work has also appeared in the Wall Street Journal, Atlantic Magazine and elsewhere.  But the forum in which Rosenberg excels in mocking anti-Semites, neo-Nazis, and other backwoods fascists is Twitter.  You might say that he is the Picasso of trolling the “alt-right” and Twitter is his canvass.

One of Rosenberg’s most common methods for mocking the alt-right is to take internet memes they have adopted to spread their views and modify those memes so that they advocate for the opposite.  For example, the alt-right has adopted “Pepe the Frog” as one of their symbols.  Rosenberg turns the meme on its head by putting a Mossad t-shirt on Pepe that declares “It’s Never an Accident.”

Another strategy Rosenberg has for mocking anti-Semites on Twitter is to treat their conspiracy theories seriously and suggest what the consequences might be if they were true.  For example:

(((Yair Rosenberg))) ‏@Yair_Rosenberg Aug 7

I know there’s lots of disinformation out there about Jews, so I try to do my best to dispel people’s misconceptions

(((Yair Rosenberg))) ‏@Yair_Rosenberg Aug 28

Oh no, I’ve said too much

 Rosenberg is also fond of mocking anti-Semitic conspiracy-theorists by acting as if those conspiracies actually exist and producing images of Jews alarmed that the goyim (Yiddish for Gentiles) have discovered the plot.  For example, when world leaders are photographed  whispering in each others’ ears, some internet folks insert captions like “Hail Hydra.”  Rosenberg inserts “The goyim know.”

(((Yair Rosenberg)))Verified account‏@Yair_Rosenberg

“The goyim know!”

 Or Rosenberg places “The goyim know” meme on stereotypical images of Jews.

Rosenberg follows in the tradition of Fasi Zaka, the Pakastani DJ and TV celebrity who I previously nominated for “The Al” for mocking the Taliban for things like smelling bad and needing deodorant.  If fascists like the Taliban or the alt-right are doing a lot to worsen the human condition, people like Yair Rosenberg and Fasi Zaka are making important strides to reverse that damage by mocking these monsters.

 Rosenberg’s efforts remind me of the old joke about two Jews sitting in the park reading newspapers.  One of them is reading an anti-Semitic rag and the other says to him: “How can you read that garbage?”  The one reading the anti-Semitic paper replies: “In your paper Jews are being persecuted and murdered.  In mine we are controlling the banks and media and running the world.  I like reading good news.”  Yair Rosenberg is giving us good news.